JPWO2018029963A1 - MAKE-UP SUPPORT DEVICE AND MAKE-UP SUPPORT METHOD - Google Patents

Abstract

Information for acquiring the first user information and the plurality of second user information from the storage device (500) storing the user information including the preference makeup face image and the makeup preference level information for each user, the makeup assisting apparatus (100) Similar user selection unit (103) that selects similar users from a plurality of second users based on the acquisition unit (101) and the preference makeup similarity calculated based on the first user information and the plurality of second user information Proposed makeup image selection unit (104) for selecting a proposed makeup image from the preference makeup face image of the similar user based on the makeup distance and the makeup similarity calculated based on the first user information and the second user information of the similar user And an information generation unit 105 that generates a simulation image in which the makeup image shown in the proposed makeup image is superimposed on the face image of the first user and outputs the simulation image to a predetermined device.

Description

  The present disclosure relates to a makeup assisting apparatus and makeup assisting method for assisting makeup of a user.

  Conventionally, a technique is known that generates a simulation image by superimposing a makeup image showing the state of makeup when makeup is applied (hereinafter referred to as "make-up") on the face image of a bare face (a face without makeup). (See, for example, Patent Document 1).

  For example, the user selects a desired one from a plurality of sample images prepared in advance (a face image of a model to which a predetermined makeup has been applied), and sets the makeup image of the sample image to the face image of the bare face of the user. A technique for generating a simulation image by superimposing (hereinafter referred to as "the prior art") is known.

Patent No. 1715357 gazette

  However, in the prior art, when the number of sample images is too large, it takes labor and time to select sample images, which may cause a burden on the user. Therefore, it is desirable to be able to propose to the user makeup that meets the user's preferences without increasing the burden of image selection.

  Therefore, one aspect of the present disclosure provides a makeup support apparatus and a makeup support method that can propose makeup that meets the user's preferences.

  The makeup assisting apparatus of the present disclosure includes a first preference makeup face image indicating a face on which a first user's preference is applied, and a first makeup preference level information indicating a degree of preference for makeup. 1 second user preference, a second preference makeup facial image showing a face on which a plurality of second users other than the first user have made a preference, and a second makeup preference showing a degree of preference for makeup Acquiring a plurality of pieces of second user information including degree information, and acquiring a face image of the first user, based on the first user information and the plurality of second user information A preference makeup similarity indicating the similarity between the preference of the first user's makeup and the preference of each of the plurality of second users is calculated, and based on the preference makeup similarity, the plurality of second users are selected. Similar user selection to select one or more similar users Calculating a makeup distance indicating a degree of ease of makeup, based on makeup information associated with the part and the preference makeup face image of the one or more similar users selected, and A cosmetic similarity indicating the degree of similarity between the makeup shown in the preference makeup face image and the makeup shown in each of the preference makeup face images of the one or more similar users is calculated, and the calculated makeup distance and the makeup similarity A proposal makeup image selection unit for selecting a proposal makeup image indicating makeup to be suggested to the first user from the preference makeup face images of the one or more similar users based on the degree of freedom; and the selected proposal makeup image And generating a simulation image by superimposing the makeup image shown in on the face image of the first user, and outputting the simulation image to the display device.

  Note that these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or recording medium, and any of the system, apparatus, method, integrated circuit, computer program, and recording medium It may be realized by any combination.

  The makeup assisting apparatus of the present disclosure can propose makeup matching the user's preference without increasing the burden of image selection.

Block diagram showing an example of the configuration of a makeup support system and makeup support apparatus according to the present disclosure A figure showing a flow chart which shows an example of user registration operation of a makeup support device concerning this indication. A figure showing a flow chart which shows an example of makeup proposal operation of a makeup support device concerning this indication. Figure showing a display example of a sample image according to the present disclosure Block diagram showing an example of another configuration of the makeup support system and makeup support apparatus according to the present disclosure

  Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings.

<System configuration>
First, the configuration of a makeup support system including the makeup support apparatus according to the present embodiment will be described using FIG. 1.

  FIG. 1 is a block diagram showing an example of a makeup support system and a makeup support apparatus.

  The makeup support system 1 shown in FIG. 1 is disposed, for example, in a cosmetics sales shop, a beauty salon, a medical institution, a makeup room for adjusting appearance, an event site, an individual house, and the like.

  As shown in FIG. 1, makeup assisting system 1 includes makeup assisting device 100, imaging device 200, display device 300, operation device 400, and storage device 500.

  The makeup assisting apparatus 100 is an apparatus for assisting the makeup of the user, and is disposed, for example, in a cosmetics sales shop, a beauty salon, a medical institution, a makeup room for adjusting appearance, an event site, an individual home, and the like. The makeup assisting apparatus 100 may be a stationary apparatus or a portable apparatus that can be easily carried. The configuration of makeup assisting apparatus 100 will be described later.

  The imaging device 200 captures an image of a face in front of the user, and outputs a face image obtained by capturing to the makeup support apparatus 100. The face image may be a still image or a moving image. Also, in the following, an image obtained by capturing a face with no makeup (hereinafter referred to as "a bare face") is referred to as "a bare face image" and an image obtained with a face with a makeup (hereinafter referred to as "a makeup face") It is called "make-up face image".

  The display device 300 may be, for example, a face image captured by the imaging device 200, a face image stored in the storage device 500 (for example, a sample image, a preference makeup face image for each user, etc. Details will be described later) indicate. The display device 300 may be, for example, a display such as a personal computer or a smartphone, or may be a large display such as a mirror stand.

  The operation device 400 receives various operations (details will be described later) by the user, and notifies the makeup support device 100 of the contents of the operations.

  The storage device 500 is provided, for example, in a server apparatus (not shown) on the network, and stores various information. Makeup assisting apparatus 100 communicates with storage device 500 via communication means such as a wireless LAN, and acquires various information. Although details will be described later, various information includes, for example, user information, a sample image, and the like.

  Note that at least one of the imaging device 200, the display device 300, the operation device 400, and the storage device 500 may be included in the makeup support device 100.

<Device configuration>
Next, the configuration of makeup assisting apparatus 100 will be described using FIG.

  As shown in FIG. 1, makeup assisting apparatus 100 includes an information acquisition unit 101, a user authentication unit 102, a similar user selection unit 103, a proposed makeup image selection unit 104, and an information generation unit 105.

  Although illustration is omitted, makeup assisting apparatus 100 includes, for example, a central processing unit (CPU), a storage medium such as a ROM (Read Only Memory) storing a control program, a working memory such as a RAM (Random Access Memory), And a communication circuit. In this case, the functions of the units shown in FIG. 1 are realized by the CPU executing a control program.

  The information acquisition unit 101 acquires information from the imaging device 200, the operation device 400, or the storage device 500. Here, "acquisition" includes the meaning of "receive (receive)". In addition, “acquisition” as referred to herein may include the meaning of “acquired from the outside of makeup assisting apparatus 100 via a communication means or the like”.

  In addition, the information acquisition unit 101 acquires the acquired information in the respective units (user authentication unit 102, similar user selection unit 103, proposed makeup image selection unit 104, or information generation unit 105) of makeup assisting apparatus 100, display device 300, Alternatively, the data is output to the storage device 500.

  For example, the information acquisition unit 101 acquires a face image (a bare face image or a makeup face image) of the user from the imaging device 200 and outputs the face image to the information generation unit 105.

  Also, for example, the information acquisition unit 101 acquires user identification information (for example, an ID and a password) from the operation device 400, and outputs the user identification information (for example, an ID and a password) to the user authentication unit 102.

  Also, for example, the information acquisition unit 101 acquires user information from the storage device 500 and outputs the user information to the similar user selection unit 103.

  The acquisition example and the output example of the information by the information acquisition unit 101 are not limited to the above.

  The user authentication unit 102 collates the user identification information (for example, ID and password) input from the operation device 400 with the user identification information (for example, ID and password) stored in the storage device 500, and makes up It is determined whether the user attempting to log in to the support device 100 is a registered user. The registered user refers to a user who has user information registered in the storage device 500 as a user who has the authority to use the makeup support apparatus 100.

  As a result of the determination, if the user who is about to log in is a registered user (that is, if the authentication is successful), the user authentication unit 102 notifies the information acquisition unit 101 that the authentication is successful. The information acquisition unit 101 that has received this notification acquires user information of the registered user from the storage device 500, and outputs the user information to the similar user selection unit 103.

  Note that the authentication by the user authentication unit 102 is not limited to one using user identification information. For example, authentication using face image recognition, voice recognition, voiceprint recognition, fingerprint recognition, iris recognition, password input or the like may be used.

  The similar user selection unit 103 selects a second user similar to the first user based on the user information received from the information acquisition unit 101. The proposed makeup image selection unit 104 will be specifically described below.

  The first user is a user who receives a proposal for makeup from makeup assisting apparatus 100 (for example, a registered user logged in to makeup assisting apparatus 100 to receive a proposal for makeup). The second user is a registered user other than the first user, and a plurality of users exist.

  As shown in FIG. 1, the similar user selection unit 103 has a preference makeup similarity calculation unit 1030.

  First, the preference makeup similarity calculation unit 1030 receives from the information acquisition unit 101 user information of a first user (hereinafter referred to as “first user information”) and user information of each of a plurality of second users (hereinafter referred to as “second user Receive information). Each piece of user information includes user identification information, preference makeup face image, makeup preference level information, makeup procedure information, selected date and time information, and the like.

  The preference makeup face image is an image showing a face to which the user's favorite makeup has been applied. The preference makeup face image is, for example, one selected by the user from among a plurality of makeup face images (details will be described later) presented by the makeup assisting apparatus 100 as options. The user may select a plurality of preference makeup face images.

  The makeup preference level information is information indicating the degree (for example, numerical value) of the preference of the user for the makeup shown in one preference makeup face image. That is, when a plurality of preference makeup face images are selected by one user, makeup preference level information is associated with each of the preference makeup face images.

  The makeup procedure information is information indicating the makeup procedure shown in one preference makeup face image. That is, when a plurality of preference makeup face images are selected by one user, makeup procedure information is associated with each preference makeup face image. In addition, the procedure of makeup is, for example, the order of application of a plurality of makeup items (for example, foundation, concealer, eyebrow, eye shadow, mascara, cheek, lip, etc.). Furthermore, the makeup procedure information may include, in addition to the makeup item, information on additional makeup techniques as to what makeup is to be applied to the makeup item. The additive makeup technology is typically a coating method, and examples of the "fundament" of a make-up item include "adding makeup using a foundation brush and lightly applying a powder foundation".

  The selection date and time information is information indicating a date and time (a date and time may be taken) by which the preference makeup face image is selected by the user. That is, when a plurality of preference makeup face images are selected by one user, selection date information is associated with each preference makeup face image.

  The user information registration operation will be described later with reference to FIG.

  Then, the preference makeup similarity calculation unit 1030 calculates the preference makeup similarity based on the first user information and the plurality of second user information. The preference makeup similarity is a value indicating the similarity between the preference of the first user's makeup and the preference of the second user's makeup. As described above, since there are a plurality of second users, a plurality of preference makeup similarities corresponding to each second user are calculated.

  Here, a specific example of calculation of preference makeup similarity will be described. In the following, the case of calculating preference cosmetic similarity between the first user and the second user a (one of the plurality of second users) will be described as an example.

  For example, it is assumed that the preference makeup face image A is included in the first user information, and the preference makeup face image A is included in the second user information (user information of the second user a). Further, for example, in the first user information, makeup preference level information corresponding to the preference makeup face image A indicates the largest value, and in the second user information, makeup preference level information corresponding to the preference makeup face image A Is the largest value. In such a case, the preference makeup similarity calculating unit 1030, for example, adds 10 points to the initial value (for example, zero) of the preference makeup similarity, and calculates the preference makeup similarity as 10 points.

  Furthermore, for example, it is assumed that the preference makeup face image B is included in the first user information, and the preference makeup face image B is included in the second user information. For example, in the first user information, the makeup preference level information corresponding to the preference makeup face image B indicates the second largest value, and in the second user information, the makeup preference level corresponding to the preference makeup face image B Suppose that the information shows the smallest value. In such a case, the preference makeup similarity calculation unit 1030 subtracts 7 points from 10 points calculated as described above, for example, to calculate the preference makeup similarity as 3 points.

  Furthermore, for example, it is assumed that the preference makeup face image C is included in the first user information, and the preference makeup face image D is included in the second user information. Further, for example, in the first user information, the makeup preference level information corresponding to the preference makeup face image C indicates the third largest value, and in the second user information, the makeup preference level corresponding to the preference makeup face image D It is assumed that the information shows the second largest value. Further, it is assumed that the makeup shown in the preference makeup face image C and the makeup shown in the preference makeup face image D are similar. In such a case, the preference makeup similarity calculating unit 1030, for example, adds 5 points to 3 points calculated as described above, and calculates the preference makeup similarity as 8 points.

  Thus, the preference makeup similarity calculation unit 1030 compares the first user information with each second user information, and calculates the preference makeup similarity by adding or subtracting a predetermined point. The more the preference makeup face images in which the same preference makeup face image or makeup type is similar are included in the first user information and the second user information, the preference makeup similarity becomes a larger value. It can be assumed that the first user and the second user have similar preference of makeup as the preference makeup similarity is a larger value.

  The specific example of the calculation of the preference makeup similarity has been described above.

  The similar user selection unit 103 is a user having similar preference of makeup to the first user among the plurality of second users based on the plurality of preference makeup similarities calculated as described above (hereinafter, “similar users ") Is selected.

  For example, the similar user selection unit 103 sequentially selects the plurality of preference makeup similarities in descending order of the value. The number of preference makeup similarities selected here corresponds to the number of similar users to be selected, and is assumed to be predetermined. Then, the similar user selection unit 103 selects the second user corresponding to the selected preference makeup similarity as a similar user.

  Then, the similar user selection unit 103 outputs the first user information and the second user information of the similar user (hereinafter referred to as “similar user information”) to the proposed makeup image selection unit 104.

  Here, it is assumed that the number of preference makeup face images included in the similar user information is plural. The plurality of preference makeup face images may be preference makeup face images associated with one similar user (in other words, included in one piece of similar user information) or may be associated with each of a plurality of similar users. It may be a preference makeup face image (in other words, included in each of a plurality of similar user information).

  The proposal makeup image selection unit 104 selects a proposal makeup image to be suggested to the first user based on the first user information and the similar user information received from the similar user selection unit 103. The proposed makeup image selection unit 104 will be specifically described below.

  As shown in FIG. 1, the proposed makeup image selection unit 104 includes a makeup distance calculation unit 1040 and a makeup similarity calculation unit 1041.

  First, the makeup distance calculation unit 1040 will be described.

  The makeup distance calculation unit 1040 calculates the makeup distance based on the makeup procedure information associated with the preference makeup face image included in the first user information or the similar user information. The makeup distance is the ease with which the makeup is shown for the makeup shown in the bare face and the predetermined preference makeup face image, or for the makeup shown in the other preference makeup face image of the makeup shown in the predetermined preference makeup face image Is a value (index) indicating the degree of Specifically, it is calculated by comparing the number of cosmetic processes (procedures) until achieving a desired makeup face and / or the degree of difficulty of individual cosmetic processes (procedures) and the like.

  Here, a specific example of calculation of the makeup distance will be described. Although the makeup distance is typically calculated by comparing the bare face of the first user (not including the makeup procedure) with the preference makeup face image of the similar user, here, the makeup of the preference makeup face image A of the similar user The calculation of the makeup distance on the assumption that the makeup of the preference makeup face image B is performed is illustrated.

  In addition, as the makeup procedure used for calculation of the makeup distance, the following makeup procedure A and makeup procedure B will be described as an example.

The cosmetic procedure A is as follows.
(1) Using a foundation brush, lightly apply the powder foundation with a brush.
(2) Draw a fluffy natural haze with eyebrow powder.
(3) Apply a white eye shadow while stretching the entire eye hole.
(4) Put a beige eye shadow slightly thicker than the eyes.
(5) Apply a dark brown eye shadow slightly darker so that the buttocks are lowered.
(6) Apply a baby pink eye shadow to the lower eyelid.
(7) Draw eyeliner with brown pencil eyeliner so that the bottom of the eye is lowered.
(8) Apply mascara to the entire eyelashes to emphasize the eyelashes of the eyes.
(9) Apply a pale pink teak on the cheek to make it oval.
(10) Apply baby pink lipstick so that your lips may look plump.

The cosmetic procedure B is as follows.
(1) Using a foundation brush, lightly apply the powder foundation with a brush.
(2) Draw a fluffy natural haze with eyebrow powder.
(3) Draw a slightly fat, raised eyeline with a black liquid eyeliner in mind.
(4) Apply mascara to the entire eyelashes to emphasize the lower eyelashes.
(5) Apply a pale pink teak on the cheek to form an oval shape.
(6) Apply a deep red lipstick with a lip brush to contour it.

  For example, the makeup procedure A is a makeup procedure shown in the preference makeup face image A of the similar user. By performing makeup in the order of (1) to (10) shown in the makeup procedure A, makeup of the preference makeup face image A is realized. The makeup procedure information indicating the makeup procedure A is associated with the preference makeup face image A.

  Also, for example, the makeup procedure B is a procedure of makeup shown in the preference makeup face image B of the similar user. By performing makeup in the order of (1) to (6) shown in the makeup procedure B, makeup of the preference makeup face image B is realized. The makeup procedure information indicating the makeup procedure B is associated with the preference makeup face image B.

  The makeup distance calculation unit 1040 compares the makeup procedure A with the makeup procedure B, and calculates the makeup distance with respect to the makeup procedure B of the makeup procedure A by adding a predetermined point.

  As an example, the cosmetic procedure is stored in the storage device 500 as a cosmetic procedure code (combination of a code indicating a make-up item and an additional cosmetic technology code), and the cosmetic distance calculation unit 1040 performs the same procedure by comparing these codes. It may be determined whether or not to add points.

  Specifically, first, when the foundation procedure (1) in the makeup procedure A and the foundation procedure (1) in the makeup procedure B are compared, “the foundation” of the makeup item in the procedure (1) of the makeup procedure A As the additional makeup technology “lightly apply powder foundation using a foundation brush so as to brush it on” is the same, the cosmetic procedure code is the same and points are not added.

  Next, comparing the eyebrow procedure (2) in the makeup procedure A with the eyebrow procedure (2) in the makeup procedure B, the points are the same as in the procedure (1) because the contents are the same.

  Next, in the eye shadow procedures (3) to (6) in the makeup procedure A, points are added because the corresponding makeup procedure code does not exist in the makeup procedure B. For example, if 1 point is added per procedure, 4 points are added to the initial value (e.g., zero). As a result, the makeup distance at this point is 4 points.

  In addition, an additional point may be added according to the difficulty of the application method (additional makeup technique) shown by the makeup procedure code | cord | chord of procedure (3)-(6) of eye shadow. In this case, the storage device 500 separately holds information on the degree of difficulty (degree of difficulty information) according to the additional makeup technology, and the makeup distance calculation unit 1040 refers to the information on the degree of difficulty to acquire points to be added. May be For example, since the application method of procedures (4) and (5) requires a technique equivalent to one point, one point is added for one procedure. As a result, 2 points are added to the total value 4 points so far, and the makeup distance at this point is 6 points.

  The difficulty level information does not have to be limited to the difference in the presence or absence of the cosmetic procedure as described above, and the difference in the points to be added is different depending on the color, the application technique, or the cosmetic tool (type of brush etc.) used for application. It may be attached.

  For example, comparing the eyeline procedure (7) in the cosmetic procedure A with the eyeline procedure (3) in the cosmetic procedure B, the color of the eyeline in the cosmetic procedure A is brown and the eyeline in the cosmetic procedure B is the eyeline Because the color of is the default black and the color is different, 1 point is added. In addition, the eyeline is drawn in such a way that the eyebrows are lowered in the cosmetic procedure A, and in the cosmetic procedure B, because it is different in how to draw up, and the difficulty is equivalent to 2 points. Is obtained from the difficulty level information of the storage device 500, and 2 points are added. As a result, 3 points (1 point due to a color difference + 2 points due to a difference in how to draw) are added to the total value 6 points so far, and the makeup distance at this point is 9 points.

  Next, comparing the mascara procedure (8) in the cosmetic procedure A with the mascara procedure (4) in the cosmetic procedure B, the cosmetic procedure A is a method of coloring that emphasizes the inside of the eyes, and in the cosmetic procedure B the lower eyelashes Since it is emphasizing and is different in the way of painting, 2 points are added to the total value of 9 points so far. The cosmetic distance at this point is 11 points.

  Next, when the teak procedure (9) in the makeup procedure A and the teak procedure (5) in the makeup procedure B are compared, the points are not added because the contents are the same. Therefore, the makeup distance at this point remains at 11 points.

  Next, comparing the lip procedure (10) in the cosmetic procedure A with the lip procedure (6) in the cosmetic procedure B, the color of the lip is baby pink in the cosmetic procedure A, and the color of the lip in the cosmetic procedure B Is a deep red, and the color is different, so one point is added. In addition, since the method of applying the lip uses the lip brush in the makeup procedure B, the makeup distance calculation unit 1040 refers to the difficulty level information of the storage device 500, and the difficulty is equivalent to one point for the makeup lip brush. Get something and add 1 point. As a result, 2 points (1 point due to color difference + 1 point due to difference in painting method) are added to the total value 11 points so far, and the final makeup distance is 13 points.

  The specific example of the calculation of the makeup distance has been described above. Here, the calculation of the makeup distance between the preference makeup face images A and B of similar users is exemplified, but in the case of performing the computation based on the bare face of the first user, Regarding the procedure, the makeup distance is obtained by performing the addition of points as described above.

  Thus, the makeup distance becomes a larger value as the makeup procedure is more (in other words, it is farther). In addition, the cosmetic distance has a larger value as the type and color of the cosmetic to be used increase. In addition, the cosmetic distance has a larger value as the use of the cosmetic product (how to paint, how to draw it) is more difficult.

  In addition, although the example of calculation of the makeup | decoration distance with respect to the makeup | decoration procedure B of the makeup | decoration procedure A was demonstrated in the said specific example, when another makeup | decoration procedure exists, a makeup | decoration distance is calculated similarly. For example, in addition to the cosmetic procedures A and B described above, when there is a cosmetic procedure C associated with the preference cosmetic face image C of the similar user, the cosmetic distance to the cosmetic procedure C of the cosmetic procedure A and the cosmetic procedure The cosmetic distance for the cosmetic procedure C of B is to be calculated.

  In addition, in addition to the cosmetic procedure information, the cosmetic face image for preference face images of the first user and similar users indicates cosmetic number information indicating the type of cosmetic used for cosmetic and / or cosmetic for indicating the price of cosmetic used for cosmetic Price information may be associated. In that case, the suggested makeup image selection unit 104 may calculate the makeup distance based on at least one of makeup procedure information, cosmetic number information, or makeup information such as cosmetic price information. For example, in the case of considering the cosmetic quantity information, the more the number of cosmetic products, the more points are added. Also, for example, in the case of considering cosmetic price information, more points are added as the cosmetic price is higher.

  Also, for example, the cosmetic price information may include information indicating the degree of difficulty in obtaining the cosmetic or information indicating a substitute for the cosmetic, in addition to information on the price. In that case, the more points the cosmetic may obtain, the more points may be added. Also, although the degree of difficulty in obtaining is high, if there is a substitute, points less than the points added when it is difficult to obtain may be added.

  In addition, since each procedure indicated in the makeup procedure information has predetermined content, it is also assumed that it does not conform to the face of the first user. Therefore, before starting the calculation of the makeup distance, the proposed makeup image selection unit 104 extracts feature quantities from the face image of the user, and based on the feature quantities, the position and shape of the face parts and the shape of the face contour are calculated. You may extract. Then, the proposed makeup image selection unit 104 adds a predetermined procedure to each procedure indicated in the makeup procedure information, as necessary, based on the position and shape of the extracted face part and the shape of the contour of the face. Or you may delete it. Thus, each procedure defined in the makeup procedure information can be adjusted to a procedure that matches the feature of the user's face.

  The makeup distance calculation unit 1040 has been described above.

  Next, the makeup similarity calculation unit 1041 will be described.

  The makeup similarity calculation unit 1041 includes a preference makeup face image (hereinafter referred to as "first preference makeup face image") included in the first user information, and a preference makeup face image (hereinafter referred to as "similar preferences") included in each similar user information. The makeup similarity is calculated based on the makeup face image. The cosmetic similarity is a value indicating the similarity between the makeup shown in the first preference makeup face image and the makeup shown in the similar preference makeup face image.

  The makeup similarity calculation unit 1041 displays one first preference makeup face image (for example, one in which the associated makeup preference information indicates the largest value, or the associated selection date information indicates the latest date). The cosmetic similarity of each of the similar preference makeup face images is calculated. In addition, the makeup similarity calculation unit 1041 displays the position, shape, size, color, etc. of the makeup portion (portion to which various makeup items are applied) shown in the first preference makeup face image and the similar preference makeup face image. The position, the shape, the size, the color, etc. of the makeup part to be done are compared, and the cosmetic similarity is calculated so that the larger the number of matching parts or similar parts, the larger the value.

  Heretofore, the makeup similarity calculation unit 1041 has been described.

  The proposed makeup image selection unit 104 selects a proposed makeup image from among the plurality of similar preference makeup face images based on the plurality of makeup distances and the plurality of makeup similarities calculated as described above. The proposed makeup image is a similar preference makeup face image showing makeup to be suggested to the first user. The number of proposed makeup images to be selected may be one or more.

  For example, the proposed makeup image selection unit 104 selects a similar preference makeup face image having the smallest makeup distance from the first user's bare face and the largest makeup similarity as the proposed makeup image. Here, the makeup distance may be calculated on the basis of the first preference makeup face image, as in the case of the calculation of the makeup similarity described above. Alternatively, the similar preference makeup face image having the largest value of makeup similarity is selected as the maximum similarity preference makeup face image first, and then the predetermined number is sequentially arranged from the smallest makeup distance based on the maximum similarity makeup face image. The similar makeup face image of may be selected as the proposal makeup image. In this case, when the number of similar makeup face images to be selected is one, the maximum similarity face makeup face image is 0 for the makeup distance, and therefore, it is selected as the proposed makeup face image.

  Alternatively, for example, the proposal makeup image selection unit 104 selects a plurality of similar preference makeup face images as proposal makeup images in descending order of makeup similarity from among the similar preference makeup face images having a makeup distance equal to or less than a predetermined value. May be

  Alternatively, for example, the proposal makeup image selection unit 104 selects a plurality of similar preference makeup face images as proposal makeup images in ascending order of the makeup distance from among the similar preference makeup face images whose makeup similarity is equal to or more than a predetermined value. May be

  Alternatively, for example, the suggested makeup image selection unit 104 randomly selects one or more suggested makeup images from among the preference makeup face images whose makeup distance is equal to or less than the predetermined value and the makeup similarity is equal to or more than the predetermined value. You may choose.

  As described above, by using the makeup distance and the makeup similarity, it is possible to select a suggested makeup image that has less makeup time and is closer to the preference of the first user's makeup.

  Then, the proposed makeup image selection unit 104 outputs the selected proposal makeup image to the information generation unit 105.

  The information generation unit 105 is a simulation image in which the image of makeup shown in the proposed makeup image received from the proposal makeup image selection unit 104 is superimposed on the first user's face image (for example, a bare face image) received from the information acquisition unit 101. Generate When the information generation unit 105 receives a plurality of proposed makeup images, the information generation unit 105 generates a plurality of simulation images in which the makeup image shown in each of the suggested makeup images is superimposed on the face image of the first user.

  Then, the information generation unit 105 causes the display device 300 (an example of a predetermined device) to display the generated simulation image. The simulation image to be displayed may be a moving image or a still image.

  In the simulation image displayed on the display device 300, the makeup image shown in the preference makeup face image selected by the second user (similar user) whose preference for makeup is similar to that of the first user is the first user's It is superimposed on the face image. Therefore, the user can receive a proposal of makeup that suits his / her preference. As a result, the user can reduce the effort of selecting one of the sample images according to his / her preference, and the time required for the selection can be shortened.

<Operation of device>
Hereinafter, an operation example of makeup assisting apparatus 100 will be described.

  First, the user registration operation of makeup assisting apparatus 100 will be described using FIG.

  FIG. 2 is a flowchart showing an example of the user registration operation of makeup assisting apparatus 100.

  In step S101, the information acquisition unit 101 acquires a plurality of sample images from the storage device 500, and causes the display device 300 to display them as options of the preference makeup face image. The sample image is, for example, an image of a face of a model to which a predetermined makeup has been applied. The model is not limited to a professional model, and may be a general person (for example, a user of makeup assisting apparatus 100).

  The sample image may be an image in which the makeup part image is superimposed on the face image of the model, or may be an image of a photo of the face of the model actually subjected to makeup. The makeup part image is an image that represents the shape and color of a makeup item (for example, an eyebrow, an eye shadow, an eye line, a mascara, a cheek, a lipstick, etc.).

  In addition, the sample image may be a simulation image suggested to another user by the makeup proposal operation (see FIG. 3) described later, or a recommended makeup prepared in advance (for example, a popular makeup) The simulation image may be a simulation image in which the image indicating the image is superimposed on the face image of the user.

  In addition, when displaying a plurality of sample images on the display device 300, the information acquisition unit 101 may arrange and display sample images indicating different makeup types so that the user can easily grasp makeup features. For example, a sample image showing makeup that makes the nose look clear and a sample image showing makeup that gives a gentle impression are displayed side by side. Here, for example, when the user selects the sample image of the former, a plurality of sample images showing makeup with different colors and application methods such as eye shadow and eye line may be displayed side by side. This makes it easy for the user to select a favorite makeup face. The sample images displayed side by side as options are prepared in advance according to which sample image the user selects.

  In addition, when displaying a plurality of sample images on the display device 300, the information acquisition unit 101 may select a sample image to be displayed based on the face image of the user acquired from the imaging device 200. For example, when the information acquisition unit 101 recognizes that the user's hairstyle is a shortcut in the user's face image, the information acquisition unit 101 may select a sample image indicating a shortcut from among a plurality of sample images and cause the display device 300 to display the sample image. . This allows the user to select a makeup face that suits his hair style.

  The user performs an operation of selecting one or more sample images of his / her choice from among a plurality of sample images displayed as options. In addition, when selecting a plurality of sample images, the user performs an operation of designating the order of preference for each of the selected sample images.

  In step S102, the information generation unit 105 determines the sample image selected by the user as the preference makeup face image of the user.

  In step S103, the information generation unit 105 sets makeup preference level information of the preference makeup face image determined in step S102. For example, when the preference makeup face image is one, the information generation unit 105 sets makeup preference level information indicating a predetermined maximum value (for example, 10 points) to the preference makeup face image. In addition, for example, when there are three preference makeup face images, the information generation unit 105 determines, for each preference makeup face image, a value (for example, 10 points) determined in advance according to the designated preference order. Set makeup preference information indicating 7 points, 5 points, etc.).

  In step S104, the information generation unit 105 associates the preference makeup face image, the makeup preference level information, and the selected date and time information (information indicating the date and time when the sample image is selected by the user) with respect to them. To generate user information.

  In step S105, the information generation unit 105 stores the user information in the storage device 500.

  A plurality of pieces of user information are stored in the storage device 500 by performing the user registration operation described above for each user.

  The user registration operation of makeup assisting apparatus 100 has been described above.

  Next, the makeup proposal operation of makeup assisting apparatus 100 will be described using FIG.

  FIG. 3 is a flowchart showing an example of the makeup proposal operation of makeup assisting apparatus 100.

  In step S201, the information acquisition unit 101 acquires, from the imaging device 200, the face image of the first user captured by the imaging device 200. Then, the information acquisition unit 101 outputs the face image of the first user to the information generation unit 105. Although the face image of the first user here may be either a bare face image or a makeup face image, it is preferably a bare face image because it is used to generate a simulation image in step S209 described later.

  Here, for example, the first user inputs his / her user identification information (for example, an ID and a password) using the operation device 400. Information acquisition unit 101 acquires the user identification information from operation device 400 and outputs the user identification information to user authentication unit 102. In addition, the information acquisition unit 101 acquires all user identification information (registered by the above-described user registration operation) stored in the storage device 500 and outputs the acquired information to the user authentication unit 102.

  In step S 202, the user authentication unit 102 authenticates the first user based on the user identification information acquired from the operation device 400 and the user identification information acquired from the storage device 500. When the user identification information acquired from the operation device 400 is included in the user identification information acquired from the storage device 500, it is determined that the first user is a registered user, and the user authentication succeeds.

  In step S 203, when the information acquiring unit 101 receives a notification indicating that the user authentication is successful from the user authenticating unit 102, the information acquiring unit 101 acquires the first user information and the second user information from the storage device 500, and sends the similar user selecting unit 103. Output.

  The second user information output to the similar user selection unit 103 is, for example, user information of each of a plurality of second users, and there are a plurality of pieces of user information. In addition, each of the first user information and the second user information includes user identification information, preference makeup face images, makeup preference information, makeup procedure information, selection date information, and the like.

  In step S204, the preference makeup similarity calculating unit 1030 of the similar user selection unit 103 calculates a preference makeup similarity based on the first user information and the second user information. For example, the preference makeup similarity is calculated corresponding to each of the plurality of second users.

  In step S205, the similar user selection unit 103 selects a similar user based on the preference makeup similarity. For example, the similar user selection unit 103 selects one or more similar users from among the plurality of second users based on the plurality of preference makeup similarities. Then, the similar user selection unit 103 outputs the first user information and the similar user information (user information of the second user selected as the similar user) to the proposed makeup image selection unit 104.

  Note that the number of similar user information output to the proposed makeup image selection unit 104 is one (when the number of similar users selected is one) or plural (the number of similar users selected is plural). In the case where it is preferable that a plurality of similar preference makeup face images (preference makeup face images of similar users) are output to the suggested makeup image selection unit 104.

  In step S206, the makeup distance calculation unit 1040 of the proposed makeup image selection unit 104 calculates the makeup distance based on the makeup procedure information associated with the similar preference makeup face image.

  In step S207, the makeup similarity calculation unit 1041 calculates makeup similarity based on the first preference makeup face image (the preference makeup face image of the first user) and the similar preference makeup face image.

  In step S208, the suggested makeup image selection unit 104 selects one or more suggested makeup images from among the plurality of similar preference makeup face images based on the makeup distance and the makeup similarity. Then, the proposed makeup image selection unit 104 outputs the selected proposal makeup image to the information generation unit 105.

  In step S209, the information generation unit 105 generates a simulation image based on the proposed makeup image and the face image of the first user, and outputs the simulation image to the display device 300 (an example of a predetermined device). Thereby, the simulation image is displayed on the display device 300. As described above, the simulation image is an image in which the image of makeup shown in the proposed makeup image is superimposed on the face image of the first user.

  Further, the information generation unit 105 may additionally store a simulation image and makeup preference level information corresponding to the simulation image with respect to the first user information stored in the storage device 500. After this, the simulation image additionally stored is treated as a preference makeup face image. The value of makeup preference level information corresponding to the preference makeup face image already stored may be recalculated and updated by the addition of the simulation image and the makeup preference level information.

  In addition, after the display of the simulation image in step S209, when the user performs an operation of instructing re-searching, the processes of steps S204 to S209 may be performed again.

  The makeup proposal operation of makeup assisting apparatus 100 has been described above. The simulation images may be displayed one by one on the display device 300 or a plurality of simulation images may be displayed simultaneously. The user who receives the makeup proposal by the display of the simulation image can input, for each of the displayed simulation images, an evaluation that he / she likes or does not like using the operation device 400 or the like. The evaluation by the user may be an evaluation on the whole for each of the simulation images, or may be an evaluation for each part, for example, although the user likes the makeup of the eyes but does not like the makeup of the mouth. By feeding back the evaluation result to the makeup support apparatus 100, the proposed makeup image selection unit 104 may correct the simulation image or generate a new one based on the feedback evaluation result. As a result, another simulation image can be displayed on the display device 300, and the user can receive a proposal of makeup more suited to his / her preference.

<Effect of this embodiment>
As described above, makeup assisting apparatus 100 of the present embodiment has the following components. The storage device 500 stores, for each user, user information including a preference makeup face image indicating a face on which the user's preference is applied, and makeup preference level information indicating the degree of preference for the makeup. The information acquisition unit 101 acquires, from the storage device 500, first user information corresponding to a first user and a plurality of second user information corresponding to each of a plurality of second users other than the first user, and The first user's face image is acquired. The similar user selection unit 103 is a preference cosmetic similarity that indicates the similarity between the preference of the first user's makeup and the preference of each of the plurality of second users based on the first user information and the plurality of second user information. The degree is calculated, and a similar user is selected from the plurality of second users based on the preference makeup similarity. The proposed makeup image selection unit 104 calculates the makeup distance indicating the ease of makeup shown in the preference makeup face image of the similar user, and the preferences of the makeup and the similar user shown in the preference makeup face image of the first user A cosmetic similarity, which indicates the similarity with makeup shown in each of the makeup face images, is calculated. Then, based on the calculated makeup distance and the degree of makeup similarity, a suggested makeup image showing makeup to be suggested to the first user is selected from the preference makeup face images of similar users. The information generation unit 105 generates a simulation image in which the image of makeup shown in the proposed makeup image is superimposed on the face image of the first user, and outputs the simulation image to a predetermined device.

  That is, according to makeup assisting apparatus 100 of the present embodiment, the makeup image shown in the preference makeup face image of the second user (similar user) whose preference for makeup is similar to that of the first user is the first user Since the simulation image superimposed on the face image of can be generated, makeup that meets the user's preference can be proposed. As a result, the user can reduce the effort of selecting one of the plurality of sample images according to his / her preference, and the time required for the selection can be shortened.

<Modification of this embodiment>
As mentioned above, although embodiment of this indication was described, this indication is not limited to the above-mentioned explanation, and various modification is possible. Each modification will be described below.

(Modification 1)
Although the case where selection of a similar user is performed when user authentication succeeds was mentioned as an example in the above-mentioned embodiment, user authentication may not be essential. That is, even if the user does not log in, the makeup proposal operation can be performed.

  In that case, makeup assisting apparatus 100 treats the user who is not logged in as an anonymous user, presents a plurality of sample images to the anonymous user, makes the preference makeup face image selected, and makes corresponding makeup preference information Set The user information (hereinafter referred to as “anonymous user information”) of the anonymous user including the preference makeup face image and the makeup preference level information is treated in the same manner as the first user information. Thereafter, based on the anonymous user information, selection of similar users, selection of a proposed makeup image, and generation of a simulation image are performed as described in the above embodiment.

  The anonymous user information may be stored in the storage device 500. Thereafter, it may be used as second user information.

  Further, makeup assisting apparatus 100 may request the anonymous user to register the user information at a predetermined timing. The predetermined timing may be, for example, the start of the makeup proposal operation, the display of the sample image, or the selection of the sample image (after the determination of the preference makeup face image).

(Modification 2)
In the above-described embodiment, the preference makeup face image of the user has been described as an example in which the user selects a plurality of sample images displayed on the display device 300, but the present invention is not limited thereto.

  The user may perform an operation of inputting a desired makeup face image into the makeup support apparatus 100. For example, the user may perform an operation of reading a makeup face published in a magazine with a scanner and inputting image data of the makeup face to the makeup support apparatus 100. Alternatively, for example, the user may perform an operation of acquiring a makeup face image published on the web page and inputting the makeup face image data into the makeup support apparatus 100. Alternatively, for example, the user may perform an operation of inputting to the makeup support apparatus 100 an image obtained by photographing a makeup face of a self, a friend, a family, or the like with a digital camera or the like.

  Thus, when the makeup face image is input to makeup assisting apparatus 100 by the user, information generation unit 105 determines the input makeup face image as the preference makeup face image of the user, and the makeup face image Set makeup preference information corresponding to. The cosmetic preference level information is set, for example, to indicate the highest value. Then, the information generation unit 105 associates the preference makeup face image with the makeup preference level information and causes the storage device 500 to store the information as user information.

(Modification 3)
In the above-described embodiment, the preference makeup face image has been described by way of example in which the user selects one of the sample images. However, the present invention is not limited to this.

  For example, makeup assisting apparatus 100 may further include an action observation unit (not shown) that observes the action of the user when viewing the sample image displayed as a choice of the preference makeup face image. The behavior observing unit determines the preference makeup face image and preference degree information of the user based on the behavior of the user, and stores the image in the storage device 500 as the user information. The details will be described below.

  In the user registration operation described above, for example, as shown in FIG. 4, a plurality of sample images 600 to 602 are displayed on the display device 300. In the example of FIG. 4, the sample image 600 is enlarged and displayed at the center. The user can switch the sample image displayed in the center by performing a predetermined operation. In the example of FIG. 4, the number of sample images to be displayed is three, but the number is not limited to this.

  Here, the behavior observation unit measures the time and / or the number of times displayed in the center for each sample image.

  Next, the behavior observing unit sets makeup preference information on each sample image according to the measured time and / or the number of times. At this time, as the makeup preference level information has a longer value, a larger value is set. In addition, as the makeup preference level information has a larger number of measurements, a larger value is set. That is, as the measurement time is longer and the number of times of measurement is larger, it is determined that the sample image shows the user's interest. A negative value may be set as makeup preference information of a sample image having a measurement time and / or the number of measurements equal to or less than a predetermined value. Moreover, when using both measurement time and the number of times of measurement, the value may be set by weighting according to which one is to be emphasized.

  Next, the behavior observing unit determines a sample image having a value of makeup preference level information equal to or more than a predetermined value as a preference makeup face image of the user.

  Next, the behavior observing unit associates the determined preference makeup face image with makeup preference degree information corresponding thereto, and causes the storage device 500 to store the information as user information.

  When the user gives an instruction to display sample images other than the sample images 600 to 602 displayed in FIG. 4, the behavior observing unit observes the observation results (measurement time and / or the number of measurements) until the instruction is received. Based on the above, makeup preference level information may be set for the sample images 600 to 602, and then other sample images may be displayed.

  In the above description, the behavior observing unit measures the time and / or the number of times displayed in the center for each sample image, but the present invention is not limited to this. For example, the behavior observing unit may track the line of sight of the user based on a face image (moving image) captured by the imaging device 200. In this case, the behavior observing unit may identify a sample image to which the user's gaze is pointing, and may measure the time and / or the number of times the gaze is turned.

(Modification 4)
In the above embodiment, although the similar user selection unit 103 calculates the preference makeup similarity and selects the similar user based on the preference makeup similarity as an example, the present invention is not limited to this. This specific example will be described below with reference to FIG.

  FIG. 5 is a diagram showing an example of the configuration of makeup assisting apparatus 100 according to the present modification. In FIG. 5, the same components as in FIG. 1 will be assigned the same reference numerals and descriptions thereof will be omitted.

  The makeup support apparatus 100 shown in FIG. 5 differs from the configuration in FIG. 1 in that the similar user selection unit 103 includes a face similarity calculation unit 1031 and an age similarity calculation unit 1032.

  In this modification, it is assumed that the first user information stored in storage device 500 includes age information of the first user in addition to the various information described in the embodiment. Further, in addition to the various information described in the embodiments, each second user information stored in storage device 500 includes the face image of the second user (for example, a bare face image of a still image) and the second user's It is assumed that age information is included. The age information may be input by the user himself or herself, for example, at the time of the user registration operation described above, or may be input by a clerk at a cosmetics store who has heard the age from the user. Alternatively, the age information may be the age estimated from the face image of the user.

  The information acquisition unit 101 acquires the first user information and the plurality of second user information from the storage device 500, and outputs them to the similar user selection unit 103. Further, the information acquisition unit 101 outputs the face image of the first user acquired from the imaging device 200 to the information generation unit 105 and also to the similar user selection unit 103.

  The face similarity calculation unit 1031 determines the face of the first user and the face of each of the plurality of second users based on the feature amount of the face image of the first user and the feature amount of the face images of each of the plurality of second users. Calculate the face similarity that indicates the similarity with.

  For example, first, the face similarity calculation unit 1031 extracts a plurality of feature points from the face image of the first user, and based on the feature points, various face parts (for example, eyes, nose, mouth, cheeks, forehead, Extract the position and shape of the jaw, etc., and the shape of the contour of the face (eg, square, inverted triangle, oval, etc.). Processing similar to this is also performed on the face image of each second user.

  The face similarity calculation unit 1031 then detects the position and shape of each face part extracted in the first user's face image, the shape of the face contour, and each face part extracted in each of the second user's face images. Compare the position and shape of the face with the shape of the contour of the face.

  As a result of this comparison, the face similarity calculation unit 1031 obtains a larger value as the position or shape of the face part becomes the same or closer, and a larger value as the shape of the face contour becomes the same or closer. Calculate the face similarity.

  The age similarity calculation unit 1032 calculates the similarity between the age of the first user and the age of each of the plurality of second users based on the age information of the first user and the age information of each of the plurality of second users. Calculate the age similarity to be shown.

  For example, the age similarity calculation unit 1032 calculates the age similarity so that the age of the first user and the age of the second user become larger as the age becomes equal to or closer to the age of the second user.

  Then, the similar user selection unit 103 calculates the user similarity corresponding to each of the second users based on the preference makeup similarity, the face similarity, and the age similarity. For example, the user similarity is a total value of preference makeup similarity, face similarity, and age similarity, but is not limited to the total value.

  Next, the similar user selection unit 103 selects a similar user from the plurality of second users based on the plurality of user similarities. For example, among the plurality of user similarities, the ones having the largest values are selected in order. The number of user similarities selected here corresponds to the number of similar users to be selected, and is assumed to be predetermined. Then, the similar user selection unit 103 selects the second user corresponding to the selected user similarity as a similar user.

  Note that each of the preference makeup similarity, face similarity, and age similarity may be weighted (for example, each similarity may be multiplied by a predetermined coefficient) to calculate the user similarity.

  In addition, after similar users are selected using the preference makeup face similarity, the similar users may be narrowed down using at least one of the face similarity and the age similarity. In this case, the user similarity is equal to the preference makeup face similarity.

  Also, the user similarity may be calculated based on the preference makeup similarity and either the face similarity or the age similarity.

  Also, as a face feature point extraction method and face part extraction method in the face similarity calculation unit 1031, known classification methods, pattern recognition methods, clustering methods, and optimization methods can be adopted. Known classification methods include, for example, decision tree analysis, neural networks (including deep learning), and naive Bayes. Known pattern recognition methods include, for example, neural networks (including deep learning) and support vector machines (SVMs). Known clustering methods include, for example, k-Nearest Neighbor (k-NN, k-nearest), k-means, and hierarchical clustering. Moreover, as a known optimization method, a genetic algorithm is mentioned, for example.

(Modification 5)
In the above embodiment, although the similar user selection unit 103 calculates the preference makeup similarity and selects the similar user based on the preference makeup similarity as an example, the present invention is not limited to this. .

  For example, similar users may be selected using preference information indicating user's preference for objects other than makeup in addition to preference makeup similarity. The preference information is, for example, information indicating a user's favorite entertainer (singer, model, actor, etc.), the user's favorite pet, the user's favorite color, and the like. The preference information is registered by the user, for example, at the time of the user registration operation described above, and is stored by being included in the user information.

  First, the similar user selection unit 103 receives the first user information and the plurality of second user information acquired by the information acquisition unit 101 from the storage device 500. Each of these pieces of user information includes preference information.

  Next, the similar user selection unit 103 compares the first preference information included in the first user information with the second preference information included in each of the second user information, and the content (preference) matches. It is determined whether or not.

  Next, the similar user selection unit 103 calculates the preference makeup similarity based on the preference makeup face image and makeup preference level information of the second user whose contents of the preference information match, and the similarity based on the preference makeup similarity Select a user.

  In addition, how to use preference information is not limited to the said description. For example, a value (for example, preference similarity) calculated based on the preference similarity and the preference makeup similarity, by calculating the similarity between the first preference information and the second preference information (hereinafter referred to as "preference similarity") A total value of preference makeup similarities (an example of user similarities) may be used to select similar users. In addition, the preference similarity may be used to calculate the user similarity together with the face similarity and the age similarity described in the fourth modification.

(Modification 6)
Although the case where the proposal makeup-image selection part 104 selects a proposal makeup image was mentioned as an example and demonstrated on the basis of both makeup distance and makeup similarity in the above-mentioned embodiment, it is not limited to this.

  The suggested makeup image selection unit 104 may select the suggested makeup image based on either the makeup distance or the makeup similarity.

  For example, the suggested makeup image selection unit 104 may select the similar preference makeup face image having the smallest makeup distance as the proposal makeup image. Alternatively, for example, the suggested makeup image selection unit 104 may select a predetermined number of similar favorite makeup face images as the suggested makeup image in ascending order of the makeup distance. Alternatively, for example, the suggested makeup image selection unit 104 may randomly select the suggested makeup image from among the similar preference makeup face images having a makeup distance equal to or less than a predetermined value.

  For example, the suggested makeup image selection unit 104 may select a similar preference makeup face image having the largest makeup similarity as a proposal makeup image. Alternatively, for example, the suggested makeup image selection unit 104 may select a predetermined number of similar favorite makeup face images as suggested makeup images in descending order of makeup similarity. Alternatively, for example, the suggested makeup image selection unit 104 may randomly select the suggested makeup image from among the similar preference makeup face images whose makeup similarity is equal to or more than a predetermined value.

  Alternatively, the suggested makeup image selection unit 104 may select the suggested makeup image without using any of the makeup distance and the makeup similarity.

  For example, the proposed makeup image selection unit 104 may select all the preference makeup face images included in the second user information of the second user selected as the similar user by the similar user selection unit 103 as the proposal makeup image.

(Modification 7)
In the above embodiment, the proposed makeup image selection unit 104 may use the following additional information when selecting the proposed makeup image.

  The additional information is, for example, information indicating at least one of the following elements: season, time of day (for example, morning, noon, evening), day of the week, or weather. The additional information is stored in the storage device 500 as user information in association with the preference makeup face image. Each of the above elements may represent a point in time when the preference makeup face image is selected by the user, or may be designated by the user or the like as optimum for makeup shown in the preference makeup face image. Good.

  First, the proposed makeup image selection unit 104 receives the first user information and the similar user information (the second user information corresponding to the similar user) including the additional information from the similar user selection unit 103, the current season, Obtain current information indicating at least one of a time of day, a day of the week, or an element of the weather.

  Next, the proposed makeup image selection unit 104 compares the current information with the additional information corresponding to each of the similar preference makeup face images (preference makeup face images included in the similar user information), and the contents (elements) match. Determine if there is something.

  Next, the suggested makeup image selection unit 104 extracts similar preference makeup face images whose contents match.

  Next, the suggested makeup image selection unit 104 calculates the makeup distance based on the makeup procedure information associated with the extracted similar preference makeup face image. Further, the suggested makeup image selection unit 104 calculates the makeup similarity based on the extracted similar preference makeup face image and the first preference makeup face image (preference makeup face image included in the first user information).

  Next, the proposed makeup image selection unit 104 selects a proposed makeup image based on the makeup distance and the makeup similarity.

  In the above description, although the case where the similar preference makeup face image is extracted by comparing the present information and the additional information before calculation of the makeup distance and the makeup similarity is described as an example, the present invention is not limited thereto. . For example, after the proposed makeup image is selected based on the makeup distance and the makeup similarity, additional information corresponding to the proposed makeup image may be compared with the current information to extract a suggested makeup image whose content matches. .

  In the above description, the case of using the cosmetic distance and the cosmetic similarity has been described as an example, but in the present modification, the modification 6 described above (when using either the cosmetic distance or the cosmetic similarity, and the cosmetic The present invention is also applicable to the case where both distance and cosmetic similarity are not used.

  According to the present variation, it is possible to propose to the user an appropriate suggested makeup image according to the season, the time zone of one day (for example, morning, noon, evening), the day of the week, or the weather.

(Modification 8)
In the above embodiment, the case where the information generation unit 105 outputs a simulation image to the display device 300 has been described as an example, but the output destination (transmission destination) is not limited to this.

  For example, the information generation unit 105 may output the simulation image to the terminal device (for example, a smartphone, a tablet, a personal computer, etc.) of the first user.

  Also, for example, the information generation unit 105 may output a simulation image to the storage device 500. In this case, the simulation image is stored as the preference makeup face image in the user information of the first user.

  Also, for example, the information generation unit 105 may output a simulation image to the printing apparatus.

(Modification 9)
In the above embodiment, after the generation of the simulation image, the information generation unit 105 generates guide information for guiding the makeup method shown in the simulation image, and outputs (sends) the guide information to a predetermined device. Good. As the output destination (transmission destination), for example, the display device 300, a terminal device (smartphone or the like) of the first user, a printing device, etc. may be mentioned.

  For example, the information generation unit 105 generates guide information based on makeup procedure information or the like associated with the proposed makeup image (similar preference makeup face image) used for generation of the simulation image.

  The guide information indicates, for example, information indicating an order of use (make-up procedure) of each makeup item, a method of applying each makeup item to the skin (for example, writing pressure such as a brush, a direction of moving a brush, etc., degree of blurring) It includes information, information indicating attributes of each makeup item (for example, model number, color, product name, features, etc.), information indicating advice when using the makeup item, and the like. The guide information may also include a simulation image in which the image of the makeup item is superimposed on the face image of the first user for each procedure. These pieces of information may be at least one of text information, still images, moving images, and audio information.

  As described above, since the guide information includes the content for guiding the method of makeup, the user can realize the makeup shown in the simulation image by performing makeup with reference to the guide information.

  When a plurality of proposed makeup images are selected, the information generation unit 105 generates a plurality of simulation images based on each of the plurality of proposed makeup images and the face image of the first user, and uses a plurality of simulation images as options. It is displayed on the display device 300. Here, the first user performs an operation of selecting a desired one from a plurality of simulation images. The information generation unit 105 generates guide information for guiding the method of makeup shown in the simulation image selected by the first user.

(Modification 10)
In the above embodiment, the information generation unit 105 may generate a simulation image in consideration of the skin condition of the first user. A specific example of this will be described below.

  First, the information generation unit 105 detects a position, a shape, a color, and the like of a discolored area (for example, a stain, acne, a bear, etc.) of the skin by known image analysis on the face image of the first user. Next, the information generation unit 105 acquires, from the storage device 500, an image (hereinafter referred to as a “care makeup image”) indicating makeup (for example, a foundation, a concealer) that hides a color-changed area predetermined according to the detection result. Next, the information generation unit 105 generates a simulation image by superimposing the care makeup image on the face image of the first user together with the makeup image shown in the proposed makeup image. In addition, the first user who looks at the simulation image may display the framed portion of the makeup portion so that the makeup of the care makeup image can be recognized. In addition, information of cosmetics used for the makeup (for example, information such as foundation, product number of concealer, color, etc.) may be displayed.

(Modification 11)
User information includes, in addition to the contents described above, for example, age, gender, nationality, residence, birthplace, skin trouble, ideal skin condition, medical institution (dermatology, cosmetic dermatology, cosmetic surgery, etc.) The contents such as the history of treatment received in (a) (name of medical institution, date of treatment, location of treatment, etc.) may be included.

  These contents are input from a predetermined device (for example, makeup assisting device 100, a terminal device used by the user, etc.) by a person concerned such as a store or a medical institution or the user himself. Then, user related information including the input content is transmitted from the predetermined device to the storage device 500, and stored in the storage device 500 in association with the user identification information.

  Then, for example, when generating a simulation image, makeup assisting apparatus 100 associates the user related information (for example, nationality, age, gender, etc.) with the generated simulation image, and generates a predetermined device (for example, , And a terminal device of a cosmetic development company or a sales shop. The association of these pieces of information makes it possible to know what kind of attribute the user prefers what kind of makeup. Therefore, it can be used for the development of cosmetics, advice for customer service, and recommendation of products.

(Modification 12)
Some components of makeup assisting apparatus 100 (information acquisition unit 101, user authentication unit 102, similar user selection unit 103, proposed makeup image selection unit 104, information generation unit 105) are physically different from other components. It may be separated. In this case, the separated components need to be provided with communication units for communicating with each other. For example, some of the functions of makeup assisting apparatus 100 may be clouded.

<Use mode of the present disclosure>
Hereinafter, assumed modes of use of makeup assisting apparatus 100 of the present disclosure will be described.

(First usage pattern)
For example, the user visits a store where the makeup assisting apparatus 100 is installed, etc., and operates the makeup assisting apparatus 100 by oneself.

(Second mode of use)
For example, the user visits a store where makeup assisting apparatus 100 is installed, and a clerk supports the operation of makeup assisting apparatus 100 of the user.

  In this case, after the simulation image proposed by the makeup support apparatus 100 is displayed on the display device 300, the clerk may actually apply the makeup shown in the simulation image to the user's face.

  The guide information may be displayed on the display device 300 together with the simulation image. The store clerk may make makeup on the user's face according to the guide information while teaching the user the content of the guide information. In addition, the store clerk may use the operation device 400 to input what he or she noticed while giving makeup, to the makeup assisting apparatus 100 using the operation device 400, and may perform an operation to add it to the guide information. The store clerk may also perform an operation of capturing the face of the user with the imaging device 200 for each makeup procedure, inputting the face image into the makeup support apparatus 100, and adding the image to the guide information.

(Third use form)
For example, the user visits a store where makeup assisting apparatus 100 is installed, and a clerk operates makeup assisting apparatus 100.

  In this case, the makeup assisting apparatus may display, for example, a store clerk screen and a user screen on each of two displays of the display device 300.

  For example, while a plurality of proposed makeup images and a plurality of simulation images are displayed on the clerk screen, only a simulation image selected by the clerk is displayed on the user screen.

  Also, for example, the guide information described in the above-described modification is not displayed on the user screen. It may be displayed only on the clerk screen. The clerk may make makeup for the user according to the guide information displayed on the clerk screen. At this time, only the face image of the user who is being dressed by the store clerk may be displayed on the user screen.

(Fourth usage form)
For example, a plurality of store clerks use a makeup assisting apparatus to share makeup knowledge.

  For example, when out of business hours of a store, a salesclerk may browse user information (for example, a simulation image, guide information, and the like) using the makeup support apparatus 100. As a result, the clerk can learn the method and knowledge of makeup performed by other clerks in the past, or a method of makeup suitable for each user having different facial features.

(Fifth usage mode)
For example, the user installs makeup assisting apparatus 100 at home and uses the makeup assisting apparatus on a daily basis.

  In this case, performing a login operation using user identification information is not compatible with everyday life, and for example, a simple login process by face authentication may be performed. In the case of single life, the login process may be omitted.

(Modification 13)
In the above-described modified example 7, data as described below may be accumulated and used for suggestion to future users, marketing, product development, sales, sales, and the like. The additional information in the seventh modification is not only the season, the time zone of one day (for example, morning, noon, evening), the day of the week, or the weather, but also the situation for making makeup (for example, party, reunion, lunch party) , Shopping, school parents' association) may be included as an element. The additional information includes at least one of the above as an element, and may include two or more elements as a combination. These pieces of additional information are stored in the storage device 500 as user information in association with the corresponding preference makeup face image.

  According to the seasons, time zone, day of the week, weather, situations, etc. as described above, cosmetic information on cosmetics used to make makeup of the recommended makeup image suitable for each cosmetic, season, time zone, day of the week, weather Depending on the situation, frequency of use, preferred color and texture, etc. are stored as data. Since the cosmetic information includes information on cosmetic product name, type, color, texture, etc., it is possible to analyze the tendency after accumulating data. Therefore, when proposing the proposed makeup image to the user, it is possible to simply present the cosmetic or alternative cosmetic to be used when making up the proposed makeup image using the data that has already been accumulated. Furthermore, depending on the season, time zone, day of the week, weather, and situation, a frequently used cosmetic may be calculated, the cosmetic may be presented to the user, and a cosmetic image using the cosmetic may be presented. Also, the cosmetic information may be presented to other similar users. In addition, accumulated data of cosmetic information may be provided to a cosmetic sales company or a cosmetic manufacturing company for future marketing, product development, sales, sales and the like.

(Modification 14)
In the above-mentioned embodiment and the above-mentioned modification, a plurality of user information may be stored in the makeup assisting apparatus so that stored data can be used for marketing of cosmetics proposal and advertisement etc. to future users.

  For example, cosmetics used for makeup of similar users are likely to be cosmetics that the first user wishes to purchase. Therefore, cosmetic information on the product name, color, price, sales destination, and the like of the cosmetic may be presented as cosmetic information to be suggested for purchase to the first user. As a method of presentation, it may be displayed on a display device such as a display, or cosmetic information or its link information may be transmitted by communication or the like to a mobile terminal such as a smartphone or a tablet possessed by the first user. Alternatively, it may be transmitted to the first user by e-mail or the like, or may be printed on a sheet of paper at a store or the like and handed over to the first user. In addition, cosmetic information associated with the proposed makeup image suggested to the first user or the simulation image selected by the first user for use in marketing and product development, and / or actually purchased by the first user Information on cosmetics may be provided to a cosmetics sales company or a cosmetics manufacturing company.

  For example, since the information on the first user's favorite cosmetics is highly likely to be the cosmetics favored by the second user a who is a similar user, the second user a who is a similar user uses the first user's favorite cosmetics If not, it may be presented to the second user a as a cosmetic proposed by the system. In addition, in order to use user information in marketing and product development in combination with cosmetic information to be presented, other cosmetic information used by the first user or a second user a in a cosmetic sales company or a cosmetic manufacturing company It is also conceivable to present cosmetic information that is being used as related cosmetic information.

  Also, for example, it is possible to learn the tendency of cosmetics used by the first user from the history information of cosmetics used by the first user and to use it from among a group of cosmetics that the first user has not used yet The high cosmetic of may be selected and presented to the first user. Also, the information on the cosmetic may be presented to the second user a who is a similar user. Also, it may be provided as a marketing or product development information to a cosmetics sales company or a cosmetics manufacturing company.

  Furthermore, when the first user selects a simulation image, situation information indicating what kind of situation (for example, party, reunion, lunch party, shopping, school parents' association) is planning to apply the makeup May be input using, for example, the operation device 400 of FIG. At this time, the cosmetic information and the situation information used for the selected simulation image may be associated with each other and stored in the storage device. Thereby, the cosmetics sales company and the cosmetics manufacturing company can analyze the tendency and trend, such as the color and texture of cosmetics which are preferred by the situation. This makes it possible to develop and propose cosmetics tailored to the situation.

  Needless to say, the above embodiment and the above modification can be combined.

<Summary of this disclosure>
The makeup assisting apparatus of the present disclosure includes a first preference makeup face image indicating a face on which a first user's preference is applied, and a first makeup preference level information indicating a degree of preference for makeup. 1 second user preference, a second preference makeup facial image showing a face on which a plurality of second users other than the first user have made a preference, and a second makeup preference showing a degree of preference for makeup Information acquisition unit for acquiring a plurality of second user information each including degree information and acquiring a face image of the first user, the first user information, and the plurality of second user information And calculating a preference makeup similarity degree indicating a similarity between the makeup preference of the first user and the preference preference of each of the plurality of second users, and based on the preference makeup similarity, the plurality of second users Similar user to select one or more similar users from The makeup distance indicating the degree of ease of makeup is calculated based on makeup information associated with the preference makeup face image of the selected one or more similar users, and the first user Calculating the similarity between the makeup shown in the preference makeup face image and the makeup shown in each of the preference makeup face images of the one or more similar users, and calculating the calculated makeup distance and the makeup distance A proposal makeup image selection unit for selecting a proposal makeup image showing makeup to be suggested to the first user from the preference makeup face images of the one or more similar users based on the makeup similarity, and the selected proposal An information generation unit configured to generate a simulation image by superimposing a makeup image shown in a makeup image on the face image of the first user, and outputting the simulation image to the display device.

  The makeup support apparatus further includes a storage device for storing the first user information and the plurality of second user information, and the information acquisition unit includes the first user information and the first user information from the storage device. A plurality of pieces of second user information may be acquired.

  Further, in the makeup support apparatus, the information acquisition unit acquires face images of each of the plurality of second users, and the similar user selection unit is configured to calculate feature amounts of face images of the first user, and Based on the feature amounts of the face images of each of the plurality of second users, a face similarity indicating the degree of similarity between the face of the first user and the face of each of the plurality of second users is calculated. The one or more similar users may be selected from the plurality of second users based on the degree and the face similarity.

  Further, in the makeup support apparatus, the information acquisition unit acquires age information of the first user and age information of each of the plurality of second users, and the similar user selection unit An age similarity indicating a similarity between the age of the first user and the age of each of the plurality of second users is calculated based on the age information of the user and each of the plurality of second users. One or more similar users may be selected from the plurality of second users based on the preference makeup similarity and the age similarity.

  Further, in the makeup support apparatus, the information acquisition unit includes face images of each of the plurality of second users, age information of the first user, and age information of each of the plurality of second users. The similar user selection unit acquires the face of the first user and the plurality of faces based on the feature amount of the face image of the first user and the face amount of each of the plurality of second users. Calculating a face similarity indicating the degree of similarity with the face of each of the second users, and based on the age information of the first user and the age information of each of the plurality of second users, Calculating the degree of similarity between the age of each of the plurality of second users and the age of each of the plurality of second users, and calculating the plurality of second plurality based on the preference makeup similarity, the face similarity, and the age similarity. One or more similar users may be selected from the users.

  Further, in the makeup assisting apparatus, the information acquisition unit is configured to: first preference information indicating a preference of the first user for a predetermined target other than a preference for makeup; and a plurality of second users for the predetermined target The plurality of second preference information indicating each preference is acquired, and the similar user selection unit is configured to acquire the plurality of second preference information based on the preference makeup similarity, the first preference information, and the plurality of second preference information. One or more similar users may be selected from the second user of.

  Further, in the makeup assisting apparatus, the information generating unit may generate guide information for guiding a makeup method shown in the simulation image, and may output the guide information to the predetermined apparatus.

  Further, in the makeup support apparatus, when the plurality of proposed makeup images are selected, the information generation unit performs the plurality of simulation images based on each of the plurality of proposed makeup images and the face image of the first user. May be generated and displayed on a predetermined display device, and the guide information for guiding the method of makeup shown in the simulation image selected by the first user among the plurality of simulation images may be generated. .

  The makeup assisting apparatus further includes an action observing unit for observing an action when the first user visually recognizes the makeup face image displayed as an option of the preference makeup face image, the action observing unit further comprising: The preference makeup face image and the preference degree information of the first user may be determined based on the action of the first user, and may be stored in the storage device as the first user information.

  In the makeup support apparatus, the suggested makeup image selection unit is based on at least one of makeup procedure information, cosmetic quantity information, or cosmetic price information, which is makeup information associated with the preference makeup face image. The makeup distance may be calculated.

  Further, in the makeup assisting apparatus, the information generating unit additionally includes the simulation image and the makeup preference degree information indicating a degree of preference of the first user with respect to makeup shown in the simulation image. It may be stored in the storage device as one user information.

  Further, in the makeup support apparatus, the preference makeup face image included in the user information includes additional information indicating at least one of a season, a time zone in one day, a day of the week, weather, and a situation in which makeup is performed. Are associated with each other, and the proposal makeup image selection unit is configured to, based on the additional information, present information indicating at least one of the current season, time zone in one day, day of the week, weather, or situation. A makeup image may be selected.

  In addition, in the makeup support apparatus, the information generation unit indicates, when the color-changed area of the skin is detected from the face image of the first user, an image showing makeup that hides the color-changed area on the proposed makeup image. The simulation image may be generated so as to be superimposed on the face image of the first user together with the makeup image to be generated.

  The makeup assisting apparatus further includes an operation unit that receives an input of evaluation by the first user for each of the simulation images displayed on the display device, the evaluation being an evaluation on the entire simulation image, At least one of the evaluations for each make part may be included.

  In the makeup support apparatus, the suggested makeup image selection unit may correct the suggested makeup image based on the evaluation received by the operation unit.

  In the makeup support apparatus, an operation unit that receives input of situation information regarding at least one of a season, a time zone, a day of the week, a weather, and a situation in which the first user and the plurality of second users make makeup. , Cosmetic information related to cosmetics used to make up in accordance with a scene (scene) indicated by the first user information, the plurality of second user information, and the situation information accumulated therein; However, each cosmetic may further include a storage unit in which usage frequency according to the situation, preferred color and texture, etc. are stored as data.

  The makeup assisting method of the present disclosure includes a first preference makeup face image indicating a face on which a first user's preference is applied and a first preference degree information indicating a degree of preference for makeup. A second preference makeup face image showing user's information, a face on which a plurality of second users other than the first user's favorite face has made a preference, and a second makeup preference degree showing a degree of preference for makeup Obtaining a plurality of pieces of second user information including information, acquiring face images of the first user, and based on the first user information and the plurality of second user information, the preference of the first user A preference makeup similarity indicating a similarity between makeup and a preference makeup of each of the plurality of second users is calculated, and one or more similar users are selected from the plurality of second users based on the preference makeup similarity. Of the selected one or more similar users The makeup distance indicating the degree of ease of makeup is calculated based on the makeup information associated with the preference makeup face image, and the makeup shown in the preference makeup face image for the first user and the one or more similar users The cosmetic similarity indicating the similarity with makeup indicated in each of the preference makeup face images is calculated, and the preference of the one or more similar users is calculated based on the calculated cosmetic distance and the cosmetic similarity. The simulation image is selected by selecting a proposed makeup image indicating makeup to be suggested to the first user from the makeup face image, and superimposing the makeup image shown in the selected suggested makeup image on the face image of the first user Are generated and output to a predetermined device.

  The makeup support apparatus and the makeup support method according to the present disclosure are useful as a makeup support apparatus and a makeup support method for supporting the makeup of the user.

DESCRIPTION OF SYMBOLS 1 makeup support system 100 makeup support apparatus 101 information acquisition unit 102 user authentication unit 103 similar user selection unit 104 proposed makeup image selection unit 105 information generation unit 200 imaging device 300 display device 400 operation device 500 storage device 1030 preference makeup similarity calculation Part 1031 Face similarity calculation part 1032 Age similarity calculation part 1040 Makeup distance calculation part 1041 Makeup similarity calculation part

Claims (17)

First user information including a first preference makeup face image indicating a face on which a first user's preference is applied, and first makeup preference level information indicating a degree of preference for makeup; and the first user A plurality of second preference makeup face images each showing a face on which each of a plurality of second users other than the second user has made their favorite makeup, and a second makeup preference level information indicating the degree of preference for makeup An information acquisition unit that acquires second user information and acquires a face image of the first user;
Based on the first user information and the plurality of second user information, preference makeup similarity is calculated which indicates the similarity between the preference of the first user's makeup and the preference of each of the plurality of second users A similar user selection unit which selects one or more similar users from the plurality of second users based on the preference makeup similarity;
Based on the makeup information associated with the preference makeup face image of the selected one or more similar users, a makeup distance indicating the degree of ease of makeup is calculated, and the preference makeup face of the first user is calculated The cosmetic similarity indicating the similarity between the makeup shown in the image and the makeup shown in each of the preference makeup face images of the one or more similar users is calculated, and the calculated makeup distance and the makeup similarity are calculated. A proposal makeup image selection unit for selecting a proposal makeup image indicating makeup to be suggested to the first user from the preference makeup face images of the one or more similar users based on the selected makeup image;
An information generation unit that generates a simulation image by superimposing a makeup image shown in the selected suggested makeup image on the face image of the first user, and outputting the simulation image to a display device;
Make-up support device. The storage device is further configured to store the first user information and the plurality of second user information, and the information acquisition unit acquires the first user information and the plurality of second user information from the storage device. ,
The makeup assisting apparatus according to claim 1. The information acquisition unit
Acquiring face images of each of the plurality of second users;
The similar user selection unit
Similarity between the face of the first user and the face of each of the plurality of second users based on the feature amount of the face image of the first user and the face amount of each of the plurality of second users Calculate the face similarity to indicate
The one or more similar users are selected from the plurality of second users based on the preference makeup similarity and the face similarity.
The makeup assisting apparatus according to claim 1. The information acquisition unit
Acquiring age information of the first user and age information of each of the plurality of second users;
The similar user selection unit
An age similarity indicating a similarity between the age of the first user and the age of each of the plurality of second users based on the age information of the first user and the age information of each of the plurality of second users Calculate
The one or more similar users are selected from the plurality of second users based on the preference makeup similarity and the age similarity.
The makeup assisting apparatus according to claim 1. The information acquisition unit
Acquiring face images of each of the plurality of second users, age information of the first user, and age information of each of the plurality of second users,
The similar user selection unit
The face of the first user and the face of each of the plurality of second users, based on the feature amount of the face image of the first user and the feature amounts of the face images of the plurality of second users Calculate the face similarity that indicates the similarity,
An age indicating a similarity between the age of the first user and the age of each of the plurality of second users based on the age information of the first user and the age information of each of the plurality of second users Calculate the similarity,
The one or more similar users are selected from the plurality of second users based on the preference makeup similarity, the face similarity, and the age similarity.
The makeup assisting apparatus according to claim 1. The information acquisition unit
Acquiring first preference information indicating the preference of the first user for a predetermined target other than the preference for makeup, and a plurality of second preference information indicating preferences of the plurality of second users for the predetermined target; ,
The similar user selection unit
The one or more similar users are selected from the plurality of second users based on each of the preference makeup similarity, the first preference information, and the plurality of second preference information.
The makeup assisting apparatus according to any one of claims 1 to 5. The information generation unit
Generating guide information for guiding the cosmetic method shown in the simulation image and outputting the guide information to the predetermined device;
The makeup assisting apparatus according to claim 1. The information generation unit
When a plurality of the proposed makeup images are selected, a plurality of the simulation images are generated based on each of the plurality of suggested makeup images and the face image of the first user, and displayed on a predetermined display device.
Generating the guide information for guiding a makeup method shown in the simulation image selected by the first user among the plurality of simulation images;
The makeup assisting apparatus according to claim 7. The first user further includes an action observation unit that observes an action when the first user visually recognizes the makeup face image displayed as the option of the preference makeup face image,
The behavior observation unit
The preference makeup face image and the preference degree information of the first user are determined based on the action of the first user, and stored in the storage device as the first user information.
The makeup assisting apparatus according to claim 2. The proposed makeup image selection unit
The makeup distance is calculated based on at least one of makeup procedure information, cosmetic quantity information, or cosmetic price information, which is makeup information associated with the preference makeup face image.
The makeup assisting apparatus according to claim 1. The information generation unit
Storing the simulation image and the makeup preference degree information indicating the degree of preference of the first user for makeup shown in the simulation image in the storage device as the additional first user information;
The makeup assisting apparatus according to claim 2. The preference makeup face image included in the user information is associated with additional information indicating at least one of a season, a time zone of one day, a day of the week, weather, and a situation in which makeup is performed.
The proposed makeup image selection unit
The proposed makeup image is selected based on current information indicating at least one of a current season, a time zone in a day, a day of the week, weather, or a situation, and the additional information.
The makeup assisting apparatus according to claim 1. The information generation unit
When a color-changed area of the skin is detected from the face image of the first user, an image showing makeup that hides the color-changed area is superimposed on the face image of the first user together with the image of makeup shown in the proposed makeup image. To generate the simulation image
The makeup assisting apparatus according to claim 1. The control unit further includes an operation unit that receives an input of evaluation by the first user for each of the simulation images displayed on the display device, the evaluation being at least one of an evaluation for the entire simulation image and an evaluation for each makeup part. Including any
The makeup assisting apparatus according to claim 1. The suggested makeup image selection unit corrects the suggested makeup image based on the evaluation received by the operation unit.
The makeup assisting apparatus according to claim 14. An operation unit that receives an input of situation information regarding at least one of a season, a time zone, a day of the week, a weather, and a situation in which each of the first user and the plurality of second users make up;
The first user information, the plurality of second user information, and the situation information are accumulated, and the cosmetic information related to the cosmetic used to make the makeup according to the scene indicated by the situation information corresponds to each cosmetic In addition, it further has a storage unit in which usage frequency according to the situation, preferred color and texture, etc. are stored as data,
The makeup assisting apparatus according to claim 1. First user information including a first preference makeup face image indicating a face on which a first user's preference is applied, and first preference degree information indicating a degree of preference for makeup, and the first user other than the first user A plurality of second preference makeup face images indicating a face on which each of the plurality of second users of the user's favorite makeup is applied, and second makeup preference level information indicating a degree of preference for makeup Get user information,
Acquiring a face image of the first user,
Based on the first user information and the plurality of second user information, preference makeup similarity is calculated that indicates the similarity between the preference makeup of the first user and the preference makeup of each of the plurality of second users. ,
One or more similar users are selected from the plurality of second users based on the preference makeup similarity,
Based on makeup information associated with the preference makeup face image of the selected one or more similar users, a makeup distance indicating a degree of ease of makeup is calculated,
Calculating a cosmetic similarity indicating the similarity between the makeup shown in the preference makeup face image of the first user and the makeup shown in each of the preference makeup face images of the one or more similar users,
Based on the calculated makeup distance and the makeup similarity, a proposal makeup image showing makeup to be suggested to the first user is selected from the preference makeup face images of the one or more similar users,
A simulation image is generated by superimposing the makeup image shown in the selected suggested makeup image on the face image of the first user, and the simulation image is output to the display device.
Makeup support method.

Images