JP2016081441A - Beauty care assist device, beauty care assist system, and beauty care assist method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a beauty care assist device with which it is possible to intuitively recognize, on the basis of similarity in sensitivity evaluation items between a face image of a user and a target face image, and how much the user's face image has approached the target face image as a result of a beauty treatment applied to the user.SOLUTION: The beauty care assist device is configured to include: a face image acquisition unit P1 for acquiring a first target face image and a face image of a user; an evaluation item selection unit P2 for selecting at least one sensitivity evaluation item in accordance with an input operation of the user; a simulation unit P3 for presenting a candidate face image for a second target face image (a synthesized face image created from the face image of the user by applying various beauty treatments) having a high degree of similarity in the sensitivity evaluation item selected by the evaluation item selection unit; and a beauty treatment presentation unit P4 for specifying, in accordance with candidate face image selection operation by the user, the second target face image of the user for which beauty treatment is to be extracted, and presenting the beauty care procedure included in the second target face image and necessary beauty care items.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a beauty support apparatus, a beauty support system, and a beauty support method for providing advice on a beauty method to a user such as a person to be dressed.

  In recent years, the types and uses of cosmetics are diverse. In addition to face-to-face sales, which are conventional sales systems, cosmetics themselves are sold in large quantities at supermarkets, convenience stores, and the like, making it possible for cosmetic customers to obtain various cosmetics more easily and in large quantities. For such a wide variety of cosmetics, cosmetic customers obtain product information attached to cosmetics, information on cosmetics and makeup techniques from fashion magazines and the Internet, etc., and use it when applying makeup themselves. Yes. However, such information alone cannot adequately determine whether, for example, the makeup actually applied is close to the target image of the subject.

  On the other hand, in Patent Document 1, a face image of a customer is received from the customer, and a simulated face image obtained by virtually applying makeup to the received face image and the same makeup as the simulated face image are actually realized. A method for providing makeup advice for transmitting information related to products and / or methods to be realized to the customer is disclosed. Specifically, a technique is described in which simulation conditions are determined according to a customer's product purchase history, system usage history, or input contents to an inquiry.

  Further, Patent Document 2 discloses a makeup advice providing method for providing advice on makeup to a customer via a network N. The customer C's face image is received from the customer, and the received face image is virtually displayed on the received face image. A technique is disclosed for transmitting information about a product and / or a method for actually realizing the same makeup face as the simulation face image to which the makeup face has been actually applied to the customer.

JP 2002-092132 A JP 2002-132916 A

According to the techniques described in Patent Documents 1 and 2, after receiving the customer's face image from the customer, a standard makeup pattern with makeup applied to the model is presented, and the makeup that is close to the impression that the customer wants to become A pattern is selected, a simulation image with the makeup applied is generated, and a product or method for realizing the same makeup is notified. In this case, if the customer's face image is similar to the face image of the model to which the standard makeup pattern is applied, a simulation image close to the desired impression is obtained as a result. However, in fact, the face images of the two are often not similar, and simply applying a standard makeup pattern applied to the model makes the atmosphere of the two different (the degree of similarity between the two is low as human sensitivity) In addition, there is a problem that it is difficult to intuitively grasp this different degree.
The present invention has been devised to solve such problems of the prior art, and its main purpose is based on the similarity of sensitivity evaluation items between the user's face image and the target face image. It is an object of the present invention to provide a beauty support apparatus, a beauty support system, and a beauty support method capable of intuitively recognizing how close a target face image is as a result of applying a beauty technique.

  The beauty support apparatus of the present invention sets a first target face image targeted by the user, and is a second combined face image of the user who is in a state of high similarity to the first target face image. A beauty support apparatus that presents a beauty technique based on a target face image, wherein the face image acquisition unit acquires a first target face image and a user's face image, and at least one according to a user's input operation An evaluation item selection unit that selects one sensitivity evaluation item, and a simulation unit that presents candidate face images of the second target face image in a state of high similarity in the sensitivity evaluation items selected by the evaluation item selection unit And specifying the second target face image of the user whose beauty technique is to be extracted in accordance with the user's selection operation of the candidate face image, and a cosmetic procedure included in the second target face image and necessary A beautiful beauty product And Beauty technique presentation unit, a configuration which includes a.

In addition, the beauty support system of the present invention sets a first target face image targeted by the user, and is a composite face image of a user who has a high degree of similarity to the first target face image. A beauty support system for presenting a beauty technique based on two target face images, a user terminal having at least an imaging unit and a display unit, and an information processing apparatus connected to the user terminal via a network; The information processing device selects at least one sensitivity evaluation item according to a face image acquisition unit that acquires the first target face image and the user's face image, and a user's input operation. An evaluation item selection unit, a simulation unit that presents candidate face images of the second target face image in a state of high similarity in the sensitivity evaluation items selected by the evaluation item selection unit, and the user's symptoms According to the face image selection operation, the second target face image of the user whose beauty technique is to be extracted is identified, and the beauty procedure included in the second target face image and the necessary beauty products are presented. And a method presentation unit.
The beauty support method of the present invention sets the first target face image targeted by the user, and is a composite face image of the user who is in a state of high similarity to the first target face image. A beauty support method for causing an information processing device to perform a process of presenting a beauty technique based on a second target face image, the step of acquiring the first target face image and a user's face image; Selecting at least one sensitivity evaluation item in accordance with an input operation; and presenting a candidate face image of the second target face image in a state of high similarity in the sensitivity evaluation items selected by this step And specifying the second target face image of the user whose beauty technique is to be extracted in accordance with the user's selection operation of the candidate face image, and a cosmetic procedure included in the second target face image and necessary Beauty products A step of presenting, the configuration with.

  According to the present invention, it is possible to acquire a beauty technique closer to the target face image based on the similarity between the candidate face image and the target face image obtained by applying different beauty techniques to the user's face image. In particular, it is possible to identify the second target face image while referring to the evaluation item based on the human sensibility for the difference between the user's face image and the target face image, so that a beauty technique with high user satisfaction is obtained. can do.

The block block diagram which shows the structure of the beauty assistance apparatus which concerns on 1st Embodiment of this invention. The flowchart explaining the process in which the beauty technique is presented in the beauty support apparatus 1 according to the first embodiment of the present invention. Flowchart of person image and first target face image acquisition unit UI screen image diagram of personal image and first target face image acquisition unit Flowchart of program module P1 of the image acquisition unit UI screen image of shooting mode UI screen image of confirmation of use of images taken in shooting mode UI screen image of selection mode Flow chart of evaluation item selection unit UI screen image of the evaluation item selection section Flow chart of simulation unit UI screen image diagram of candidate face image presentation of second target face image Illustration of radar chart with selective evaluation items UI screen image diagram of list display of candidate face images of second target face image UI screen image of candidate face image editing mode UI screen image when comparing the candidate face image being edited and the whole image before applying the beauty technique UI screen image when comparing a specific part enlarged view of the candidate face image being edited and the person image before applying the beauty technique UI screen image diagram when comparing the overall view of the candidate face image being edited and the first target face image UI screen image when comparing a specific part enlarged view of the candidate face image being edited and the first target face image Flow chart of beauty technique presentation unit UI screen image of beauty technique presentation unit UI screen image when the current whole image is enlarged and displayed UI screen image at the time of enlarged display of a specific part enlarged view of the current person image UI screen image diagram when comparing the whole image of the current person image and the second target face image UI screen image at the time of comparison of a specific part enlarged view of the current person image and the second target face image UI screen image when comparing the whole image of the current person image and the first target face image UI screen image at the time of comparing a specific part enlarged view of the current person image and the first target face image

  According to a first aspect of the present invention for solving the above problems, a first target face image targeted by a user is set, and a user who is in a state of high similarity with the first target face image is synthesized. A beauty support apparatus for presenting a beauty technique based on a second target face image that is a face image, a face image acquisition unit that acquires the first target face image and a user's face image; An evaluation item selection unit that selects at least one sensitivity evaluation item according to an input operation, and the second target face image candidate that is in a high similarity state in the sensitivity evaluation item selected by the evaluation item selection unit A second target face image of a user whose beauty technique is to be extracted is identified according to a simulation unit that presents a face image, and a user's selection operation of the candidate face image, and the second target face image Beauty procedures and A configuration that includes and Beauty technique presentation section for presenting the necessary cosmetic products, the.

  According to this, it is possible to acquire a beauty technique closer to the target face image based on the similarity between the candidate face image and the target face image obtained by applying different beauty techniques to the user's face image. Since the second target face image can be identified while referring to the evaluation item based on the human sensitivity for the difference between the user's face image and the target face image, a beauty technique with high user satisfaction is acquired. Can do.

  The second aspect of the present invention further includes a candidate face image editing unit, which allows the candidate face image presented by the simulation unit to be edited by a user's operation input.

  According to this, since the user can manually edit the selected candidate face image, the second target face image intended by the user can be set.

  The third invention is configured to display on the user's operation input screen the face image of the user before applying the beauty technique and the evaluation value corresponding to the sensitivity evaluation item.

  According to this, the difference between the face image of the user and the first target image can be objectively grasped by the evaluation value corresponding to the sensitivity evaluation item.

  According to a fourth aspect of the present invention, at least two images of the user's face image before application of the beauty technique, the first target face image, and the second target face image are displayed on the user's operation input screen. Are displayed side by side.

  According to this, in the process of acquiring the final beauty technique, the user compares the face image of the user before applying the beauty technique, the first target face image, and the second target face image, The simulation of the method can be performed, and an unintended beauty technique is prevented from being acquired.

  According to a fifth aspect of the present invention, when the candidate face image editing unit edits the candidate face image, the face image of the user before applying a beauty technique, the first target face image, and the second target face image Among them, an enlarged image related to the editing part of at least one face image is displayed.

  According to this, in the process of acquiring the final beauty technique, it is possible to perform a more detailed simulation of the beauty method while referring to the enlarged image relating to the edited region.

  According to a sixth aspect of the present invention, a face image of a user before applying a beauty technique, a thumbnail image of the first target face image, and a thumbnail image of the second target face image are displayed on a user operation input screen. And

According to this, in the process of acquiring the beauty technique, the thumbnail images of the user's face image before the application of the beauty technique, the first target face image, and the second target face image are arranged. The selection state of each face image selected by the person can be constantly reviewed.
According to a seventh aspect of the present invention, a first target face image targeted by the user is set, and the second target face image is a composite face image of a user who is in a state of high similarity to the first target face image. A beauty support system for presenting a beauty technique based on a target face image, comprising: a user terminal including at least an imaging unit and a display unit; and an information processing apparatus connected to the user terminal via a network. And an evaluation item for selecting at least one sensitivity evaluation item according to the user's input operation, and a face image acquisition unit that acquires the first target face image and the user's face image A selection unit, a simulation unit that presents a candidate face image of the second target face image in a state of high similarity in the sensitivity evaluation items selected by the evaluation item selection unit, and the candidate face image of the user Selection of A beauty technique presentation unit that specifies the second target face image of the user whose beauty technique is to be extracted according to the work, and presents the cosmetic procedure and necessary cosmetics included in the second target face image; It is set as the structure provided with.

  According to this, similarly to the first invention, based on the similarity between the candidate face image and the target face image obtained by applying different beauty methods to the user's face image, the beauty method closer to the target face image is obtained. It can be acquired.

According to an eighth aspect of the present invention, a first target face image targeted by the user is set, and a second combined face image of a user who is in a state of high similarity to the first target face image. A beauty support method for causing an information processing apparatus to perform a process of presenting a beauty technique based on a target face image, the step of acquiring the first target face image and a user face image, and a user input operation And selecting at least one sensitivity evaluation item, and presenting a candidate face image of the second target face image in a state of high similarity in the sensitivity evaluation items selected by this step; According to the selection operation of the candidate face image by the user, the second target face image of the user whose beauty technique is to be extracted is specified, and the cosmetic procedure and the necessary beauty that are included in the second target face image Present A structure having a step, the.

  According to this, similarly to the first invention, based on the similarity between the candidate face image and the target face image obtained by applying different beauty methods to the user's face image, the beauty method closer to the target face image is obtained. It can be acquired.

  FIG. 1 is a block configuration diagram showing a schematic configuration of a beauty support apparatus 1 according to the first embodiment of the present invention. Hereinafter, the configuration and functions of the beauty support apparatus 1 according to the first embodiment will be described with reference to FIG. In the present embodiment, a case where a user who makes makeup himself as a user uses this apparatus will be described as an example. However, the same configuration can be adopted in a beauty technique other than makeup.

  In FIG. 1, the beauty support apparatus 1 includes an imaging unit 2, an image input unit 3, a storage unit 4, a display unit 5, a user interface unit 6, an overall control unit 7, and an image processing unit 8. The beauty support device 1 introduces an application program into a PC as an information processing device in advance and configures it as a dedicated device. In addition, the beauty support device 1 records an application program operating on a general-purpose OS (Operating System) on a recording medium, It may be provided to the user via. In that case, a terminal such as a tablet terminal or a smartphone that has been widely used in recent years can be used.

  The imaging unit 2 includes an imaging device configured by, for example, a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor). For example, the imaging unit 2 captures an image with a period of 15 fps (Frames Per Second) and outputs image data. Is output. For image data captured by the imaging unit 2, for example, an image processing circuit (not shown) built in the imaging unit 2 performs known processing such as demosaic, color correction, white balance correction, and γ correction as preprocessing. A function to perform processing is installed.

  Thereafter, various processes are executed by the image processing unit 8 with reference to the data stored in the storage unit 4 for the pre-processed image data. Note that the image data to be subjected to image processing may be stored in a memory (not shown) once predetermined processing is performed by the image processing unit 8, and the next component may be activated. Hardware or software may be configured such that the component operates as a pipeline.

  The image input unit 3 corresponds to a face image acquisition unit that acquires the first target face image and the user's face image, and the user's own face image via a portable storage medium, the Internet, or the like. Or an image obtained by photographing the face of a third person with a digital camera or the like, or an image obtained by searching for a face photograph of an actress or the like.

  The storage unit 4 stores an application program for starting up the beauty support apparatus 1 and data on the subject, and particularly stores an application program for operating the beauty support apparatus 1 on a general-purpose OS. The storage unit 4 is a hard disk drive that is a so-called large-capacity storage device, or in the case of handling the data of a plurality of people in the device configuration, the storage unit 4 is configured with a plurality of hard disk drives to improve reliability and high-speed data transfer RAID (Redundant Arrays of Inexpensive Disks) that is designed to be used may be used.

The display unit 5 is configured by a liquid crystal display or the like, and is used for browsing a function operation screen generated by an application program, a face image captured via the image input unit 3, and the like. The beauty support apparatus 1 is provided with a user interface unit 6. Specifically, the user interface unit 6 includes input devices such as a keyboard, a mouse, and a touch panel, and the subject gives instructions to the beauty support apparatus 1 through the overall control unit 7 by operating the user interface unit 6. . A specific example of this instruction is, for example, an image selection instruction.

  Next, the overall control unit 7 will be described. The beauty support apparatus 1 includes a CPU (Central Processing Unit) or MPU (Micro-Processing Unit) as an arithmetic device (not shown), an ASIC (Application Specific Integrated Circuit), a work piece as hardware for controlling the operation of each component described above. Known configurations such as a memory, a program memory, and a parallel / serial bus for exchanging data between these components are provided.

  Note that it is not necessary to configure all the functional blocks shown in FIG. 1 by hardware, and the above-described arithmetic unit and software for operating the same may be implemented as appropriate. It may be mounted on a typical notebook PC (Personal Computer) with a camera, a smartphone, a tablet terminal or the like. When using such a notebook PC with a camera (Personal Computer), a smartphone, a tablet terminal, etc., an application program that operates on a general-purpose OS (Operating System) is recorded on a recording medium and provided to the user via a network. You may be made to do.

  Next, the outline of the image processing function of the image processing unit 8 will be described. A composite face image (hereinafter referred to as “candidate face image”) obtained by applying (that is, simulating) various makeup techniques to a user's face image. A large number of candidate face images (the first) are determined, and the similarity is determined with the first target face image that the user wants to make such a face by applying makeup. Beauty method such as a makeup method used for generating the second target face image) is extracted.

  FIG. 2 is a flowchart for explaining a process of presenting a beauty technique in the beauty support apparatus 1 according to the first embodiment of the present invention.

  The flowchart shown in FIG. 2 represents a flow according to the execution order of the four program modules P1, P2, P3, and P4.

  P1 is a program module of the image acquisition unit, and a first target face that points to a user's own face image that points to the face image of the user and a target face image that the user wants to make such a face before makeup. It functions as an image acquisition unit that acquires images. The beauty support apparatus 1 according to the first embodiment allows an image captured by the imaging unit 2 as an image input source and any source captured by the image input unit 3.

  P2 is a program module of the evaluation item selection unit, which is an index for determining the degree of similarity between the person image and the first target face image (or a second target face image described later). It functions as an evaluation item selection unit for the user to select "different evaluation items based on". In the following description, “different evaluation items based on human sensitivity” are simply referred to as “evaluation items”.

P3 is a program module of the simulation unit and the candidate face image editing unit, and generates a plurality of candidate face images obtained by applying various makeup methods (that is, simulation) to the principal image, and the generated candidate face images And a simulation unit that presents the evaluation value of the image corresponding to the evaluation item selected by the program module P2 for each candidate face image. From the candidate face images presented by the simulation unit, the user
A target face image (hereinafter referred to as “second target face image”) based on one's face is selected and determined.

  When the user himself wishes to edit the candidate face image, the mode shifts to a candidate face editing mode that enables editing of the candidate face image selected by the candidate face image editing unit. Further, it also functions as a candidate face image editing unit that manually performs a simulation by a user's operation input and creates a second target face image.

  P4 is a program module of the beauty technique presentation unit that extracts the beauty technique (such as cosmetics and makeup procedures) used to generate the candidate face image corresponding to the second target face image, and stores the beauty information related to this beauty technique. It functions as a beauty technique presentation section to present.

  Hereinafter, the operation of the beauty support apparatus 1 according to the first embodiment will be described in detail with reference to FIG. In the explanation, important points will be explained by adding drawings. The beauty support apparatus 1 according to the first embodiment is described with an example in which an application program related to a makeup simulation is mounted on a tablet terminal on which a touch panel in which the display unit 5 and the user interface unit 6 are integrated is mounted. To do.

  First, the person image and the first target face image acquisition unit will be described. FIG. 3 shows a detailed flowchart of the image acquisition unit.

  First, the UI screen 40 shown in FIG. 4 is displayed (ST101), and the user selects the user's face button 43 or the target face button 44 to acquire the person image 41 or the first target face 42. To do. When the image has been acquired, a face image corresponding to the acquired person image 41 or the first target face 42 is displayed next to the user face button 43 or the target face button 44. When the user's face button 43 or the like is selected on the UI screen 40, a button frame is highlighted as shown in FIG. 4, and a selection frame (not shown) is included so as to include the user's image 41 and the person's face button 43. You may make it highlight by.

Next, whether to capture the current face image using the imaging unit 2 (imaging mode) or to use a face image acquired in the past by the imaging unit 2 or the image input unit 3 (selection mode) is captured. The user selects with the mode button 45 or the selection mode button 46 (ST104). The instruction button 47 is used when various operations such as a scroll operation of evaluation items described later and a page change on the UI screen are executed.
Here, when either the shooting mode button 45 or the selection mode button 46 is selected, the program module P11 or the program module P12 shown in FIG. 5 is called.

  When the program module P11 is called, first, the UI screen 60 of FIG. 6 is displayed (ST111). A face image 61 in FIG. 6 is a current face image taken by the imaging unit 2. The face area is detected by the image processing unit 8 and a face area frame 62 indicated by a solid line is displayed. The face area frame 62 is within the range of the photographing face frame 63 indicated by a dotted line and is a certain size or larger. Shooting is possible if there is. The permissible range enabling this shooting can be adjusted by the user using a setting screen (not shown). A face image 61 (the person image 41 or the first target face image 42 in FIG. 4) captured when the user presses the shooting button 64 displayed with the camera icon is acquired (ST112).

After the photographing button 64 is pressed, the UI screen 70 of FIG. 7 is displayed (ST113). The photographed face image 71 in FIG. 7 is the acquired person image or the first target face image. When the user confirms whether to use this image and the YES button 72 is selected, the presented face image is acquired as the person's face image (or first target face image). When the NO button 73 is selected, the UI screen 60 of FIG. 6 is displayed again and the mode is changed to the shooting mode (ST114).

  Next, a selection mode (P102 in FIG. 3) using a face image captured in the past will be described with reference to FIG.

  When the program module P12 is called, first, the UI screen 80 of FIG. 8 is displayed (ST121521). The face image list 81 on the UI screen 80 in FIG. 8 is a display of face images acquired from the image input unit 3 as images of a predetermined size. An image selection frame 82 indicated by a solid line is displayed around the face image selected by the user. When the user presses the enter button 83, the face image selected by the image selection frame 82 is acquired as the user image or the first target face image.

  When both the user image and the first target face image have been acquired, the program module P2 that is the evaluation item selection unit can be transferred (ST102 in FIG. 3). When the instruction button 47 is pressed on the screen UI, the screen shifts to the evaluation item selection unit (ST103).

  Next, processing of the program module P2 that is an evaluation item selection unit will be described. FIG. 9 shows a flowchart of the program module P2 which is an evaluation item selection unit.

  First, the UI screen 100 shown in FIG. 10 is displayed (ST201). The user selects an evaluation item 101 and selects it. The selected evaluation item 101 is highlighted using a selection frame 102 indicated by a solid line, for example.

  If the user wants to change the person image or the first target face image, the user can press the image reacquisition button 103 to return to P1 and acquire the person image or the first target face image again. (ST202).

  It is also possible to estimate attributes such as age and race, and highlight the recommended evaluation item 101 as indicated by the star symbol 104 in FIG.

  When the next page button 105 is pressed, options other than the displayed evaluation item 101 are displayed. Similarly, when the previous page button 106 is pressed, the evaluation item 101 displayed so far is displayed again.

  Note that the evaluation item 101 may be newly added by the user.

  When the number of selected evaluation items 101 is one or more, the simulation button 107 can be pressed (ST203). When the simulation button 107 is pressed, the process moves to the P3 simulation unit (ST204).

  Next, processing of the program module P3 of the simulation unit and candidate face image editing unit will be described. FIG. 11 shows a flowchart of the program module P3 of the simulation unit and candidate face image editing unit.

  First, the UI screen 120 shown in FIG. 12 is displayed (ST301). A candidate face image 121 of the second target face image is presented, and at the same time, the person image 122 and the first target image 123 before applying the beauty technique are displayed.

In addition, the candidate face image 121, the identity image 122 before applying the beauty technique,
The evaluation value in the evaluation item 101 (selected in FIG. 10) during selection of the first target image 123 is shown.

  FIG. 13 is an explanatory diagram of the radar chart 124. As shown in FIG. 13, the evaluation value of the candidate face image 121 is indicated by a broken line, the evaluation value of the person image 122 before applying the beauty technique is indicated by a one-dot chain line, and the evaluation value of the first target image 123 is indicated by a two-dot chain line. In the actual UI screen 120, each image and its evaluation value may be associated with each other using colors or the like. According to this radar chart, the difference between the user's face image and the first target image can be objectively grasped by the evaluation value corresponding to the sensitivity evaluation item.

  Returning to FIG. 12, when the next candidate button 125 is pressed, the candidate face image 121 having the highest similarity with the first target image 123 next to the displayed candidate face image 121 is presented. When the previous candidate button 126 is pressed, the candidate face image 121 displayed so far is displayed again. That is, the arrangement order of the candidate face images 121 is controlled using a similarity score calculated based on the evaluation value for each candidate face image.

  If it is desired to change the evaluation item 101, it is possible to return to the program module P2 and select the evaluation item 101 again by pressing the evaluation item selection button 127 (ST302).

  Next, FIG. 14 shows a UI screen 140 when the list of candidate face images 121 is displayed. When the list display button 128 of FIG. 12 is pressed (ST303), the candidate face image 121 and the radar chart 124 showing the evaluation values of the selected evaluation item 101 of the image are displayed in a list (ST304).

  The candidate face image 121 selected by the user from the list of candidate face images 121 is highlighted using an image selection frame 141 indicated by a solid line. At the same time, the evaluation of the selected evaluation item 101 of the person image 122 and the first target image 123 before applying the beauty technique, the selected candidate face image 121, the person image 122 and the first target image 123 before applying the beauty technique. A radar chart 124 having a value is enlarged and displayed upward. When the enter button 142 is pressed, the screen shifts to the UI screen 120 shown in FIG. 12, and the selected candidate face image 121 is presented (ST305). Then, the candidate face image 121 whose selection is confirmed here is set as the second target image.

  Here, based on the presented candidate face image 121, the user himself can edit and create the second target face. As a result, the user can manually edit the selected candidate face image 121, so that the second target face image intended by the user can be set. When executing the editing process, the editing operation can be performed by pressing the editing mode button 129. When the edit mode button 129 is pressed (ST306), a UI screen 150 as shown in FIGS. 15 to 19 is displayed, and the mode is shifted to the edit mode (ST307). 16 to 19, the same components as those in FIG. 15 are denoted by the same reference numerals, and redundant description thereof is omitted.

  In FIG. 15, on the UI screen 150, a personal image thumbnail 151 before applying the beauty technique, a thumbnail 152 of the candidate image being edited, and a thumbnail 153 of the first target face image are arranged and displayed at the upper left.

When the user presses the thumbnail 152 of the candidate image being edited, the thumbnail 152 of the candidate image being edited is highlighted as shown by a selection frame 154 indicated by a solid line, and the entire candidate face image to be edited is displayed. 155 and the enlarged image 157 of the part being edited displayed in the part selection frame 156 indicated by the dotted line are arranged and displayed in the lower left part of the UI screen 150.
Next, a part to be edited is selected by a part selection button 158, and a part to be edited of the enlarged part is selected by a layer button 159. Note that the part selection frame 156 based on the operation of the part selection button 158 specifies a part of the face image (such as eyes, nostrils, and eyebrows) by a known image recognition technique so that the part is selected to include the specific part. A frame 156 is set. In addition, the region selection frame 156 can be set arbitrarily by the user by performing operations such as pinch-in and pinch-out.

  The user selects a cosmetic and its color and shade using a cosmetic color selection button 161 and a shade bar 162 as shown on the right side of the UI screen 150 shown in FIG. Can be changed, and the entire image 155 of the candidate face image can be edited into an image according to preference. Reference numeral 163 denotes a cosmetic icon. In addition to the cosmetics being displayed, the user can select favorite cosmetics from a cosmetic list (not shown) by tapping the cosmetic icon 163 or the like and display the cosmetics on the UI screen 150. Reference numeral 164 denotes an editing end button for ending the editing process.

  As described above, when the user applies various makeup methods, the evaluation value in the evaluation item 101 during the selection of the candidate face image 155 and the candidate face image indicated by the dotted line on the radar chart 124 is edited. It changes with.

  When the user image thumbnail 151 before application of the beauty technique is pressed on the upper left of the UI screen 150, the UI screen 150 shown in FIG. 16 is displayed, and the entire candidate face image 155 and the entire person image before application of the beauty technique are displayed. The image 165 is displayed in parallel.

  When the detail comparison button 166 in FIG. 16 is pressed, the UI screen 150 shown in FIG. 17 is displayed, and an enlarged image 157 of the candidate face image of the selected part and an enlarged image 167 of the person image before applying the beauty technique are displayed. To do. When the overall comparison button 168 is pressed, the screen shifts again to the UI screen 150 shown in FIG.

  When the thumbnail 153 of the first target face image at the upper left in each UI screen 150 is pressed, the UI screen 150 shown in FIG. 18 is displayed, and the entire candidate face image 155 and the entire first target face image are displayed. The image 169 is displayed in parallel.

  When the detail comparison button 166 is pressed, the UI screen 150 shown in FIG. 19 is displayed, and an enlarged image 157 of the candidate face image of the selected part and an enlarged image 171 of the first target face image are displayed. When the overall comparison button 168 is pressed, the screen shifts again to the UI screen 150 shown in FIG.

  When the edit end button 164 is pressed (ST309), the edit mode is ended and the UI screen 120 of FIG. 12 is displayed (ST301).

  As described above, since the user can manually edit the selected candidate face image, the second target face image intended by the user can be set.

  When the beauty technique presentation button 131 is pressed, the program module P4 that is a beauty technique presentation unit is entered (ST310).

  Next, the process of the program module P4 of the beauty technique presentation unit will be described. FIG. 20 shows a flowchart of the second beauty technique presentation unit.

First, the UI screen shown in FIG. 21 is displayed (ST401). The entire image 165 of the current user's own image captured by the imaging unit 2 and an enlarged image 167 of the selected part are displayed. In addition, a cosmetic technique 172 corresponding to the part selection button 158 and the layer button 159 is displayed as a cosmetic icon. (Cosmetics information) 163 and displayed. Here, a comment about the procedure content is described as the beauty technique 172, but a beauty movie may be presented by popping up a movie playback screen or the like.

  Further, as shown in the upper left part of the UI screen 150, the current principal image thumbnail 151, the candidate face image thumbnail 152 that becomes the second target face image (the set candidate image), and the thumbnail 153 of the first target face image. Is displayed.

  In addition, the evaluation value of the selected evaluation item 101 corresponding to the current face image is displayed on the radar chart 124. At this time, the radar chart 124 also displays the evaluation values of the selected evaluation item 101 regarding the first target face image and the second target face image.

  If it is desired to change the second target face image, the user can return to the program module P3 by pressing the re-simulation button 173 to select and edit the candidate face image again and set the second target face image. It is possible (ST402).

  When the entire display button 174 is pressed, the screen shifts to the UI screen 150 shown in FIG. 22, and the entire image 165 of the current user's personal image captured by the imaging unit 2 is displayed.

  When the parallel display button 175 of the UI screen 150 shown in FIG. 22 is pressed, the UI screen shown in FIG. 21 is displayed, and the whole image 165 of the current user's own image captured by the imaging unit and the enlarged image of the selected part are displayed. 167 is displayed again.

  When the detail display button 176 is pressed, the screen shifts to the UI screen 150 shown in FIG. 23, and an enlarged image 167 of the selected portion of the current user's own image taken by the imaging unit 2 is displayed.

  When the thumbnail 152 of the second target face image is pressed, the screen shifts to the UI screen 150 shown in FIG. 24, and the whole image 165 of the current user photographed by the imaging unit 2 and the second target face image are displayed. The whole image 155 of the candidate face image is displayed in parallel.

  When the detail comparison button 177 is pressed, the screen shifts to the UI screen 150 shown in FIG. 25, and an enlarged image 167 of the entire image 165 of the current user imaged by the imaging unit 2 and a selected portion of the second target face image Are displayed in parallel. Here, an overall comparison button 178 appears instead of the detail comparison button 177, and when the overall comparison button 178 is pressed, the screen returns to the UI screen 150 of FIG.

  When the thumbnail 153 of the first target face image is pressed, the UI screen shown in FIG. 26 is displayed, and the entire image 165 of the current user and the entire image of the first target face image captured by the imaging unit 2. 169 are displayed in parallel.

  When the detail comparison button 177 is pressed, the screen shifts to the UI screen 150 shown in FIG. 27, and the enlarged image 167 of the current user and the enlarged image 171 of the selected portion of the first target face image taken by the imaging unit 2. Are displayed in parallel.

  When the current user image thumbnail 151 is pressed, the screen shifts to the UI screen 150 shown in FIG. 21, and the entire image 165 of the current user image captured by the imaging unit 2 and the enlarged image 167 of the selected portion are displayed again. indicate.

  When makeup end button 179 is pressed, the application is terminated (ST403).

  As described above, according to the first embodiment, a beauty technique closer to the target face image is acquired based on the similarity between the candidate face image and the target face image obtained by applying different beauty techniques to the user's face image. In particular, the second target face image can be identified while referring to the evaluation item based on the human sensitivity for the difference between the user's face image and the target face image, so that the user's satisfaction is high. A beauty technique can be acquired.

  Further, since the user can manually edit the selected candidate face image, the second target face image intended by the user can be set.

  Also, the difference between the user's face image and the first target image can be objectively grasped by the evaluation value corresponding to the sensitivity evaluation item shown in the radar chart.

  Further, in the process of acquiring the final beauty technique, the user compares the face image of the user before applying the beauty technique, the first target face image, and the second target face image while comparing the beauty method. Simulation can be performed, and unintended beauty techniques are prevented from being acquired.

  Further, in the process of acquiring the final beauty technique, a more detailed simulation of the beauty method can be performed while referring to an enlarged image relating to the edited region.

  In addition, in the process of acquiring the beauty technique, the user's face image before applying the beauty technique, the thumbnail images of the first target face image, and the thumbnail images of the second target face image are arranged. The selected state of each selected face image can be constantly reviewed.

  In the first embodiment, an application program related to a makeup simulation is mounted on a tablet terminal on which a touch panel in which the display unit 5 and the user interface unit 6 are integrated is mounted as the beauty support device 1 (user terminal). Explained with an example. As 2nd Embodiment of this invention, it can also be comprised as a beauty assistance system as described below.

  That is, the user side terminal includes a camera (imaging unit) and a touch panel (display unit) that captures the user's face, and starts browsing software such as a web browser (browsing unit) to display an IP network (not shown) A cloud computer (information processing apparatus) provided on the provider side that provides a service relating to makeup simulation is accessed via a communication medium (network) such as a LAN. Then, similarly to the function executed by the beauty support apparatus 1, a makeup simulation program for beauty support may be executed. In this case, the cloud computer can share processing with a large load necessary for the makeup simulation.

  For example, in such a configuration, at least a process with a large amount of computation among processes necessary for makeup simulation, for example, a process of generating a composite face image that is a candidate face image 121 is performed at high speed with a cloud computer provided on the operator side. Can be performed. With this configuration, the user only needs to process the operation instruction input and display output, so there is no need for a high-speed information processing device on the user side, and the cost of the user is greatly reduced. Can do.

  In addition, makeup simulation can be performed even on inexpensive portable terminals such as smartphones and tablet terminals, so that it is possible to receive beauty support not only at home but also at any place such as a place to go.

  A beauty support apparatus, a beauty support system, a beauty support method, and a beauty support program according to the present invention are based on the similarity between a candidate face image and a target face image obtained by applying different makeup methods to a user's face image. Thus, it is possible to provide a makeup method that is closer to the target face image. In addition, the difference between the user's face image and the target face image can be displayed as a specific numerical value as an evaluation item based on human sensitivity, and intuitively recognize how close the target face image is to the target face image. It is useful as a makeup simulator.

DESCRIPTION OF SYMBOLS 1 Beauty support apparatus 2 Imaging part 3 Image input part 4 Storage part 5 Display part 6 User interface part 7 Overall control part 8 Image processing part P1 Program module P2 of an image acquisition part Program module P3 of an evaluation item selection part Simulation part and candidate face Program module P4 of the image editing unit Program module 41 of the beauty technique presentation unit 41 Personal image 42 Target face 43 Personal face button 44 Target face button 45 Shooting mode button 46 Selection mode button 47 Instruction button 61 Face image 62 Face region frame 63 Shooting face Frame 64 Shooting button 71 Shooting face image 72 YES button 73 NO button 81 Face image list 82 Image selection frame 83 Determination button 101 Evaluation item 102 Selection frame 103 Image reacquisition button 105 Next page button 106 Previous page button 107 Simulation button 121 Candidate Image 122 User image 123 First target image 124 Radar chart 125 Next candidate button 126 Previous candidate button 127 Evaluation item selection button 128 List display button 129 Edit mode button 131 Beauty technique presentation button 141 Image selection frame 142 Determination button 151 User image thumbnail 152 Thumbnail (second target face image)
153 Thumbnail (second target face image)
154 Selection frame 155 Whole image 156 Part selection frame 157 Enlarged image 158 Part selection button 159 Layer button 161 Cosmetic color selection button 162 Shading bar 164 Edit end button 165 Whole image 166 Detail comparison button 167 Enlarged image 168 Whole comparison button 169 Whole image 171 Enlarged image 172 Beauty technique 173 Re-simulation button 174 Whole display button 175 Parallel display button 176 Detail display button 177 Detail comparison button 178 Whole comparison button 179 Makeup end button

Claims (8)

A cosmetic method based on a second target face image which is a composite face image of a user who sets a first target face image targeted by the user and which has a high degree of similarity to the first target face image A beauty support device that presents
A face image acquisition unit for acquiring the first target face image and the user's face image;
An evaluation item selection unit that selects at least one sensitivity evaluation item according to a user's input operation;
A simulation unit that presents candidate face images of the second target face image in a state of high similarity in the sensitivity evaluation items selected by the evaluation item selection unit;
According to the selection operation of the candidate face image by the user, the second target face image of the user whose beauty technique is to be extracted is specified, and the cosmetic procedure and the necessary beauty that are included in the second target face image A beauty technique presentation unit for presenting products,
A beauty support device characterized by comprising: The beauty support apparatus according to claim 1, further comprising a candidate face image editing unit, wherein the candidate face image presented by the simulation unit can be edited by a user operation input. The beauty support apparatus according to claim 1, wherein a face image of the user before applying a beauty technique and an evaluation value corresponding to the sensitivity evaluation item are displayed on a user operation input screen. At least two images of the user's face image before the application of the beauty technique, the first target face image, and the second target face image are displayed side by side on the user's operation input screen. The beauty support apparatus according to claim 1. At the time of editing the candidate face image by the candidate face image editing unit, at least one of the face image of the user before applying a beauty technique, the first target face image, and the second target face image. The beauty support apparatus according to claim 2, wherein an enlarged image of the editing part is displayed. The user's face image before applying a beauty technique, the first target face image, and the thumbnail images of the second target face image are displayed on the user's operation input screen. The beauty support apparatus according to claim 5. A cosmetic method based on a second target face image which is a composite face image of a user who sets a first target face image targeted by the user and which has a high degree of similarity to the first target face image A beauty support system that presents
A user terminal having at least an imaging unit and a display unit;
An information processing apparatus connected to the user terminal via a network;
Have
The information processing apparatus is
A face image acquisition unit for acquiring the first target face image and the user's face image;
An evaluation item selection unit that selects at least one sensitivity evaluation item according to a user's input operation;
A simulation unit that presents candidate face images of the second target face image in a state of high similarity in the sensitivity evaluation items selected by the evaluation item selection unit;
According to the selection operation of the candidate face image by the user, the second target face image of the user whose beauty technique is to be extracted is specified, and the cosmetic procedure and the necessary beauty that are included in the second target face image A beauty technique presentation unit for presenting products,
Beauty support system characterized by having A cosmetic method based on a second target face image which is a composite face image of a user who sets a first target face image targeted by the user and which has a high degree of similarity to the first target face image A beauty support method for causing an information processing apparatus to perform a process of presenting
Obtaining the first target face image and the user's face image;
A step of selecting at least one sensitivity evaluation item according to a user's input operation, and a candidate face image of the second target face image in a state of high similarity in the sensitivity evaluation items selected in this step Presenting steps;
According to the selection operation of the candidate face image by the user, the second target face image of the user whose beauty technique is to be extracted is specified, and the cosmetic procedure and the necessary beauty that are included in the second target face image Presenting goods,
A beauty support method characterized by comprising:

Images