JP5085636B2 - Makeup face image generating apparatus, method thereof, server and program - Google Patents

Description

The present invention is a cosmetic facial image generating apparatus for generating user-specific model makeup face image from the user's face image, the way, about the server and a program.

  Patent Documents 1 to 5 disclose techniques for providing a user with a makeup method that meets a user's request. For example, in the makeup method presentation method described in Patent Document 1, a makeup method selected by the user is applied to the face image sent from the user to create a makeup face image, and a makeup procedure according to the makeup face image is performed. Present to the user.

In the makeup advice method described in Patent Document 2, the face of the user and the face of the makeup model are combined to create a makeup face image modeled by the user.
The makeup advice system disclosed in Patent Literature 3 acquires a user's face image from an image input unit, classifies the user's face impression into one of a plurality of impression patterns based on the feature amount of the face image, The classification result is output together with typical face images of a plurality of impression patterns. Then, a simulation image of the user's makeup face is formed for each of the typical impression patterns, and is output together with the user's face image.

  The makeup advice device described in Patent Literature 4 inputs a user's face image, determines the necessary makeup from the difference between the face image and the ideal face registered in the second database, and determines the makeup procedure. The data is read from one database and displayed on a display device.

In the makeup simulation method described in Patent Literature 5, image processing such as morphing and tweening is performed on the user's face image to mix the user's face and the desired face. A plurality of makeup face candidates are obtained by changing the mixing ratio of the user's face and the desired face. Among the obtained makeup face candidates, a makeup face is selected as desired by the user from among makeup face candidates within a range of similarity levels that can be obtained by makeup.
JP 2000-339331 A Japanese Patent No. 3408524 Japanese Patent No. 3582458 JP 2002-123585 A JP-A-10-255066

  In the conventional makeup advice, a makeup model image of a makeup model or a composite image of a makeup model net makeup image and a user face image is used as a model. When compositing a net makeup image of a makeup model with a user's face image, the position and shape of parts such as the face shape, eyes, and mouth are matched between the user's face image and the net makeup image of the makeup model. In addition, a process of deforming the net makeup image of the makeup model has been performed. However, when a user's model makeup face is created from the net makeup image of the makeup model that has been subjected to the deformation process, a model makeup face that does not match the user is obtained. For example, when a model makeup face image is generated by transforming a net makeup image of a makeup model having an egg-shaped face to fit a user with a square face, the model makeup face image is expected by the user. Sometimes it was far from the makeup face. Therefore, the user has to select a makeup face of another makeup model having a face shape similar to the user's face shape.

In Patent Document 5, there is a problem that only makeup face candidates that are separated from the desired face are obtained.
An object of the present invention, the net makeup image of the sample cosmetic face when synthesized in the face image of the user to generate a model decorative face of the user only can generate a model decorative face of the user dedicated closer to the user's preference The object is to provide a makeup face image generation apparatus, a method thereof, a server, and a program .

The present invention is an apparatus for generating a user-specific model makeup face image from a user face image, the sample makeup face image selected by the user, the sample makeup face image, and the Storage means for storing a net makeup image corresponding to the difference between the sample makeup face image and the sample makeup face image, face image acquisition means for obtaining the user face image, and the net makeup Matching processing means for generating a matched net makeup image by performing processing for matching an image with the user's face image, and each of the sample makeup face image and the user face image performing feature extraction to obtain the characteristics of each real face, resembling a real face difference acquisition means for acquiring a difference in the characteristics of both real face, the real face of the user in real face of the sample decorative face Wherein the real face correcting cosmetic image acquisition hand stage you get a decorative image for real face correction a cosmetic to be applied to the real face representing the image of the makeup to reduce the difference in characteristics of the real face of the user, the real face corrected for comprising the use makeup image, said matching already net makeup image, before SL performs synthetic processing to add the user's real face image and an image synthesizing means for generating an image of the user-specific model makeup face.

The image synthesizing means, prior SL includes an image correction means for generating a positive Ajika粧image compensation after by adding Matched net makeup the real face correcting cosmetic image on an image, the corrected net makeup image Is preferably added to the user's face image to generate the user-specific model face image .

It is preferable that the net makeup image includes a plurality of layers corresponding to a plurality of makeup layers from the face of the sample makeup face to the sample makeup face.
The real face difference obtaining means, from the real face of the image of the image with the sample decorative face of the real face of the user, and the outline of both faces to obtain a difference different contours of both real face, the real face correction makeup image acquisition unit , it is preferable to obtain the real face contour correction cosmetic picture image for showing less difference in the contour as the real face correction cosmetic image.

The face difference acquisition means performs feature extraction on each of the user's face image and the sample makeup face's face image to acquire the shape of both face parts, and the corresponding parts between the face faces. get the differences in shape, the real face correction makeup image acquisition unit preferably acquires a part shape correction image image for showing less difference in the shape of the part as the real face correction cosmetic image.

In one example, the part is at least one of eyes, eyebrows, nose and mouth.
The face difference acquisition means acquires a face part layout from each of the user's face image and the sample makeup face's face image, acquires a face part layout difference between both face faces, and corrects the face correction. cosmetic image obtaining means, it is preferable to obtain the real face parts layout correction cosmetic picture image for showing less difference in the face parts layout as the real face correction cosmetic image.

The apparatus preferably has before the image synthesizing unit further comprising a selection means for selecting in accordance with the operation of the user whether to perform a process of pressurizing with the real face correction cosmetic image.

In one example, the storage unit stores the difference eliminating image associated with the type and extent of the difference in characteristics of the real face and the location for adding the difference solution extinguishing images in real face images of the users The face correction makeup image acquisition means reads the difference-resolving image and the position according to the type and degree of difference in the features of the face from the storage means, and the image composition means reads the difference-resolving image. An image is added to the read position in the user's face image.

In one example, the storage means stores procedure information for adding the difference elimination image to the position, and the face correction makeup image acquisition means stores the difference elimination image, the position, and the procedure information. Read from the storage means, and the image composition means adds the read difference eliminating image to the read position in the user's face image according to the procedure information.

The apparatus stores a first makeup procedure recipe for explaining a procedure for applying makeup from a bare face corresponding to the sample makeup face to a makeup face corresponding to the sample makeup face using at least one of a document, sound, and image. And a makeup procedure creation for creating a second makeup procedure recipe that describes at least one of a document, a sound, and an image of a procedure for applying makeup from the user's face to the user-specific model makeup face. The makeup procedure creating means reads out the first makeup procedure recipe corresponding to the sample makeup face from the recipe storage means, and corresponds to the makeup image for face correction in the first makeup procedure recipe. It is preferable that the second makeup procedure recipe is created by correcting a part of the recipe.

In one example, the real face correction makeup image and the previous Matched positive Ajika粧image is composed of a plurality of makeup step and a plurality of layers respectively associated with said device further allows the image of the real face of the user And a makeup intermediate face image generating means for sequentially synthesizing the layers as the makeup step proceeds to create an image of a model makeup face in the middle of each makeup step.

  In one example, a makeup intermediate face image acquisition means for acquiring an image of an actual makeup intermediate face of the user in each makeup step and an image of a model makeup face in the middle of each makeup step, and the actual makeup intermediate face An intermediate makeup face difference acquisition unit for acquiring a difference between the image and the image of the intermediate model makeup face; a makeup procedure recipe for reducing the difference acquired by the intermediate makeup face difference acquisition unit; And advice means for generating makeup advice described in at least one of the images.

  In one example, the sample makeup face image is an image of another person's makeup face that is selected by the user from a plurality of others' makeup face images and has a user-preferred makeup, and the storage means A plurality of other person's makeup face images and the plurality of other person's natural faces are stored in association with each other.

In one example, the device is a server connected via a client terminal and a network, wherein the server, when receiving a start request from said client terminal, generate process of the user-specific model makeup face image The user's face image is received from the client terminal, and the user-specific model makeup face image is transmitted to the client terminal.

In one example, the apparatus is a client terminal that is connected to a server via a network, and that includes a processor that executes a makeup face image generation processing program downloaded from the server, and the storage unit is connected to the client terminal. The sample makeup face image, the sample makeup face image, and the net makeup image received from the server are stored, and the makeup image acquisition means for face correction is provided in the client terminal. And accessing the server to acquire the makeup image for face correction according to the difference between the features of the faces .

The present invention further provides a method for generating a user-specific model makeup face image. The method includes: a sample makeup face image selected by the user; a sample makeup face image; and a user preference corresponding to a difference between the sample makeup face image and the sample makeup face image. A step of storing a net makeup image, a step of obtaining an image of the user's face, and a process of matching the net makeup image with the image of the user's face to generate a matched net makeup image a step of a step of performing feature extraction to obtain the characteristics of each real face, obtains a difference in the characteristics of both real face for each of the sample decorative face real face image and the user of the real face of the image, element representing the image of the makeup to reduce the difference in characteristics of the real face of the sample decorative face of the to resemble real face said a cosmetic to be applied to the real face of the user real face of the user A step you get a correction makeup image by performing said real face correcting cosmetic image, and the matching previously net makeup image, the synthesis process to be added to the real face of the image before Symbol user, user-specific model makeup Generating a face image.

In one example, the feature of the both faces is at least one of a face outline, a face part shape, and a face part layout.
The present invention further provides a server for generating a user-specific model makeup face image from a user face image. The server generates a user-specific model makeup face image from the user face image, the sample makeup face image selected by the user, the sample makeup face image, and Storage means for storing a user-preferred net makeup image corresponding to the difference between the sample makeup face image and the sample makeup face image, and a face image for obtaining the user face image from the client terminal An acquisition means; a matching processing means for generating a matched net makeup image by performing processing for matching the net makeup image with the user's face image; and an image of the sample makeup face and the user's face performing feature extraction for each of the real face of the image, acquires the characteristics of each real face, the real face difference acquisition means for acquiring a difference in the characteristics of both real face, the You get a decorative image for real face correction representing an image of the makeup a cosmetic to be applied to the real face of the user real face of over THE to resemble real face of the sample decorative face to reduce the differences in the characteristics of the real face performing a real face correcting cosmetic image acquisition hand stage, and the real face correcting cosmetic image, and the matching previously net makeup image, the synthesis process to be added to the real face of the image before Symbol user, user-specific model makeup face Image synthesizing means for generating the image and means for transmitting the image of the user-specific model makeup face to the client terminal.

The present invention further provides a program for executing the method on a computer.

Hereinafter, preferred embodiments of the present invention will be described with reference to FIGS.
FIG. 1 shows a makeup advice system 10 for providing makeup advice. The makeup advice system 10 includes a client terminal 12 that is communicably connected to each other through a network 11 such as the Internet, and a server computer 20 (hereinafter simply referred to as a server). The client terminal 12 can be a mobile station such as a personal computer that can be connected to the network 11 by wire or wirelessly, or a mobile phone that can be connected to the network 11 via the mobile phone network 13. Connected to the network 11 is a computer 15 installed in a makeup or beauty / health related organization. The computer 15 provides information related to makeup, beauty information, and health information to the server 20 through the network 11. The makeup information includes information on cosmetics (new product information on cosmetics, characteristics, usage, ingredients, etc.), information on makeup procedures and techniques, and the like. Examples of the network 11 are the Internet, a LAN (Local Area Network), and a VAN (value added circuit network).

  The client terminal 12 includes an input unit 12a (for example, a pointing device such as a keyboard or a mouse). The input unit 12 a is used when the user 14 wants to receive makeup advice and inputs the face image of the user 14 to the client terminal 12. The client terminal 12 transmits the face image of the user 14 to the server 20 through the network 11. The server 20 performs image processing on the face image of the user 14 received from the client terminal 12, generates a model makeup face image dedicated to the user 14, and transmits the model makeup face image to the client terminal 12 of the user 14. To do. The client terminal 12 displays the model makeup face image on the display unit 12b. The client terminal 12 is provided with a camera 16. The camera 16 is used to photograph the user's 14 face and makeup face. The camera 16 is built in the client terminal 12 or externally connected.

  After providing the model makeup face image to the client terminal 12, the server 20 will explain at least one of a document, sound, and image a makeup procedure for making the user's 14 face as a makeup face according to the model makeup face. The makeup procedure recipe is transmitted to the client terminal 12 of the user 14, and the makeup procedure recipe is output to an output device such as the display unit 12b. The makeup procedure recipe includes at least sentences. For example, when the makeup procedure recipe includes sound, a sound describing the makeup procedure is output from the speaker of the client terminal 12.

  The makeup procedure consists of a plurality of steps. The user 14 sequentially applies makeup according to the makeup procedure recipe. When the target makeup face for each step, that is, the face during makeup, is completed, the user 14 captures the face during makeup and transmits it to the server 20 from the client terminal 12. The server 20 that received the makeup intermediate face evaluates the makeup intermediate face in the current makeup step. Then, the server 20 prepares makeup advice for bringing the current makeup intermediate face closer to the model makeup face of the step, and transmits the makeup advice to the client terminal 12 of the user 14. For example, the user can correct the makeup with reference to the makeup advice displayed on the display unit 12b.

  As shown in FIG. 1, the server 20 includes a main body 21 and first to seventh storage units 22 to 28 (seven types of databases). The storage units 22 to 28 may be stored in one storage device or may be stored separately in a plurality of storage devices. A storage device that stores the storage units 22 to 28 may be built in the main body 21 or may be externally attached. The contents that require specialized knowledge about makeup such as cosmetics and procedures and techniques for makeup in the database are created using information acquired from the computer 15 through the network 11.

  The server 20 performs a process of generating a model makeup face image dedicated to the user 14, a process of evaluating the makeup performed by the user 14, and a process of giving makeup advice according to various program codes stored in the storage means. The server 20 stores in the storage means images for displaying the screens shown in FIGS. 4, 5, 12, 13, 15, and 17 on the display unit 12 b of the client terminal 12, and makeup support processing in FIG. 3. The program is stored. The makeup support processing program includes a makeup face image generation processing program and a makeup advice processing program.

  The server 20 includes a central processing unit (CPU), a ROM (read only memory), and a RAM (random access memory). The CPU executes various control programs stored in the ROM or hard disk drive. The RAM temporarily stores various data such as calculation results. The server 20 includes a transmission / reception unit 29 that transmits / receives data to / from the client terminal 12 connected to the Internet 11. For example, the transmission / reception unit 29 transmits web screen data in response to an access request from the client terminal 12 and causes the display unit 12b of the client terminal 12 to display the web screen shown in FIGS. Part.

  FIG. 2 is a block diagram showing a functional configuration of a makeup support apparatus constructed by the server when the server 20 executes the makeup support processing program (makeup face generation processing program and makeup advice processing program). The transmission / reception unit 29 is configured by the web server unit as described above, and data to be transmitted to the client terminal 12 is generated and transmitted in advance as web data. The makeup support apparatus 30 illustrated in FIG. 2 includes a makeup face image generation apparatus that performs makeup face image generation processing, makeup procedure screen generation processing, and makeup advice processing.

  The makeup support apparatus 30 includes a control unit 31 that performs overall control of the apparatus, a matching processing unit 32, a net makeup image correction processing unit 33, a composition processing unit 34, a recipe correction unit 35, first to seventh storage units 22 to 28, and Storage units 36 to 38 functioning as buffers for temporarily storing data such as images are provided.

The storage units 22 to 28 will be described.
A sample makeup face database is constructed in the first storage unit 22. The sample makeup face database is obtained by dividing an image 64 of a large number of sample makeup faces (for example, a makeup face of another person such as a makeup model) and an image 72 of the sample makeup face from the sample makeup face images 64. And a net makeup image 73 of the sample makeup face.

  In the second storage unit 23, a sample face database including sample face images 72 of the sample makeup face respectively corresponding to the many sample makeup face images 64 stored in the first storage unit 22 is constructed.

  The third storage unit 24 stores a facial contour correction image used to compensate for a facial shape difference (for example, a difference between the facial contour of the user 14 and the facial contour of the sample selected by the user 14). Thus, a face contour correction database is constructed.

  The fourth storage unit 25 stores a part shape correction image used to compensate for a difference in facial part shape (for example, a difference between the nose of the user 14 and the nose of the sample selected by the user 14). A database for shape correction has been established.

  In the fifth storage unit 26, a face used to compensate for a difference in face part layout (see FIG. 11) (for example, a difference between the eye position of the user 14 and the eye position of the sample selected by the user 14). A part layout correction image is stored, and a face part layout correction database is constructed.

The face contour correction image, the part shape correction image, and the face part layout correction image stored in the third to fifth storage units 24 to 26 are each assigned one identifier.
A sixth storage unit 27 serving as a recipe storage means describes a procedure for applying makeup from a sample face to a sample makeup face. A makeup procedure recipe (first makeup procedure) including at least one of a document, an image, and a voice is described. Recipe) is stored, and a makeup procedure recipe database is constructed. The makeup procedure recipe includes at least a document. The makeup procedure recipe is preferably a combination of a sentence and an image (still image or video), but only a sentence, only a voice, a combination of a voice and an image, or a combination of a sentence, a voice and an image But you can.

  The seventh storage unit 28 stores a correction document for correcting the text of the first makeup procedure recipe in accordance with the user 14, and a recipe correction database is constructed. This recipe correction document is inserted into the document of the first makeup procedure recipe or used to replace a part (eg, word, phrase, sentence, paragraph). The correction document is stored in each of the third to fifth storage units 24 to 26. The recipe correction document is a contour correction net makeup image and part shape adapted to correct the face contour. The part shape correction net makeup image adapted to be corrected and the layout correction net makeup image adapted to correct the face part layout are associated with identifiers. If the server 20 searches the procedure correction database based on the identifier attached to the correction net makeup image, the correction document corresponding to the identifier can be acquired. The seventh storage unit 28 includes an address for specifying a document position in the document where the correction document may be replaced for correction, or a position where additional correction document may be inserted. A link table that associates the identifier attached to the net makeup image is stored. When the net makeup image for correction is determined, the replacement area address or additional insertion position address of the correction document corresponding to the identifier can be acquired by referring to the link based on the identifier attached thereto. Details of documents and links will be described later.

  The storage unit 36 stores a face image of the user 14's face. The face image of the user 14 is received by the transmission / reception unit 29 from the client terminal 12 and stored in the storage unit 36 in accordance with an instruction from the control unit 31. The transmission / reception unit 29, the control unit 31, and the storage unit 36 constitute a face image acquisition unit that acquires an image of the face of the user 14.

  The memory | storage part 37 memorize | stores the image acquired in the process of producing | generating the corrected net makeup image 100 which should be synthesize | combined with the user's 14 face image in order to produce the user's 14 example makeup face image. The corrected net makeup image 100 is a difference between a net makeup image 74 (FIG. 7 (b)) after matching processing, which will be described later, and the face image 72 of the user 14 and the sample makeup face. And a makeup image (corrected image) added to the net makeup image 74 in order to compensate for a difference in contour, a difference in part shape, and a difference in face part layout.

  The storage unit 38 temporarily stores document data in the process of creating a makeup procedure for making the user's 14 face into a model makeup face. Each of the storage units 36 to 38 includes a predetermined storage area of a memory such as a RAM.

  Hereinafter, using various data stored in the first to seventh storage units 22 to 28, a makeup face image generation process for generating a model makeup face image of the user 14, and the user 14 models the model makeup face Each of the units 32 to 35 that perform makeup advice processing for advice on makeup to be performed will be described. The operations of the units 32 to 35 are controlled by the control unit 31.

  The makeup support apparatus 30 uses the face image 71 of the user 14, the face image 72 of the sample makeup face, and the net makeup image 73 of the sample makeup face, and matches the net makeup image to the contour of the face of the user 14 and the like. Thus, the net makeup image for user 14 (corrected net makeup image 100) is generated. Then, the corrected net makeup image 100 is combined with the user's 14 face image to generate a model 14 makeup face image. This processing is performed by the matching processing unit 32, the net makeup image correction processing unit 33, and the composition processing unit 34. The generated model makeup face image of the user 14 is used as a model when the user 14 applies makeup.

  The matching processing unit 32 as a matching processing means performs a matching process for processing the net makeup image 73 (see FIG. 6) of the sample makeup face so as to match the face size, the face outline, and the like of the user 14. That is, the net makeup image 73 (see FIG. 6) of the sample makeup face selected by the user 14 is read from the first storage unit 22. The net makeup image 73 is stored as a part of the layer (net makeup image layer group) constituting the sample makeup face image 64, or is associated with the sample makeup face image 64 and the sample makeup face image 64. Or stored as a separate image. Conventionally, when a model makeup face image of the user 14 is generated, it is performed until the matching process is performed on the net makeup image 73. However, in the preferred embodiment, the net makeup image 74 after the matching process (FIG. 7B). ))) Is further corrected by the net makeup image correction processing unit 33.

  The net makeup image correction processing unit 33 serving as a correction unit extracts a difference between the sample facial face of the sample makeup face and the user's 14 facial face, and performs matching processing on an image (correction image) representing makeup to make the difference less visible. Correction processing to be added to the subsequent net makeup image 74 is performed. In order to realize this correction processing, the net makeup image correction processing unit 33 includes a difference extraction unit 40 and an image processing unit 45. The difference extraction unit 40 extracts a difference between the sample facial face of the sample makeup face and the user's 14 facial face, and a correction image acquisition function of acquiring an image (correction image) representing makeup for making the difference less visible have. Further, the image processing unit 45 performs image processing for adding the corrected image to the net makeup image 74 after the matching processing to generate the corrected net makeup image 100. In a preferred embodiment, the difference extraction unit 40 functions as a face difference acquisition unit and a corrected image acquisition unit.

  The difference extraction unit 40 uses facial contours, part shapes, and face part layouts as extraction targets of differences, and for each difference extraction target, a facial contour extraction unit 41, a part shape extraction unit 42, and a face part layout extraction unit 43 Is provided. Further, there are three types of correction images that the image processing unit 45 adds to the net makeup image 74 in accordance with differences in face contour, part shape, and face part layout. The image processing unit 45 includes a first image processing unit 46, a second image processing unit 47, and a third image processing unit 48 according to the type of the correction image added to the net makeup image 74.

  The face contour extraction unit 41 extracts the feature of each face contour from the sample makeup face image and the user's 14 face image, and extracts the difference between the feature-extracted sample and the user's 14 face profile. . Then, in order to obtain an image representing makeup (hereinafter referred to as a face contour correction image 79 (see FIG. 8)) for showing the difference in the face contour less, 3. A correction image corresponding to the difference, procedure information of the correction image, and layer position information are searched from the face contour correction database in the storage unit 24. The corrected image is a color image representing a makeup color or density determined according to the makeup layer or makeup type, for example, an image such as a shadow or a highlight. Depending on the image, a density distribution such as a gradient is predetermined. It is an image applied in the direction. The procedure information is the area designation information that designates the area to be corrected by the positional relationship with the area corresponding to the difference, and the density designation that designates the direction in which the density distribution is applied in the case of an image having a density distribution such as gradient. Contains information. The position designation information may specify the area itself corresponding to the difference as the area to which the correction image is applied, or the position relative to the area corresponding to the difference, such as when the area corresponding to the difference does not exist on the user's 14 face. In some cases, a specific area in the face area of the user 14 is designated as an area to be subjected to the correction image. Then, the first image processing unit 46 generates a layer of the face contour corrected image by applying the corrected image acquired by the face contour extracting unit 41 to the region specified by the procedure information by a specified procedure. At the same time, a face contour correction process is performed to add the layer of the face contour correction image to the net makeup image 74 after the matching process.

  For example, as shown in the upper part of FIG. 6, the sample makeup face image 64 includes a sample makeup face image (ie, a face image of the sample makeup face before makeup) 72, and a sample makeup face net makeup image 73. Can be divided into In the preferred embodiment, the net makeup image 73 of the sample makeup face shown in FIG. 6 is composed of a plurality of layers Lm1 to Lmn, as shown on the lower side in FIG. The plurality of layers Lm1 to Lmn are images classified according to makeup types and makeup layers such as foundation, concealer, teak (blusher), lipstick (lip), eye shadow, eyeline, mascara, eyebrow, and the like. A net makeup image 74 obtained by matching the net makeup image 73 with the contour of the user's face, the face size, and the like is similarly composed of a plurality of layers Lu1 to Lun (see FIG. 7). reference). The face contour correction image 79 (see FIG. 8) is also composed of one or a plurality of layers.

  The first image processing unit 46 designates the layer of the face contour correction image 79 generated according to the correction image and the procedure information by the layer position information among the plurality of layers Lu1 to Lun constituting the net makeup image 74. The process of inserting and adding to the appropriate layer position is performed. The layers Lu1 to Lun of the net makeup image 74 are stacked in a predetermined order for each makeup type, and at least a plurality of makeup layers in the order in which makeup layers are stacked on the face background. The insertion position is determined to be a layer position corresponding to a layer corresponding to the makeup type and layer (for example, foundation, cheek, etc.) of the facial contour correction image 79. In a preferred embodiment, the storage unit 24 stores layer position information for specifying a layer position to be inserted in association with each corrected image related to the face outline. When the face contour extraction unit 41 obtains the correction image and procedure information related to the face contour according to the difference in the face contour, the face contour extraction unit 41 reads out the layer position information corresponding to the correction image from the storage unit 24 together. The corrected image related to the contour, the procedure information, and the layer position information are sent as a set to the first image processing unit 46. The first image processing unit 46 applies the layer of the face contour correction image 79 generated by applying the correction image to the position area according to the procedure information by the specified method to the layers Lu1 to Lun constituting the net makeup image 74. On the other hand, it is added to the layer position specified by the layer position information.

  The reason for performing the correction to add makeup that compensates for the difference in the facial contour is as follows. The outline of the face includes an egg shape, an inverted triangle, a quadrangle, a round shape, and a triangle. For example, if the outline of the sample is an egg shape, the makeup applied to the sample is a makeup that matches the egg shape. The composition of this egg-shaped face and makeup suitable for the egg-shaped face is a sample makeup face. For example, if this net makeup image is given to the face image of the user 14 with a square face, even if the net makeup image is a net makeup image after matching processing according to the square face of the user 14, the model makeup The face image gives the impression that the user's 14 square face is more stretched than the sample makeup face image 64. Therefore, the obtained model makeup face image of the user 14 gives an impression that is greatly different from the sample makeup face image 64, and is suitable even if the user 14 applies the makeup of the sample makeup face to his / her face. There will be no makeup.

  In the preferred embodiment, in consideration of the combination of the face contour and the makeup determines the makeup face, between the sample facial image of the sample makeup face and the sample obtained by feature extraction from the user's 14 face image and the user 14 The image processing is performed to extract the difference from the face contour and make up the makeup (corrected image) representing the makeup for making the difference appear small on the net makeup image 74 after the matching processing. For example, assuming that the contour of the sample is “oval” and the contour of the face of the user 14 is “square”, a makeup that makes the contour of the square face appear as an oval facial contour is added to the net makeup image 74. The contour correction process for the face to be performed is performed. Specifically, the face contour of the user 14 is made close to the sample egg-shaped face contour, for example, the face contour shape of the part from the jaw toward both sides is rounded to a rounded shape. The makeup to be shown is added, the makeup of the square face of the user 14 is made close to the sample egg-shaped face, and then the net makeup image of the sample makeup face (however, the net makeup image 74 after the matching process) Is given. For this reason, the model makeup face image of the user 14 gives an impression close to the image of the sample makeup face.

  The part shape extraction unit 42 extracts features of each part shape from the sample facial image of the sample makeup face and the user's 14 facial image, and also extracts the difference between the sampled part shape of the sample and the part shape of the user 14. Then, a correction image related to the part shape used to generate an image (referred to as a part shape correction image) representing makeup for showing a small difference in part shape, procedure information of the correction image, and layer position information The part shape correction database in the fourth storage unit 25 is searched. Then, the second image processing unit 47 generates a layer of the part shape correction image by applying the correction image acquired by the part shape extraction unit 42 to the specified position area in accordance with the procedure information by a specified method, and generates the generated part shape. A part shape correction process for adding the corrected image to the net makeup image 74 after the matching process is performed. In a preferred embodiment, eyes, eyebrows, nose and mouth are used as parts. That is, the difference in the shape of parts such as eyes, eyebrows, nose, and mouth between the sample makeup face and the user's 14 face is also significant in the impression of the makeup face obtained by synthesizing the net makeup image 74 with the user's 14 face image. Affect. Therefore, in a preferred embodiment, a difference in part shape between the sample and the user 14 is extracted, and an image (part shape correction image) representing makeup for making the difference appear small is a net makeup image 74 after the matching process. By performing the part shape correction process applied to the model 14, the model makeup face image of the user 14 is brought close to the impression of the sample makeup face from the part shape surface.

  Specifically, when the part is an eye, the sample eye shape and the user 14 eye shape are extracted from each of the sample makeup face image and the user 14 face image, and the feature extracted sample and user 14 is extracted. Then, it is used to generate an image (corrected image) representing makeup for making the eye shape difference look smaller from the part shape correction database in the fourth storage unit 25 based on the eye shape difference. A corrected image, procedure information of the corrected image, and layer position information are searched. Then, an eye shape correction image is generated by applying the searched correction image to a specified position region according to the procedure data information by a specified method, and the eye shape correction image is added to the net makeup image 74. When the part is an eyebrow, the eyebrow shape of the sample and the user 14 is extracted, and the difference in the eyebrow shape between the sample whose features are extracted and the user 14 is extracted. Then, based on the difference, a correction image used for generating an image (eyebrow shape correction image) representing makeup for showing the difference in the eyebrow shape from the part shape correction database in the fourth storage unit 25, and The procedure information and the layer position information are retrieved, and the eyebrow shape corrected image is generated by applying the retrieved corrected image to the specified region according to the procedure information in a specified method, and the eyebrow shape corrected image is converted into the net makeup image 74. Append. When the part is a nose, the nose shape of the sample and the user 14 is extracted, and the difference in the nose shape between the sample whose features are extracted and the user 14 is extracted. Then, based on the difference, from the part shape correction database of the fourth storage unit 25, a correction image used to generate a makeup image (nose shape correction image) that makes the difference in the nose shape appear less The procedure information and the layer position information are searched, and the nose shape correction image generated by applying the searched correction image to the specified area according to the procedure information by the specified method is added to the net makeup image 74. Further, when the part is a mouth, the mouth shape of the sample and the user 14 is extracted, and the difference in mouth shape between the extracted sample and the user 14 is extracted. Then, based on the difference in mouth shape, the correction used to generate an image (mouth shape correction image) representing makeup for making the mouth shape difference appear less from the part shape correction database in the fourth storage unit 25. The image, procedure information, and layer position information are searched, and the searched corrected image is applied to a specified area according to the procedure information by a specified method to generate a mouth shape corrected image, and the mouth shape corrected image is a net makeup image. 74.

  For example, when the eyes of the sample are “hanging eyes” slightly raised at the corners of the eyes, whereas the eyes of the user 14 are “sagging eyes” slightly lowered at the corners of the eyes, the net makeup image 73 of the sample makeup face is: Since it is a makeup suitable for some hanging eyes (for example, makeup that makes the corners of the eyes look slightly lower than the actual makeup), even if the net makeup image 74 is applied to the user 14 as it is even after the matching processing, Sagging eyes will be further emphasized. Therefore, the model makeup face image of the user 14 gives an impression far from the sample makeup face. Therefore, in this part shape correction process, a makeup image in which the user's 14 sag is brought close to the catch of the sample makeup face on the net makeup image 74 after the matching process (that is, a makeup image that makes the corner of the eye look higher than the actual makeup). ) Is added. By the correction makeup of the parts, the impression of the model makeup face image of the user 14 obtained by synthesizing the corrected net makeup image on the face image of the user 14 approaches the impression of the sample makeup face.

  The face part layout extraction unit 43 extracts a face part layout representing a part position and a part interval from each of the sample face image of the sample makeup face and the user face image, and the face part between the sample and the user 14. Calculate the difference from the layout. The face part layout extraction unit 43 stores a correction image, procedure information, and layer position information used to generate an image (referred to as a face part layout correction image) representing makeup for showing the difference to a fourth storage unit. Search from 25 face part layout correction databases. Then, the third image processing unit 48 generates a face part layout corrected image by applying the corrected image acquired by the face part layout extracting unit 43 to a specified region in accordance with the procedure information in a specified method, and the face part layout corrected image. Is added to the net makeup image 74 after the matching process. The difference in the facial part layout determined by the layout of parts such as the eyes, eyebrows, nose and mouth in the entire face between the sample makeup face and the user's 14 face is also obtained by synthesizing the net makeup image with the user's 14 face image. This greatly affects the impression of the makeup face. Therefore, in a preferred embodiment, the difference between the facial part layout between the sample and the user 14 is extracted, and the makeup for making the difference appear small, that is, the facial part layout of the user 14 is changed to the facial part layout of the sample cosmetic face. A face part layout correction process is performed in which the makeup image (face part layout correction image) to be brought closer is added to the net makeup image 74. By this face part layout correction process, the impression of the model makeup face image of the user 14 is brought close to the impression of the sample makeup face from the face part layout side.

  As the face part layout, the face part layout in each of the horizontal and vertical faces is obtained for each of the sample and the user 14. The horizontal face part layout is obtained, for example, as the ratio of the width of each area obtained by dividing the face width at the eye height into three at the positions of both eyes. For example, if the distance between both eyes of the user 14 is wider than the distance between both eyes of the sample in the horizontal or horizontal direction of the face, the makeup image (corrected image) that makes the distance between the eyes narrower is displayed as the net makeup image 74. Append. The vertical face part layout is obtained as a ratio of the distance from the hairline to the eyes, the distance from the eyes to the mouth corner of the mouth, and the distance from the mouth corner to the tip of the jaw in the face length, for example. After comparing the face part layout in the vertical direction between the sample and the user 14, the makeup image (corrected image) that makes the face part layout of the user 14 look closer to the face part layout of the sample makeup face is subjected to the matching process The image processing to be added to the net makeup image 74 is performed. For example, if the hairline of the user 14 and the gap between eyes are wider than that of the sample, a makeup image (corrected image) that makes the gap between the hairline and eyes look narrower is added to the net makeup image 74. , The impression of the face part layout is brought closer to that of the sample makeup face.

  A two-dimensional image may be used for the face contour and part shape extraction processing, but a three-dimensional image is preferably used. For example, it is possible to create a three-dimensional image of the face outline and part shape using a plurality of images obtained by photographing a face from several different angles. From the three-dimensional image, it is possible to detect a three-dimensional difference in the face between the user 14 and the sample, for example, the height of the nose is lower than the nose of the sample and, for example, a makeup image that makes the nose look higher (Corrected image) is added to the net makeup image 74. If the user's 14 cheekbone comes out of the sample, a makeup image that suppresses the protrusion of the cheekbone is added to the net makeup image 74. In a preferred embodiment, hair style advice is provided in addition to makeup. For example, if the user's forehead is wider than the sample, it is recommended to remove the bangs, and if the hair volume is small, it is recommended to apply hair, etc., and advice other than makeup is advised to resemble the impression of the sample makeup face. Advise the colors of accessories and clothes, and advise on the accessories that change the impression of the user's face closer to the sample makeup face, the color of clothes that emphasize the face color, the clothes and patterns of clothes that look like small faces.

  The composition processing unit 34 as an image composition unit corrects the net makeup image 74 to generate a corrected net makeup image 100 (see FIG. 7C), and the corrected net makeup image 100 is displayed as the face of the user 14. Composite to image. That is, the user's 14 face image is read from the storage unit 36, the corrected net makeup image 100 is read from the storage unit 37, and the combined process of overlaying the corrected net makeup image 100 on the face image is performed. A model makeup face image is generated.

  The makeup support apparatus 30 creates a makeup procedure recipe (in this example, a document) that is referred to when the user 14 applies makeup on his / her own face by looking at the model makeup face image of the user 14 generated by the composition processing unit 34. To do. Then, in a plurality of makeup steps until reaching the makeup of the model makeup face image, a screen including a makeup intermediate face image targeted in each makeup step and a description of a makeup method for finishing the makeup intermediate face (See FIGS. 13 and 15) is displayed on the client terminal 12.

  The procedure correction unit 35 as a makeup procedure creation means reads a makeup procedure recipe corresponding to the selected sample makeup face from the sample makeup procedure database in the sixth storage unit 27, and converts the recipe document into a net makeup image 74. By making corrections according to the attached correction image, a makeup procedure document suitable for the model makeup face image of the user 14 is created.

  The procedure correction unit 35 corrects the makeup procedure recipe according to the type of correction (face contour, part shape, face part layout) applied to the net makeup image 74, and the part shape procedure correction. Unit 52 and face part layout procedure correction unit 53. An identifier (a face contour image identifier, a part shape image identifier, a face part layout image identifier) is included in the correction image (face contour correction image, part shape correction image, face part layout correction image) extracted by the difference extraction unit 40. Is attached. The procedure correction unit 35 acquires the identifier of the correction image added to the net makeup image 74 from the difference extraction unit 40.

  The face contour procedure correction unit 51 searches the procedure correction database in the seventh storage unit 28 based on the face contour image identifier to obtain a face contour correction document according to the face contour image identifier. This face contour correction document indicates the correction contents of the makeup procedure recipe that is necessary because the face contour correction image with the identifier is added to the net makeup image 74. The part shape procedure correction unit 52 searches the procedure correction database in the seventh storage unit 28 based on the part shape image identifier, and acquires the part shape correction document corresponding to the part shape image identifier. This part shape correction document indicates the correction contents of the makeup procedure recipe that is necessary because the part shape correction image with the identifier is added to the net makeup image 74. The face part layout procedure correction unit 53 searches the procedure correction database in the seventh storage unit 28 based on the face part layout image identifier, and acquires a face part layout correction document according to the face part layout image identifier. The face part layout correction document indicates the correction contents of the makeup procedure recipe that is necessary because the face part layout correction image with the identifier is added to the net makeup image 74. When performing the procedure correction process, the procedure correction unit 35 first reads a makeup procedure recipe corresponding to the sample makeup face from the sixth storage unit 27 and stores it in the storage unit 38. Then, part or all of the sample makeup procedure recipe (first makeup procedure recipe) temporarily stored in the storage unit 38 is replaced with a correction document obtained by searching the database of the seventh storage unit 28, or the sample makeup procedure A correction document is added to the recipe to create a makeup procedure recipe (second makeup procedure recipe) for the user 14.

  FIG. 14A shows a sample makeup procedure recipe stored in the sixth storage unit 27. In this example, the makeup procedure recipe includes n recipes D1 to Dn from “Step 1” to “Step n”. Each makeup procedure recipe includes a sentence explaining a makeup procedure using a cosmetic or a makeup tool used to complete a makeup intermediate face to be targeted in each step. Each of the makeup procedure recipes D1 to Dn includes a sentence portion (face contour related document) explaining makeup that affects the face contour and makeup that affects the shape of parts such as eyes, eyebrows, nose and mouth. A sentence part to be explained (part shape related document) and a sentence part (face part layout related document) explaining makeup that affects the face part layout are included. Each of these sentence parts may be included only in one step of sentence, or may be included in a plurality of documents in several steps.

  In a preferred embodiment, the makeup instructions D1 to Dn are composed of a document explaining a makeup method for each step by dividing a makeup procedure from a sample makeup face to a sample makeup face into a plurality of steps. As shown in FIG. 14A, an address 56 is assigned to a predetermined document area 55 or position in the document of each makeup instruction D1 to Dn. The document area 55 specified by the address 56 is a candidate for replacement destination of the corrected document corresponding to the corrected image, and the position specified by the address 56 is a candidate for additional insertion destination of the corrected document corresponding to the corrected image. . In the example of FIG. 14, the address 56 is represented by 5-bit binary data, but an appropriate number of bits can be adopted.

  A link table that associates the identifier of the corrected image with the address 56 is stored in the predetermined storage area of the seventh storage unit 28. By referring to the link table based on the identifier of the corrected image, the address 56 corresponding to the identifier can be acquired. In other words, the face contour image identifier includes an address 56 for specifying a document area 55 to be replaced or an address 56 for specifying a position to be additionally inserted into a corrected document that affects the face contour searched based on the identifier. Linked. The eye shape image identifier is linked to an address 56 that specifies a document area 55 to be replaced or an address 56 that specifies a position to be additionally inserted into a corrected document that affects the searched eye shape based on the identifier. Yes. Hereinafter, similarly, the nose shape image identifier includes an address 56 for specifying a replacement document area 55 or an additional insertion destination position of the corrected document that affects the nose shape searched based on the identifier, and a mouth shape image. The identifier is linked to the document area 55 that is the replacement destination of the corrected text that affects the mouth shape searched based on the identifier or the address 56 that specifies the position of the additional insertion destination. The face part layout image identifier is linked to an address 56 that specifies the position of the replacement insertion destination document area 55 or the additional insertion destination that affects the face part layout searched based on the identifier.

  If the address 56 acquired by referring to the link table based on the identifier of the corrected image specifies the document area 55, the procedure correcting unit 35 uses the identifier for the procedure correction in the seventh storage unit 28. When the corrected document retrieved from the database is replaced with the document in the document area 55 at the address 56, and the address 56 specifies the position, the corrected document corresponding to the identifier is additionally inserted at the position specified by the address 56. To do. Specifically, the face contour procedure correction unit 51 acquires an address 56 by referring to a link based on the face contour image identifier, and the document in the document area 55 specified by the acquired address 56 and the corrected document are obtained. The correction document is additionally inserted at the position specified by the address 56. The part shape procedure correction unit 52 refers to the link based on the part shape image identifier, acquires the address 56, and replaces the document in the document area 55 specified by the acquired address 56 with the corrected document, or the address A correction document is additionally inserted at the position specified by 56. The face part layout procedure correcting unit 53 obtains an address 56 by referring to the link based on the face part layout image identifier, and replaces the document in the document area 55 specified by the acquired address 56 with the corrected document, A correction document is additionally inserted at the position specified by the address 56.

  The amendment document related to the shape should contain information that corrects which part should have shadows and highlights when applying makeup that affects the shape, and what the area, shape, and density should be as appropriate. It has become. For example, a makeup procedure recipe for a sample makeup face to be used as a template includes a document “please apply a XX foundation”, and the address 56 in the document area 55 of this sentence is an identifier of a face contour correction image. Is linked to. Then, the face contour procedure correction unit 51 accesses the procedure correction database based on the identifier acquired from the difference extraction unit 40, for example, “a tone of a tone slightly darker than the makeup base so as to put a shadow from the chin to the side. “A color foundation is applied” is acquired, and an address 56 is acquired by referring to the link table from the identifier. Then, the face contour procedure correction unit 51 replaces the corrected document with a document in the document area 55 specified by the address 56, that is, “please apply a foundation of color XX”. Similarly, the makeup procedure recipe is corrected by the correction document for the part shape and the face part layout.

  For example, if each corrected image related to the face outline and the face part layout is related to the same foundation, even if the identifiers are different from each other, the same document area 55 or position relating to how to apply the foundation is specified. The address 56 is acquired. When the address 56 obtained by referring to the link table based on the identifier becomes the same between different identifiers, the procedure correction unit 35 preferentially uses one correction document according to a predetermined rule, The document content in a specific area in the document is analyzed, and processing for combining both corrected documents is performed based on the analysis result.

  In this way, the procedure correction unit 35 corrects the template document, that is, the explanation document of the makeup procedure recipe for the sample makeup face, so that the makeup procedure recipe when the user 14 performs makeup using the makeup face image as a model. An explanatory document is created. Then, the makeup procedure recipes D1 to Dn for the sample makeup face shown in FIG. 14 (a) are replaced or additionally inserted as shown in FIG. 14 (b) by the correction described above. The makeup procedure recipes X1 to Xn are created.

  When the description of the makeup procedure recipe includes voice or an image other than text, for correction, for example, the address of the description part of the part that affects the face outline in the voice is linked to the identifier of the face outline corrected image The address of the explanation image part of the part that affects the outline of the face in the image may be linked to the identifier of the face outline corrected image. In this way, even if the makeup procedure recipe is sound or image, the makeup procedure recipe customized for the user 14 can be created by correcting the makeup procedure recipe for the sample makeup face.

  Next, the makeup face image generation process and the makeup advice process performed in the makeup support apparatus 30 by the CPU of the server 20 executing the makeup support process program will be described with reference to the flowchart shown in FIG. The CPU first executes a makeup face image generation program.

  First, when using the makeup advice system 10, the user 14 registers as a user, obtains an ID and a password, and inputs from an initial screen (not shown) displayed on the display unit 12b of the client terminal 12. The server 20 is accessed by inputting the ID and password by operating the unit 12a. When there is an access from the client terminal 12, the CPU of the server 20 displays a main screen for makeup support on the display unit 12 b of the client terminal 12. When the user selects to receive makeup support on this main screen, a screen having an image registration button for acquiring an image of the user's 14 face is displayed.

  In step S1, processing for capturing the face image of the user 14 is performed. The user 14 takes a picture of his / her own face (may be slightly decorated) with the camera 16 attached to the client terminal 12 and stores the face image in the memory of the client terminal 12 or digitally. The face image of his / her own face photographed by the camera is stored in the memory of the client terminal 12 in advance. When the user 14 operates the input unit 12a and operates the image registration button, an input field for sequentially specifying an image storage destination folder and an image file is displayed on the screen. After specifying a storage address (for example, a path), a registration execution button (not shown) is operated. In this way, the storage address (path) of the own face image stored in the memory of the client terminal 12 is designated. When the image registration button is operated, a shooting screen for shooting with the camera 16 is displayed. When the input unit 12a is operated on the shooting screen, shooting with the camera 16 is performed and the captured face image is displayed on the client terminal 12. It may be configured to be stored in a predetermined folder managed by an installed processor.

  When the user 14 operates the input unit 12a of the client terminal 12 and operates the registration execution button of the user 14's face image (face image), the server 20 that has received an access request based on the operation is for the sample makeup face selection screen. 4 is transmitted to the client terminal 12, and the web screen 60 shown in FIG. 4 is displayed on the display unit 12 b of the client terminal 12. The web screen 60 includes a sample makeup face selection screen 61 in the content area. The sample makeup face selection screen 61 includes a list display area 62 in which images of a plurality of sample makeup faces are arranged in a matrix, for example, and an OK button. 63.

  In the next step S2, a sample makeup face is selected. That is, the user 14 operates the input unit 12 a to select a favorite sample makeup face from the list display area 62, and then operates the OK button 63. In the example of FIG. 4, the third sample makeup face image 64 from the upper left in the list display area 62 is selected. When the OK button 63 is operated in this selected state, the server 20 that has received an access request based on the operation stores the sample facial face image before makeup corresponding to the sample cosmetic face image 64 in the second storage unit 23. Web data including the image data for the selection screen is created and transmitted to the client terminal 12. As a result, the web screen 65 shown in FIG. 5 is displayed on the display unit 12 b of the client terminal 12. The web screen 65 includes a selection screen 66 in the content area. The selection screen 66 includes a first display area 67 in which the face image 71 of the user 14 is displayed and a face image 72 of the sample makeup face. 2 includes a display area 68, an item selection area 69 for selecting whether or not to perform a makeup correction process, or an item selection area 69 for selecting a makeup correction target when performing the makeup correction process, an execution button 70, and the like. It is. The client terminal 12 receives the program executed when the OK button 63 is operated on the sample makeup face selection screen 61 together with the previous web data, and executes the program when the OK button 63 is operated. As a result, the user's own face image is read from the previously designated path, and the user's 14 face image 71 is displayed in the first display area 67 of the selection screen 66. Of course, based on the operation of the OK button 63, the user's 14 face image is read from the specified path and transmitted to the server 20, and the server 20 creates web data including both the user 14's face image and the sample makeup face's face image. And the structure which transmits it to the client terminal 12 is also employable.

  In step S3, an item is selected. That is, the user 14 operates the input unit 12a to select a desired item from the items in the item selection area 69 on the selection screen 66 shown in FIG. The user 14 prepares items prepared in the item selection area 69 on the selection screen 66, that is, “face outline”, part shapes “eyebrow shape”, “eye shape”, “nose shape”, “mouth shape”, “ At least one desired item can be selected from “Face parts layout” and “Select all”. In this example, the description will proceed assuming that “select all” is selected. After the item is selected, the user 14 operates the input button 12a to operate the execution button 70 to execute the makeup face creation process. Then, the process after step S4 which synthesize | combines the net makeup image 73 of a sample makeup face with the face image of the user 14, etc. according to the outline of the user 14, etc. is advanced. The control unit 31 that receives the selection information of the item selection area 69 input by the operation of the input unit 12a and instructs the makeup correction processing of the content according to the selection information constitutes a selection unit.

  In step S4, a matching process for matching the sample makeup face net makeup image 73 with the face of the user 14 is performed. Since the sample and the user 14 have differences in face size (including those caused by the image size), face contour, part shape, face part layout (part placement), etc., the sample makeup face net makeup image 73 is displayed as the user. The matching process is performed so as to match the 14 face size, face outline, part shape, and face part layout, and a net makeup image 74 (see FIG. 7) that matches the face of the user 14 is acquired.

  FIG. 7A schematically shows the face image 71 of the user 14 and the face image 72 of the sample makeup face. First, feature extraction is performed on both eyes for both face images 71 and 72. Next, the center point (coordinates) of both eyes is obtained, and the sample makeup face net makeup image 73 is expanded and contracted so that the center points of both eyes of the sample makeup face coincide with the center points of both eyes of the user 14, The net makeup image 73 is adjusted to the face size of the user 14. The features of the face contour, mouth and nose are extracted from both face images 71 and 72, and the positions of the eyes and the positions of the mouth and nose (for example, each face image 71 in FIG. , 72), the net makeup image 73 is matched to the face of the user 14 so that they match within a predetermined tolerance. The above matching process is performed.

  In this way, the net makeup image 74 is acquired by performing the matching process so that the net makeup image 73 matches the face size, face contour, parts layout, and the like of the user 14. As shown in FIG. 7B, the net makeup image 74 after the matching processing is a net makeup image that fits the face size, face contour (face contour), eyes, eyebrows, nose, and mouth positions of the user 14. 73, since this net makeup image 74 is synthesized on the user's 14 face image 71, the makeup of the net makeup image 74 is correctly applied on the user's 14 face without any misalignment. The net makeup image 74 is processed with respect to the face image 71 of the user 14 in a form that can be accurately applied to, for example, lipstick on the lips, cheek on the cheeks, eye shadow on the eyelids, and eyebrow on the eyebrows. . Since the net makeup image 74 is obtained by performing matching processing on the layers Lm1 to Lmn constituting the net makeup image 73 shown in FIG. 6, the same number of layers as the layer of the net makeup image 73 is shown in FIG. It is composed of a plurality of layers Lu1 to Lun shown. Each layer Lu1 to Lun corresponds to each layer Lm1 to Lmn shown in FIG. 6 before the matching process. In the above matching processing, only the center point of both eyes or only the center point of both eyes and the face outline may be employed. The position and number of points (dots in the bare face images 71 and 72 in FIG. 7) used for alignment in the matching process can be set as appropriate.

  The net makeup image 74 after the matching process ignores that the face contour, part shape (eye, eyebrow, nose, mouth shape), and face part layout differ between the sample makeup face and the user 14 face. It has become. Therefore, even if the net makeup image 74 is directly synthesized with the user's 14 face image 71 as it is, the user's makeup face obtained by the synthesis has a much different impression from the sample makeup face. In order to prevent this, in the preferred embodiment, the difference between the face of the sample makeup face and the user's face is different in the net makeup image 74 in consideration of the difference in the face outline, part shape, and face part layout. Correction is performed to add makeup (corrected image) to compensate for.

  In step S5, it is determined whether face contour correction is performed. If it is determined that face contour correction is performed, the process proceeds to step S6. If it is not determined that face contour correction is performed, the process proceeds to step S7. When the execution button 70 in FIG. 7 is operated, the selection information in the item selection area 69 on the selection screen 66 is transmitted to the server 20, and the face outline is selected based on the selection information temporarily stored in the memory. Judge whether or not. When the face contour is selected, it is determined that the face contour is corrected, and when the face contour is not selected, it is determined that the face contour is not corrected. Since an example in which “select all” is selected in the item selection has been described, the process proceeds to step S6.

  In step S6, face contour correction processing is first performed. That is, the feature of the face contour is extracted from the sample facial image of the sample makeup face and the user's 14 face image, the difference between the sample and the user 14 is extracted, the difference is reduced, and the user 14 is reduced. Image processing is performed to add a makeup image (face contour correction image) that makes the contour of the face closer to the contour of the sample makeup face to the net makeup image 74 after the matching processing. As shown in FIGS. 5 and 7A, in this example, the contour of the sample makeup face is “oval”, whereas the contour of the face of the user 14 is “square”. ”Is added to the net makeup image 74 of the user 14 to make up the face contour of“ oval ”. The face contour correction process is performed in cooperation with the face contour extraction unit 41 and the first image processing unit 46. Specifically, the facial images of the sample makeup face and the user's 14 facial image are read out from the storage units 23 and 36, respectively, and the facial contours are characterized from the both facial images after the face size is adjusted as necessary. Extract the difference in facial contour between the sample and the user 14. As a result, the face outline 75 of the user 14 and the sample face outline 76 as shown in FIG. 8 are acquired. In FIG. 8, in addition to the contours 75 and 76, the eyes, nose, and mouth are shown, but the eyes, nose, and mouth are not actually shown.

  Next, a process of taking the difference between the obtained face contours 75 and 76 is performed, and a portion where the contour 75 of the user 14 protrudes from the sample contour 76 or a portion recessed inward is taken as a difference in the face contour. get. For example, when the contour of the sample is “oval” and the contour of the user 14 is “square face”, the contour 75 of the user 14 protrudes from the contour 76 of the sample as shown in FIG. The area of the error area is obtained as a positive difference 77. On the other hand, as shown in FIG. 8, a region of the face jaw portion where the contour 76 of the sample makeup face protrudes from the contour 75 of the user 14 is obtained as a negative difference 78. In the case of a positive difference 77, the face contour database in the third storage unit 24 based on the position of the area of the difference 77 (for example, both ends of the error and the forehead) and the positive value that is the sign of the difference. To obtain the corrected image corresponding to the search information “Era, Plus”, the procedure information and the layer position information of the correction image, for example. Then, the corrected image, procedure information, and layer position information are sent to the first image processing unit 46. On the other hand, when the difference 78 is negative, the contour of the face of the third storage unit 24 is based on the position (part) (for example, the jaw) to which the region of the difference 78 belongs and the negative value that is the sign of the difference. For example, the correction image corresponding to the search information “jaw, minus”, the procedure information of the correction image, and the layer position information are acquired by accessing the database. Then, the corrected image, procedure information, and layer position information are sent to the first image processing unit 46.

  The first image processing unit 46 applies the correction image corresponding to each of the regions corresponding to the differences 77 and 78 under the conditions specified by the respective procedure information, and generates the face outline correction image 79 shown in FIG. . The corrected image corresponding to the difference 77 is, for example, a shadow image (color image) that has been subjected to gradation, and the procedure information is that the shadow image is applied to the region of the difference 77 with a predetermined density distribution that becomes darker toward the contour side. This is information indicating the condition of the procedure. On the other hand, the corrected image corresponding to the difference 78 is, for example, a highlight image (color image) that has been subjected to gradation, and the procedure information has a density in the specified direction in the jaw region of the user 14 adjacent to the region of the difference 78. This is information indicating the condition of the procedure of applying the highlight image so that the distribution appears. In accordance with the procedure information, the first image processing unit 46 applies a shadow 80 or a highlight 81 to a designated area corresponding to the area of the differences 77 and 78 so that the density distribution is in the designated direction as a corrected image. A shadow 80 layer and a highlight 81 layer are generated. In this example, the first image processing unit 46 is applied to the error areas on both sides corresponding to the difference 77 and both end areas of the forehead with a density distribution in which the density gradually decreases from the contour side (outside) toward the inside. A layer L1 of the shadow 80 is generated. The first image processing unit 46 also applies the layer L2 of the highlight 81 applied to the jaw region adjacent to the face region side of the user 14 with the density distribution in the designated direction with respect to the region of the difference 78 (both in FIG. 7 ( c)). The face outline correction image 79 is composed of two layers, a layer L 1 of the shadow 80 and a layer L 2 of the highlight 81. Then, each layer L1, L2 of the face contour correction image 79 at the layer insertion position determined from the layer position information corresponding to the correction image among the layers Lu1 to Lun constituting the net makeup image 74 read from the storage unit 37. Process to add each. And the net makeup image which added each layer L1, L2 is memorize | stored in the memory | storage part 37. FIG. In this example, the layer L1 of the shadow 80 and the layer L2 of the highlight 81 constituting the facial contour correction image 79 added to bring the facial contour of the user 14 closer to the contour of the sample are, for example, foundation or concealer correction. It is an image and is added to the layer position of the decorative layer corresponding to the foundation or concealer. In the face contour corrected image 79 in FIG. 8, the expression of the density distribution in the shadow 80 and the highlight 81 is omitted.

  In step S7, it is determined whether or not there is a part shape correction. If it is determined that there is part shape correction, the process proceeds to step S8. If it is not determined that there is part shape correction, the process proceeds to step S9. In other words, based on the selection information temporarily stored in the memory, it is determined whether or not one of the part shapes (eyes, eyebrows, nose, mouth) is selected. If it is determined that there is no part shape correction, it is determined that there is no part shape correction. Since the case where “select all” is selected in the item selection has been described, the process proceeds to step S8.

  In the next step S8, a part shape correction process is performed. In other words, the part shape is extracted from the sample face image and the user's 14 face image, the difference in the part shape between the sample and the user 14 is extracted, and the part shape of the user 14 is reduced by reducing the difference. A part shape correction process is performed in which makeup (corrected image) that makes the sample appear close to the part shape of the sample is applied to the net makeup image 74 after the matching process. This part shape correction processing is performed in cooperation with the part shape extraction unit 42 and the second image processing unit 47. Specifically, the parts (eyes, eyebrows, nose) selected as correction targets from the sample facial face of the sample makeup face and the face of the user 14 that have been previously read from the storage units 23 and 36 and have the same face size, respectively. , At least one selected from the mouth) is extracted, and the shape of each part of the user 14 and the sample is acquired. For a specific part, the angle of the part (eg, eye inclination) is further obtained from the part shape, the density of the part (density distribution) (eg, eyebrow density, lip density) and the high density line around the part. (Single fence, double fence, firewood, etc.)

  Next, using the obtained user 14 and the sample part shape, a process of obtaining a difference in the part shape between the same parts is performed. For example, the part of the user 14 protruding or recessed from the sample part is acquired as a difference in part shape.

  FIG. 9 illustrates a case where makeup (corrected image) that compensates for the difference in shape between the user 14 and the sample eye is added to the net makeup image 74 when the eyes are selected as parts. Hereinafter, the eye shape correction process will be described with reference to FIG. As shown in FIG. 9, in this example, it is assumed that the sample is slightly “hanging” while the user 14 is “hanging”.

  The face image of the sample makeup face and the face image of the user 14 read out from the storage units 23 and 36, respectively, after aligning the face size as necessary, each eye from the face image of the user 14 and the face image of the sample makeup face. Is extracted. The upper part in FIG. 9 shows the eyes 83 and sample eyes 84 of the user 14 whose features have been extracted. By extracting the eye shape, the shape and size of the eye can be grasped. It is also possible to obtain the eye angle (hanging eyes, leaning eyes, etc.) based on the extracted information related to the eye shape, and to obtain high density lines (single eyelids, double eyelids, etc.) around the eyes.

  When obtaining the eye difference, the eye 83 of the user 14 and the sample eye 84 are overlapped so that the center point of the eye (the center point of the pupil) matches. The eye 14 of the user 14 has a larger shift amount inward toward the eye side (right side in FIG. 9) on the lower side of the sample eye 84, and a negative difference 85 is obtained in the portion shifted inward, On the upper side of the eye, the shift toward the inner side is performed toward the outer corner of the eye (left side in FIG. 9), and a negative difference 85 is obtained at the portion shifted to the inner side. When the user's 14 eye 83 is narrower than the sample eye 84, a negative difference 85 is generated around the user's 14 eye on at least one of the upper side and the lower side of the eye. If the region where the difference is negative at the lower side of the eye is narrower at the outer corner of the eye and wider at the outer side of the eye, and the region of lower difference at the upper side of the eye is wider at the outer corner of the eye and narrower at the upper side of the eye, the user 14 Is determined. In this case, the greater the difference in the amount of shift inward between the top of the eye and the outside of the eye, the greater the degree of sag. In this example, the degree of eye inclination is expressed in nine levels. The breakdown of the degree of inclination of the eyes is represented by, for example, the degree of slacking in four stages, the standard inclination (substantially horizontal) is represented in one stage, and the degree of lifting is represented in four stages. A part shape correction image corresponding to the search condition is acquired by accessing the part shape database of the fourth storage unit 25 using the stagnation and its degree as a search condition. In this case, a corrected image of the eye shadow 88 added to the upper area of the outer corner of the eye is obtained as one layer constituting the part shape corrected image 87.

  Whether the eye is a single eyelid or a double eyelid can be extracted. If the user's 14 eye 83 is a single eyelid and the sample eye 84 is a double eyelash, a double portion line is obtained as the difference 86. Since the user 14 is single-legged with respect to the double fold of the sample and the user 14 is short of the single-legged, it is evaluated as “minus-single”, and this is used as the search condition. If the sample is single and the user 14 is double, then “plus single”. In the standard setting, correction to make the double is single is not performed, but correction to make the double is also possible. Using the minus single as a search condition, the part shape database in the fourth storage unit 25 is accessed, and a corrected image corresponding to the search condition is acquired. In this case, a corrected image of the line 89 is obtained as a layer constituting the part shape corrected image 87 to be added to the upper side of the eye to form a double eyelid. As described above, the part shape correction image 87 in FIG. 9 is an image in which the eye shadow layer and the double eyelid layer are combined, and includes, for example, the layers L3 and L4 in FIG.

  FIG. 10 illustrates a case where makeup (corrected image) that makes the difference in the shape of the mouth between the user 14 and the sample appear small is applied to the net makeup image 74 when the mouth is selected as a part. . The mouth shape correction process will be described below with reference to FIG. The face size of the sample makeup face and the user's 14 face image read out from the storage units 23 and 36, respectively, are adjusted as necessary, and the mouth is extracted from each of the user's 14 face image and the sample face image. To do. The upper row in FIG. 10 shows the mouth 91 and the sample mouth 92 of the user 14. By this feature extraction, the shape and size of the mouth can be grasped. As shown in FIG. 10, in this example, the mouth 91 of the user 14 has a slightly thin lip and a wide horizontal width, and the sample mouth 92 has a slightly thick lip and a narrow horizontal width.

  As shown in the lower center of FIG. 10, when the mouths 91 and 92 are overlapped and the difference between them is extracted, the mouth 91 of the user 14 is slightly higher on the upper side than the mouth 92 of the sample makeup face, and considerably lower on the lower side. A negative difference 93 is obtained which is shifted inwardly by the amount. In addition, a positive difference 94 is obtained in which the mouth 91 of the user 14 protrudes from the sample mouth 92 at the left end and the right end of the mouth 91. A plurality of areas are set along the periphery of the mouth. In the example of FIG. 10, for example, three areas near the center of the seven areas set on the upper side of the mouth and the lower side of the mouth are set. Among the seven areas, information on the position of each of the five areas closer to the center, the left area, and the right area, plus or minus is acquired as a search condition. Then, the part shape database in the fourth storage unit 25 is accessed, and a corrected image corresponding to the search condition is acquired. In this case, as shown on the lower right side in FIG. 10, based on the information on the upper and lower region positions of the user's 14 mouth and the negative information, it is added to the upper center portion and the lower center portion of the user's lips. A corrected image (layer) of the lipstick 96 is obtained, and an image 97 for deleting the lipstick at the left end and the right end of the lip of the user 14 based on the information on the left and right area positions of the user 14 and the plus information. A corrected image (layer) is obtained. The corrected image 95 shown in FIG. 10 includes a lipstick 96 layer and an image 97 layer that erases the lipstick.

  Then, these corrected images are sent to the second image processing unit 47. The second image processing unit 47 grasps the layer insertion position corresponding to the makeup layer corresponding to the corrected image among the layers Lu1 to Lun constituting the net makeup image 74, and the part shape is determined at the grasped layer insertion position. Processing for adding each layer L3, L4, etc. of the corrected image 87 is performed. Then, the processed net makeup image is stored in the storage unit 37. Instead of the image 97, image processing for deleting the lipstick corresponding to the left and right ends of the lips from the lipstick layer constituting the net makeup image 74 may be performed.

  In step S9, it is determined whether or not there is face part layout correction. If it is determined that there is face part layout correction, the process proceeds to step S10, and if it is not determined that face part layout correction is performed, the process proceeds to step S11. That is, it is determined whether or not a face part layout item is selected based on the selection information temporarily stored in the memory. If the face part layout is selected, it is determined that the face part layout is corrected, and the face part layout is determined. If is not selected, it is determined that there is no face part layout correction. Since the case where “select all” is selected in the item selection is described, the process proceeds to step S10.

  In the next step S10, face part layout correction processing is performed. That is, the face size of the sample makeup face and the user's 14 facial image are read out from the storage units 23 and 36, respectively, and the face size is aligned as necessary. Then, the face outline and a plurality of parts (both eyes and mouth) are extracted, and the sample and the face part layout of the user 14 are obtained from the position of the face outline and the plurality of parts. Then, a difference between both face part layouts is extracted, and a face part layout correction process is performed for applying makeup to the net makeup image 74 so as to make the difference appear small. For example, as shown in FIG. 11, the face contour obtained by performing feature extraction from the face image 71 of the user 14 and the face image 72 of the sample makeup face, and the positions of both eyes and mouth are acquired. Then, based on the face contour and the positions of both eyes, the ratio of three intervals obtained by dividing the width of the face in the horizontal direction at each position of both eyes is obtained as a horizontal part layout ratio (face part layout horizontal ratio). Also, based on the face contour and the positions of both eyes and mouth, the distance from the hairline to the eye position in the vertical direction of the face, the distance from the eyes to the mouth corner of the mouth, and the mouth corner to the tip of the jaw A vertical part layout ratio (face part layout vertical ratio), which is a ratio to the interval, is obtained.

  For example, as shown in the upper part of FIG. 11, the face part layout horizontal ratio, which is the ratio of each area obtained by dividing the face width into three areas at the positions of both eyes in the horizontal direction of the face, is 20:60:20 ( %) Whereas the sample is 25:50:25 (%). Further, in the vertical direction of the face, the face part layout vertical ratio, which is the ratio of the distance from the hairline to the eyes, the distance from the eyes to the mouth corners, and the distance from the mouth corners to the tip of the chin, is the user 14. Is 30%: 45%: 25%, whereas the sample is 45%: 35%: 20%.

  When the difference between the sample and the user 14 is calculated with respect to the face part layout aspect ratio, the position of both eyes of the user 14 is the direction from the center of the face toward the outside by 5% of the face width with respect to the positions of both eyes of the sample. A difference of “minus 5%” is obtained, which means that the shift is in the minus direction. A makeup (corrected image) is added to the net makeup image 74 so that the ratio of the distance between both eyes of the user 14 approaches the ratio of the distance between both eyes of the sample so as to compensate for the difference. Perform image processing.

  Further, when the difference between the sample and the user 14 is calculated regarding the face part layout vertical ratio, the ratio of the interval from the hairline of the user 14 to the position of the eyes is the length of the face in the vertical direction relative to that of the sample. A difference of “minus 15%” is obtained, which means that the face length is shifted to the narrow side (minus direction) by 15% of the face length. However, there is a limit to the rate at which makeup can be seen as if the parts are shifted in a certain direction from the actual, and the difference of “plus 5%” is to make this limit value (for example, 5%) the maximum value. Given.

  Similarly, it means that the ratio of the interval from the eye position of the user 14 to the mouth position is shifted to the wide side (plus direction) by 10% of the face length with respect to that of the sample “plus 10%”. Is obtained. In this case, since minus 5%, which increases the ratio between the hairline and the distance between the eyes, has already been given, “minus 5%” is given in this case. Similarly, it means that the ratio of the distance from the mouth position of the user 14 to the tip of the jaw is shifted to the wide side (minus direction) by 5% of the face length with respect to that of the sample “plus 5%”. Is obtained. In this case, the difference between the mouth and the tip of the jaw is calculated as a difference of “minus 5%” which narrows the ratio of the distance by 5% of the face length. Image processing for adding makeup (corrected image) that approximates the facial part layout of the user 14 to the sample facial part layout to the net makeup image 74 is performed so as to compensate for the difference in facial part layout in the vertical direction of the face.

  In addition to the above face part layout, for example, the distance from the hairline to the lower end of the eyebrows, the distance between the eyebrows and the eyes, the distance between the eyes and the lower end of the nose, and the distance from the lower end of the nose to the tip of the chin The facial part layout represented by the ratio of the user 14 is also calculated, the difference in the facial part layout between the sample and the user 14 is calculated, and the makeup (corrected image) in which the ratio of the user 14 approaches the ratio of the sample is calculated as the net makeup. Image processing to be added to the image 74 can also be performed. For example, in the face of the user 14, the ratio between the hairline and the lower end of the eyebrows is smaller than that of the sample, the ratio of the distance between the lower end of the eyebrows and the eyes is substantially equal to that of the sample, and The proportion of the interval between is approximately equal to that of the sample and the proportion of the nose-to-mouth spacing is greater than that of the sample. Therefore, image processing is performed in which makeup (corrected image) that increases the proportion of the eyebrow and eye spacing of the user 14 and makeup (corrected image) that decreases the proportion of the nose and mouth spacing are applied to the net makeup image 74.

  In order to compensate for the difference in the face part layout, a corrected image acquired from the face part layout database is shown in the lower part of FIG. 11, for example. In this example, in order to show a wide interval from the hairline to the eyes of the user 14, the face part layout database is accessed based on the search condition of the interval from the hairline to the eyes and the plus 5%. A correction image of the highlight 102 to be added to the forehead portion is acquired. In order to make the interval from the eyes to the mouth narrow, the correction image of the shadow 104 added along the nose is accessed by accessing the face parts layout database based on the search condition of the interval from the eyes to the mouth and minus 5%. To be acquired. In order to make the gap between the mouth and the tip of the jaw narrow, a correction image of the shadow 104 added to the jaw portion is acquired. In order to reduce the ratio of the distance between the lower end of the nose and the mouth, a corrected image of the shadow 104 added to the upper part of the mouth can be acquired. It is also possible to acquire a correction image of the highlight 103 added to the outer corner of the eye in order to make the distance between both eyes of the user 14 narrower in the lateral direction of the face. By this face part layout correction process, for example, the layer L5 of the highlight 102, the layer L6 of the shadow 104, and the layer L7 of the highlight 103 are acquired as shown on the lower right side in FIG.

  The acquired corrected image is sent to the third image processing unit 48. The third image processing unit 48 reads the net makeup image 74 from the storage unit 37 and adds a correction image related to the face part layout to the read net makeup image 74 to the layer position specified by the face part layout identifier. Then, the net makeup image 74 after the addition is stored in the storage unit 37 again. A corrected net makeup image 100 shown in FIG. 7C obtained by correcting the net makeup image 74 after the matching processing so that the facial contour, part shape, and face part layout of the user 14 are close to those of the sample is obtained. It is stored in the storage unit 37. When each correction image related to the face outline, part shape, and face part layout overlaps in the same makeup layer, or even when the face of the user 14 is brought close to the sample makeup face in the same target even if the makeup layer is different, it is due to the overlap To prevent excessive correction.

  As a method of making the facial part layout difference closer to the sample facial part layout of the user 14, the facial part layout difference is converted into the shift direction and shift amount of the eyebrow, eye, nose, and mouth positions. Based on the conversion information, a method of acquiring a correction image that makes the part appear to be shifted by the shift direction and the shift amount from the actual part and applying the correction image to the net makeup image 74 can be adopted.

  In step S11, the corrected net makeup image 100 and the face image 71 are combined and displayed. That is, the corrected net makeup image 100 shown in FIG. 7C is read from the storage unit 37, and the user's 14 face image 71 is read from the storage unit 36, and the corrected net makeup image 100 is displayed on the user's 14 face image 71. The composition process to overlap is performed. Since it is sufficient that the two points coincide with each other during the composition processing, for example, the corrected net makeup image 100 may be pasted onto the bare face image 71 by aligning the center positions of both eyes.

  The control unit 31 creates web data including the makeup face image synthesized by the synthesis processing unit 34 and transmits the web data from the transmission / reception unit 29 to the client terminal 12 through the network 11. As a result, the web screen 105 shown in FIG. 12 is displayed on the display unit 12 b of the client terminal 12. The web screen 105 includes a makeup face display screen 106 in its content area, and the makeup face image of the user 14 is displayed in the makeup face display area 107 of the makeup face display screen 106. The makeup face display screen 106 is provided with a plurality of selection fields (pull-down menus) for selecting items to be additionally corrected. Specifically, three types of face contour, part shape, and face part layout are prepared as items that can be additionally corrected, and the eyes, eyebrows, nose, and mouth can be selected as the part shape. In the face outline pull-down menu, you can select the desired face outline. For example, you can select the face outline such as oval, square, triangle, etc. It is also possible to select the shape. For example, when an egg shape is selected from the outline of the face, for example, the next screen (detailed setting screen) is displayed, and a desired one can be selected from a plurality of types of egg shapes, or the ratio (%) of approaching the egg shape. Specification is also possible. In the eyes of the part shape, you can select large, thin, cut length, leaning, hanging, almond, etc. from the pull-down menu, as well as selection of double eyelids, single eyelids, long eyelashes, etc. When any one is selected, a detailed setting screen is displayed in the same manner, and detailed setting is possible. In addition, various shapes and partial shapes and textures of the eyebrows, nose, and mouth (for example, if the mouth is moist, glossy, wet, elastic, etc.) can be selected. For the facial parts layout, in addition to the golden ratio, 3: 4: 2, 4: 3: 3, etc., the forehead is widened or narrowed, the distance between eyes and mouth is widened or narrowed, and the distance between mouth and chin is It is possible to select to change the size of a partial area such as wide or narrow, and similarly, detailed setting is possible for each. When the correction execution button 111 is operated, an additional correction process selected and specified by the user 14 is executed. When the color change button 112 is operated, a detailed color change setting screen is displayed, and the foundation color, concealer color, lipstick color, cheek color, eye shadow color, and the like can be changed.

  In this way, the shape, ratio, and color are additionally corrected as necessary. If you like the model makeup face, you can save the makeup face data in the memory of the client terminal 12 by operating the save button 113. For example, it is also possible to save favorite model makeup faces, display them on the screen in order or as a list, and select a model makeup face image that is most preferred from them.

  When the model makeup face image is generated in this way, the user 14 operates the input unit 12a to operate the makeup procedure recipe display button 109. When the server 20 accepts a request based on the operation of the makeup procedure display button 109, the CPU of the server 20 executes the makeup advice process from step S12 by executing the makeup advice process program in the makeup support process program.

  By the way, in the makeup face image generation process, identifiers acquired by the face contour extraction unit 41, the part shape extraction unit 42, and the face part layout extraction unit 43 (face contour image identifier, part shape image identifier, face part layout image identifier). Is sent to the procedure correction unit 35. In the procedure correction unit 35, the face contour image identifier is sent to the face contour procedure correction unit 51, the part shape image identifier is sent to the part shape procedure correction unit 52, and the face part layout image identifier is sent to the face part layout procedure correction unit 53. Sent to each.

  In step S12, procedure correction processing is performed. That is, the procedure correction unit 35 refers to the link table stored in the seventh storage unit 28 and acquires an address corresponding to the identifier of the corrected image. Then, the procedure correction unit 35 accesses the procedure correction database stored in the seventh storage unit 28, and acquires procedure correction data corresponding to the identifier of the corrected image. That is, the face contour procedure correction unit 51 accesses the procedure correction database and acquires face contour procedure correction data corresponding to the face contour image identifier. The part shape procedure correction unit 52 accesses the procedure correction database, and acquires part shape procedure correction data corresponding to the part shape image identifier. The face part layout procedure correction unit 53 accesses the correction database, and acquires face part layout procedure correction data corresponding to the face part layout image identifier.

  In performing the procedure correction process, the procedure correction unit 35 first reads the sample makeup procedure recipes D1 to Dn corresponding to the sample makeup face image 64 from the sixth storage unit 27 and temporarily stores them in the storage unit 38. In the procedure correction process, corrections are made to the texts of the sample makeup procedure recipes D1 to Dn stored in the storage unit 38. In the configuration in which the voice from the speaker is read out, the voice is also corrected by correcting the text. Of course, it is possible to employ a configuration in which sound is prepared and the sound is corrected.

  The face contour procedure correction data, the part shape procedure correction data, and the face part layout procedure correction data are sentence data for correcting sentences in the sample makeup procedure recipe. Face contour procedure correction data and part shape procedure correction data include face contour (egg-shaped face, square face, inverted triangle face, round face, triangle face, etc.), part shape (specifically eyebrow shape, eye shape, nose shape) , Mouth shape) is linked to a specific document area 55 or an address 56 specifying a specific position in the text of the sample makeup procedure recipe. Each procedure correction data is replaced with the document area 55 specified by the corresponding address 56 or additionally inserted at a specific position specified by the corresponding address 56.

  That is, the facial contour procedure correction data is replaced with the text in the document area 55 specified by the address 56 linked to the facial contour image identifier in the text of the sample makeup procedure recipe shown in FIG. Further, it is inserted at a position designated by the address 56 linked to the face outline image identifier. The procedure correction data such as the eye shape, the nose shape, and the mouth shape are stored in the document area 55 designated by the address 56 linked to the image identifiers of the eye shape, the nose shape, and the mouth shape in the text shown in FIG. Replaced with text. When the sample makeup procedure recipes D1 to D include an image other than the document, the image corresponding to the identifier is specified as in the case of the document, and the image is replaced with a correction image, or the image An image layer for correction is added.

  In FIG. 13, model makeup face images I <b> 1 to In that provide a completed image at a step in the middle of makeup are created using the layers constituting the corrected net makeup image 100. The layers constituting the corrected net makeup image 100 are divided into makeup layers (for example, different layers if the makeup layer is different even for the same concealer) and makeup types (for example, foundation and cheek). The layers constituting the corrected net makeup image 100 are numbered, for example, in the order in which makeup is performed, and the makeup step to which the layer belongs is managed from the layer number. When creating the model makeup face images I1 to In, the layer belonging to the procedure 1 is first read out from the storage unit 37 from the layers constituting the corrected net makeup image 100, and the read layer is the face of the user 14 The model makeup face image I1 of the procedure 1 is generated by performing a composition process to be superimposed on the image 71. Next, the layer belonging to the procedure 2 is read from the storage unit 37, and the sample makeup face image I2 of the procedure 2 is generated by performing a composition process of superimposing the read layer on the model makeup face image I1. Thereafter, the same process is repeated from the procedure 3 to the procedure n to generate model makeup face images I3 to In. The control unit 31 that generates a model makeup face image using the layer of the corrected net makeup image 100 constitutes a makeup intermediate face image generation means.

  Thus, in all makeup steps, the model makeup face images I1 to In and the makeup procedure recipes X1 to Xn of the user 14 composed of sentences are created. Then, the control unit 31 uses the model makeup face images I1 to In and the makeup procedure recipes X1 to Xn to arrange the model makeup face image and the makeup method explanation document for each procedure as shown in FIG. Web data for displaying the makeup procedure recipe display screen 116 is created. Then, the control unit 31 transmits the web data from the transmission / reception unit 29 to the client terminal 12 through the network 11.

  A web screen 115 shown in FIG. 13 is displayed on the display unit 12 b of the client terminal 12. The makeup procedure recipe display screen 116 in the web screen 115 displays a model makeup face image and a sentence based on the makeup procedure recipes X1 to Xn explaining the makeup method for each step from step 1 to step n. . The makeup procedure recipe display screen 116 is provided with a redo button 117 and a registration button 118. When the redo button 117 is operated, a screen for selecting the makeup face of the user 14 is displayed, and the makeup procedure recipe for the makeup face reselected on the screen can be displayed again. If the user operates the registration button 118 when he / she likes to make makeup according to the makeup procedure recipe, the makeup procedure recipe can be stored in the client terminal 12 or the server 20. The stored (registered) makeup procedure recipe can be displayed in a list on the screen of the client terminal 12 by accessing the server 20 from the client terminal 12, and the user can select a favorite makeup procedure recipe from the registered ones. Thus, the makeup procedure recipe can be displayed on the display unit 12b immediately without the makeup face generation process.

  In the makeup procedure recipe display screen 116 shown in FIG. 13, the uppermost procedure number is linked. When a desired procedure number is selected, a makeup execution screen corresponding to the selected procedure is displayed. For example, when “procedure 1” is selected and operated, a web screen 120 including a makeup execution screen 121 of procedure 1 is displayed as shown in FIG. The makeup execution screen 121 includes a relay screen 122 on which a moving image of the user 14 performing makeup with the camera 16 is displayed, and a model screen 123 on which a model makeup face image I1 in the procedure is displayed. And a sentence based on the makeup procedure recipe X1 explaining the makeup method is displayed below them. The relay screen 122 serves as a mirror, and the user 14 performs makeup according to the explanation document of the presented makeup method using the model makeup face in the model screen 123 as a model while viewing the relay screen 122. When the NEXT button 124 is operated on the makeup execution screen 121, it is possible to proceed to the next procedure. When the makeup advice button 125 is operated, makeup advice based on a comparison between the actual makeup face image and the model makeup face image in the procedure can be received.

  The process of creating makeup advice for each makeup step is the same as the processing method described in Citation 2. That is, the user 14 takes a picture of his / her face during makeup with the camera 16 each time makeup for each makeup step is completed. The photographed makeup face image K1 (see FIG. 16) is written in a predetermined storage area of the client terminal 12. After that, when the makeup advice button 125 shown in FIG. 15 is operated by operating the input unit 12a, the actual makeup face image is transmitted to the server 20 and stored in the memory in the server 20.

  The control unit 31 of the server 20 includes an actual makeup face image K1 (makeup face image or makeup completed face image) photographed for each procedure, and a model makeup face image I1 (model makeup face image or model). ) And the makeup state difference between the makeup face image K1 and the model makeup face image I1 is calculated as shown in FIG. Then, based on this difference, the control unit 31 creates makeup advice for bringing the makeup face image K1 of the user 14 closer to the model makeup face image I1. The server 20 transmits makeup advice to the client terminal 12 and displays a makeup advice display screen Jj (j = 1, 2,..., N) shown in FIG. 17 for each procedure. In the preferred embodiment, the makeup advice includes at least a sentence. Of course, the makeup advice may be at least one of a document, sound, and image. The camera 16 and the transmission / reception unit 29 on the client terminal 12 side and transmission / reception unit 29 constitute a makeup intermediate face image acquisition means. Moreover, the control part 31 which calculates the difference in makeup | decoration state between a makeup face image and a model makeup face image comprises a makeup face difference acquisition means. The control unit 31 that creates makeup advice based on the difference in makeup face constitutes advice means. The display unit 12b of the client terminal 12 that displays makeup advice constitutes output means.

  In addition to the actual makeup face images K1 to Kn and the model makeup face images I1 to In displayed on the makeup advice display screens J1 to Jn shown in FIG. 17, the model makeup face images I1 to In and the makeup face images Cosmetic advices R1 to Rn (documents) created by the control unit 31 in the server 20 based on the difference from K1 to Kn are displayed. The makeup advices R1 to Rn correspond to the model makeup face image Ij (j = 1, 2,...) With respect to the actual makeup face image Kj (j = 1, 2,. Advice for bringing the makeup face of the user 14 close to the model makeup face image Ij, such as pointing out a different part from n) and a method for correcting the part, is displayed. The user 14 repeats this operation for each procedure, and the makeup advice display screens J1 to Jn are displayed for each procedure 1 to n.

  By viewing the makeup advice display screens J1 to Jn, the user 14 makes an appropriate judgment as to whether or not makeup has been applied according to the makeup face image of the user according to the makeup method provided. It becomes possible to receive. Further, since the makeup advice is received for each procedure, for example, compared with a case where the makeup state is compared with the model makeup face image after all makeup is completed, it is not necessary to make a significant correction at the time of makeup correction.

As described above in detail, according to the preferred embodiment, the following effects can be obtained.
(1) Since makeup (corrected image) that compensates for the difference in facial contour between the user 14 and the sample is added to the net makeup image 74 after the matching process, a makeup face image of the user 14 that is close to the sample makeup face is provided. it can.

  (2) Since makeup (corrected image) that compensates for differences in the shape of parts such as eyes, eyebrows, nose, and mouth between the user 14 and the sample is added to the net makeup image 74 after the matching process, the sample makeup face A makeup face image of the user 14 closer to the image can be provided.

  (3) Since makeup (corrected image) that compensates for the difference in the facial part layout between the user 14 and the sample is added to the net makeup image 74 after the matching process, the user who has applied makeup that is closer to the sample makeup face 14 makeup faces can be provided.

  (4) Since the text in the makeup procedure recipe for the sample makeup face is corrected and the makeup procedure recipe text of the user 14 is created, it is not necessary to prepare all possible explanatory texts as a database, and for correction Since it is only necessary to have the sentence portion as a database, the storage capacity required for the database of makeup procedure recipes can be reduced.

  (5) The layers constituting the corrected net makeup image 100 are classified with numbers and the like for each makeup step. Therefore, if a layer for each makeup step is added to the face image of the user 14 or an image process is performed to add to the makeup face image in the previous makeup step, the makeup face image can be easily created for each makeup step. .

  (6) When the user 14 completes a makeup intermediate face in each makeup step, by reading a photo image of the makeup intermediate face, the makeup intermediate face of the user 14 who actually applied makeup and the makeup intermediate to be targeted A makeup method that extracts the difference from the face and compensates for the extracted difference is provided as makeup advice. Therefore, the user 14 can complete the makeup face as the target in each makeup step and proceed to the next makeup procedure, and can finish the user 14's face as a makeup face according to the makeup face image. Therefore, it is possible to reduce the frequency with which the finished makeup face is far from the desired makeup face.

  (7) A configuration in which at least one of items such as a face outline, a part shape, and a face part layout can be selected on the makeup support screen, whether makeup to compensate for the difference between the user 14 and the sample is added did. For this reason, comparing the makeup face image of the user 14 created by various trials such as adding makeup that compensates for this kind of difference, not adding, or adding makeup that compensates for only a certain item, The user 14 can select a desired makeup face as the target makeup face. For this reason, the user 14 can select a desired makeup face image as a model from variously tested makeup face images. Further, when it is desired to leave a favorite feature of one's own face, if the feature portion of the desired face is removed from the item, a makeup face image of the user 14 maintaining the feature can be generated. Therefore, it is possible to acquire a makeup face image that is close to the sample makeup face for a portion other than the feature of the face while leaving the feature of the face that the user likes.

  (8) When the part shape is selected, since at least one desired part shape can be selected from the eyebrow, eye, nose, and mouth part shapes, the user 14 can select the eyebrow, eyes, nose, mouth. Various makeup face images can be tried while selecting at least one desired shape from the respective part shapes, and a desired makeup face image can be selected as a model. Also, if you want to keep the features of your favorite part of your face, remove the face part you want to leave from the item, and you can generate the makeup face image of the user 14 in the form that the feature of that part remains. it can. Therefore, it is possible to acquire a makeup face image that is close to the sample makeup face for a portion other than the part of the face while keeping the features of the face parts that the user likes.

  (9) When a three-dimensional image is used as the user 14 and the sample image instead of the two-dimensional image, three-dimensional information such as the height of the nose, the depression of the eyes, the cheek, the forehead, and the protruding amount of the mouth is acquired. And makeup that compensates for the difference in the three-dimensional information in the sample can be added to the net makeup image. Therefore, the makeup face of the user 14 that is close to the sample makeup face in three dimensions can be provided.

The preferred embodiment may be modified as follows.
In the preferred embodiment, the correction image is added as a layer. However, the processing method for applying the correction net makeup image to the net makeup image is not limited to the method of adding a layer. For example, a configuration in which a correction image is directly added to a layer constituting the net makeup image 74 or image processing such as partial deletion of the image of the layer can be employed. In short, a configuration in which a makeup image that compensates for at least one of the contour, part shape, and face part layout of the user 14 and the sample makeup face is sufficient for the net makeup image. Further, it is not essential that the net makeup images 73 and 74 have a layer structure.

  In the preferred embodiment, the corrected net makeup image 100 is added to the net makeup image to compensate for the differences in facial contour, part shape, and facial part layout. However, the present invention is limited to adopting all three above. Not. For example, only the face outline, only the part shape, only the face part layout, or the face outline and part shape, the face outline and face part layout, the part shape and face part layout, etc. May be. Moreover, although eyes, eyebrows, noses, and mouths are employed as the part shapes, only one, only two, or only three of these may be employed. Furthermore, the structure which adds the makeup | decoration which compensates for the difference of another element is also employable. For example, a configuration in which makeup that compensates for the difference in background color between the user 14 and the sample may be added to the net makeup image. In other words, if the user's 14 skin color is darker than the sample's facial skin color, the foundation color is made lighter than that of the sample, and conversely if the user 14's facial skin color is lighter than that of the sample, Make the color darker than the sample. For example, when the user's 14 skin color is darker than the sample skin color, the foundation color is darker than that of the sample, and vice versa. If the skin color is lighter than that of the sample, make the foundation color lighter than that of the sample. According to this configuration, by using the foundation having the same color as that of the sample, it is possible to prevent the user 14 from having an unnatural facial color because the color of the face background of the user 14 and the foundation are too different.

  The sample makeup face image is not limited to the makeup image of an actual person, and may be a makeup face image obtained by image processing. For example, a makeup face obtained by morphing and synthesizing another person's makeup face image such as a makeup model and the user's 14 face image may be selected as the user's favorite makeup face. When performing morphing synthesis, at least one of the eyes, eyebrows, nose, mouth, and face outline is selected, and the face image of the user 14 and the makeup face image of the makeup model are selected only for the selected parts and face portions. You may synthesize. The face image of the makeup model can be obtained by combining the face image of the user 14 and the face image of the makeup model by performing a morphing process under the same conditions as when the user's favorite makeup face image is combined. The net makeup image is acquired by performing a process of subtracting the face image generated by morphing from the user's favorite makeup image generated by morphing.

  In the preferred embodiment, the server 20 performs makeup support processing and transmits the processing result to the client terminal 12. However, the client terminal downloads the makeup support processing program (makeup face image generation processing program downloaded from the server). In other words, the client terminal 12 can be configured as a makeup face image generation device. The client terminal downloads a part of the makeup support program from the server, and the CPU of the client terminal executes the program to execute the makeup support process, and for the process that requires access to the database, the necessary data is stored in the server. A configuration is also adopted in which the server accesses the database based on the data, acquires necessary data (image or document), performs predetermined processing as necessary, and then transmits the data to the client terminal 12. it can.

  The makeup advice system is not limited to being configured by a server and a client terminal connected to each other through a network. For example, by installing a makeup support processing program from a storage medium such as a CD-ROM in a personal computer, workstation, PDA (Personal Digital Assistants), or mobile phone, and causing the built-in CPU to execute the makeup support processing program, A makeup support apparatus can also be configured. This makeup support apparatus performs correction processing on the net makeup image to compensate for the difference between the user's 14 face image and the makeup model's face image using the input user's 14 face image and the makeup model's face image. Then, it functions as a makeup face image generation device that displays the makeup face image of the user 14 generated by combining the corrected net makeup image with the face image of the user 14 on the monitor. In this case, a camera (image photographing means) is connected or assembled to a personal computer or the like, and an image photographing means for photographing the user's 14 face or a makeup intermediate face at the time of makeup advice, and a user's favorite makeup face or makeup advice content, etc. It can also be configured as a makeup support apparatus having display means such as a monitor for displaying.

  In a preferred embodiment, makeup advice is given, but a configuration without makeup advice may be used. For example, a makeup face image generation apparatus that only displays the makeup face image (model makeup face) of the user 14 may be used. The case where makeup advice is performed is not limited to the configuration in which the makeup procedure recipe is created by correcting the sample makeup procedure recipe as in the embodiment. For example, the makeup face image of each procedure may be analyzed, and the computer may create a makeup procedure recipe based on the analysis result.

  In the preferred embodiment, it is realized by software by causing a computer to execute a program. However, it may be realized by hardware such as a control circuit (custom IC or the like), or may be realized by cooperation of hardware and software. Good.

The schematic diagram which shows the makeup | decoration advice system according to preferable embodiment of this invention. The block diagram of a makeup | decoration assistance apparatus. The flowchart of a makeup | decoration assistance process. An example of a sample makeup face selection screen. Example of selection screen. Explanatory drawing explaining a sample makeup face and its net makeup image. Explanatory drawing explaining the matching process of a net makeup image. An example of face contour correction. An example of eye shape correction. An example of mouth shape correction. Explanatory drawing explaining a face parts layout correction process. An example of a makeup face display screen. An example of a makeup procedure recipe display screen. (A), (b) Explanatory drawing of a makeup | decoration procedure recipe. An example of a makeup execution screen. Explanatory drawing explaining makeup advice processing. The example of the makeup | decoration advice display screen in each procedure.

Claims (20)

An apparatus for generating a user-specific model makeup face image from a user face image,
The sample makeup face image selected by the user, the sample makeup face image, and the net makeup image corresponding to the difference between the sample makeup face image and the sample makeup face image are stored. Storage means;
Face image acquisition means for acquiring an image of the user's face;
Matching processing means for generating a matched net makeup image by performing processing for matching the net makeup image with the user's face image,
Performing feature extraction on each of the sample makeup face image and the user's face image, obtaining the feature of each face, and obtaining a difference between the features of both faces;
You get a real face correction cosmetic image representing an image of the makeup to reduce the difference in characteristics of the real face of the sample decorative face of the to resemble real face said a cosmetic to be applied to the real face of the user real face of the user and the real face correction cosmetic image acquisition hand stage,
Said real face correcting cosmetic image, wherein the Matched net makeup image, before SL performs synthetic processing to add the user's real face image and an image synthesizing means for generating an image of the user-specific model makeup face A makeup face image generation apparatus comprising the makeup face image generation apparatus . The image synthesizing means, prior SL includes an image correction means for generating a positive Ajika粧image compensation after by adding Matched net makeup the real face correcting cosmetic image on an image, the corrected net makeup image The makeup face image generating apparatus according to claim 1 , wherein the image of the user-specific model makeup face is generated by performing a composition process for adding the image to the user's face image . 3. The makeup face image according to claim 1, wherein the net makeup image includes a plurality of layers corresponding to a plurality of makeup layers from the face of the sample makeup face to the sample makeup face. 4. Generator . The real face difference obtaining means, from the real face of the image of the image with the sample decorative face of the real face of the user, acquires the contour of both the face, the difference between different contours of both real face,
The real face correction makeup image acquisition unit, any one of claims 1 to 3, characterized in that to obtain the real face contour correction cosmetic picture image for showing less difference in the contour as the real face correction cosmetic image The makeup face image generation apparatus according to the item . The face difference acquisition means performs feature extraction on each of the user's face image and the sample makeup face's face image to acquire the shape of both face parts, and the corresponding parts between the face faces. To get the difference in the shape of
The real face correction makeup image acquisition unit, any one of claims 1 to 4, characterized in that to obtain the part shape correction image image for showing less difference in the shape of the part as the real face correction cosmetic image The makeup face image generation apparatus according to the item . The makeup part image generation apparatus according to claim 5, wherein the part is at least one of eyes, eyebrows, nose, and mouth. The face difference acquisition unit acquires a face part layout from each of the user's face image and the sample makeup face's face image, and acquires a difference in face part layout between both face faces,
The real face correction makeup image acquisition unit of claim 1 to 6, characterized in that to obtain the real face parts layout correction cosmetic picture image for showing less difference in the face parts layout as the real face correction cosmetic image The makeup | decoration face image generation apparatus as described in any one of Claims . Any one of claims 1 to 7, characterized in that said image synthesizing means further comprising selection means for selecting in accordance with the operation of the user whether to perform a process of pressurizing with the real face correction cosmetic image The makeup face image generation device described in 1 . The storage means may store a difference eliminating image associated with the type and extent of the difference in characteristics of the real face and the location for adding the difference solution extinguishing images in real face image of the user,
The face correction makeup image acquisition means reads from the storage means the difference elimination image and position according to the type and degree of difference in the features of the face.
The image synthesizing means, cosmetic face image according to any one of claims 1 to 8, characterized in that the addition of the read differences resolved image to the read position in the image of the real face of the user Generator . The storage means stores procedure information for adding the difference eliminating image to the position,
The face correction makeup image acquisition means reads the difference elimination image, the position and the procedure information from the storage means,
The makeup image generating apparatus according to claim 9, wherein the image synthesis unit adds the read difference-resolving image to the read position in the user's face image according to the procedure information. . Recipe storage means for storing a first makeup procedure recipe for explaining a procedure for applying makeup from a bare face corresponding to the sample makeup face to a makeup face corresponding to the sample makeup face by at least one of a document, sound and image When,
A makeup procedure creating means for creating a second makeup procedure recipe for explaining a procedure for applying makeup from the user's face to the user-specific model makeup face using at least one of a document, sound and image;
The makeup procedure creation means reads the first makeup procedure recipe corresponding to the sample makeup face from the recipe storage means, and corrects a part corresponding to the makeup image for face correction in the first makeup procedure recipe. Te, cosmetic facial image generating apparatus according to any one of claims 1 to 10, characterized in that to create the second cosmetic procedures recipe. The real face correction makeup image and the previous Matched positive Ajika粧image is composed of a plurality of makeup step and a plurality of layers respectively associated with said apparatus further
A makeup intermediate face image generating means is provided for sequentially synthesizing the layers with the face image of the user as the makeup step proceeds to create an image of a model makeup face in the middle of each makeup step. The makeup face image generation device according to any one of claims 1 to 11 . A makeup intermediate face image acquisition means for acquiring an image of the user's actual makeup intermediate face in each makeup step and an image of a model makeup face in the middle of each makeup step;
A midway makeup face difference obtaining means for obtaining a difference between the actual makeup midway face image and the image of the model makeup face on the way;
An advice means for generating makeup advice that explains the makeup procedure recipe for reducing the difference acquired by the midway makeup face difference acquisition means by at least one of a document, a sound, and an image is further provided. The makeup face image generation device according to claim 12 . The sample makeup face image is an image of another person's makeup face selected by the user from a plurality of other person's makeup face images and having a user's favorite makeup, and the storage means is the plurality of others The makeup face image generation apparatus according to any one of claims 1 to 13, wherein the makeup face image and the plurality of others' face faces are stored in association with each other. The apparatus is a server connected to a client terminal via a network, and the server
When receiving the start request from the client terminal starts generate processing of the user-specific model makeup face image,
Receiving an image of the user's face from the client terminal;
Makeup face image generating apparatus according to any one of claims 1 to 14 and transmits the image of the user-specific model makeup face to the client terminal. The apparatus is a client terminal that is connected to a server via a network and incorporates a processor that executes a makeup face image generation processing program downloaded from the server,
The storage means is provided in the client terminal and stores the sample makeup face image, the sample makeup face image and the net makeup image received from the server,
The face correction makeup image acquisition means is provided in the client terminal, and accesses the server to acquire the face correction makeup image according to the difference between the features of the faces. The makeup face image generation device according to any one of claims 1 to 15 . A method for generating a user-specific model makeup face image from a user face image,
The sample makeup face image selected by the user, the sample makeup face image, and the user-desired net makeup image corresponding to the difference between the sample makeup face image and the sample makeup face image Storing
Obtaining an image of the user's face;
Performing a process of matching the net makeup image with the face image of the user to generate a matched net makeup image;
Performing feature extraction on each of the sample makeup face image and the user's face image, obtaining the features of each face, and obtaining the difference between the features of both faces;
You get a real face correction cosmetic image representing an image of the makeup to reduce the difference in characteristics of the real face of the sample decorative face of the to resemble real face said a cosmetic to be applied to the real face of the user real face of the user Process,
Said real face correcting cosmetic image, and the matching previously net makeup image, before SL performs synthetic processing to add the user's real face image, and a step of generating an image of a user dedicated model makeup face A method for generating a makeup face image . The makeup face image generation method according to claim 17, wherein the feature of the both faces is at least one of a face outline, a face part shape, and a face part layout. A server that generates an image of a user-specific model makeup face from an image of the user's face,
The sample makeup face image selected by the user, the sample makeup face image, and the user-desired net makeup image corresponding to the difference between the sample makeup face image and the sample makeup face image Storage means for storing
A face image acquiring means for acquiring an image of the real face of the user from a client terminal,
Matching processing means for generating a matched net makeup image by performing processing for matching the net makeup image with the user's face image,
Performing feature extraction on each of the sample makeup face image and the user's face image, obtaining the feature of each face, and obtaining a difference between the features of both faces;
You get a real face correction cosmetic image representing an image of the makeup to reduce the difference in characteristics of the real face of the sample decorative face of the to resemble real face said a cosmetic to be applied to the real face of the user real face of the user and the real face correction cosmetic image acquisition hand stage,
Said real face correcting cosmetic image, wherein the Matched net makeup image, before SL performs synthetic processing to add the user's real face image, an image combining means for generating an image of the user-specific model makeup face,
Means for transmitting an image of the user-specific model makeup face to the client terminal. A program for causing a computer to execute the method of claim 17 .

Images