WO2018088243A1 - 情報処理装置、および情報処理方法、並びにプログラム - Google Patents

情報処理装置、および情報処理方法、並びにプログラム Download PDF

Info

Publication number
WO2018088243A1
WO2018088243A1 PCT/JP2017/038862 JP2017038862W WO2018088243A1 WO 2018088243 A1 WO2018088243 A1 WO 2018088243A1 JP 2017038862 W JP2017038862 W JP 2017038862W WO 2018088243 A1 WO2018088243 A1 WO 2018088243A1
Authority
WO
WIPO (PCT)
Prior art keywords
face image
user
makeup
image
face
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2017/038862
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
治彦 岸
陽子 深田
亜香里 星
隆浩 坂井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to US16/329,552 priority Critical patent/US11321764B2/en
Publication of WO2018088243A1 publication Critical patent/WO2018088243A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0633—Managing shopping lists, e.g. compiling or processing purchase lists
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/0076—Body hygiene; Dressing; Knot tying
    • A—HUMAN NECESSITIES
    • A45—HAND OR TRAVELLING ARTICLES
    • A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
    • A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
    • A45D44/005—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms for selecting or displaying personal cosmetic colours or hairstyle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0631—Recommending goods or services
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/60—Static or dynamic means for assisting the user to position a body part for biometric acquisition
    • G06V40/67—Static or dynamic means for assisting the user to position a body part for biometric acquisition by interactive indications to the user
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/0023—Colour matching, recognition, analysis, mixture or the like
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • A—HUMAN NECESSITIES
    • A45—HAND OR TRAVELLING ARTICLES
    • A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
    • A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
    • A45D2044/007—Devices for determining the condition of hair or skin or for selecting the appropriate cosmetic or hair treatment
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to an information processing apparatus, an information processing method, and a program, and in particular, an information processing apparatus and an information processing method that can support makeup that makes the user approach an ideal arrangement of each part of a face. , As well as programs.
  • an image of makeup (makeup) is generated based on the face image of the face, and makeup is applied in accordance with the movement of the captured face image.
  • a technique for assisting makeup by tracking and displaying an image and allowing a user to make makeup while viewing the face image (see Patent Document 1).
  • the present disclosure has been made in view of such a situation, and in particular, makes it possible to propose makeup (makeup) in which each part constituting the user's face approaches an ideal arrangement. .
  • An information processing apparatus includes an imaging unit that captures a user's face image, and the user's face image.
  • An information processing apparatus including a makeup setting unit configured to set makeup to be applied to the user's face so as to be viewed close to an optimal face image arranged at a position.
  • the optimum face image can be a face image in which the parts constituting the user's face image are each of a predetermined size and arranged at a predetermined position.
  • the optimum face image may be a face image in which the parts in the face images of a plurality of persons are arranged in an average size and an average position.
  • the optimum face image can be a face image in which the parts are arranged in a size and a position set based on a predetermined theory.
  • the optimum face image may be a face image in which the parts are set in a size and a position set based on a face image of a predetermined person.
  • a feature amount detecting unit for detecting a feature point for recognizing the part of the user's face image; and recognizing the part based on the feature amount; and An evaluation unit that evaluates by scoring the deviation for each part, and the makeup setting unit includes the face image of the user based on the evaluation result of the evaluation unit.
  • the makeup that should be applied to the user's face is set so that the parts constituting the user's face image have an optimal size and can be seen close to the optimal face image arranged at an optimal position. can do.
  • the makeup setting unit generates a face image applied with various makeup applied to the face image of the user, and the evaluation unit performs the face image applied with various makeup applied, and
  • the makeup that minimizes the score of the evaluation result obtained by scoring the deviation of each part from the optimum face image can be set as the makeup to be applied to the user's face.
  • An analysis unit for analyzing the skin color of the user's face image can be further included, and the makeup setting unit is based on the color analysis result analyzed by the analysis unit and the evaluation result.
  • the user's face image is displayed on the user's face so that the parts constituting the user's face image are close in size to the optimal face image arranged at an optimal position.
  • the makeup to be applied can be set.
  • a recipe image generation unit that generates an image showing the set makeup procedure as a recipe image for each part.
  • the recipe image generation unit can sequentially display the recipe image set by the makeup setting unit for each part.
  • a registration unit that registers the user's face image and the recipe image corresponding to the makeup set based on the user's face image in a database in association with information for identifying the user. You can make it.
  • the user's face image and the recipe image corresponding to the makeup set based on the user's face image are associated with information for identifying the user and transmitted to an external server device,
  • a communication unit registered as a database can be further included.
  • An information processing method acquires a user's face image, and places the user's face image in an optimal position with parts that constitute the user's face image having an optimal size. It is an information processing method including the step of setting the makeup to be applied to the user's face so that the user can see it close to the optimal face image.
  • the program according to one aspect of the present disclosure includes an imaging unit that captures a user's face image, and the user's face image in an optimal position with parts that constitute the user's face image having an optimal size. It is a program that causes a computer to function as a makeup setting unit that sets makeup to be applied to the user's face so as to make it appear close to the optimal face image arranged.
  • a user's face image is captured, and the user's face image has an optimal size in which parts constituting the user's face image are optimally sized and arranged at an optimal position.
  • the makeup to be applied to the user's face is set so as to make it appear close to the face image.
  • FIG. 11 is a diagram illustrating a configuration example of a general-purpose personal computer.
  • a terminal device composed of a tablet terminal captures a user's face image by an imaging unit composed of a built-in camera, Based on the captured face image, each part constituting the face image is recognized, and a best balance face image that provides an optimal part arrangement is obtained from each recognized part and the contour of the face image.
  • the terminal device obtains a difference between the captured face image of the user and the best balance face image, scores it, sets makeup (makeup) that approaches the best balance face image based on the score, A recipe image showing a makeup (makeup) procedure is generated and displayed like a recipe showing the above procedure, thereby assisting the user in makeup (makeup).
  • the server device registers information including the face image of the user of the terminal device and the recipe image corresponding to the set makeup as a database in association with information for identifying the user.
  • FIG. 1 illustrates a configuration example of an information processing system including a server device and a terminal device for supporting a user's makeup (makeup) to which the technology of the present disclosure is applied.
  • the information processing system 11 in FIG. 1 includes a server device 31, terminal devices 32-1 to 32-n, and a network 33.
  • the terminal devices 32-1 to 32-n are simply referred to as the terminal device 32 unless otherwise distinguished, and the other configurations are also referred to in the same manner.
  • the terminal device 32 is a tablet terminal or the like possessed by the user, for example, as shown by the terminal device 32-n.
  • the terminal device 32 captures the face image of the user and recognizes each part of the captured face image. Then, for each part of the captured face image of the user, the best balance face image in which the arrangement is optimum is set. Then, the terminal device 32 evaluates the user's face image by scoring the difference between the best balance face image and the actual user's face image, and determines the best balance face image based on the evaluation result.
  • a makeup (makeup) is set so as to approach, and a recipe image composed of an image showing the makeup procedure is presented.
  • the terminal device 32 sends the captured real user's face image and a recipe image that supports makeup to the user's face image to the server device 31 via the network 33 represented by the Internet. Send.
  • the server device 31 stores each user's face image and recipe image transmitted via the network 33 in a database in association with information for identifying the user.
  • the server device 31 includes a control unit 51, a communication unit 52, and a storage unit 53.
  • the control unit 51 is, for example, an arithmetic processing device including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and the CPU is a ROM, HDD (Hard Disc Drive) or The program and data stored in the storage unit 53 including SSD (Solid State Drive) or the like are expanded in the RAM, and predetermined processing is executed.
  • a CPU Central Processing Unit
  • RAM Random Access Memory
  • ROM Read Only Memory
  • HDD Hard Disc Drive
  • SSD Solid State Drive
  • control unit 51 controls the communication unit 52 including an Ethernet board and the like, and information on the user's face image and recipe image transmitted from the terminal device 32 via the network 33 represented by the Internet. Receive. Then, the control unit 51 controls the database management unit 61 to store the received user face image and recipe image information in the storage unit 53 in association with user information for identifying the user. Stored in the in-server database 111.
  • the terminal device 32 includes an imaging unit 81, a display unit 82, a control unit 83, a storage unit 84, a communication unit 85, and an operation unit 86.
  • the imaging unit 81 is an image sensor made of CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor), etc., provided on the upper part of the terminal device 32 made of a tablet terminal as shown by the terminal device 32-n. Then, an image in a direction facing the imaging surface is captured.
  • the tablet-shaped terminal device 32 is provided with a camera or the like having a function of imaging the direction facing the user and the opposite direction, but the imaging unit 81 here is opposed to the user. It consists of a camera that picks up the direction of the image. Accordingly, when the terminal device 32 is being held by the user, the imaging unit 81 is normally configured to capture the user's face image.
  • the display unit 82 is provided on the surface of the tablet terminal as shown by the terminal device 32-n, and is composed of LCD (Liquid Crystal Display) or organic EL (Electro Luminescence). And an image picked up by the image pickup unit 81 is displayed.
  • LCD Liquid Crystal Display
  • organic EL Electro Luminescence
  • the control unit 83 is, for example, an arithmetic processing device including a CPU, a RAM, and a ROM.
  • the CPU expands programs and data stored in the storage unit 53 including a ROM, an HDD, an SSD, and the like into the RAM, and performs predetermined processing. Execute the process. More specifically, the control unit 83 executes a program or data, thereby performing a feature point extraction unit 91, a balance evaluation analysis unit 92, a makeup setting unit 93, a color analysis unit 94, a recipe image display control unit 95, and a database. It functions as the management unit 96.
  • the feature point extraction unit 91 extracts feature points for extracting parts constituting the user's face such as the face outline and the eyes, nose, lips, and eyebrows from the face image captured by the imaging unit 81.
  • the balance evaluation analysis unit 92 recognizes the contour of the user's face and various parts based on the feature point information, and obtains the best balance face image having the optimal arrangement and size for each part corresponding to the recognized contour.
  • the balance evaluation analysis is performed by scoring the difference between the various parts of the face image of the actual user that has been obtained and captured and the best balance face image.
  • the color analysis unit 94 analyzes the skin color from the actual user's face image captured by the imaging unit 81.
  • the makeup setting unit 93 includes a balance evaluation analysis result obtained by scoring a difference in arrangement and size of various parts between the actual user's face image captured by the image capturing unit 81 and the best balance face image, and a color analysis result. Based on the above, makeup (makeup) is set so that the face image of the user looks close to the best balance face image.
  • the recipe image display control unit 95 generates a recipe image indicating a makeup procedure for each part for performing makeup set by the makeup setting unit 93, and sequentially displays the generated recipe image for each of various parts. In this way, the user's makeup is supported.
  • the database management unit 96 associates the recipe image generated by the recipe image display control unit 95 and the captured face image of the user with information for identifying the user, and stores the storage unit 84 including an HDD, an SSD, or the like. Registered in the terminal internal database 131 and managed. Further, the database management unit 96 controls the communication unit 85 including an Ethernet board and the like, together with information for identifying the user, the recipe image generated by the recipe image display control unit 95 and the captured face image of the user. Then, the data is transmitted to the server device 31 via the network 33.
  • the operation unit 86 includes an operation button displayed on the display unit 82 when the display unit 82 includes a touch panel and the like, an operation button provided below the display unit 82 of the terminal device 32-n, and the like. It accepts the user's operation and generates an operation signal according to the operation content.
  • the in-terminal database 131 includes user information 151, balance face information 152, and recipe display information 153.
  • the user information 151 is information unique to the user that identifies the user.
  • the user information 151 includes a user ID (Identification) 161, a name nickname (user name and nickname) 162, a user gender 163, and usage. Information of the person's age 164 is included.
  • the balance face information 152 includes user face images 171-1 to 171-n imaged by the imaging unit 81, and is registered in association with the face image management information 172-1 to 172-n, respectively.
  • the face image management information 172 includes a face image ID (Identifier) 181, feature point correction information 182, balance analysis result information 183, balance recipe result information 184, and color analysis result information 185.
  • the face image ID is information for identifying the face image of the user imaged by the imaging unit 81.
  • the feature point correction information 182 is information on feature points for recognizing parts such as eyes, nose, lips, and eyebrows constituting a face image to be described later, and further includes information on feature points corrected by the user. That is, the feature point information extracted from the face image may be detected with a deviation from the actually required position (may be erroneously detected), so that it can be corrected by the user.
  • the feature point correction information 182 is information on all feature points including the feature points corrected by the user.
  • the balance analysis result information 183 is information indicating a balance evaluation analysis result obtained by scoring the difference between the best balance face image and the face image of the user actually captured by the balance evaluation analysis unit 92.
  • the balance recipe result information 184 is information indicating a balance evaluation analysis result obtained by scoring the difference between the face image when the makeup set to be close to the best balance face image by simulation and the best balance face image is scored. is there.
  • the color analysis result information 185 is an analysis result based on the user's face image, and is information indicating which of the skin colors corresponding to the user's face color set in advance in a palette shape.
  • the recipe display information 153 includes recipe images 191-1 to 191-n of the face images 171-1 to 171-n and corresponding recipe image management information 192-1 to 192-n.
  • the recipe image management information 192 includes application part information 201 that is information relating to a part corresponding to a part to be applied among the parts constituting the face, color information 202 that is information relating to the color of makeup applied to each part, It includes shape information 203, which is information on the shape of the parts, and arrangement information 204, which is information relating to the arrangement of each part.
  • the in-server database 111 includes user management information 231, user balance face information 232, and recipe display update information 233.
  • the user management information 231 is information for managing the user information 151 of each terminal device 32 that is transmitted from each terminal device 32 via the network 33, and uses information corresponding to the user information 151. Stored as person information 241-1 to 241-m.
  • the user balance face information 232 includes face images 261-1 to 261 -m and face image management information 262-1 to 262 -m corresponding to the face image 171 and the face image management information 172 of the balance face information 152.
  • the face image management information 262-1 to 262-m includes the user ID 161 of the user information 241 corresponding to the user information 151, respectively.
  • the updated recipe display image 233 includes recipe images 281-1 to 281-m and recipe image management information 282-corresponding to the recipe image 191 and the recipe image management information 192 of the recipe display information 153 in the terminal database 131. 1 to 282-m are stored in association with each other.
  • the best balance face image is generally expressed by making each part such as eyes, nose, lips, and eyebrows of a large number of human faces an average size and a face image arranged at an average position. It is said that you can. Therefore, the best balance face image may be a face image configured by arranging parts such as the eyes, nose, lips, and eyebrows of many human faces in an average size and an average position. .
  • a face image that is an average size of each part such as eyes, nose, lips, and eyebrows of a large number of human faces and that is arranged at an average position is also referred to as an average face image.
  • the best balance face image may be a face image made up of arrangements of parts other than the average face image.
  • an expert related to makeup such as a prominent makeup artist, defines the optimum arrangement. It may be a face image based on a predetermined theory.
  • the balance of the size and arrangement of each part in the best balance face image is as follows. As shown in FIG. 4 to FIG. From bottom to jaw length condition, eye size condition, eye position condition, nose size condition, nose position condition, ear height condition, lip size condition, lips It is also possible to satisfy at least one of a position condition, a profile condition, an eye balance condition, a mouth balance condition, an eyebrow balance condition, and a cheek balance condition.
  • the vertical length of the face includes a length AD from the hairline to the bottom of the eyebrow, a length DE from the eyebrow to the bottom of the nose, and the bottom of the nose.
  • the lengths EF from the tip of the jaw to the tip of the jaw are substantially the same length.
  • the width condition of the face includes a left eye width, a right eye width, and a width between the left and right eyebrows in a length BC between the left and right eye corners.
  • the width of each is approximately five times the width of the pupil.
  • the length condition from the bottom of the nose to the chin is, for example, as shown in FIG. 4, the length EG from the bottom of the nose to the bottom of the lower lip and the length GF from the bottom of the lower lip to the chin. are substantially the same.
  • the ratio of the vertical length to the horizontal length of the eye is approximately 1: 3.
  • the condition of the eye position is, for example, as shown in FIG. 5, the length AB from the hairline to the eye and the length BC from the eye to the center position of the lips are substantially the same length, and The width of the left eye, the width of the right eye, and the width between the left and right eyebrows are substantially the same.
  • the nose size condition is that the nose wing (width of the nose) DE is substantially the same as the length of the width of the eye.
  • the length FG from the lower nose to the lower lip and the length GH from the lower lip to the chin are substantially the same.
  • the condition of the height of the ear is, for example, the height between the extension line B of the outer corner of the eye and the extension line F of the nose as shown in FIG.
  • the width of the lips is approximately the same as the distance AB between the left and right pupils, and the upper and lower lips are approximately 1: 1.3 to 1 in the vertical width. : 1.5 ratio.
  • the condition of the position of the lips is, for example, as shown in FIG. 6, when the length CE from the nose to the chin is divided into three equal parts, the center D of the lips is set to a position of 1/3 from the top.
  • the profile of the profile is that the angle B between the nose and the nose is 90 degrees, and the tip of the upper lip touches a straight line C connecting the tip of the chin and the tip of the nose, or slightly
  • the distance is between the inner side (face side) of the straight line C and the upper end A of the eyes and nose is about 5 mm.
  • the balance condition of the eyes is, for example, as shown in the range A of FIG. 8, the ratio of the vertical width to the horizontal width of the eyes is approximately 1: 3, and as shown by the range B, It is assumed that the ratio of the width of the eyelid is 1: 1, and the angle between the eyes and the corners of the eyes is approximately 10 degrees as indicated by the angle C.
  • the condition of the mouth balance is, for example, as shown by a dotted line extending from the range A in FIG. 9, where the peak of the upper lip is the same as the center position of the width of the nostril, and the angle B from the valley of the upper lip to the peak is approximately 10 It is assumed that the base of the lower lip is substantially the same position as the tip of the chin and is longer than the width of the upper lip crest as shown in the range C of FIG. Further, as shown by the range D in FIG. 9, the shape of the lower lip is substantially parallel to the chin line, and as shown by the range E in FIG. And the balance between the upper lip and the lower lip is set to a ratio of approximately 1: 1.3 to 1: 1.5, as indicated by a range F in FIG.
  • the condition of the eyebrow balance is, for example, as shown by a straight line A in FIG. 10, where the position of the eyebrow head a starts directly above the eye d and the position of the eye corner c as shown by a straight line B in FIG. 10. Is on the extension line (straight line B) connecting the nose f and the corner of the eye c, and as shown by the straight line C connecting the eyebrow crest a to the eyebrow crest of FIG. And the end of the white eye, as shown by the straight line D in FIG. 10, the eye head d and the corner of the eye c are in a straight line, and as shown by the angle E in FIG. The angle is assumed to be approximately 10 degrees.
  • the condition of the cheek balance is such that the length of the cheek is approximately the ratio of the horizontal width from the side of the nose p to the contour line and the vertical width from just below the pupil to the contour. It shall be 1: 2.
  • the best balance face image may be set according to various other definitions. For example, a predetermined actor or a famous person or a famous person who is regarded as a beautiful face is selected. It may be defined based on each part of size and arrangement in the face image.
  • the balance evaluation analysis unit 92 sets a corresponding best balance face image based on the contour recognized based on the feature point information of the captured user's face image based on the above-described conditions, and sets the best balance face
  • the balance evaluation result is generated by scoring the difference between the image and the user's face image.
  • the makeup setting unit 93 sets makeup that brings the user's face image closer to the best balance face image based on the balance evaluation result. Then, the recipe image display control unit 95 generates a recipe image for realizing the set makeup.
  • makeup for the balance evaluation result is set to make up an impression that brings the user's face image closer to the best balance face image based on the balance evaluation result. That is, the user's face becomes a face that gives an impression similar to that when viewing the best balance face image by applying the set makeup.
  • makeup setting section 93 sets makeup (makeup) to which light and shade is added so as to bring the user's face image closer to the best balance face image based on the balance evaluation result.
  • each part that exists symmetrically is arranged so as to be separated from the center position to the left and right outside.
  • makeup as shown in the image P12 is set.
  • the face image arranged so that the parts present on the left and right are separated from the center position on the left and right of the face is an external face image. It shall also be called.
  • makeup setting section 93 has eyebrows A1-R, A1-L, eyes A2-R, A2-L, cheeks A3-R, A3-L, and lips A4-R existing on the left and right in image P12.
  • A4-L is a makeup that adds a darker color to a position closer to the center position on the left and right, and a makeup that adds a lighter color to a position far from the center position on the left and right.
  • the image P13 Makeup as shown by is set.
  • the face image in which various parts constituting the face image are arranged near the left and right center positions is also referred to as a centripetal face image.
  • makeup setting section 93 has eyebrows B1-R, B2-L, eyes B2-R, B2-L, cheeks B3-R, B3-L, and lips B4-R existing on the left and right in image P13.
  • B4-L is a makeup that adds a darker color at a position farther from the left and right center positions, and a lighter color is set at a position closer to the left and right center positions.
  • the makeup setting unit 93 determines, based on the balance evaluation analysis result, the score indicating the difference between the user's face image and the best balance face image, and the outer face image is a range close to the center of various parts. Set a darker color makeup, set a lighter color makeup in the area away from the center of each part, and for centripetal facial images, lighten the area near the center of each part, away from the center Set the makeup to darken the range.
  • step S11 the control unit 83 controls the imaging unit 81 to capture a preview image of the face image, and controls the display unit 82 to display the face image captured by the imaging unit 83 as a preview image.
  • step S12 the control unit 83 controls the operation unit 86 to determine whether or not imaging has been instructed by a user operation. If imaging is not instructed, the process returns to step S11. That is, the control unit 83 continues to display the preview image of the face image on the display unit 82 until imaging is instructed. During this time, the user adjusts the orientation of the imaging unit 81 of the tablet-shaped terminal device 32 so that an appropriate facial image can be captured while viewing the preview image of the displayed facial image.
  • step S12 for example, when the user adjusts the orientation of the imaging unit 81 of the terminal device 32 and operates the operation unit 86 to instruct imaging, the process proceeds to step S13.
  • step S13 the control unit 83 controls the image capturing unit 81 to capture a face image, and causes the face image 171 in the new balance face information 152 to be registered in the in-terminal database 131 of the storage unit 84.
  • the face image management information 172 corresponding to the new face image 171 is also provided in the balance face information 152, and the face image ID 181 is registered.
  • step S14 the control unit 83 causes the display unit 82 to display the captured face image, for example, as indicated by the face image F in FIG. With such display, the user can check the captured face image. At this time, if the face image is not the one that the user likes, it may be possible to capture the image again.
  • step S15 the control unit 83 controls the feature point extracting unit 91 to detect feature points for extracting parts such as the eyes, nose, lips, and eyebrows that constitute the face from the captured face image.
  • step S16 for example, as shown in FIG. 15, the feature point extraction unit 91 superimposes the position information detected as the feature point on the captured face image and causes the display unit 82 to display the face part recognition result.
  • feature point information is displayed on the display unit 82 as a black dot on the captured face image.
  • the feature point extraction unit 91 may display the feature points of the black points shown in FIG. 15 in different colors for each part such as the eyes, nose, lips, and eyebrows constituting the face, for example.
  • the feature points are set on parts such as the eyes, nose, lips, and eyebrows that make up the face. However, in some cases, those that protrude from the part range may be detected. There are feature points to be detected). Therefore, the feature points that have protruded can be corrected by the user, and the feature point extraction unit 91 accepts a correction operation using the operation unit 86 by the user and corrects the feature points. . For example, the user may be able to make corrections by tapping and shifting feature points on the display unit 82 including a touch panel of the tablet-like terminal device 32 or the like. At this time, the feature point extraction unit 91 registers information on all feature points including information on the corrected feature points as the feature point correction information 182. Therefore, feature point information is registered in the feature point correction information 182 regardless of the presence or absence of correction.
  • step S17 the control unit 83 controls the balance evaluation analysis unit 92 to execute balance evaluation processing and registers it as balance analysis result information 183.
  • step S31 the balance evaluation analysis unit 92 recognizes parts such as an outline and eyes, nose, lips, and eyebrows based on the feature point information of the feature point correction information 182.
  • step S32 the balance evaluation analysis unit 92 detects the outline of the face constituting the face image based on the feature point correction information 182 and controls the display unit 82, for example, the outline as shown in FIG. Display an image.
  • the balance evaluation analysis unit 92 obtains the ratio of the vertical direction to the horizontal direction of the outline of the face image as the face vertical and horizontal balance, and displays it superimposed on the face image.
  • the ratio of the vertical direction to the horizontal direction of the contour of the face image as the face vertical and horizontal balance is displayed as 4: 3.
  • the balance evaluation analysis unit 92 provides a comment field 301 at the lower part of the face image, that is, the lower part of the display unit 82, and displays a comment regarding the face vertical and horizontal balance.
  • “face vertical / horizontal balance (standard)” is displayed in the comment field 301, indicating that the facial vertical / horizontal balance is standard.
  • step S33 the balance evaluation analysis unit 92 generates a best balance face image based on the above-described conditions based on the feature point correction information 182.
  • the balance evaluation analysis unit 92 determines the size and arrangement of various parts so as to satisfy the conditions defined as the best balance face image according to the contour detected based on the feature points, thereby obtaining the best balance face image. Is generated.
  • step S34 the balance evaluation analysis unit 92 detects the vertical balance of the face image based on the feature point correction information 182 and controls the display unit 82, for example, as shown in FIG. Display a balanced image in the vertical direction.
  • the eyes, nose, lips, and eyebrows of the best balance face image are indicated by dotted lines, and as shown on the left side of the figure, from the hairline to the eyebrows of the best balance face image.
  • the length A1, the length A2 from the eyebrow head to the bottom of the nose, and the length A3 from the bottom of the nose to the chin are shown.
  • the length B1 from the hairline of the best balance face image to the corner of the eye and the length B2 from the corner of the eye to the chin are shown.
  • the length C1 from the hairline of the best balance face image to the bottom of the lips and the length C2 from the bottom of the lips to the chin are shown.
  • the length D1 from the bottom of the nose to the bottom of the lips and the length D2 from the bottom of the lips to the chin of the best balance face image are shown.
  • the balance evaluation analysis unit 92 calculates the length of the jaw, the length of the nasal muscles, the length of the jaw, and the eyebrow. Are displayed in the comment field 301 with respect to the best balance face image of the vertical position of the eye, the vertical position of the black eye, the vertical position of the nose, and the vertical position of the mouth.
  • the length of the jaw is “[short]”, which indicates that it is shorter than the best balance face image.
  • the length of the nose is “[long]”, which indicates that it is long with respect to the best balance face image.
  • the length of the jaw is “[long]”, which indicates that the jaw is long with respect to the best balance face image.
  • the vertical position of the eyebrow is “high on both left and right”, which indicates that the left and right are higher than the best balance face image.
  • the upper and lower positions of the black eyes are “high on both the left and right”, indicating that both the left and right are higher than the best balance face image.
  • the upper and lower positions of the nose are set to “[high]”, which indicates that it is higher than the best balance face image.
  • the vertical position of the mouth is “[high]”, which indicates that it is higher than the best balance face image.
  • step S35 the balance evaluation analysis unit 92 detects the horizontal (horizontal) direction balance of the face image based on the feature point correction information 182, and controls the display unit 82, for example, as shown in FIG. Display a horizontal balance image.
  • the eyes, nose, lips, and eyebrows of the best balance face image are indicated by dotted lines, the right eye width G1, the left eye width G2, and the left eye width G3 of the best balance face image. Are shown respectively.
  • the width H1 of the nose and the width H2 of the lips are shown.
  • an interval based on the width of the black eye in the horizontal direction is shown.
  • the balance evaluation analysis unit 92 calculates the width of the outer corner of the eye, the width of the nose, the width of the mouth, and the width between the eyes. The relationship between the width of the corner of the eye and the edge of the face with respect to the best balance face image is displayed in the comment field 301.
  • the width of the outer corner of the eye is “right is (standard), left is [wide]”, and the right side is the standard width with respect to the best balance face image, but the left side is It is expressed as wide.
  • the width of the nose is “[wide]”, indicating that it is wider than the best balance face image.
  • the width of the mouth is “[wide]”, which indicates that it is wider than the best balance face image.
  • the width between the eyes is “[wide]”, which indicates that it is wider than the best balance face image.
  • the width of the corners of the eyes and the edges of the face is “[wide]”, which indicates that it is wider than the best balance face image.
  • step S36 the balance evaluation analysis unit 92 detects the eye proportion based on the feature point correction information 182, and controls the display unit 82 to display, for example, an eye proportion image as shown in FIG.
  • the eyes in the best balance face image are indicated by dotted lines, and straight lines J1 and J2 indicating the height of the corners of the user's face image are shown.
  • the balance evaluation analysis unit 92 determines the best balance with respect to the thickness of the eyes and the angle between the eyes and the eyes based on the positions of the eyes in the best balance face image and the straight lines J1 and J2 indicating the heights of the eyes in the user's face image. The relationship with the face image is displayed in the comment field 301.
  • step S37 the balance evaluation analysis unit 92 detects the lip shape based on the feature point correction information 182 and controls the display unit 82 to display, for example, a lip shape image as shown in FIG.
  • the mouth in the best balance face image is indicated by a dotted line, and the width interval between the crests and valleys in the lateral direction of the lips in the mouth of the best balance face image is shown.
  • the balance evaluation analysis unit 92 determines the best balance for the entire lip thickness, upper and lower lip thickness, and lip peak position. The relationship with the face image is displayed in the comment field 301.
  • the thickness of the entire lips is “[thick]”, indicating that the lips are thicker than the best balance face image.
  • the thickness of the upper and lower lips is “[thick]”, which indicates that the upper and lower lips are thicker than the best balance face image.
  • Regarding the position of the lip crest “right is [outside] left is [inside]”, and the position of the lip crest with respect to the best balance face image is that the right is outside and the right is inside. It is represented.
  • step S38 the balance evaluation analysis unit 92 detects the right / left position range of the eyebrows based on the feature point correction information 182 and controls the display unit 82, for example, the left / right position of the eyebrows as shown in FIG. Display the range.
  • the balance evaluation analysis unit 92 includes a best balance face image for the right and left positions of the eyebrows, the left and right positions of the eyebrows, and the range of the eyebrows based on the line segments of the eyebrows, the eyebrows, and the eyebrows in the face image of the user. Are displayed in the comment field 301.
  • the right and left positions of the eyebrow are “right (standard) and left [inside]”, the right and left positions of the eyebrow are the standard positions on the right balance face image, and the left side is the inside. It is expressed that there is.
  • the left and right positions of Mt. Bizan “right is (standard) left is [inside]”, and the right and left positions of Mt. Has been.
  • the range of the eyebrows “right is (standard) left is [inside]”
  • the right eyebrow range is the standard position on the right side and the left side is the inside of the best balance face image. Has been.
  • step S39 the balance evaluation analysis unit 92 obtains a comprehensive evaluation based on the balance evaluation analysis results so far, and controls the display unit 82 to display, for example, a comprehensive evaluation image as shown in FIG.
  • each part in the best balance face image is represented by a dotted line and is displayed superimposed on the user's face image.
  • the balance evaluation analysis unit 92 displays a comprehensive evaluation regarding makeup for each part in the comment column 301.
  • the left and right position of the black eye corresponding to the eye makeup is set to “[extracentric] score 100”, and the left and right position of the black eye is outside the center of the left and right with respect to the position in the best balance face image. It is shown that the score is 100.
  • the upper and lower positions of the black eyes are “[high] score 70”, which indicates that the upper and lower positions of the black eyes are higher than the position of the best balance face image, and the score is 70.
  • the angle of both eyes is “[down eye] score 90”, which indicates that the angle of both eyes is smaller than that in the best balance face image, the eye is down, and the score is 90.
  • the vertical balance of the forehead corresponding to the eyebrow is “[narrow] score 90”, which indicates that the vertical balance of the forehead is narrower than the width in the best balance face image and the score is 90.
  • the width between both eyes corresponding to the nose makeup is “[wide] score 100”, which indicates that the width between both eyes is wider than the width in the best balance face image and the score is 100.
  • the nose vertical balance is “[long] score 70”, which indicates that the nose vertical balance is longer than the length in the best balance face image and the score is 70.
  • the vertical balance of the jaw corresponding to shading is “[narrow] score 90”, which indicates that the vertical balance of the jaw is narrower than the length in the best balance face image and the score is 90.
  • the lateral balance of the chin (chin) is “[long] score 90”, which indicates that the lateral balance of the jaw is longer than the length in the best balance face image and the score is 90.
  • the width of the corner of the eye and the contour is “[wide] score 60”, which indicates that the width of the corner of the corner and the contour is wider than the width of the best balance face image, and the score is 60.
  • the vertical balance of the chin (jaw) with respect to the cheek is “[long] score 90”, indicating that the vertical balance of the jaw is longer than the length in the best balance face image and the score is 90.
  • the width of the mouth corresponding to the lip makeup is set to “[wide] score 50”, which indicates that the width of the mouth is wider than the width in the best balance face image and the score is 50.
  • the thickness of the upper lip is “[thick] score 50”, which indicates that the thickness of the upper lip is thicker than the thickness in the best balance face image and the score is 50.
  • the thickness of the lower lip is “[thick] score 50”, which indicates that the thickness of the lower lip is thicker than the thickness in the best balance face image and the score is 50.
  • step S40 the balance evaluation analysis unit 92 generates the right half composite image generated by using the right half as the left half with respect to the horizontal center line of the user's face image, and the left half as the right half.
  • the left half composite image generated by using is generated.
  • step S41 the balance evaluation analysis unit 92 controls the display unit 82 to display the right half image in FIG. 24 and the left half image in FIG. 25, for example.
  • the right half composite image in FIG. 24 is composed of a face image in which the right half face image of the user's face image is also applied to the left half.
  • the balance evaluation analysis unit 92 displays “right is [smaller]” for the left and right size of the face in the comment field 301. This means that the right half of is smaller than the left half.
  • the left half composite image in FIG. 25 is composed of a face image in which the left half face image of the user's face image is also applied to the right half.
  • the balance evaluation analysis unit 92 displays “right is [smaller]” for the left and right size of the face in the comment field 301. This means that the right half of is smaller than the left half.
  • balance evaluation processing is executed, and balance evaluation information for generating makeup to be applied is generated so that the user's face image approaches the best balance face image, and the evaluation result is shown.
  • step S18 the control unit 83 controls the color analysis unit 94 to analyze the skin color of the user's face based on the user's face image. For example, in FIG. A color test image as shown is displayed on the display unit 82 as a color analysis result.
  • the user's face image is displayed at the top, and a color palette 321 in which a plurality of 3 ⁇ 3 skin colors are arranged in a palette shape is displayed at the bottom.
  • a pointer 321a is shown on the palette corresponding to the skin color of the face image.
  • the color analysis unit 94 registers the color analysis result in the color analysis result information 184 in the balance face information 152.
  • step S ⁇ b> 19 the makeup setting unit 93 performs a simulation for setting makeup so that each part of the user's face image approaches each part of the best balance face image based on the balance evaluation information and the color analysis result. , And the simulation result when the makeup obtained by the simulation is applied to the face image of the user is displayed as a simulation image as shown in FIG. 27, for example.
  • the balance evaluation analysis unit 92 evaluates the result of scoring such that the higher the deviation between the arrangement of each part of the user's face image and the arrangement of each part in the best balance face image, the higher the score. It is said. Therefore, the makeup setting unit 93 executes a simulation for applying various makeups to the user's face image, and compares the face image after makeup with the best balance face image to obtain the makeup with the minimum score. As a result, a makeup that approaches the best balance face image is obtained by simulation. In this simulation, makeup setting section 93 repeats while changing makeup based on the concept of the makeup setting method for the balance evaluation result described with reference to FIG. 12 described above. At this time, makeup setting unit 93 registers score information based on makeup obtained by simulation in balance recipe result information 184 in balance face information 152.
  • the simulation image of FIG. 27 shows a face image when the makeup obtained by the simulation is applied to the user's face image in the upper part. Also, in the comment column 301 in the lower part of FIG. 27, “Simulation image” is written, indicating that the face image of the user in the upper part is a face image obtained by applying the makeup determined by the simulation. Has been.
  • step S20 the recipe image display control unit 95 executes a recipe image display process and sequentially displays recipe images.
  • the comment column 301 “Simulation drawing” is described, which indicates that the face image with the makeup determined by the simulation is displayed on the face image of the user. Yes.
  • each step after step S20 shall advance each step sequentially by operating the operation part 86.
  • the process of each step after step S20 shall advance each step sequentially by operating the operation part 86.
  • the content displayed by the user is confirmed, and the process proceeds to the next step at the timing requested by the user.
  • the description of the operation is redundant and the description is omitted.
  • each process of the subsequent steps including step S20 can return to the immediately preceding step by operating the operation unit 86 as necessary so that the user can look back on the immediately preceding process. Also good.
  • step S51 the recipe image display control unit 95 generates a recipe image 191 indicating a procedure for applying the corresponding makeup from the simulation result of the makeup based on the balance evaluation information and the color analysis result, and the corresponding recipe image.
  • Management information 192 is generated and registered in the in-terminal database 131 as recipe display information 153.
  • the recipe image display control unit 95 displays the entire recipe image on the display unit 82 as shown in FIG. 29, for example.
  • the entire makeup recipe to be applied to each part is displayed superimposed on the user's face image.
  • the entire makeup recipe image of FIG. 29 includes face shading recipes 401-1 to 401-3, nose recipes 402-1, 402-2, 403, eyebrow recipes 404-1, 404-2, eye Shadow recipes 405-1, 405-2, 406-1, 406-2, 407-1, 407-2, cheek recipes 408-1, 408-2, cheek highlight recipes 409-1, 409-2, lip recipe
  • a specific recipe for makeup to be applied to each of 410 is shown. That is, the user applies each makeup based on each recipe of the entire recipe image shown in FIG. 29 to each part of his / her face, so that the face image corresponding to the simulation result image shown in FIG. Such makeup can be realized.
  • step S52 the recipe image display control unit 95 displays a face shading recipe image as shown in FIG.
  • the face shading recipe image of FIG. 30 shows the direction of shading to be applied, as indicated by the arrow directions of the face shading recipes 401-1 to 401-3 in the user's face image.
  • step S53 the recipe image display control unit 95 displays a nose recipe image on the display unit 82, for example, as shown in FIG.
  • nose recipes 402-1, 402-2, and 403 show specific methods for making nose shadows. That is, nose recipes 402-1 and 402-2 show how to put shadows on the left and right nose, and nose recipe 403 shows how to put highlights on the nose.
  • step S54 the recipe image display control unit 95 displays the eyebrow recipe image on the display unit 82 as shown in FIG.
  • eyebrow recipes 404-1 and 404-2 indicate specific methods of eyebrow makeup.
  • step S55 the recipe image display control unit 95 displays the eye shadow (blocking) recipe image on the display unit 82 as shown in FIG.
  • eye shadow (blocking) recipes 405-1, 405-2, 406-1, 406-2, 407-1, and 407-2 are respectively eye shadows (blocking).
  • eye shadows (blocking) As a specific method of makeup, the color of the palette used for each position is shown.
  • step S56 the recipe image display control unit 95 displays an eye shadow (base coating) recipe image on the display unit 82 as shown in FIG.
  • the arrows in the eye shadow (base coating) recipes 405-1 and 405-2 indicate the specific directions in which gradation is applied in the makeup of the eye shadow (base coating). It is shown.
  • step S57 the recipe image display control unit 95 displays the eye shadow (frame coating) recipe image on the display unit 82 as shown in FIG.
  • the arrows in the eye shadow (frame painting) recipes 406-1 and 406-2 indicate the specific directions in which gradation is applied in the eye shadow (frame painting) makeup, respectively. It is shown.
  • step S58 the recipe image display control unit 95 displays, for example, an eye shadow (undercoat) recipe image on the display unit 82 as shown in FIG.
  • the arrows in the eyeshadow (undercoat) recipes 407-1 and 407-2 are gradations as recipes for making eyeshadow (undercoat), respectively. The direction to apply is shown.
  • step S59 the recipe image display control unit 95 displays an eyeline recipe image on the display unit 82, for example, as shown in FIG.
  • the arrows in the eyeline recipes 421-1 and 421-2 indicate the direction in which a specific eyeline in eyeline makeup is extended.
  • step S60 the recipe image display control unit 95 displays, for example, an eyelash recipe image on the display unit 82 as shown in FIG.
  • the eyelash recipes 431-1 and 431-2 show specific eyelash positions in eyelash makeup.
  • step S61 the recipe image display control unit 95 displays a cheek recipe image on the display unit 82 as shown in FIG.
  • the cheek recipes 408-1 and 408-2 are colored as recipes for making cheek makeup, respectively.
  • step S62 the recipe image display control unit 95 displays a cheek highlight makeup recipe image on the display unit 82, for example, as shown in FIG.
  • the cheek highlight recipes 409-1 and 409-2 are each provided with an arrow indicating the specific direction of the highlight outline in the makeup of the cheek highlight.
  • step S63 the recipe image display control unit 95 displays a lip recipe image on the display unit 82, for example, as shown in FIG.
  • the width of the lips is wide and thick both at the top and bottom, so let's darken the center of the lips and blur the top and bottom contours of the lips with a concealer.
  • the thickness of the top and bottom lips is about 1: 1.5
  • the points of lip makeup based on the evaluation results are shown.
  • the makeup method for each part in the user's face image is displayed so as to indicate the procedure sequentially at the timing when the operation unit 86 such as a button operated when the user proceeds with the next processing. It will be done. For this reason, the user can make makeup that approaches each best balance face image in order while looking at the makeup recipe image displayed for each part in order, and the impression close to the best balance face image on his / her face It is possible to easily apply makeup that gives the appearance.
  • the process of displaying the recipe regarding makeup of each part is not limited to the order described above, and may be displayed in another order.
  • the operation button may be operated to return to the previous recipe display and to return to the previous part recipe display.
  • step S21 the database management unit 96 controls the communication unit 85 to read out the new balance face information 152 and recipe display information 153 stored in the in-terminal database 131 by the series of processes described above. Together with the user information 151, the information is transmitted to the server device 31 via the network 33.
  • control unit 51 of the server device 31 controls the communication unit 54 to receive the transmitted balance face information 152 and recipe display information 153 together with the user information 151. Then, the control unit 51 controls the database management unit 61 to register in the user management information 231 as user balance face information 232 and recipe display update information 233 in the in-server database 111 of the storage unit 53. Registration is performed in association with user information 241.
  • the user can use the tablet-like terminal device 32 to capture his / her own face image, and while looking at the recipe image for making makeup close to the best balance face image, It is possible to achieve makeup that is close to a balanced face image. In addition, it is possible to achieve makeup that also considers skin color, so if the skin color is dark due to sunburn etc., make the recipe to realize makeup that makes the skin color white Alternatively, it is possible to realize a makeup that prevents the face from being visually recognized even if the face color is not good due to poor physical condition.
  • the best balance face image may be a face image in which the size and arrangement of each part constituting the face image are set based on various theories, for example, based on the theory of a famous makeup artist or the like. You may make it be the face image specified by conditions.
  • the best balance face image may be a face image of a celebrity such as an actor or a singer that the user likes. Further, makeup recommended by a plurality of famous makeup artists may be selected, or any one of a plurality of best balance face images may be selected.
  • the colors used for makeup may be adapted to the destination and season of the user, such as weddings, funerals, parties, spring, summer, autumn, winter, etc. Depending on the situation, it is also possible to set and select a variation in which the hue and / or density are changed.
  • the server device 31 converts the user's face image into the user's face image. Based on this, a balance evaluation process may be executed, a recipe image may be generated, and sequentially supplied to the terminal device 32 for display.
  • either the terminal device 32 or the server device 31 via the network 33 identifies the user and the corresponding recipe display information 153. May be read out and supplied to the terminal device 32 to display only the recipe image.
  • the recipe image for makeup may be displayed by switching sequentially for each part at a predetermined time interval, or each time the operation unit 86 including operation buttons is operated, the part is switched.
  • the recipe image may be displayed, and each time the operation unit 86 is operated, the parts are switched and the recipe image is displayed so that the recipe image is displayed according to the timing when the user makes up. It becomes possible to do.
  • ⁇ Example executed by software> By the way, the series of processes described above can be executed by hardware, but can also be executed by software.
  • a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.
  • FIG. 42 shows a configuration example of a general-purpose personal computer.
  • This personal computer incorporates a CPU (Central Processing Unit) 1001.
  • An input / output interface 1005 is connected to the CPU 1001 via a bus 1004.
  • a ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.
  • the input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 for outputting a processing operation screen and an image of the processing result to a display device, programs, and various types.
  • a storage unit 1008 including a hard disk drive for storing data, a LAN (Local Area Network) adapter, and the like are connected to a communication unit 1009 that executes communication processing via a network represented by the Internet.
  • magnetic disks including flexible disks
  • optical disks including CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)), magneto-optical disks (including MD (Mini Disc)), or semiconductors
  • a drive 1010 for reading / writing data from / to a removable medium 1011 such as a memory is connected.
  • the CPU 1001 is read from a program stored in the ROM 1002 or a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 to the RAM 1003. Various processes are executed according to the program.
  • the RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.
  • the CPU 1001 loads the program stored in the storage unit 1008 to the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, for example. Is performed.
  • the program executed by the computer (CPU 1001) can be provided by being recorded on the removable medium 1011 as a package medium, for example.
  • the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be installed in the storage unit 1008 via the input / output interface 1005 by attaching the removable medium 1011 to the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. In addition, the program can be installed in advance in the ROM 1002 or the storage unit 1008.
  • the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.
  • the input unit 1006 and the output unit 1007 correspond to the imaging unit 81 and the operation unit 86
  • the communication unit 1009 corresponds to the communication unit 85, respectively.
  • the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .
  • the present disclosure can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is processed jointly.
  • each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • this indication can also take the following structures.
  • an imaging unit that captures a user's face image; The user's face image is applied to the user's face so that the parts constituting the user's face image have an optimal size and appear close to the optimal face image arranged at an optimal position.
  • An information processing apparatus including a makeup setting unit configured to set makeup to be applied.
  • the optimum face image is a face image in which the parts constituting the user's face image are each of a predetermined size and arranged at a predetermined position. apparatus.
  • ⁇ 3> The information processing apparatus according to ⁇ 2>, wherein the optimum face image is a face image in which the parts of the face images of a plurality of persons are arranged in an average size and an average position. .
  • ⁇ 4> The information processing apparatus according to ⁇ 2>, wherein the optimum face image is a face image arranged at a size and a position where the parts are set based on a predetermined theory.
  • ⁇ 5> The information processing apparatus according to ⁇ 2>, wherein the optimum face image is a face image arranged at a size and a position where the parts are set based on a face image of a predetermined person.
  • ⁇ 6> a feature amount detection unit that detects a feature point for recognizing the part of the user's face image;
  • An evaluation unit that recognizes the parts based on the feature amount and evaluates by scoring a deviation of each part of the user's face image with respect to the optimum face image;
  • the makeup setting unit based on the evaluation result of the evaluation unit, the user's face image, an optimal face in which the parts constituting the user's face image are optimally sized and arranged at an optimal position
  • the information processing apparatus according to any one of ⁇ 1> to ⁇ 5>, wherein makeup to be applied to the user's face is set so as to be viewed close to an image.
  • the makeup setting unit generates a face image applied with various makeup applied to the user's face image
  • the evaluation unit scored the deviation of each of the parts of the face image applied by changing the makeup in various ways and the optimal face image, and the makeup with the minimum score
  • the information processing apparatus according to ⁇ 6>, which is set as a makeup to be applied to a user's face.
  • An analysis unit that analyzes a skin color of the user's face image
  • the makeup setting unit based on the color analysis result analyzed by the analysis unit and the evaluation result, the user's face image, the parts constituting the user's face image are optimally sized.
  • the information processing apparatus in which makeup to be applied to the user's face is set so as to be viewed close to the optimal face image arranged at an optimal position.
  • the information processing apparatus according to any one of ⁇ 1> to ⁇ 8>, further including a recipe image generation unit that generates an image indicating the set makeup procedure as a recipe image for each part.
  • the recipe image generation unit sequentially displays the recipe image set by the makeup setting unit for each part.
  • a registration unit that registers the face image of the user and the recipe image corresponding to the makeup set based on the face image of the user in a database in association with information for identifying the user
  • the information processing apparatus further including: ⁇ 12>
  • the user's face image and the recipe image corresponding to the makeup set based on the user's face image are associated with information for identifying the user, and the external server device
  • the information processing apparatus further including a communication unit that transmits and registers as a database.
  • ⁇ 13> Acquire user face images, The user's face image is applied to the user's face so that the parts constituting the user's face image have an optimal size and appear close to the optimal face image arranged at an optimal position.
  • An information processing method including a step of setting a makeup to be performed.
  • an imaging unit that captures a user's face image; The user's face image is applied to the user's face so that the parts constituting the user's face image have an optimal size and appear close to the optimal face image arranged at an optimal position.
  • a program that causes a computer to function as a makeup setting section that sets makeup to be applied.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
PCT/JP2017/038862 2016-11-11 2017-10-27 情報処理装置、および情報処理方法、並びにプログラム Ceased WO2018088243A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/329,552 US11321764B2 (en) 2016-11-11 2017-10-27 Information processing apparatus and information processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016220618A JP6753276B2 (ja) 2016-11-11 2016-11-11 情報処理装置、および情報処理方法、並びにプログラム
JP2016-220618 2016-11-11

Publications (1)

Publication Number Publication Date
WO2018088243A1 true WO2018088243A1 (ja) 2018-05-17

Family

ID=62109549

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/038862 Ceased WO2018088243A1 (ja) 2016-11-11 2017-10-27 情報処理装置、および情報処理方法、並びにプログラム

Country Status (3)

Country Link
US (1) US11321764B2 (https=)
JP (1) JP6753276B2 (https=)
WO (1) WO2018088243A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021049082A (ja) * 2019-09-24 2021-04-01 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108305317B (zh) * 2017-08-04 2020-03-17 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质
CN109426767A (zh) * 2017-08-24 2019-03-05 丽宝大数据股份有限公司 眼线描绘指引装置及其方法
US11157985B2 (en) * 2017-11-29 2021-10-26 Ditto Technologies, Inc. Recommendation system, method and computer program product based on a user's physical features
CN111275650B (zh) * 2020-02-25 2023-10-17 抖音视界有限公司 美颜处理方法及装置
WO2022146615A1 (en) * 2020-12-30 2022-07-07 L'oreal Digital makeup palette
JP7708013B2 (ja) * 2022-06-20 2025-07-15 トヨタ自動車株式会社 教育支援装置、通信システム、プログラム、及び教育支援方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008102440A1 (ja) * 2007-02-21 2008-08-28 Tadashi Goino 化粧顔画像生成装置及び方法
JP2014149678A (ja) * 2013-02-01 2014-08-21 Panasonic Corp 美容支援装置、美容支援システム、美容支援方法、並びに美容支援プログラム

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009064423A (ja) * 2007-08-10 2009-03-26 Shiseido Co Ltd メイクアップシミュレーションシステム、メイクアップシミュレーション装置、メイクアップシミュレーション方法およびメイクアップシミュレーションプログラム
US8884980B2 (en) * 2010-09-24 2014-11-11 Taaz, Inc. System and method for changing hair color in digital images
JP4760999B1 (ja) * 2010-10-29 2011-08-31 オムロン株式会社 画像処理装置、画像処理方法、および制御プログラム
JP4862955B1 (ja) * 2010-10-29 2012-01-25 オムロン株式会社 画像処理装置、画像処理方法、および制御プログラム
JP2012181688A (ja) * 2011-03-01 2012-09-20 Sony Corp 情報処理装置、情報処理方法、情報処理システムおよびプログラム
JP6008323B2 (ja) * 2013-02-01 2016-10-19 パナソニックＩｐマネジメント株式会社 メイクアップ支援装置、メイクアップ支援方法、およびメイクアップ支援プログラム
JP6132232B2 (ja) * 2013-02-01 2017-05-24 パナソニックＩｐマネジメント株式会社 メイクアップ支援装置、メイクアップ支援システム、およびメイクアップ支援方法
JP6128309B2 (ja) * 2013-02-01 2017-05-17 パナソニックＩｐマネジメント株式会社 メイクアップ支援装置、メイクアップ支援方法、およびメイクアップ支援プログラム
WO2014118842A1 (ja) 2013-02-01 2014-08-07 パナソニック株式会社 美容支援装置、美容支援システム、美容支援方法、並びに美容支援プログラム
JP5991536B2 (ja) * 2013-02-01 2016-09-14 パナソニックＩｐマネジメント株式会社 メイクアップ支援装置、メイクアップ支援方法、およびメイクアップ支援プログラム
US9524028B2 (en) * 2013-03-08 2016-12-20 Fastvdo Llc Visual language for human computer interfaces
EP3039990B1 (en) * 2013-08-30 2019-07-24 Panasonic Intellectual Property Management Co., Ltd. Makeup assistance device, makeup assistance system, makeup assistance method, and makeup assistance program
CN104637078B (zh) * 2013-11-14 2017-12-15 腾讯科技（深圳）有限公司 一种图像处理方法及装置
JP6476608B2 (ja) * 2014-06-30 2019-03-06 カシオ計算機株式会社 動作処理装置、動作処理方法及びプログラム
US20160357578A1 (en) * 2015-06-03 2016-12-08 Samsung Electronics Co., Ltd. Method and device for providing makeup mirror
EP3485762A4 (en) * 2016-07-14 2019-07-24 Panasonic Intellectual Property Management Co., Ltd. MAKE-UP ASSISTANCE AND MAKE-UP ASSISTANCE PROCEDURES
WO2018079255A1 (ja) * 2016-10-24 2018-05-03 パナソニックＩｐマネジメント株式会社 画像処理装置、画像処理方法、および画像処理プログラム

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008102440A1 (ja) * 2007-02-21 2008-08-28 Tadashi Goino 化粧顔画像生成装置及び方法
JP2014149678A (ja) * 2013-02-01 2014-08-21 Panasonic Corp 美容支援装置、美容支援システム、美容支援方法、並びに美容支援プログラム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HAYASHI, HIROYUKI ET AL.: "Presentation of makeup method using animation and proposal and implementation of the makeup supporting system corresponding to changes in facial expression of the user", PROCEEDINGS OF THE MULTIMEDIA, DISTRIBUTED, COOPERATIVE AND MOBILE (DICOM02015) SYMPOSIUM, vol. 2015, no. 1, 1 July 2015 (2015-07-01), pages 1450 - 1457, ISSN: 1882-0840 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021049082A (ja) * 2019-09-24 2021-04-01 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム
JP7413691B2 (ja) 2019-09-24 2024-01-16 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム

Also Published As

Publication number Publication date
JP2018077766A (ja) 2018-05-17
US11321764B2 (en) 2022-05-03
US20190254408A1 (en) 2019-08-22
JP6753276B2 (ja) 2020-09-09

Similar Documents

Publication Publication Date Title
JP6753276B2 (ja) 情報処理装置、および情報処理方法、並びにプログラム
US12444110B2 (en) System and method for digital makeup mirror
US20160042557A1 (en) Method of applying virtual makeup, virtual makeup electronic system, and electronic device having virtual makeup electronic system
JP3779570B2 (ja) 化粧シミュレーション装置、化粧シミュレーション制御方法、化粧シミュレーションプログラムを記録したコンピュータ読み取り可能な記録媒体
CN101779218B (zh) 化妆模拟系统及其化妆模拟方法
US11776187B2 (en) Digital makeup artist
JP5991536B2 (ja) メイクアップ支援装置、メイクアップ支援方法、およびメイクアップ支援プログラム
JP3984191B2 (ja) 仮想化粧装置及びその方法
CN103430208B (zh) 图像处理装置、图像处理方法以及控制程序
US20100189357A1 (en) Method and device for the virtual simulation of a sequence of video images
JP2010211308A (ja) メイクアップアドバイス装置、メイクアップアドバイス方法及びプログラム
JP5503725B2 (ja) 認証装置、認証プログラム、および認証方法
JPH10255066A (ja) 顔画像の修正方法、化粧シミュレーション方法、化粧方法、化粧サポート装置及びファンデーション転写膜
CN103180873A (zh) 图像处理装置、图像处理方法及控制程序
US20190066348A1 (en) Image processing system and method
CN104822292A (zh) 化妆辅助装置、化妆辅助系统、化妆辅助方法以及化妆辅助程序
WO2021197186A1 (zh) 辅助化妆方法、终端设备、存储介质及程序产品
US11961169B2 (en) Digital makeup artist
CN103679767A (zh) 图像生成装置以及图像生成方法
WO2022146766A1 (en) Digital makeup artist
CN108234770A (zh) 一种辅助化妆系统、辅助化妆方法、辅助化妆装置
KR101507410B1 (ko) 모바일 단말의 라이브 메이크업 촬영 방법 및 장치
JP6078896B2 (ja) メイクアップ支援装置およびメイクアップ支援方法
KR20230117240A (ko) 디지털 메이크업 팔레트

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17869427

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17869427

Country of ref document: EP

Kind code of ref document: A1