JP6427297B2 - Makeup simulation system, makeup simulation method, and makeup simulation program - Google Patents

Description

  The present invention relates to a makeup simulation system for performing makeup simulation on a face image, a makeup simulation method, and a makeup simulation program.

  Conventionally, various techniques have been proposed to support the makeup of the user. For example, Patent Document 1 describes a technique relating to a display method in the case where makeup is applied or appearance is adjusted while confirming the display content of a captured image of a user. Further, Patent Document 2 describes a technology for assisting the user's makeup using three-dimensional information acquired using a three-dimensional measurement unit.

JP, 2013-223001, A JP, 2015-197710, A

  However, in the method of Patent Document 1, there is a possibility that the photographed image and the comparison target image can not be superimposed properly due to the performance of the recognition means for recognizing the specific part. In particular, when it is necessary to apply makeup in accordance with the shape of the eye, such as eye makeup, it is important to superimpose a makeup image in accordance with the contour of the constituent part of the face of the individual.

  Further, in the method of Patent Document 2, it is considered that the use of three-dimensional information makes it easier to recognize the state of the face as compared to Patent Document 1, but it is necessary to project the measurement pattern by the light projector. Therefore, there is a problem that the apparatus becomes expensive.

  When performing makeup simulation, if the outline of the component parts of the face can not be recognized automatically and properly, the user manually inputs a line of the outline and designates an area to apply makeup and a mask area not to apply makeup. Is required. At the time of manual input by the user, there is a problem that the accuracy of the line varies depending on the user's skill level and dexterity.

  Furthermore, in both Patent Document 1 and Patent Document 2, the makeup applied by the user is only a configuration in which the image is confirmed by oneself, but a plurality of people share the screen and can check the image and operate the image. It is required to By sharing the screen, appropriate feedback from experts such as an adviser and an operator is considered to be able to perform makeup simulation with high user satisfaction.

  Therefore, in the present invention, in view of the above problems, it is an object of the present invention to provide a makeup simulation system, a makeup simulation method, and a makeup simulation program for performing makeup simulation on a face image by accepting designation of a plurality of feature points by plotting. I assume.

  The present invention provides the following solutions.

The invention according to the first feature is
A makeup simulation system that performs makeup simulation on a face image, and
Configuration part type specification receiving means for receiving specification of the type of configuration part of the face image;
Feature point specification receiving means for receiving specification of a plurality of feature points by plotting on the contour of the component part received the specification;
Contour drawing means for drawing the contour of the component part based on a plurality of feature points designated by the plot;
Providing a makeup simulation system characterized by comprising:

  According to the invention as set forth in the first aspect, in a makeup simulation system for performing makeup simulation on a face image, a composition part type specification accepting means for receiving designation of the kind of composition part of the face image, and composition parts receiving the designation A feature point specification receiving unit that receives specification of a plurality of feature points by plotting on the contour of the image, and an outline drawing unit that draws an outline of the component based on the plurality of feature points specified by the plot; Prepare.

  The invention relating to the first feature is a category of a makeup simulation system, but the same action and effect can be obtained even with a makeup simulation method and a makeup simulation program.

An invention according to a second feature is the makeup simulation system according to the first feature, wherein
There is provided a makeup simulation system characterized in that the feature points to be specified by the plot include four points on the top, the bottom, the left, and the right of the component part.

  According to the invention as set forth in the second feature, the makeup simulation system as the invention as set forth in the first feature is characterized in that the top, bottom, leftmost, rightmost of the component parts are specified as feature points to be specified by the plot. Includes 4 points.

An invention according to a third feature is the makeup simulation system according to the first or second feature, wherein
Attribute information input receiving means for receiving input of attribute information of the face image;
The contour drawing means provides a makeup simulation system characterized by drawing the contour of the component part in consideration of the input attribute information.

  According to the invention of the third aspect, the makeup simulation system according to the first or second aspect includes attribute information input receiving means for receiving input of attribute information of the face image, and the contour drawing means The contour drawing of the component part is drawn in consideration of the input attribute information.

An invention according to a fourth feature is the makeup simulation system according to any one of the first to third features, wherein
An eyebrow drawing means for predicting and drawing an eyelid outline from the eye outline drawn by the outline drawing means when the specified component part is an eye;
Providing a makeup simulation system characterized by comprising:

  According to the invention of the fourth aspect, the makeup simulation system, which is the invention of any of the first to third features, is drawn by the contour drawing means when the designated component is an eye. The eyelid drawing means is further provided to predict and draw the eyelid outline from the eye outline.

An invention according to a fifth feature is the makeup simulation system according to any of the first through fourth features, wherein
Drawing contour fine adjustment means for performing fine adjustment of the drawn contour when there is a deviation between the drawn contour and the contour of the component part of the face image;
Providing a makeup simulation system characterized by comprising:

  According to the invention as set forth in the fifth aspect, in the makeup simulation system according to any one of the first to fourth aspects, there is a gap between the drawn outline and the outline of the component of the face image. And a drawing contour fine adjustment means for finely adjusting the drawn contour.

An invention according to a sixth feature is the makeup simulation system according to any one of the first to fifth features, wherein
Screen sharing means for sharing a screen with a terminal connected via a network;
A remote control unit that can operate the shared screen from a terminal connected via the network;
Providing a makeup simulation system characterized by comprising:

  According to the invention of the sixth aspect, the makeup simulation system, which is the invention of any of the first to fifth features, comprises: a screen sharing means for sharing a screen with a terminal connected via a network; And a remote control unit that can operate the shared screen from a terminal connected via a network.

An invention according to a seventh feature is a makeup simulation method for performing makeup simulation on a face image,
A configuration part type specification receiving step of receiving specification of a type of configuration part of the face image;
A feature point specification accepting step of accepting specification of a plurality of feature points by plotting on the contour of the component part received the specification;
An outline drawing step of drawing an outline of the component part based on a plurality of feature points specified by the plot;
Providing a makeup simulation method characterized by comprising:

An invention according to an eighth feature relates to a makeup simulation system for performing makeup simulation on a face image,
Configuration part type specification receiving step for receiving specification of the type of configuration part of the face image,
A feature point specification step of accepting specification of a plurality of feature points by plotting on the contour of the component part received the specification;
An outline drawing step of drawing an outline of the component part based on a plurality of feature points specified by the plot;
Provide a makeup simulation program for executing the program.

  According to the present invention, it is possible to provide a makeup simulation system, a makeup simulation method, and a makeup simulation program for performing makeup simulation on a face image by accepting specification of a plurality of feature points by plotting.

FIG. 1 is a schematic diagram of a preferred embodiment of the present invention. FIG. 2 is a diagram showing the relationship between functional blocks of the terminal 100 and each function. FIG. 3 is a flowchart of the makeup simulation process of the terminal 100. FIG. 4 is a diagram showing the relationship between the functional blocks of the terminal 100 having the attribute information input acceptance function and the respective functions. FIG. 5 is a flowchart of makeup simulation processing of the terminal 100 provided with the attribute information input acceptance function. FIG. 6 is a diagram showing the relationship between the functional blocks of the terminal 100 having the eyelid drawing function and the respective functions. FIG. 7 is a flowchart of makeup simulation processing of the terminal 100 having the eyelid drawing function. FIG. 8 is a diagram showing the relationship between the functional blocks of the terminal 100 having the contour fine adjustment function and the respective functions. FIG. 9 is a flowchart of makeup simulation processing of the terminal 100 provided with the contour fine adjustment function. FIG. 10 is a diagram showing the relationship between functional blocks of the connection source terminal 100a and the connection destination terminal 100b having the screen sharing function and the remote control function. FIG. 11 is a flowchart of the connection source terminal 100a and the connection destination terminal 100b having the screen sharing function and the remote control function. FIG. 12 is a diagram showing the relationship between the functional blocks of the terminal 100 and the server 300 and the functions when the server is provided with the contour database. FIG. 13 is a flowchart of the terminal 100 and the server 300 when the server is provided with a contour database. FIG. 14 is an example of the display of the face image acquisition screen on the terminal 100. FIG. 15 is an example of a display of a component part specification specification acceptance screen on the terminal 100. FIG. 16 is an example of the display of the feature point specification reception screen on the terminal 100. FIG. 17 is an example of display of a contour drawing screen on the terminal 100. FIG. 18 is an example of the display of the makeup drawing screen on the terminal 100. FIG. 19 is an example of the display of the attribute information input acceptance screen on the terminal 100. FIG. 20 is an example of display of a contour drawing screen of eyelids on the terminal 100. FIG. 21 is an example of the display of the contour fine adjustment screen on the terminal 100. FIG. 22 is an example of display when screen sharing and remote control are performed between the connection source terminal 100a and the connection destination terminal 100b. FIG. 23 shows an example of the data structure of the contour database. FIG. 24 shows an example of the data structure of the contour database in the case of classification for each attribute information.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

[System overview of makeup simulation system]
The outline of the present invention will be described based on FIG. As shown in FIG. 2, the terminal 100 includes a camera unit 110, an input unit 120, an output unit 130, a storage unit 140, and a control unit 150. The input unit 120 cooperates with the control unit 150 to implement a component part specification specification acceptance module 121 and a feature point specification reception module 122. Further, the output unit 130 cooperates with the control unit 150 to implement the contour drawing module 131.

  When performing contour drawing processing for makeup simulation, the terminal 100 first acquires a face image (step S01). Here, a face image for performing makeup simulation is determined by performing imaging with the camera unit 110 or selecting an image stored in the terminal 100.

  Next, the configuration part type specification receiving module 121 of the terminal 100 receives specification of a configuration part from the user (step S02). Here, the user is allowed to select the type of face configuration part to be subjected to the makeup simulation from options such as the right eye, the left eye, the nose, the mouth, and the entire face. By specifying the component parts, it is possible to perform appropriate processing according to each part thereafter.

  Next, the feature point specification receiving module 122 receives specification of a feature point from the user (step S03). The feature points are, for example, four points at the leftmost, rightmost, topmost, and bottom of the component parts, and the like, which are a plurality of assisting points for deducing the outline from the face image. Here, the specification of feature points is performed by the user plotting points on the face image.

  Finally, the contour drawing module 131 performs contour analogy based on the specified plurality of feature points, and draws a contour on the face image (step S04). In the analogy processing of the contour of the component part, a database in which the contour data of the component part is stored is prepared, and the feature data of the designated contour is collated with the database, whereby the contour data matching the feature point is obtained. get. As a result, it is possible to draw an appropriate contour and perform makeup simulation at an appropriate position in accordance with the contour only by plotting a feature point without requiring a special device.

[Description of each function]
FIG. 2 is a diagram showing the relationship between functional blocks of the terminal 100 and each function. The camera unit 110, the input unit 120, the output unit 130, the storage unit 140, and the control unit 150. The input unit 120 cooperates with the control unit 150 to implement a component part specification specification acceptance module 121 and a feature point specification reception module 122. Further, the output unit 130 cooperates with the control unit 150 to implement the contour drawing module 131.

  The terminal 100 may be a general information terminal that allows a user to capture an image by a camera, and is an information device or an electrical appliance having a function to be described later. General information appliances such as tablet PCs, notebook PCs, wearable devices, or PCs with displays that can be connected to mobile phones, smartphones, camera functions, or external cameras such as web cameras etc. It may be an appliance. The smartphone illustrated as the terminal 100 is just an example.

  The terminal 100 includes a camera in the camera unit 110. The camera unit 110 converts a captured image into digital data and stores the digital data in the storage unit 140. The captured image may be a still image or a moving image. In the case of a moving image, the control unit 150 functions to cut out part of the moving image and store it as a still image in the storage unit 140. It is also possible to Further, it is assumed that an image obtained by photographing is a precise image having an amount of information required by the user, and that the number of pixels and the image quality can be specified.

  The input unit 120 cooperates with the control unit 150 to implement a component part specification specification acceptance module 121 and a feature point specification reception module 122. Further, it is assumed that the camera unit 110 has a function necessary to designate imaging by the camera unit 110. As an example, it is possible to provide a liquid crystal display for realizing a touch panel function, a keyboard, a mouse, a pen tablet, hardware buttons on the device, a microphone for voice recognition, and the like. The present invention is not particularly limited in function by the input method.

  The output unit 130 implements the contour drawing module 131 in cooperation with the control unit 150. In addition, it has a function necessary to display a captured image or a simulation image. As an example, forms such as a liquid crystal display, a display of a PC, a projection on a projector, and the like can be considered. The present invention is not particularly limited in function by the output method.

  The storage unit 140 includes a storage unit of data using a hard disk or a semiconductor memory, and stores captured moving images and still images, data necessary for contour analogization processing, data of a makeup method, and the like. Also, a contour database is stored in which contour data is stored as needed.

  In addition, the control unit 150 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like.

Makeup simulation processing
FIG. 3 is a flowchart of the makeup simulation process of the terminal 100. The process executed by the module of each device described above will be described together with this process.

  First, the terminal 100 acquires a face image to be subjected to makeup simulation (step S101). Here, a face image for performing makeup simulation is determined by performing imaging with the camera unit 110 or selecting an image stored in the terminal 100.

  FIG. 14 is an example of the display of the face image acquisition screen on the terminal 100. In FIG. 14, for example, it is assumed that imaging by the camera unit 110 is started by selecting the camera icon 1401. You may shoot by yourself using the in-camera on your smartphone or tablet, or use the back-side camera (out-camera) to have someone else shoot you, or shoot yourself in a mirror You may Alternatively, it is also possible to select an image stored in the library of the terminal 100 by selecting the photo icon 1402. When the determination button 1403 is selected, an image on which makeup simulation is to be performed is determined. When the cancel button 1404 is selected, the displayed image is discarded, and it is assumed that the image is taken or selected from the library again.

  Next, the configuration part type specification receiving module 121 of the terminal 100 receives specification of a configuration part from the user (step S102). Here, the user is allowed to select the type of face configuration part to be subjected to the makeup simulation from options such as the right eye, the left eye, the nose, the mouth, and the entire face. By specifying the component parts, it is possible to perform appropriate processing according to each part thereafter.

  FIG. 15 is an example of a display of a component part specification specification acceptance screen on the terminal 100. In FIG. 15, as the radio button 1501, the user is allowed to select constituent parts from the eyes (both eyes), right eye, left eye, nose, mouth, cheeks, and entire face. The options listed here are merely examples and may be set according to the system. In FIG. 15, the right eye is selected by the radio button. When the decision button 1502 is selected, a component part to perform makeup simulation is decided. The decision button 1502 may be grayed out until a radio button selection is made. When the cancel button 1503 is selected, the designation of the component part is discarded, and the component part selection screen is ended. In this case, the process may return to step S101 again to acquire a face image.

  Next, the feature point specification receiving module 122 of the terminal 100 receives specification of a feature point from the user (step S103). The feature points are, for example, four points at the leftmost, rightmost, topmost, and bottom of the component parts, and the like, which are a plurality of assisting points for deducing the outline from the face image. Here, the specification of feature points is performed by the user plotting points on the face image.

  FIG. 16 is an example of the display of the feature point specification reception screen on the terminal 100. In FIG. 16, it is illustrated that the terminal 100 is a smart phone, and a characteristic point is plotted using a touch panel. The right eye, which is the component part selected in step S102, is displayed, and the user is allowed to input four feature points of the leftmost, the rightmost, the topmost, and the bottommost. The user plots the leftmost feature point 1601, the rightmost feature point 1602, the topmost feature point 1603, and the bottommost feature point 1604. The feature points may be plotted by the user one by one in order, or temporary feature points may be displayed on the image to allow the user to perform adjustment. When the determination button 1605 is selected, the feature point is determined. When the cancel button 1606 is selected, the input of the feature point is discarded and the input screen of the feature point is ended. In this case, the process may return to step S102 to select a component part.

  Next, the contour drawing module 131 performs contour analogy based on the specified plurality of feature points, and draws a contour on the face image (step S104). In the analogy processing of the contour of the component part, a database in which the contour data of the component part is stored is prepared, and the feature data of the designated contour is collated with the database, whereby the contour data matching the feature point is obtained. get.

  FIG. 23 shows an example of the data structure of the contour database. Contour data is stored for each component, such as the right eye, left eye, nose, mouth, whole face, etc. The contour data includes data necessary to draw the contour of the component part together with the face image. The data necessary for drawing the contour may be data of pixel positions of the contour or may be vector data. The contour drawing module 131 collates the designated feature points with each contour data to obtain suitable contour data, and performs drawing on the output unit 130 of the terminal 100 based on the contour data. As the data structure of the contour database, attribute information such as gender, age, race, etc. may be stored together with the contour data. Details of the attribute information will be described later.

  FIG. 17 is an example of display of a contour drawing screen on the terminal 100. An outline is drawn on the image in the display field 1705 based on the inferred outline data in accordance with the input feature points. When the determination button 1701 is selected, contour data is determined. When the cancel button 1702 is selected, the contour data is discarded, and the contour drawing confirmation screen is ended. In this case, the process may return to step S103 to input feature points. If the change button 1703 is selected, collation with the contour database is performed again, another contour data is acquired and displayed. Alternatively, even in the stage of step S104, a plurality of contour data are acquired as the first candidate, the second candidate, the third candidate, and switching to display of the next candidate contour data by selecting the change button 1703 Good. When the fine adjustment button 1704 is selected, the user is allowed to perform fine adjustment of the contour. Details of the fine adjustment of the contour will be described later.

  Finally, the terminal 100 draws a makeup image along the outline on the output unit 130 (step S105). The user is made to select a desired makeup method, and drawing is performed according to the contour determined in step S104.

  FIG. 18 is an example of the display of the makeup drawing screen on the terminal 100. The makeup method selected by the user is drawn according to the contour. In FIG. 18, since the right eye is selected as the component part, the user is allowed to select the eye makeup. When the user selects a desired makeup method from the makeup method display column 1801 and selects the determination button 1802, the makeup method is reflected on the image of the display column 1804. When the cancel button 1803 is selected, the selected makeup method is discarded and the makeup drawing screen is ended. In this case, the process may return to step S104 to check the contour. Further, in FIG. 18, the image displayed in the display field 1804 is only the right eye, but a makeup method may be drawn on the image of the entire face. In addition, by repeating step S102 to step S105, makeup simulation may be sequentially performed for each component part, and makeup simulation of the entire face may be finally completed.

[Attribute information input acceptance processing]
FIG. 4 is a diagram showing the relationship between the functional blocks of the terminal 100 having the attribute information input acceptance function and the respective functions. In addition to the functions described with reference to FIG. 2, the input unit 120 and the control unit 150 cooperate to realize the attribute information input acceptance module 123.

  FIG. 5 is a flowchart of makeup simulation processing of the terminal 100 provided with the attribute information input acceptance function. The face image acquisition (step S201), the configuration part type specification acceptance (step S203), the feature point specification acceptance (step S204), the contour drawing (step S205), and the makeup drawing along the contour (step S206) in FIG. This corresponds to face image acquisition (step S101), component part specification specification acceptance (step S102), feature point specification reception (step S103), contour drawing (step S104), and makeup drawing along the contour (step S105) in FIG. . Therefore, here, the attribute information input acceptance in step S202 will be mainly described.

  When performing attribute information input acceptance processing, after acquiring the face image in step S201, the attribute information input acceptance module 123 receives input of attribute information from the user (step S202).

  FIG. 19 is an example of the display of the attribute information input acceptance screen on the terminal 100. The attribute information here is information of a person of a face image, such as gender, age, race, etc., which is considered to affect the contour of the component part of the face. In FIG. 19, the radio button 1901 is used to select whether the gender is male or female. The radio button 1902 is used to select one of the ages: teens, 20s, 30s, 40s, 50s, 60s or more. A radio button 1903 is used to select whether the race is Japanese or not. These options do not necessarily have to be described above, and may be appropriately set according to the system. In addition to gender, age, race, and any other element that affects the contour of the constituent part of the face, it may be added as attribute information. In addition, as a system, it is not necessary to input this attribute information without fail, and it may be an arbitrary input of the user. When the user selects attribute information from radio button 1901, radio button 1902, and radio button 1903 and selects determination button 1904, attribute input is accepted. When the cancel button 1905 is selected, the selected attribute information is discarded, and the attribute information input screen is ended. In this case, the process may return to step S201 again to acquire a face image, or may proceed to the next step S203.

  The processing of configuration part type specification acceptance in step S203 and the processing of feature point specification acceptance in step S204 is the same as the processing of configuration part type specification acceptance in step S102 of FIG. 3 and feature point specification reception in step S103.

  When performing contour drawing in step S205, the contour drawing module 131 draws a contour based on the input attribute information. FIG. 24 shows an example of the data structure of the contour database in the case of classification for each attribute information. The data of the contour database shown in FIG. 23 is classified for each attribute information. In FIG. 24, the contour data of the right eye of Japanese teenagers are classified according to whether they are male, female or gender unknown. If the attribute information input in step S202 is a Japanese male of teens and the designation of the component is a right eye, the right eye of "male + teen + Japanese" at the top of FIG. The database of contour data may first be queried. Therefore, in the database of the contour data of the right eye, which is considered to have the next appropriate data, such as "Gender unknown + teen + Japanese" or "male + 20 + Japanese", when an appropriate thing is not found. We shall make a query. The order in which the database for each attribute information is to be inquired can be set for each system. Also, if it is not possible to find suitable contour data, contour data that is considered closest may be used. At this time, in the case where there is a plurality of candidate contour data, it is possible to prioritize and select those having similar attribute information.

The process of makeup drawing along the contour of the last step S206 is the same as the process of step S105 of FIG. 3.

  As described above, by receiving the input of attribute information of the face image that is considered to affect the contour of the component part of the face and using it for the contour drawing, it is possible to draw the contour of the component part with higher accuracy.

[瞼 drawing process]
FIG. 6 is a diagram showing the relationship between the functional blocks of the terminal 100 having the eyelid drawing function and the respective functions. In addition to the functions described in FIG. 2, the output unit 130 and the control unit 150 cooperate to realize the eyelid drawing module 132.

  FIG. 7 is a flowchart of makeup simulation processing of the terminal 100 having the eyelid drawing function. The face image acquisition (step S301), the configuration part type specification acceptance (step S302), the feature point specification acceptance (step S303), the contour drawing (step S304), and the makeup drawing along the contour (step S307) in FIG. This corresponds to face image acquisition (step S101), component part specification specification acceptance (step S102), feature point specification reception (step S103), contour drawing (step S104), and makeup drawing along the contour (step S105) in FIG. . Therefore, here, the description will be made focusing on step S305 and step S306.

  When performing the eyelid drawing process, after drawing the contour in step S304, the eyelid drawing module 132 confirms whether the component designated in step S302 is an eye (step S305). If the designated component is not the eye, the process proceeds to step S307.

  If the specified component is an eye, the eyelid drawing module 132 performs eyelid outline drawing (step S306).

  FIG. 20 is an example of display of a contour drawing screen of eyelids on the terminal 100. Here, an example in which the right eye is specified as a component part is shown. The eyelid drawing module 132 draws the outline of the eyelid on the image of the display field 2005 by analogizing the outline of the eyelid of the right eye according to the outline of the right eye already drawn. FIG. 20 shows an example in which the outline of the eyebrow is drawn by a broken line. When the determination button 2001 is selected, the outline data of the eyebrows is determined. When the cancel button 2002 is selected, the eyelid contour data is discarded, and the eyelid contour drawing confirmation screen is ended. In this case, the process may return to step S304 to perform contour drawing of the right eye. When the change button 2003 is selected, collation with the contour database is performed again, and contour data of another eyelid is acquired and displayed. Alternatively, at the stage of step S306, a plurality of eyelid contour data are acquired as the first candidate, the second candidate, the third candidate, and selection of the change button 2003 displays the contour data of the next candidate eyelid You may switch to When the fine adjustment button 2004 is selected, the user is allowed to perform fine adjustment of the eyelid contour. Details of the fine adjustment of the outline of the eyebrow will be described later.

  Here, the eyelid contour analogy processing by the eyelid drawing module 132 may be performed by predicting the eyelid thickness based on the eye contour, or the eyelid contour as attached information in the eye data of the contour database. Data may be registered, and the data may be acquired.

  Finally, a makeup image is drawn along the drawn contours of the right eye and eyebrows (step S307). The user is made to select a desired makeup method, and drawing is performed according to the contours determined in step S304 and step S306.

  In this way, by drawing the eyebrow which is an important part for eye makeup, it is possible to provide a makeup simulation system more useful to the user.

[Contour fine adjustment process]
FIG. 8 is a diagram showing the relationship between the functional blocks of the terminal 100 having the contour fine adjustment function and the respective functions. In addition to the functions described in FIG. 2, the input unit 120 and the control unit 150 cooperate to realize the contour fine adjustment module 124.

  FIG. 9 is a flowchart of makeup simulation processing of the terminal 100 provided with the contour fine adjustment function. The face image acquisition (step S401), the configuration part type specification acceptance (step S402), the feature point specification acceptance (step S403), the contour drawing (step S404), and the makeup drawing along the contour (step S407) in FIG. This corresponds to face image acquisition (step S101), component part specification specification acceptance (step S102), feature point specification reception (step S103), contour drawing (step S104), and makeup drawing along the contour (step S105) in FIG. . Therefore, the description here will focus on step S405 and step S406.

  When the contour fine adjustment process is performed, after the contour drawing in step S404, the contour fine adjustment module 124 confirms whether or not to finely adjust the contour (step S405). Here, whether or not to perform fine adjustment may be selected by the user, or image analysis is performed to determine whether there is a deviation between the contour drawn in S404 and the contour of the component part of the actual face image, Fine adjustment may be performed when there is a deviation. If fine adjustment of the contour is to be performed, the process proceeds to step S406. If fine adjustment of the contour is not to be performed, the process proceeds to step S407.

  The contour fine adjustment module 124 finely adjusts the contour of the component part based on the input from the user (step S406). FIG. 21 is an example of the display of the contour fine adjustment screen on the terminal 100. Here, an example in which the right eye is specified as a component part is shown. A fine adjustment of the outline is performed by designating and moving the point on the outline of the right eye drawn in the display field 2103 to the user. The point to be designated and moved does not have to be the previously input feature point, and it is possible to designate and move a certain point on the contour. If a user moves a point, the outline fine adjustment module 124 connects smoothly the point after movement and the point on the original outline in the vicinity of the point and redraws the outline; Make the line uninterrupted. In addition, the enlargement button 2104 and the reduction button 2105 may be provided, and the user may freely enlarge or reduce the image in the display field 2103 to facilitate fine adjustment of the outline. When the enter button 2101 is selected, contour data on which fine adjustment has been performed is stored. When the cancel button 2102 is selected, the finely adjusted contour data is discarded, and the contour fine adjustment screen is ended. In this case, the process may return to step S404 to perform contour drawing. Although the example of normal outline fine adjustment was shown in FIG. 21, it is also the same as when processing the outline fine adjustment of eyebrows.

  Finally, the user draws a makeup image of a desired makeup method along the contour (step S407).

  As described above, it is possible to provide a makeup simulation system with higher accuracy by drawing the makeup image after performing the fine adjustment processing of the contour.

[Remote control processing by screen sharing]
FIG. 10 is a diagram showing the relationship between blocks of the connection source terminal 100a and the connection destination terminal 100b having the screen sharing function and the remote control function, and the respective functions. The connection source terminal 100 a and the connection destination terminal 100 b each include a communication unit 160 in addition to the functions of the terminal 100 described in FIG. 2. The communication unit 160 implements the screen sharing module 161 and the remote control module 162 in cooperation with the control unit 150. It is assumed that the terminals are connected via the communication network 200. The communication network 200 may be a public communication network or a dedicated communication network. Here, it is assumed that the connection source terminal 100a is a terminal that executes a screen sharing request to the connection destination terminal 100b on the network via the communication network 200.

  The communication unit 160 may be a device for enabling communication with other devices, for example, an IMT-2000 standard such as a WiFi (Wireless Fidelity) compliant device compliant with IEEE 802.11 or a third generation or fourth generation mobile communication system. A wireless device or the like conforming to It may be a wired LAN connection.

  FIG. 11 is a flowchart of the connection source terminal 100a and the connection destination terminal 100b having the screen sharing function and the remote control function. The process executed by the module of each device described above will be described together with this process. Here, although the description is made that the connection source terminal 100a and the connection destination terminal 100b directly communicate with each other, the server may be interposed between them depending on the system.

  First, the screen sharing module 161 of the connection source terminal 100a notifies the connection destination terminal 100b of the start of screen sharing (step S501). Here, only notification of the start of screen sharing from the connection source terminal 100a to the connection destination terminal 100b is illustrated for simplification of the flow, but authentication processing for maintaining security, screen sharing approval on the connection destination terminal 100b side It is also assumed that processing necessary for the normal screen sharing system, such as processing and response processing from the connection destination terminal 100b to the connection source terminal 100a, is also performed. The screen sharing module 161 of the connection destination terminal 100b performs the response process from the connection destination terminal 100b to the connection source terminal 100a.

  Next, the screen sharing module 161 of the connection source terminal 100a creates shared data for performing screen sharing (step S502). When the makeup simulation system is performed on the connection source terminal 100a after the screen sharing process is started, in this shared data creation process, the respective flowcharts of FIG. 3, FIG. 5, FIG. 7, and FIG. At intervals, the screen data of the connection source terminal 100a is converted into a format suitable for display on the connection destination terminal 100b as shared data.

  Next, the screen sharing module 161 of the connection source terminal 100a transmits the created shared data to the connection destination terminal 100b (step S503).

  The screen sharing module 161 of the connection destination terminal 100b receives the shared data (step S504), and displays the shared screen based on the received shared data (step S505).

  Here, the remote control module 162 of the connection destination terminal 100b confirms whether or not the operation is performed on the shared screen (step S506).

  If the operation is not performed, the process proceeds to step S508. When the operation is performed, the remote control module 162 of the connection destination terminal 100b receives an operation input from the user (step S507). It is assumed that the operation on the connection destination terminal 100b here is equivalent to the operation that can be performed on the input unit 120 of the connection source terminal 100a.

  Next, the remote control module 162 of the connection destination terminal 100b transmits the operation data to the connection source terminal 100a (step S508). When the operation in step S507 is not performed, only the response may be transmitted instead of transmitting the operation data.

  The screen sharing module 161 of the connection source terminal 100a receives the operation data from the connection destination terminal 100b (step S509).

  Then, the screen sharing module 161 of the connection source terminal 100a updates the display of the screen based on the received operation data (step S510). For example, when fine adjustment of the outline is performed at the connection destination terminal 100b, the screen after the fine adjustment is displayed on the output unit 130 of the connection source terminal 100a.

  Thereafter, the screen sharing module 161 of the connection source terminal 100a checks whether to end the screen sharing (step S511).

  If the screen sharing is not finished, the process returns to step S502 and continues. When the screen sharing is finished, the end of the screen sharing is notified to the connection destination terminal 100b, and the screen sharing process is finished (step S512).

  Here, an example is shown in which only the connection source terminal 100a confirms whether the screen sharing is to be ended, but the connection destination terminal 100b may also confirm whether the screen common process is to be ended. Further, according to the system or the user's request, the screen of the connection destination terminal 100b may be shared instead of sharing the screen of the connection source terminal 100a.

  FIG. 22 is an example of display when screen sharing and remote control are performed between the connection source terminal 100a and the connection destination terminal 100b. The connection source terminal 100 a and the connection destination terminal 100 b are sharing the screen via the communication network 200. The user 2201 of the connection source terminal 100a selects the right eye as a component part and tries to execute a makeup simulation, but contour drawing is not successful, and the above-mentioned fine adjustment processing of the contour can not be considered as well. . In such a case, it becomes possible for the user 2202 of the connection destination terminal 100b to finely adjust the contour on the screen of the connection destination terminal 100b to perform makeup simulation in accordance with the desired contour. Here, the user 2202 may be a friend of the user 2201 or the like, may be a help desk or an operator of a makeup simulation system, or an expert such as a makeup artist who gives advice on makeup simulation. It may be.

  Thus, by performing the remote control process by screen sharing, it is possible to provide a makeup simulation system that is more convenient for the user.

  [A cosmetic simulation system with a contour database on the server]

  FIG. 12 is a diagram showing the relationship between the functional blocks of the terminal 100 and the server 300 and the functions when the server is provided with the contour database. Here, an example in which the terminal 100 is provided with all the aforementioned modules is illustrated. It is assumed that the terminal 100 and the server 300 are connected via the communication network 200. The communication network 200 may be a public communication network or a dedicated communication network. The server 300 includes a communication unit 310, a control unit 320, and a storage unit 330. The communication unit 310 can communicate with other devices by wire or wirelessly. The control unit 320 includes a CPU, a RAM, a ROM, and the like. The storage unit 330 includes a storage unit of data such as a hard disk or a semiconductor memory, and further includes a contour database 331.

  FIG. 13 is a flowchart of the terminal 100 and the server 300 when the server is provided with a contour database. Face image acquisition in FIG. 13 (step S601), attribute information input acceptance (step S602), configuration part type specification acceptance (step S603), feature point specification acceptance (step S604), contour drawing (step S611), along the contour The makeup drawing (step S612) is the face image acquisition in FIG. 3 (step S101), the attribute information input acceptance in FIG. 5 (step S202), the configuration part type specification acceptance in FIG. 3 (step S102), and the feature point specification acceptance ( Step S103) corresponds to contour drawing (step S104) and cosmetic drawing along the contour (step S105). Therefore, the description here will focus on steps S605 to S610.

  After receiving the feature point specification in step S604, the contour drawing module 131 transmits the types of component parts and data of a plurality of specified feature points to the server 300 (step S605). If attribute information is input, the attribute information is also transmitted. When a face image is also used in the analogy process of the contour of the component part, the face image may be transmitted together.

  The server 300 receives data necessary for the contour analogy process from the terminal 100 (step S606).

  Next, the server 300 refers to the contour database 331 based on the received data, and performs a contour analogization process (step S 607). By collating the feature points of the designated contour with the database, contour data matching the feature points is acquired. At this time, if the attribute information exists, as described above in the processing of FIG. 5, the contour data is preferentially acquired from the database corresponding to the attribute information.

  The server 300 performs contour data creation processing based on the inferred contour data (step S608). The actual size of the face image used for the makeup simulation and the image registered in the outline database 331 may differ, and in this case, the feature points of the face image and the outline data of the outline database 331 are used. The comparison needs to be done by scaling. When the resizing is performed to perform contour analogization processing, the contour data is scaled according to the size suitable for the size of the actual face image.

  Next, the server 300 transmits the created contour data to the terminal 100 (step S609).

  The contour drawing module 131 of the terminal 100 receives contour data from the server 300 (step S610).

  Next, the contour drawing module 131 draws a contour on the face image based on the received contour data (step S611).

  Finally, the terminal 100 draws a makeup image along the outline on the output unit 130 (step S612).

  As described above, by providing the outline database in the server 300, it is not necessary to provide the database in the terminal 100, and therefore, the capacity of the storage unit 140 of the terminal 100 is not compressed. Further, since contour data of the database of the server 300 can be increased as needed, it is possible to provide a makeup simulation system with higher accuracy.

  The above-described means and functions are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program. The program is provided, for example, in the form of being recorded on a computer readable recording medium such as a flexible disk, a CD (CD-ROM or the like), a DVD (DVD-ROM, DVD-RAM or the like), a compact memory or the like. In this case, the computer reads the program from the recording medium, transfers the program to an internal storage device or an external storage device, stores it, and executes it. Alternatively, the program may be recorded in advance in a storage device (recording medium) such as, for example, a magnetic disk, an optical disk, or a magneto-optical disk, and may be provided from the storage device to the computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects according to the present invention are limited to those described in the embodiments of the present invention is not.

  100 terminals, 100a connection source terminals, 100b connection destination terminals, 200 communication networks, 300 servers,

Claims (8)

A makeup simulation system that performs makeup simulation on a face image, and
Attribute information input receiving means for receiving input of attribute information that is information of a person of the face image that is considered to affect the contour of the constituent parts of the face;
Configuration part type specification receiving means for receiving specification of the type of configuration part of the face image;
Feature point specification receiving means for receiving specification of a plurality of feature points by plotting on the contour of the component part received the specification;
An outline drawing unit that draws the outline of the component part in consideration of the input attribute information based on a plurality of feature points specified by the plot;
A cosmetic simulation system characterized by comprising:   The makeup simulation system according to claim 1, wherein the attribute information includes any of gender, age, and race.   The makeup simulation system according to any one of claims 1 or 2, characterized in that the feature points to be specified by the plot include the top, bottom, leftmost, and rightmost four points of the component part. . An eyebrow drawing means for predicting and drawing an eyelid outline from the eye outline drawn by the outline drawing means when the specified component part is an eye;
The makeup simulation system according to any one of claims 1 to 3, further comprising: Drawing contour fine adjustment means for performing fine adjustment of the drawn contour when there is a deviation between the drawn contour and the contour of the component part of the face image;
The makeup simulation system according to any one of claims 1 to 4, comprising: Screen sharing means for sharing a screen with a terminal connected via a network;
A remote control unit that can operate the shared screen from a terminal connected via the network;
The makeup simulation system according to any one of claims 1 to 5, comprising: A makeup simulation method executed by a makeup simulation system that performs makeup simulation on a face image,
An attribute information input accepting step of receiving an input of attribute information which is information of a person of the face image considered to affect the contour of a constituent part of the face;
A configuration part type specification receiving step of receiving specification of a type of configuration part of the face image;
A feature point specification accepting step of accepting specification of a plurality of feature points by plotting on the contour of the component part received the specification;
An outline drawing step of drawing an outline of the component part in consideration of the input attribute information based on a plurality of feature points specified by the plot;
A cosmetic simulation method comprising: A makeup simulation system that performs makeup simulation on a face image,
An attribute information input accepting step of receiving an input of attribute information which is information of a person of the face image considered to affect the contour of a component of the face;
Configuration part type specification receiving step for receiving specification of the type of configuration part of the face image,
A feature point specification step of accepting specification of a plurality of feature points by plotting on the contour of the component part received the specification;
An outline drawing step of drawing an outline of the component part in consideration of the input attribute information based on a plurality of feature points specified by the plot;
Computer readable makeup simulation program for running.