JP2007216000A - Makeup simulation system, makeup simulation device, makeup simulation method and makeup simulation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a makeup simulation system, a makeup simulation device, a makeup simulation method and a makeup simulation program, accurately achieving makeup for the face of a user included in a dynamic image with small processing load. <P>SOLUTION: This makeup simulation system includes: a photographing means 2 for outputting a photographed dynamic image; a control means 8 for image-processing the output dynamic image and outputting the same; and a display means 7 for displaying the dynamic image output from the control means 8, wherein the control means 8 includes: a face recognition processing means for recognizing the face of a user from the dynamic image based on a predetermined tracking point; and a makeup processing means for conducting predetermined makeup for the face of a user included in the dynamic image based on the tracking point and outputting the same to the display means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a makeup simulation system, a makeup simulation device, a makeup simulation method, and a makeup simulation program, and in particular, a makeup simulation system, a makeup simulation device for applying makeup to a user's face included in a moving image, The present invention relates to a makeup simulation method and a makeup simulation program.

2. Description of the Related Art Conventionally, a technique for simulating a face after makeup on a computer without actually applying makeup for sales of products for applying makeup is known (see, for example, Patent Document 1). However, in Patent Document 1, since the simulation result is displayed as a still image, the face after makeup when the user's facial expression is changed cannot be easily confirmed. Accordingly, development of a technique for simulating makeup in a moving image that captures a change in the facial expression of a user is underway (see, for example, Patent Document 2).
JP 2001-346627 A JP 2003-44837 A

  However, the makeup simulation apparatus described in Patent Document 2 identifies a change in the user's facial expression by a pixel area corresponding to the mouth and both eyes, and tracks the pixel area by template matching to calculate a makeup area to which face makeup is applied. (See paragraph [0028], for example). As described above, tracking the change of the user's facial expression in the pixel area corresponding to the mouth and both eyes has a problem that the processing load on the computer is large and it is difficult to accurately deal with the case where the eyes are closed.

  The present invention has been made in view of the above points. A makeup simulation system, a makeup simulation apparatus, a makeup simulation method, and a makeup simulation system that can accurately make up a user's face included in a moving image with a small processing load. An object is to provide a makeup simulation program.

  In order to solve the above problems, the present invention is a makeup simulation system for applying makeup to a moving image obtained by photographing a user's face, the photographing means for photographing a user's face and outputting a moving image, A control unit that receives the moving image output from the photographing unit, performs image processing on the moving image, and outputs the moving image; and a display unit that displays the moving image output from the control unit, and the control unit includes: Face recognition processing means for recognizing a user's face from a moving image based on a predetermined tracking point, and applying predetermined make-up to the user's face included in the moving image based on the tracking point and outputting to the display means And makeup processing means.

  In the present invention, the face recognition processing means for recognizing the user's face from the moving image based on a predetermined tracking point, and the display means by applying a predetermined makeup to the user's face included in the moving image based on the tracking point. By providing the makeup processing means for outputting, the user's face is recognized from the moving image based on the tracking point with a small processing load, and the makeup on the user's face included in the moving image is accurately based on the tracking point. Can be applied.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  According to the present invention, it is possible to provide a makeup simulation system, a makeup simulation apparatus, a makeup simulation method, and a makeup simulation program that can accurately make up a user's face included in a moving image with a small processing burden. is there.

  Next, the best mode for carrying out the present invention will be described based on the following embodiments with reference to the drawings. FIG. 1 is an external view of a first embodiment of a makeup simulation apparatus according to the present invention. The makeup simulation apparatus 1 includes a camera 2, an operation panel 4, a printer 5, an illumination 6, and a monitor 7.

  The camera 2 captures a user standing in front of the makeup simulation apparatus 1 and outputs a moving image. The operation panel 4 displays an operation image, accepts an operation from the user, and outputs operation information. The printer 5 prints an image (for example, an image image after makeup) and information (for example, product information for makeup like an image image) displayed on the monitor. The illumination 6 performs dimming after, for example, a makeup simulation is started.

  FIG. 2 is a cross-sectional view of the first embodiment of the makeup simulation apparatus. The makeup simulation apparatus 1 includes a camera 2, an operation panel 4, a printer 5, a monitor 7, and a control unit 8.

  The camera 2 outputs the captured moving image to the control unit 8. The operation panel 4 displays the operation image output from the control unit 8, receives an operation from the user, and outputs operation information to the control unit 8. The printer 5 prints the image and information output from the control unit 8. The monitor 7 displays the moving image (main image) output from the control unit 8. The control unit 8 receives the moving image output from the camera 2 and performs image processing on the moving image as will be described later, thereby applying makeup to the user's face included in the moving image and outputting it to the monitor 7. .

  The makeup simulation apparatus 1 shown in FIGS. 1 and 2 is based on the concept of a mirror essential item for makeup, and has an interactive function with a user. That is, the makeup simulation apparatus 1 has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  The makeup simulation apparatus 1 performs image processing on the moving image output from the camera 2 by the control unit 8, performs makeup on the user's face included in the moving image, and displays the image on the monitor 7 that is a digital mirror. The makeup simulation apparatus 1 can display various product information, beauty information, and an image with makeup applied to the user's face on the monitor 7.

  FIG. 3 is an external view of a second embodiment of the makeup simulation apparatus according to the present invention. In addition, the same code | symbol is attached | subjected about the same part as FIG.

  The makeup simulation apparatus 1 in FIG. 3 includes an operation panel 4, a printer 5, an illumination 6, and a transparent plate 9. The transparent plate 9 transmits light from the outside of the makeup simulation apparatus 1 and transmits light from the inside of the makeup simulation apparatus 1.

  FIG. 4 is a sectional view of a second embodiment of the makeup simulation apparatus. In addition, the same code | symbol is attached | subjected about the same part as FIG. The makeup simulation apparatus 1 shown in FIG. 4 includes a camera 2, a half mirror 3, an operation panel 4, a printer 5, a monitor 7, a control unit 8, and a transparent plate 9.

  The half mirror (semi-transparent mirror) 3 reflects the hit light and transmits a part of the hit light. The camera 2 is installed at a position where the user standing in front of the makeup simulation apparatus 1 can be photographed via the half mirror 3 and the transparent plate 9. The camera 2 is installed at the height of the user's line of sight. The camera 2 captures a user standing in front of the makeup simulation apparatus 1 through the half mirror 3 and the transparent plate 9 and outputs a moving image.

  The monitor 7 is installed at a position where a user standing in front of the makeup simulation apparatus 1 can see through the half mirror 3 and the transparent plate 9. The light output from the monitor 7 is reflected by the half mirror 3 and output to the outside of the makeup simulation apparatus 1 through the transparent plate 9. Therefore, the user can view the moving image displayed on the monitor 7 from the outside of the makeup simulation apparatus 1.

  The makeup simulation apparatus 1 shown in FIGS. 3 and 4 is based on the concept of an indispensable item, which is a mirror, and has an interactive function with the user. That is, the makeup simulation apparatus 1 has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  Since the makeup simulation apparatus 1 has the camera 2 installed at the height of the line of sight, the face of the user standing in front of the makeup simulation apparatus 1 is more natural than the installation at the position of the camera 2 in the first embodiment. Can shoot.

  The makeup simulation apparatus 1 performs image processing on the moving image output from the camera 2 by the control unit 8, performs makeup on the user's face included in the moving image, and displays the image on the monitor 7 that is a digital mirror. The makeup simulation apparatus 1 can display various product information, beauty information, and an image with makeup applied to the user's face on the monitor 7.

  FIG. 5 is an external view of a third embodiment of the makeup simulation apparatus according to the present invention. In addition, the same code | symbol is attached | subjected about the same part as the previous figure. The makeup simulation apparatus 1 in FIG. 5 includes a camera 2, a half mirror 3, an operation panel 4, a printer 5, and an illumination 6.

  The half mirror 3 transmits a part (for example, 50%) of light from the brighter side and reflects the rest of the light (for example, 50%) from the brighter side. Since there is no light in the darker side, the half mirror 3 does not transmit or reflect light from the darker side.

  The illumination 6 is dimmed so that the monitor 7 side of the half mirror 3 becomes bright after the makeup simulation is started. Accordingly, the half mirror 3 provided on the display direction side of the monitor 7 functions as a mirror by reflecting light from the user side (outside the makeup simulation apparatus 1) before the makeup simulation is started.

  After the makeup simulation is started, the half mirror 3 transmits light from the monitor 7 side (inside the makeup simulation apparatus 1) and functions as glass. Therefore, the user can see the moving image displayed on the monitor 7 through the half mirror 3.

  FIG. 6 is a sectional view of a third embodiment of the makeup simulation apparatus. The makeup simulation apparatus 1 shown in FIG. 6 includes a camera 2, a half mirror 3, an operation panel 4, a printer 5, a monitor 7, and a control unit 8. In addition, the same code | symbol is attached | subjected about the same part as the previous figure. The moving image displayed on the monitor 7 passes through the half mirror 3 functioning as glass. The user can view the moving image displayed on the monitor 7 from the outside of the makeup simulation apparatus 1.

  The makeup simulation apparatus 1 shown in FIGS. 5 and 6 is based on the concept of a mirror essential item for makeup, and has an interactive function with the user. That is, the makeup simulation apparatus 1 has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  The makeup simulation apparatus 1 performs image processing on the moving image output from the camera 2 by the control unit 8, performs makeup on the user's face included in the moving image, and displays the image on the monitor 7 that is a digital mirror. The makeup simulation apparatus 1 can display various product information, beauty information, and an image with makeup applied to the user's face on the monitor 7.

  FIG. 7 is an external view of a fourth embodiment of the makeup simulation apparatus according to the present invention. FIG. 8 is a sectional view of a fourth embodiment of the makeup simulation apparatus. In addition, the same code | symbol is attached | subjected about the same part as the previous figure. The makeup simulation apparatus 1 includes a camera 2, a printer 5, an illumination 6, and a touch panel monitor 15.

  The makeup simulation apparatus 1 in FIGS. 7 and 8 is different from the makeup simulation apparatus 1 in FIGS. 1 and 2 in that the touch panel monitor 15 is included instead of the operation panel 4 and the monitor 7. The touch panel monitor 15 functions as the operation panel 4 and the monitor 7.

  The touch panel monitor 15 displays an operation image, receives an operation from the user, and outputs operation information. The touch panel monitor 15 displays the operation image output from the control unit 8, accepts an operation from the user, and outputs operation information to the control unit 8. The touch panel monitor 15 displays the moving image (main image) output from the control unit 8. The control unit 8 receives the moving image output from the camera 2 and performs image processing on the moving image as will be described later, thereby applying makeup to the user's face included in the moving image and outputting it to the touch panel monitor 15. .

  The makeup simulation apparatus 1 shown in FIGS. 7 and 8 is based on the concept of a mirror essential item for makeup, and has an interactive function with the user. That is, the makeup simulation apparatus 1 has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  The makeup simulation apparatus 1 performs image processing on the moving image output from the camera 2 by the control unit 8, performs makeup on the user's face included in the moving image, and displays the makeup on the touch panel monitor 15 that is a digital mirror. . The makeup simulation apparatus 1 can display various product information, beauty information, and an image with makeup applied to the user's face on the touch panel monitor 15.

  The makeup simulation apparatus 1 of the fourth embodiment is a self-sale by providing a display case for displaying a tester product (hereinafter referred to as a tester product) to be used by a user, as shown in FIGS. 9 and 10, for example. It can also be used as a housing for the camera. FIG. 9 is an external view of a fifth embodiment of the makeup simulation apparatus according to the present invention. FIG. 10 is a cross-sectional view of a fifth embodiment of the makeup simulation apparatus. In addition, the same code | symbol is attached | subjected about the same part as the previous figure.

  The makeup simulation apparatus 1 includes a camera 2, a printer 5, an illumination 6, a touch panel monitor 15, a display case 16, and an IC tag reader / writer 17. The makeup simulation apparatus 1 of FIGS. 9 and 10 is different from the makeup simulation apparatus 1 of FIGS. 7 and 8 in that it includes a display case 16 and an IC tag reader / writer 17.

  The display case 16 displays a plurality of tester products. An IC (RFID) tag is attached to the tester product. The IC tag attached to the tester product stores identification information for identifying the tester product. When the user takes out the tester product from the display case 16 and brings it closer to the IC tag reader / writer 17, the IC tag reader / writer 17 reads the identification information of the tester product from the IC tag.

  The IC tag reader / writer 17 transmits the identification information of the tester product read from the IC tag to the control unit 8. The control unit 8 receives the moving image output from the camera 2 and performs image processing on the moving image as will be described later, thereby using the tester product corresponding to the identification information read from the IC tag to include the moving image. Then, an image with makeup applied to the user's face is output to the touch panel monitor 15.

  The correspondence table for associating the tester product with the identification information may be provided in the makeup simulation apparatus 1 or may be provided in another apparatus that can be acquired from the makeup simulation apparatus 1 via a network.

  Further, in the makeup simulation apparatus 1 of FIGS. 9 and 10, an example is shown in which an IC tag is used to identify a tester product, but a bar code, a two-dimensional code, a label, or the like may be used. Further, the makeup simulation apparatus 1 shown in FIGS. 9 and 10 is provided with a mechanism (for example, position recognition using a sensor such as an optical sensor) in the display case 16 so that the taken-out tester product can be understood. You may make it notify to the control part 8. FIG.

  The makeup simulation apparatus 1 shown in FIGS. 9 and 10 is based on the concept of a mirror essential item for makeup, and has an interactive function with the user. That is, the makeup simulation apparatus 1 has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  The makeup simulation apparatus 1 performs image processing on the moving image output from the camera 2 by the control unit 8 and applies makeup to the user's face included in the moving image using a tester product selected from the display case 16 by the user. The applied image is displayed on the touch panel monitor 15 which is a digital mirror. The makeup simulation apparatus 1 can display various product information, beauty information, and an image with makeup applied to the user's face on the touch panel monitor 15. By taking a log of the tester product selected by the user from the display case 16, the makeup simulation apparatus 1 can acquire data such as the user's preference.

  For example, in the makeup simulation apparatus 1 provided with a shelf for displaying not only a tester product but also a product, by displaying the product displayed on the touch panel monitor 15 on the shelf, It can be effectively used as a housing.

  Hereinafter, the makeup simulation apparatus 1 of the first embodiment will be described as an example. FIG. 11 is a hardware configuration diagram of an embodiment of the makeup simulation apparatus. In addition, the same code | symbol is attached | subjected about the same part as FIG.1 and FIG.2.

  The makeup simulation apparatus 1 shown in FIG. 11 includes a camera 2, an operation panel 4, a printer 5, a monitor 7, an arithmetic processing device 10, a memory device 11, a drive device 12, and an auxiliary storage device 13 that are connected to each other via a bus B. Consists of. Note that the arithmetic processing device 10, the memory device 11, the drive device 12, and the auxiliary storage device 13 of FIG. 11 constitute the control unit 8 of FIG.

  The makeup simulation program of the present invention is at least a part of various programs for controlling makeup simulation apparatus 1. The makeup simulation program is provided by distributing the recording medium 14, for example.

  The recording medium 14 on which the makeup simulation program is recorded is a recording medium such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. for recording information optically, electrically or magnetically, a ROM, a flash memory, etc. Various types of recording media such as a semiconductor memory that electrically records information can be used.

  When the recording medium 14 on which the makeup simulation program is recorded is set in the drive device 12, the makeup simulation program is installed from the recording medium 14 to the auxiliary storage device 13 via the drive device 12. The auxiliary storage device 13 stores the installed makeup simulation program and also stores necessary files, data, and the like. The memory device 11 reads and stores the makeup simulation program from the auxiliary storage device 13 at the time of activation. The arithmetic processing unit 10 implements various processes as described later according to a makeup simulation program stored in the memory device 11.

  FIG. 12 is a flowchart of an embodiment showing an outline of processing performed by the makeup simulation apparatus. FIG. 13 is an image diagram of an example of a main screen displayed on the monitor and an operation screen displayed on the operation panel. Screen images 100 to 111 in FIG. 13 are those of the main screen displayed on the monitor 7. Screen images 200 to 210 are those of an operation screen displayed on the operation panel 4.

  The control unit 8 continuously receives moving images taken by the camera 2. At this time, the control unit 8 displays the screen image 100 on the monitor 7 and displays the screen image 200 on the operation panel 4. Screen images 100 and 200 represent examples in which a screen saver is displayed.

  In step S1, the control unit 8 continuously determines whether or not the received moving image includes the user's face. The control unit 8 repeats the process of step S1 until it recognizes that the user's face is included in the moving image (NO in S1).

  If it is recognized that the user's face is included in the moving image (YES in S1), the control unit 8 proceeds to step S2 and activates software including a makeup simulation program for performing makeup simulation. At this time, the control unit 8 displays the screen image 101 on the monitor 7 and displays the screen image 201 on the operation panel 4.

  A screen image 101 represents an example in which a moving image obtained by photographing the user's face with the camera 2 is displayed. A screen image 201 represents an example in which a welcome comment moving horizontally is displayed.

  In step S3, the control unit 8 starts a makeup simulation based on the software activated in step S2. At this time, the control unit 8 displays the screen image 102 on the monitor 7 and displays the screen image 202 on the operation panel 4. The screen image 102 represents an example in which a moving image obtained by photographing the user's face with the camera 2 is displayed with a magical mirror effect. The screen image 202 represents an example in which a welcome comment that moves to the side is displayed, like the screen image 201.

  In step S4, the control unit 8 executes makeup simulation as will be described later. At this time, the control unit 8 sequentially displays the screen images 103 to 106 on the monitor 7 and displays the screen images 203 to 206 on the operation panel 4.

  Screen images 103 to 106 represent an example in which a user's face to which four makeup patterns (images) are applied by a makeup simulation is displayed. Screen images 203 to 206 represent examples in which the contents (for example, names) of the makeup patterns of the screen images 103 to 106 displayed on the monitor 7 at that time are displayed. The control unit 8 sequentially displays the screen images 103 to 106 on the monitor 7 and displays the screen images 203 to 206 on the operation panel 4 until a predetermined time elapses or the user touches the operation panel 4.

  When the predetermined time has elapsed or the user touches the operation panel 4, the control unit 8 proceeds to step S 5, displays the screen image 107 on the monitor 7, and displays the screen image 207 on the operation panel 4. The screen image 107 represents an example in which a moving image obtained by photographing the user's face with the camera 2 is displayed, like the screen image 101. A screen image 207 represents an example in which an image selection screen capable of selecting one image from four makeup patterns (images) is displayed. The user can select one image from the image selection screen by operating the operation panel 4.

  In step S6, the control unit 8 repeats the process of step S6 until one image is selected from the image selection screen (NO in S6). When the user selects one image from the image selection screen, the operation panel 4 receives an operation from the user and outputs operation information to the control unit 8.

  If it is determined that the user has selected one image from the image selection screen (YES in S6), control unit 8 proceeds to step S7, and displays the image screen of the selected image on monitor 7 and operation panel 4. At this time, the control unit 8 displays the screen image 108 on the monitor 7 and displays the screen image 208 on the operation panel 4.

  The screen image 108 is based on one image selected from the image selection screen, an image image of the face of the user after makeup having a different color pattern, and product information for makeup like an image image. Are displayed in order. The screen image 208 displays the content of one image selected from the image selection screen, and product information for makeup like the image image of the screen image 108 displayed on the monitor 7 at that time. An example is shown.

  Note that the user can also instruct printout by operating the operation panel 4. When receiving a printout instruction from the user, the control unit 8 displays the screen image 109 on the monitor 7 and displays the screen image 209 on the operation panel 4. A screen image 109 represents an example in which an image to be printed out is displayed. A screen image 209 represents an example in which a comment being printed out is displayed. The control unit 8 controls the printer 5 to print out the image displayed on the monitor 7.

  The user can also instruct display and printout of a comparison screen composed of image images before and after make-up by operating the operation panel 4. When receiving the display instruction of the comparison screen from the user, the control unit 8 displays the screen image 110 on the monitor 7. A screen image 110 represents an example in which a comparison screen including image images before and after makeup is displayed. When receiving a printout instruction from the user while the comparison screen is displayed, the control unit 8 controls the printer 5 to print out the comparison screen displayed on the monitor 7.

  When the makeup simulation by the user is finished, the control unit 8 displays screen images 111 and 210 as screen savers on the monitor 7 and the operation panel 4 and finishes the process.

  In the example of FIGS. 12 and 13, four images are displayed by the makeup simulation, but other images may be displayed. Further, the makeup simulation apparatus 1 shows an example in which makeup is applied to the user's face included in the moving image output from the camera 2, but the moving image captured in advance is stored as a moving image file in the auxiliary storage device 13 or the like. In addition, makeup may be applied to the user's face included in the moving image file.

  The makeup simulation apparatus 1 can also use the moving image output from the camera 2 other than the makeup simulation. FIG. 14 is an image diagram showing processing other than makeup simulation performed by the makeup simulation apparatus.

  A screen image 300 represents an example in which a moving image of a user's face photographed by the camera 2 is displayed. The makeup simulation apparatus 1 recognizes the user's face included in the moving image and cuts out a still image 301 of the user's face.

  The makeup simulation apparatus 1 can perform a face diagnosis and a skin color diagnosis of the still image 301 using the face analysis logic and the skin analysis logic, and display a screen image 302 representing the result on the monitor 7. Note that the processing for performing the face diagnosis and the skin color diagnosis of the still image 301 using the face analysis logic and the skin analysis logic is a known technique described in, for example, Japanese Patent Application Laid-Open No. 2001-346627.

  The makeup simulation apparatus 1 can also display a course selection screen 303 such as a screen image 303 on the monitor 7 to allow the user to select each course (for example, trend, basic, and free). The makeup simulation apparatus 1 displays screen images 304 to 309 based on the course selected by the user on the monitor 7 to perform simulation and advice.

  Screen images 304 to 306 represent examples in which a simulation screen of each course is displayed. Screen images 307 to 309 represent examples in which an advice screen for each course is displayed.

  For example, the basic course is a simulation and advice of an optimal makeup technique based on the results of the face diagnosis and the skin color diagnosis. The trend course is a simulation and advice on the latest trend makeup. The free make-up course is a simulation and advice for items corresponding to the eyes, mouth, cheeks, eyebrows, and individual parts.

  When receiving a printout instruction from the user while the simulation screen or advice screen is displayed, the control unit 8 controls the printer 5 to print out the simulation screen or advice screen displayed on the monitor 7. You can also.

  Next, details of a makeup simulation system for realizing the makeup simulation apparatus 1 as described above will be described. FIG. 15 is a system configuration diagram of an embodiment of a makeup simulation system according to the present invention.

  The makeup simulation system 20 of FIG. 15 includes an analog camera 21, a USB capture device 22, a moving image file 23, a makeup camera 24, a still image system 25, a moving image file 26, a shared memory 27, a simulator main application 28, and an interface application 29. It is a configuration.

  The analog camera 21 outputs, for example, a captured NTSC moving image via the USB capture device 22. The moving image output from the USB capture device 22 is input to the simulator main application 28 using, for example, DirectX 31 as an example of an API (Application Programming Interface). The moving image file 23 is also input to the simulator main application 28 using DirectX 31.

  The makeup camera 24 outputs the captured moving image via IEEE1394 as an example of a serial interface. The moving image output from the makeup camera 24 is input to the simulator main application 28 using the dedicated API 32. The simulator main application 28 obtains a moving image resolution original image and a still image resolution still image from a moving image input using the dedicated API 32.

  The simulator main application 28 uses a moving image and moving image file 23 input using DirectX 31 and a moving image with a moving image resolution obtained from a moving image input using the dedicated API 32 as an original image. Trim and reduce the image.

  The simulator main application 28 trims the original image to obtain a pre-makeup image. Further, the simulator main application 28 performs a reduction process on the original image to obtain a face recognition processing image. The face recognition processing unit 33 obtains a tracking point 34 to be described later for recognizing the user's face from the face recognition processing image by FFT (Fast Fourier Transform).

  Based on the tracking point 34, the makeup processing unit 35 performs makeup including foundation, eyebrows, shadow, lipstick, and teak on the user's face included in the pre-makeup image to obtain a post-makeup image. The makeup processing unit 35 includes a foundation processing unit 36, an eyebrow processing unit 37, a shadow processing unit 38, a lipstick processing unit 39, and a cheek processing unit 40.

  It should be noted that the makeup processing unit 35 can include product information 41 for makeup as in the post-makeup image in the post-makeup image. The moving image server 42 can write the pre-makeup image and the post-makeup image in the shared memory 27 and output the pre-makeup image and the post-makeup image as the moving image file 26.

  The interface application 29 includes a moving image control object 52, a moving image display object 53, and other controllers 54 that use the ActiveX controller 50 and the ActiveX viewer 51. Note that the interface application 29 and the simulator main application 28 are linked by ActiveX & Socket.

  The interface application 29 can display the pre-makeup image and the post-makeup image written in the supply memory 27 on the monitor 7 using the ActiveX viewer 51.

  The simulator main application 28 performs trimming and reduction processing on a still image having a still image resolution obtained from a moving image input using the dedicated API 32. The simulator main application 28 trims a still image having a still image resolution. Further, the simulator main application 28 performs a reduction process on the still image having the still image resolution to obtain a face recognition processing image. The face recognition processing unit 43 obtains a tracking point 44 described later for recognizing the user's face from the face recognition processing image.

  Based on the tracking point 44, the simulator main application 28 extracts a detailed part from the user's face included in the trimmed moving image, and obtains additional information such as the type and skin color used for the face diagnosis and the skin color diagnosis from the detailed part. Then, the tracking point 45 with additional information added to the tracking point 44 and the still image with the still image resolution obtained from the moving image input using the dedicated API 32 are output to the still image system 25.

  The still image system 25 can use the tracking point 45 to perform the face diagnosis and the skin color diagnosis of the still image 301 by using the face analysis logic and the skin analysis logic described above, and display a screen image 302 representing the result on the monitor 7. In addition, the still image system 25 can also display the screen images 303 to 309 on the monitor 7.

  FIG. 16 is a screen image diagram showing processing performed by the simulator main application. A screen image 400 represents an example of displaying a pre-makeup image. A screen image 401 represents an example in which the tracking point 34 obtained from the face recognition processing image is displayed superimposed on the pre-makeup image. A screen image 402 represents an example in which a post-makeup image obtained by applying makeup to the user's face included in the pre-makeup image based on the tracking point 34 is displayed.

  Hereinafter, details of the face recognition process and the makeup process among the processes performed by the simulator main application 28 will be described in order with reference to the drawings. In this embodiment, a makeup process including a foundation process, an eyebrow process, a shadow process, a lipstick process, and a cheek process is described as an example of the makeup process, but other combinations may be used.

(Face recognition processing)
FIG. 17 is an image of the user's face included in the pre-makeup image. FIG. 18 is an image diagram of an example showing tracking points. The face recognition processing unit 33 of the simulator main application 28 obtains 45 tracking points 34 as shown in FIG. 18 for recognizing the user's face from the image as shown in FIG. The tracking point 34 in FIG. 18 is an example, and may be adjusted according to the processing capability of the control unit 8 and the definition of the monitor 7.

  In this manner, by obtaining the tracking point 34 from the user's face included in the pre-makeup image, the makeup processing unit 35 associates the tracking point 34 with the makeup method and color as shown in FIG. It can be set in the processing parameter file.

  FIG. 19 is a block diagram of an example of a makeup process parameter file. The makeup process parameter file in FIG. 19 is a file in which makeup methods and colors are set in association with the tracking points 34 for each eye, mouth, cheek, and the like. The makeup process parameter file is set for each makeup pattern (image). The makeup process parameter file in FIG. 19 represents an example of class elegance.

(Lipstick treatment)
As shown in FIG. 20, the lipstick processing unit 39 included in the simulator main application 28 refers to the tracking points 34 of the eight lips and the three noses to perform lipstick processing. FIG. 20 is an image diagram showing tracking points referred to by the lipstick process.

  FIG. 21 is a flowchart illustrating an example of lipstick processing. The lipstick process is roughly divided into a preparatory process in step S10 and a color painting process in step S20. The pre-preparation process includes a cut-out and rotation process in step S11, a contour extraction image creation process in step S12, a contour extraction process in step S13, a spline curve generation process in step S14, a rotation return process in step S15, and a color in step S16. And a paint map generation process. The color painting process includes a color painting process based on the color painting map in step S21 and a debug & design drawing process in step S22.

  Proceeding to the cutout and rotation process in step S11, the lipstick processing unit 39 cuts out the partial image 500 including the lips of the user's face from the face recognition processing image, and rotates the partial image 500 to the processing posture. Get 501.

  Proceeding to the contour extraction image creation process in step S 12, the lipstick processing unit 39 creates a contour extraction image from the partial image 501. Proceeding to the contour extraction process in step S 13, the lipstick processing unit 39 extracts the lip contour from the contour extraction image using points as shown in the partial image 502.

  FIG. 22 is an image diagram showing an example of the contour extraction process. The contour extraction image 600 is displayed with 8 lips and 3 nose tracking points superimposed. The lipstick processing unit 39 re-searches based on the tracking points of 8 lips and 3 noses, and extracts the points of the contour extraction image 601.

  FIG. 23 compares the tracking points of the lips 8 points and 3 noses of the contour extracting image 600 with the points of the contour extracting image 601 extracted by re-searching based on the tracking points of 8 lips and 3 noses. It is a screen. When the re-search is not successful, the default points as shown in FIG. 24 calculated from the tracking points of 8 lips and 3 noses of the contour extraction image 600 are employed.

  FIG. 24 is an image diagram showing an example of processing for obtaining a default point from tracking points of 8 lips and 3 noses. The lipstick processing unit 39 refers to the three tracking points of the nose and obtains default points for the five upper lips.

  Proceeding to the spline curve generation process in step S14, the lipstick processing unit 39 interpolates the points of the contour extraction image 601 or the default points as shown in FIG. 24 with a quadratic spline to complete the contours as shown in FIG. FIG. 25 is an image diagram showing processing for completing a contour from a point or a default point. When the contour of the lips is completed as in the partial image 503, the lipstick processing unit 39 proceeds to step S15, and performs a rotation return process that restores the rotation performed in the cut-out and rotation process in step S11.

  Proceeding to the color painting map generation process of step S16, the lipstick processing unit 39 generates a color painting map that determines the strength of color painting from the brightness and saturation of the partial image 700. FIG. 26 is an image diagram illustrating an example of a color painting map generation process. Specifically, the lipstick processing unit 39 generates a gray scale image 701 representing the intensity of color coating from the brightness and saturation. The lipstick processing unit 39 cuts out only the partial image 702 surrounded by the outline of the lips completed by the spline curve generation processing in step S 14 from the gray scale image 701 as the color painting map 504.

  Proceeding to the color painting process based on the color painting map in step S21, the lipstick processing unit 39 is set in the color painting map 504 generated in the color painting map generation process in step S16 and the makeup process parameter file as shown in FIG. The pre-makeup image is colored based on the makeup method and the designated color.

  FIG. 27 is an image diagram illustrating an example of a color painting process based on the color painting map. The lipstick processing unit 39 is based on the makeup application method and designated color set in the makeup processing parameter file 801 as shown in FIG. 19 and the color painting map 802 generated by the color painting map generation processing in step S16. Then, the pre-makeup image 803 is colored to obtain a post-makeup image 804.

  Then, the process proceeds to the debug & design drawing process in step S22. After the lipstick processing unit 39 performs the debug & design drawing process, the lipstick process ends.

(Shadow processing)
As shown in FIG. 28, the shadow processing unit 38 included in the simulator main application 28 refers to the tracking points 34 of three eyes and one eyebrow separately for left and right, and performs shadow processing. FIG. 28 is an image diagram showing tracking points referred to by shadow processing.

  FIG. 29 is a flowchart illustrating an example of shadow processing. The shadow process is roughly divided into a preparatory process in step S30 and a color painting process in step S40. The pre-preparation process includes the basic contour generation process in step S31.

  The color painting process is repeated for the color painting pattern, based on the color painting outline generation process in step S41, the color painting center generation process in step S42, the color painting map generation process in step S43, and the color painting map in step S44. It consists of a color painting process and a debug & design drawing process in step S45.

  Proceeding to the basic contour generation processing in step S31, the shadow processing unit 38 obtains the shape of the eyes of the user's face from the face recognition processing image as in the partial image 900. FIG. 30 is an image diagram illustrating an example of basic contour generation processing.

  Since the shadow processing unit 38 generates a contour that is the basis of the color painting contour, as shown in the partial image 1001, the vertical direction is searched from the center of the eye to recognize two points of the eye contour (upper boundary & lower boundary). To do. The shadow processing unit 38 adds four points generated by spline interpolation to the recognized two points of the outline of the eye, the corner of the eye, and the four points of the eye, and generates a polygon with a total of eight points as in the partial image 1002.

  Proceeding to the color painting contour generation process in step S41, the shadow processing unit 38 creates a color painting contour like the partial image 901. FIG. 31 is an image diagram illustrating an example of the color painting contour generation process. Since the shadow processing unit 38 generates a colored outline, as shown in the partial image 1101, it generates a colored outline based on the corners of the eyes and the eyes. It should be noted that parameters such as contour expansion and vertex shift may be designated by GUI.

  Proceeding to the color painting center generation process in step S 42, the shadow processing unit 38 generates the color painting center position as “●” in the partial image 902. Proceeding to the color painting map generation process of step S43, the shadow processing unit 38 generates a color painting map that determines the strength of color painting as in the partial image 903.

  Specifically, in the color painting map generation process, the strength of painting is determined in accordance with the distance from the color painting center to the side of the polygon. For example, the shadow processing unit 38 determines the color coating map so that the closer to the side, the weaker the coating strength. It should be noted that the color map generation process is targeted for a portion obtained by removing the basic contour from the color contour.

  As shown in FIG. 32, the shadow processing unit 38 generates a smoother gradation by applying a blurring process to the generated color painting map. FIG. 32 is an image diagram of an example showing a color map that has not been subjected to the blurring process and a color map that has been subjected to the blur process.

  Proceeding to the color painting process based on the color painting map in step S44, the shadow processing unit 38 is set in the color painting map 903 generated in the color painting map generation process in step S43 and the makeup process parameter file as shown in FIG. The pre-makeup image is colored on the basis of the makeup application and the designated color, thereby obtaining a post-makeup image.

  Then, the process proceeds to the debug & design drawing process in step S45, and after performing the debug & design drawing process, the shadow processing unit 38 ends the shadow process. Note that the shadow processing unit 38 can also realize multicolor coating by repeatedly performing the processing of steps S41 to S45 for the color coating pattern.

(Cheek treatment)
As shown in FIG. 33, the cheek processing unit 40 included in the simulator main application 28 performs cheek processing with reference to the tracking points 34 of the corners of the eyes and the corners of the mouth (separate left and right) and the center of the eyes and the nose (for stabilization). FIG. 33 is an image diagram showing tracking points referred to by the cheek processing.

  FIG. 34 is a flowchart illustrating an example of cheek processing. The cheek process is composed of a color painting outline generation process in step S50, a color painting process in step S51, and a debug & design drawing process in step S52.

  Proceeding to the color painting contour generation processing in step S50, the cheek processing unit 40 creates a contour polygon as a color painting contour based on the corners of the eyes and the mouth corners in order to generate a color painting contour. FIG. 35 is an image diagram showing an example of a color outline. Note that the number, size, shape, position, etc. of the contour polygon may be designated by a GUI.

  Proceeding to the color painting process in step S51, the cheek processing unit 40 determines the strength of the paint corresponding to the distance from the color painting center to the side of the contour polygon. Note that if the processing cost is too high for determining the coating strength, the coating strength may be determined by reducing the resolution (parameter specification using the GUI) (thinning in a mosaic pattern). The cheek processing unit 40 performs coloring on the pre-makeup image based on the determined coating strength, the makeup method and the designated color set in the makeup processing parameter file as shown in FIG. Get an image after up.

  Then, the process proceeds to the debug & design drawing process in step S52, and the cheek processing unit 40 ends the cheek process after performing the debug & design drawing process.

(Eyebrow processing)
As shown in FIG. 36, the eyebrow processing unit 37 included in the simulator main application 28 refers to the right and left eye corners, the eye center, and the two eyebrow tracking points 34 to perform eyebrow processing. FIG. 36 is an image diagram showing tracking points referred to by eyebrow processing.

  FIG. 37 is a flowchart illustrating an example of eyebrow processing. The eyebrow process is roughly divided into an eyebrow outline extraction process in step S60, an original eyebrow area erasing process in step S70, an eyebrow shape deforming process in step S80, and a deformed eyebrow pasting process in step S90.

  The original eyebrow area erasing process includes the area expanding process in step S71 and the eyebrow erasing process in step S72. The eyebrow shape deformation process includes an instruction curve creation process corresponding to the deformation designation in step S81 and a deformed eyebrow generation process in step S82.

  Proceeding to the eyebrow contour extraction process in step S60, the eyebrow processing unit 37 obtains the shape of the eyebrow of the user's face from the face recognition processing image as a partial image 2001. FIG. 38 is an image diagram illustrating an example of an eyebrow contour extraction process. In order to extract the outline of the eyebrows, the eyebrow processing unit 37 searches the eyebrows to the left and right around the buttocks (actually near the center of the eyebrows) as shown in the partial image 2101. As shown in the partial image 2102, the eyebrow processing unit 37 recognizes the outline shape of the eyebrows from the search result.

  Proceeding to the expansion processing of the region in step S71, the eyebrow processing unit 37 expands the region representing the recognized outline shape of the eyebrow. Proceeding to the eyebrow erasing process in step S72, the eyebrow processing unit 37 erases the eyebrows by painting the expanded region using the skin color in the vicinity. In addition, the eyebrow processing unit 37 performs a process of blending in the boundary portion of the expanded region.

  Proceeding to the instruction curve creation process corresponding to the deformation designation in step S81, the eyebrow processing unit 37 deforms the recognized area (skeleton) representing the outline shape of the eyebrow according to the designation. FIG. 39 is an image diagram showing an example of a process for creating an instruction curve corresponding to the deformation designation.

  As shown in the partial image 2201, the eyebrow processing unit 37 replaces the region representing the recognized outline shape of the eyebrow with a skeleton 2202 composed of a horizontal axis and a plurality of vertical tangs, and the shape of the axis and the height of the tangs. For example, the transformation process can be performed like a skeleton 2203.

  Proceeding to the deformed eyebrow generation process in step S <b> 82, the eyebrow processing unit 37 generates a deformed eyebrow from the skeleton 2203. Proceeding to the process of attaching the deformed eyebrows in step S90, the eyebrow processing unit 37 attaches the deformed eyebrows to the pre-makeup image to obtain an image after make-up.

(Foundation treatment)
As shown in FIG. 40, the foundation processing unit 36 included in the simulator main application 28 refers to the right and left eye corners, the eyes and one eyebrow, and the tracking points 34 between the eyes and the center of the nose to perform the foundation processing. FIG. 40 is an image diagram showing tracking points referred to by the foundation process.

  FIG. 41 is a flowchart illustrating an example of the foundation process. The foundation process includes a contour generation process in step S101, a blurring process on the target image in step S102, and an image pasting process in step S103. Note that the processing of steps S101 to S103 is repeated for the target area.

  Proceeding to the contour generation processing in step S101, the foundation processing unit 36 generates three types (four locations) of the forehead, nose, and cheek (left and right) as shown in the image of FIG. FIG. 42 is an example of an image representing an outline. It should be noted that the size, position, etc. of each contour may be designated by a GUI.

  Proceeding to the blurring process on the target image in step S102, the foundation processing unit 36 performs the blurring process on the target image corresponding to the generated contour as shown in FIG. FIG. 43 is an image diagram illustrating an example of an original image before blurring processing and an original image after blurring processing. By the blurring process on the target image, the foundation processing unit 36 can smooth the fine roughness of the skin.

  Proceeding to the image pasting process in step S103, the foundation processing unit 36 pastes the target image after the blurring process on the three types (four locations) of the forehead, nose, and cheek (left and right) contours of the pre-makeup image, Get an image after makeup.

  As described above, according to the present invention, makeup can be accurately applied to a user's face included in a moving image with a small processing load. The photographing means described in the claims corresponds to the camera 2, the control means corresponds to the control unit 8, the display means corresponds to the monitor 7, and the face recognition processing means corresponds to the face recognition processing unit 33. The makeup processing means corresponds to the makeup processing unit 35, the operation means corresponds to the operation panel 4, the half mirror means corresponds to the half mirror 3, and the printing means corresponds to the printer 5.

  Further, the present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims.

  Since the makeup simulation apparatus 1 of the present invention is a real-time simulation, unlike the one based on a still image or the conventional makeup simulation apparatus 1, the face tracking point is instantly recognized, and the simulation is performed based on the tracking point. Since it can, it also enables the following things that were not possible before.

  The makeup simulation apparatus 1 of the present invention can perform real-time simulation. Since the makeup simulation apparatus 1 of the present invention can perform simulations not only from the front face but also from the side face as before, it is easy to check the simulation effect and technique such as blusher.

  Since the makeup simulation apparatus 1 of the present invention is a real-time simulation, it has become possible to analyze a face as a three-dimensional object and to express a three-dimensional feeling and texture from the conventional planar expression.

  In addition, since the makeup simulation apparatus 1 of the present invention can recognize a number of faces at the same time, a number of real-time simulations are possible at the same time. Since the makeup simulation apparatus 1 according to the present invention has an excellent face recognition function, makeup according to each feature or classification can be made according to each feature or automatically classifying men and women. For example, the makeup simulation apparatus 1 of the present invention is capable of performing each makeup simulation simultaneously with a couple.

1 is an external view of a first embodiment of a makeup simulation apparatus according to the present invention. It is sectional drawing of the 1st Example of a makeup simulation apparatus. It is an external view of the 2nd Example of the makeup simulation apparatus by this invention. It is sectional drawing of the 2nd Example of a makeup simulation apparatus. It is an external view of the 3rd Example of the makeup simulation apparatus by this invention. It is sectional drawing of the 3rd Example of a makeup simulation apparatus. It is an external view of 4th Example of the makeup simulation apparatus by this invention. It is sectional drawing of 4th Example of a makeup simulation apparatus. It is an external view of 5th Example of the makeup simulation apparatus by this invention. It is sectional drawing of the 5th Example of a makeup simulation apparatus. It is a hardware block diagram of 5th Example of a makeup simulation apparatus. It is a flowchart showing the outline of the process which a makeup simulation apparatus performs. It is an image figure of an example of the main screen displayed on a monitor, and the operation screen displayed on an operation panel. It is an image figure showing processes other than the makeup simulation which a makeup simulation apparatus performs. 1 is a system configuration diagram of an embodiment of a makeup simulation system according to the present invention. It is a screen image figure showing the process which a simulator main application performs. It is an image of a user's face included in a pre-makeup image. It is an image figure of an example showing a tracking point. It is a block diagram of an example of a makeup process parameter file. It is an image figure which shows the tracking point referred by a lipstick process. It is an example flowchart which shows a lipstick process. It is an image figure of an example showing an outline extraction process. It is a comparison screen between the tracking points of 8 lips and 3 noses of the contour extracting image 600 and the points of the contour extracting image 601 extracted by re-searching based on the tracking points of 8 lips and 3 noses. . It is an image figure of an example showing the process which calculates | requires a default point from the tracking points of eight lips and three noses. It is an image figure showing the process which completes an outline from a point or a default point. It is an image figure of an example showing the color painting map production | generation process. It is an image figure of an example showing the color painting process based on a color painting map. It is an image figure which shows the tracking point referred by shadow processing. It is an example flowchart which shows a shadow process. It is an image figure of an example showing the production | generation process of a basic outline. It is an image figure of an example showing the production | generation process of a color painting outline. It is an image figure of an example showing the color painting map which has not applied the blurring process, and the color painting map which has applied the blurring process. It is an image figure which shows the tracking point referred by a cheek process. It is a flowchart of an example which shows a cheek process. It is an image figure of an example showing a color painting outline. It is an image figure which shows the tracking point referred by eyebrow processing. It is an example flowchart which shows an eyebrow process. It is an image figure of an example showing an eyebrow outline extraction process. It is an image figure of an example showing the creation process of the instruction | indication curve corresponding to a deformation | transformation designation | designated. It is an image figure which shows the tracking point referred by foundation process. It is a flowchart of an example which shows a foundation process. It is an example image image showing an outline. It is an image figure of an example showing the original image before a blurring process, and the original image after a blurring process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Makeup simulation apparatus 2 Camera 3 Half mirror 4 Operation panel 5 Printer 6 Illumination 7 Monitor 8 Control part 9 Transparent board 10 Arithmetic processing apparatus 11 Memory apparatus 12 Drive apparatus 13 Auxiliary storage apparatus 14 Recording medium 15 Touch panel monitor 16 Display case 17 IC Tag reader / writer 25 Still image system 27 Shared memory 28 Simulator main application 29 Interface application 33 Face recognition processing unit 34 Tracking point 35 Makeup processing unit 36 Foundation processing unit 37 Eyebrow processing unit 38 Shadow processing unit 39 Lipstick processing unit 40 Cheek processing unit 41 Product information

Claims (15)

A makeup simulation system for applying makeup to a moving image of a user's face,
Photographing means for photographing a user's face and outputting a moving image;
Control means for receiving the moving image output from the photographing means, processing the moving image, and outputting the processed image;
Display means for displaying a moving image output from the control means,
The control means includes face recognition processing means for recognizing a user's face from the moving image based on a predetermined tracking point;
A makeup simulation system comprising: makeup processing means for applying predetermined makeup to a user's face included in the moving image based on the tracking point and outputting the result to the display means. The makeup processing means includes a foundation processing means for performing a foundation process on a user's face included in the moving image based on the tracking point;
Lipstick processing means for performing lipstick processing on the face of the user included in the moving image based on the tracking point;
Eyebrow processing means for performing eyebrow processing on the face of the user included in the moving image based on the tracking point;
Cheek processing means for performing cheek processing on the face of the user included in the moving image based on the tracking point;
The makeup simulation system according to claim 1, further comprising at least one shadow processing unit that performs shadow processing on a user's face included in the moving image based on the tracking point.   The makeup processing unit generates a contour for applying a predetermined makeup to a user's face included in the moving image based on the tracking point, and performs a color painting process based on the contour. 1. The makeup simulation system according to 1.   The makeup processing unit sequentially performs a plurality of types of makeup on a user's face included in the moving image at predetermined time intervals, and outputs an image image after makeup to the display unit. Item 1. A makeup simulation system according to item 1. It further comprises storage means for storing product information for makeup such as the image image for each type of makeup,
5. The makeup simulation system according to claim 4, wherein the makeup processing means acquires product information for makeup such as the image image from the storage means and outputs the product information to the display means.   2. The makeup simulation system according to claim 1, wherein the makeup processing means outputs a comparison screen comprising image images before and after makeup of the user's face included in the moving image to the display means. . Reading means for reading the identification information of the article selected by the user;
A correspondence means for associating the article with the identification information;
The makeup processing means identifies the article corresponding to the identification information read from the reading means based on the correspondence means, and uses the article to perform a predetermined makeup on the user's face included in the moving image. 2. The makeup simulation system according to claim 1, wherein the image image subjected to the processing is output to the display means. A makeup simulation apparatus for applying makeup to a moving image obtained by photographing a user's face,
Photographing means for photographing a user's face and outputting a moving image;
Control means for receiving the moving image output from the photographing means, processing the moving image, and outputting the processed image;
Display means for displaying a moving image output from the control means,
The control means includes a face recognition processing means for recognizing a user's face from the moving image based on a predetermined tracking point;
A makeup simulation apparatus comprising: makeup processing means for applying predetermined makeup to a user's face included in the moving image based on the tracking point and outputting the result to the display means. Further comprising half mirror means provided on the display direction side of the display means;
The half mirror means transmits light from the display means side to the user side when a moving image is displayed on the display means, and the user side when a moving image is not displayed on the display means The makeup simulation apparatus according to claim 8, wherein the makeup simulation apparatus reflects light from the apparatus. Further comprising half mirror means provided on the display direction side of the display means;
The photographing means is arranged at a position where the user's face can be photographed if passing through the half mirror means,
9. The makeup simulation apparatus according to claim 8, wherein the half mirror means transmits light from the display means side to the user side and transmits light from the user side to the photographing means side.   9. The makeup simulation apparatus according to claim 8, further comprising printing means for printing an image after makeup of the user's face displayed on the display means.   9. The makeup simulation apparatus according to claim 8, further comprising display means for displaying a product for makeup such as an image image after makeup of the user's face displayed on the display means. . Reading means for reading the identification information of the article selected by the user;
A correspondence means for associating the article with the identification information;
The makeup processing means identifies the article corresponding to the identification information read from the reading means based on the correspondence means, and uses the article to perform a predetermined makeup on the user's face included in the moving image. 9. The makeup simulation apparatus according to claim 8, wherein the image image subjected to the processing is output to the display means. A makeup simulation method in a makeup simulation system for applying makeup to a moving image obtained by photographing a user's face,
The control means recognizes the user's face from the moving image taken by the photographing means and starts a makeup simulation;
Recognizing the face of the user included in the moving image based on a predetermined tracking point;
And a step of applying a predetermined makeup to a face of a user included in the moving image based on the tracking point and outputting the makeup to the display unit. A makeup simulation program for applying makeup to a moving image obtained by photographing a user's face, which is executed in a computer having at least an arithmetic processing device, a storage device, and an output device,
The arithmetic processing unit recognizes a user's face from a moving image captured by an imaging unit, and starts a makeup simulation;
Recognizing a user's face included in the moving image based on a predetermined tracking point;
A makeup simulation program for executing a predetermined makeup on a user's face included in the moving image based on the tracking point and outputting it to the output device.