JP2005044283A - Cosmetics guidance system, server apparatus, terminal device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a cosmetics guidance system, which can provide guidance for make-up methods using cosmetics desired by a customer who wants to purchase cosmetics, server apparatus, terminal devices and programs thereof. <P>SOLUTION: A customer terminal 12 transmits attribute information, such as ages of customers, and static face image data of customers to a server 13. The server 13 transmits a cosmetics list suitable for a customer, which is extracted based on the attribute information, and a list of processes which are patterns of make-up methods to the customer terminal 12. The customer terminal 12 transmits cosmetics and process names selected by the customer to the server 13. According to data related to the cosmetics and the processes selected by the customer, the server 13 sequentially performs color update processings to the static face image data of the customer, and creates animation data by time-sequentially arranging the results. The server 13 transmits the created animation data to the customer terminal 12. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a cosmetic guidance system, a server device, a terminal device, and a program for providing guidance regarding cosmetic selection and a makeup method to those who wish to purchase cosmetics.

  Many cosmetic dealers provide cosmetic trial corners to help customers purchase cosmetics suitable for them and at the same time increase customer motivation. However, such trial use of cosmetics has the following problems.

(A) Makeup takes time and effort.
(B) As a result of the sales clerk directly contacting the customer, the customer is given a psychological feeling of having to purchase something.
(C) Because the customer may withdraw from trial or purchase cosmetics that he / she does not want for the reasons described above, the dealer cannot obtain sufficient and accurate marketing related information.
(D) It is difficult for the reasons of cost and the like to ensure that the sales staff can always provide appropriate advice regarding the selection of cosmetics and makeup methods according to the characteristics and needs of various customers.

In order to solve the above problems, as a system for presenting a makeup method suitable for the purchaser, image data when a makeup similar to the makeup applied to the model face is applied to the user's face is virtually The system which produces | generates is proposed (for example, refer patent document 1).
JP 2001-268594 A

In addition, the makeup procedure is indicated by characters and the like, and based on the image data of the user's face, image data virtually created by the procedure is generated, and the image data and the user actually perform the procedure. A system has been proposed in which the user presents points to be improved in the makeup procedure by comparing the image data of the face after performing makeup according to the method (for example, see Patent Document 2). .
JP 2002-224049 A

  Since the system according to the prior art presents the state of the face that has been decorated with a still image, there has been a problem that the state of the change of the face during the makeup cannot be sufficiently provided to the user.

  In view of the above situation, the present invention supports the selection of cosmetics suitable for himself / herself to those who wish to purchase cosmetics, and also provides the progress of makeup using the cosmetics selected by the buyers The present invention provides a system, a server device, a terminal device, and a program that can support acquisition of a makeup method of a purchase applicant.

  In order to achieve the above object, the present invention comprises a terminal device and a server device, wherein the terminal device transmits and receives image data via a communication network, and image data received via the communication network. And the server device stores cosmetic data storage means for storing cosmetic data including at least information relating to the color of the cosmetic, and process data storage means for storing process data indicating a process in which makeup is applied to the body. And still image data receiving means for receiving still image data indicating an image of a customer's body from the terminal device via the communication network, cosmetic data stored in the cosmetic data storage means, and the process data Process data stored in storage means, and still image data indicating an image of the customer's body received by the still image data receiving means Based on the video data creation means for creating video data representing temporal changes in the surface of the customer's body when applying makeup to the customer, the video data created by the video data creation means, There is provided a cosmetic guidance system comprising: moving image data transmission means for transmitting to the terminal device via a communication network. This is called the first aspect of the present invention.

  Furthermore, the present invention comprises cosmetic data storage means for storing cosmetic data including at least information relating to the color of cosmetics, process data storage means for storing process data indicating a process in which makeup is applied to the body, and a terminal device. Still image data receiving means for receiving still image data indicating a customer's body image via a communication network, cosmetic data stored in the cosmetic data storage means, and process data stored in the process data storage means And the temporal change of the color on the surface of the customer's body when applying makeup to the customer based on the still image data indicating the image of the customer's body received by the still image data receiving means. Moving image data creating means for creating moving image data to be represented, and moving image data created by the moving image data creating means Via network, to provide a server apparatus, characterized in that it comprises a video data transmission means for transmitting to the terminal device. This is called the second aspect of the present invention.

  2nd aspect of this invention WHEREIN: The process data memorize | stored in the said process data memory | storage means contain the time series data which show operation | movement of the makeup | decoration tool in the time series in the process in which makeup | decoration is given to a body, The said moving image data preparation The means obtains a temporal change in color on the surface of the customer's body based on time series data included in the process data stored in the process data storage means, and obtains moving image data representing the temporal change in the color. It is good also as a structure to create. This is called the third aspect of the present invention.

  In the second aspect of the present invention, the image processing apparatus includes process identification data receiving means for receiving process identification data capable of specifying one process data from the terminal device via the communication network, The moving image data may be created based on the process data specified by the process identification data received by the process identification data receiving means. This is called the fourth aspect of the present invention.

  In the second aspect of the present invention, the moving image data creation includes: cosmetic identification data receiving means for receiving one or more cosmetic identification data capable of specifying one cosmetic data from the terminal device via the communication network. The means may create moving image data based on the cosmetic data specified by the cosmetic identification data received by the cosmetic identification data receiving means. This is called the fifth aspect of the present invention.

  5th aspect of this invention WHEREIN: The condition data receiving means which receives the condition data which shows the conditions regarding the attribute of cosmetics from the said terminal device via the said communication network, One or more memorize | stored by the said cosmetics data storage means Cosmetic data extracting means for extracting cosmetic data relating to cosmetics that satisfy the condition indicated by the condition data received by the condition data receiving means, and the cosmetic data via the communication network to the terminal device It is good also as a structure provided with the cosmetics list transmission means which transmits the cosmetics list containing one or more cosmetics identification data which can specify each of the cosmetics data extracted by the extraction means. This is called the sixth aspect of the present invention.

  In the fifth aspect of the present invention, the terminal device has one or more cosmetic identification data that can identify one piece of cosmetic data stored in the cosmetic data storage unit via the communication network, and the one or more It is good also as a structure provided with the cosmetics list transmission means which transmits the cosmetics list | wrist including one or more message data which shows the message matched with either of cosmetics identification data. This is called the seventh aspect of the present invention.

  5th aspect of this invention WHEREIN: The attribute data receiving means which receives the attribute data which show a customer's attribute from the said terminal device via the said communication network, The cosmetics identification data received by the said cosmetics identification data receiving means It is good also as a structure provided with the marketing relevant data storage means which matches the data regarding the cosmetics data specified by this with the attribute data received by the said attribute data receiving means, and memorize | stores it as marketing relevant data. This is called the eighth aspect of the present invention.

  In a second aspect of the present invention, it includes store identification data that can specify a cosmetic store, cosmetic identification data that can specify a cosmetic, and price data that indicates a price associated with the cosmetic identification data. The store data storage means for storing at least one store data, the cosmetic identification data that can identify one cosmetic data from the terminal device via the communication network, and the cosmetic identification data. Purchase application data receiving means for receiving purchase application data, including one or more sets of quantity data indicating the quantity obtained, and purchase application data stored in the store data storage means and received by the purchase application data receiving means The cost data is based on the price data included in the store data including the same content as the cosmetic identification data included in the product and the quantity data included in the purchase application data. Calculation means for calculating the cost data, cost data calculated by said calculation means, via the communication network may be configured to include a cost data transmission means for transmitting to the terminal device. This is called the ninth aspect of the present invention.

  Furthermore, the present invention provides a sensor unit for detecting an operation performed using a cosmetic tool, and a cosmetic tool on the surface of a virtual human body based on the contents of the operation of the cosmetic tool detected by the sensor unit. Provided is a terminal device comprising processing data generation means for generating processing data which is data representing a contact position and a contact pressure, and storage means for storing the processing data generated by the processing data generation means. . This is referred to as a tenth aspect of the present invention.

  In a tenth aspect of the present invention, the sensor unit includes a plurality of pressure sensors for measuring pressure, and each of the plurality of pressure sensors is disposed on a surface of a three-dimensional object imitating the virtual human body. It is good also as a structure which measures the pressure in the position currently made. This is referred to as the eleventh aspect of the present invention.

  In a tenth aspect of the present invention, the sensor unit includes a plurality of touch sensors that detect a pressure equal to or greater than a predetermined value, and a pressure sensor that measures a pressure applied to the cosmetic device, and the plurality of touch sensors Each may be arranged on the surface of a three-dimensional object imitating the virtual human body and detect the pressure at the arranged position. This is referred to as a twelfth aspect of the present invention.

  10th aspect of this invention WHEREIN: The display means which displays the two-dimensional or three-dimensional image showing the said virtual human body is provided, The said sensor part has multiple pressure sensors which measure a pressure, These multiple pressures Each of the sensors may be arranged on the surface of a panel that covers the display screen of the display unit, and measure the pressure at the position where the sensor is arranged. This is referred to as a thirteenth aspect of the present invention.

  In a tenth aspect of the present invention, the display device includes a display unit that displays a two-dimensional or three-dimensional image representing the virtual human body, and the sensor unit includes a plurality of touch sensors that detect a pressure greater than a predetermined value, Each of the plurality of touch sensors is disposed on the surface of the panel that covers the display screen of the display unit, and detects the pressure at the position where the touch sensor is disposed. It is good also as a structure. This is referred to as the fourteenth aspect of the present invention.

  In the thirteenth or fourteenth aspect of the present invention, the display means generates an image representing a trajectory of the operation of the cosmetic device based on the processing data stored in the storage means, and displays the virtual human body. It is good also as a structure displayed with the image to represent. This is called the fifteenth aspect of the present invention.

  Furthermore, the present invention is a cosmetic data storage process for storing cosmetic data including at least information relating to the color of cosmetics, a process data storage process for storing process data indicating a process in which makeup is applied to the body, and a terminal device. Still image data reception processing for receiving still image data representing an image of a customer's body via a communication network, cosmetic data stored in the cosmetic data storage processing, and process data stored in the process data storage processing And temporal change in color on the surface of the customer's body when applying makeup to the customer based on the still image data indicating the image of the customer's body received in the still image data receiving process. Movie data creation process for creating movie data to be represented, and a movie created in the movie data creation process The chromatography data, via the communication network, to provide a program for executing a video data transmitting process of transmitting to the terminal device to the computer. This is called the sixteenth aspect of the present invention.

  According to the cosmetic guidance system, the server device, the terminal device and the program according to the present invention, the makeup is virtually applied by the makeup artist using the cosmetics selected by the customer on the customer's body. Provided to customers as real-time video. Therefore, the customer can easily select the cosmetics suitable for him and can easily learn the makeup method using the cosmetics.

[1. First Embodiment]
[1.1. Configuration of guidance system]
FIG. 1 is a diagram showing an overall configuration of a guidance system 1 according to an embodiment of the present invention. The guidance system 1 includes an Internet 11, a plurality of customer terminals 12 connected to the Internet 11, a server 13, a plurality of manufacturer terminals 14, and a plurality of store terminals 15. In FIG. 1, only one customer terminal 12, manufacturer terminal 14, and store terminal 15 are shown for simplification of the drawing.

  The Internet 11 is a communication network group in which a plurality of LANs (Local Area Network) capable of communicating various data such as characters and images are connected to each other by a gateway server or the like.

  The customer terminal 12 is a terminal device used by a customer who wants to purchase cosmetics. The customer terminal 12 can acquire still image data of the customer's face and can use a communication protocol, for example, HTTP (HyperText Transfer Protocol) that can be used in the Internet 11 with other communication devices connected to the Internet 11. ) And the like. The customer terminal 12 can also be realized by causing a general computer or mobile terminal to perform processing according to a specific program. In the present embodiment, as an example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an HD (Hard Disk), a display unit, an operation unit, and a communication unit as illustrated in FIG. The customer terminal 12 is realized by causing a general-purpose computer having a digital camera to perform processing according to Web browser software.

  The CPU of the customer terminal 12 receives Web page data described in a language such as HTML (HyperText Markup Language) from the server 13 according to the Web browser software stored in the HD, and displays the Web page according to the Web page data. To display. The Web page data can include still image data, moving image data, and the like as attached data. In addition to characters, symbols, graphics, and the like, the Web page data displays still images and reproduces moving images.

  The server 13 is a server device that is managed by the staff of a service provider that provides the information providing service of the guidance system 1. Similarly to the customer terminal 12, the server 13 is a device that can transmit and receive various data to and from other communication devices connected to the Internet 11. In the present embodiment, the server 13 is realized by causing a general-purpose computer as shown in FIG. 2 to perform processing according to Web server software. However, the general-purpose computer that functions as the server 13 does not need to have a digital camera. The CPU of the server 13 transmits various Web page data to the requesting communication device in response to a request from another communication device according to the Web server software stored in the HD.

  The manufacturer terminal 14 is a terminal device used by the staff of a cosmetic manufacturer. The store terminal 15 is a terminal device used by the staff of the cosmetic store. Like the customer terminal 12, the manufacturer terminal 14 and the dealer terminal 15 are devices that can transmit and receive various data to and from other communication devices connected to the Internet 11. In the present embodiment, the maker terminal 14 and the store terminal 15 are also realized by causing a general-purpose computer as shown in FIG. 2 to perform processing according to Web browser software. However, the general-purpose computer that functions as the manufacturer terminal 14 and the store terminal 15 does not need to have a digital camera.

[1.2. Configuration of data stored by server]
The HD of the server 13 stores various programs such as Web server software, an OS (Operation System), and a guidance program for causing the server 13 to perform an information providing service in the guidance system 1. As shown in FIG. 3, the HD of the server 13 includes a customer database 21, a cosmetic database 22, a face still image data group 23, model face feature point data 24, model face shadow data 25, a process database 26, and coloring characteristics. A database 27, an HSB-RGB correspondence table 28, a manufacturer database 29, a store database 30, a marketing related database 31, and a web page data group 32 are stored.

  The customer database 21 includes a plurality of customer data, which are data related to customers. The customer data is created by the server 13 based on data transmitted from the customer terminal 12 to the server 13. FIG. 4 is a diagram illustrating the configuration of the customer database 21. The customer data includes items of “customer basic information”, “customer skin quality information”, “face still image data file name”, “cosmetic condition information”, and “selected cosmetic name”.

  “Basic customer information” has sub-items such as “customer ID”, “customer password”, “age”, “gender”, “profession”, and “address” of the customer.

  “Customer skin quality information” has sub-items of “skin quality”, “repellent cosmetic name” and “symptom” of the customer. Here, “skin quality” is information that specifies the skin quality of the customer according to a classification such as normal skin, dry skin, and the like. “Repellent cosmetic product name” means a cosmetic product that has been used by a customer in the past. This “repellent cosmetic name” may be plural. “Symptom” indicates the type of trouble such as itching or swelling that occurs when the cosmetic product is used with respect to each “repellent cosmetic product name”.

  “Face still image data file name” is a file name of still image data obtained by photographing a customer's face with a digital camera. This file name indicates one of the face still image data included in the face still image data group 23.

  The “cosmetic condition information” has sub-items corresponding to various cosmetic categories such as “skin lotion”, “foundation (liquid)”, “eye shadow (accent color)”, and “lipstick”. Each sub-item includes condition data indicating conditions desired by the customer regarding the corresponding cosmetic classification. For example, if the sub-item “Lipstick” contains the condition data “Price: High or Medium”, this condition data indicates that the customer is considering purchasing a lipstick with a higher price or a medium price lipstick. It shows that. As the type of condition data, in addition to “Price”, for example, “Hue”, “Saturation”, “Brightness”, “Gloss”, “Price”, “Water resistance”, “UV protection”, “Release date” Various names such as “similar color cosmetic name” and “manufacturer name” are conceivable.

  The “selected cosmetic name” has sub-items corresponding to various cosmetic classifications. Each sub-item indicates the name of the cosmetic selected by the customer as the cosmetic used for virtual makeup using the still image data of the customer's face described later.

  The cosmetic database 22 includes a plurality of cosmetic data relating to cosmetics sold at that time. The cosmetic data is created by the server 13 based on data transmitted from the manufacturer terminal 14 to the server 13. FIG. 5 is a diagram showing the configuration of the cosmetic database 22. The cosmetic data relating to one cosmetic is the “cosmetic name”, “manufacturer name”, “cosmetic classification”, “oil content ratio”, “component”, “symptom”, “hue”, “saturation”, “lightness” of the cosmetic. ”,“ Transparency ”,“ Viscosity ”,“ Content ”,“ Price ”,“ UV Transmissivity ”,“ Water Resistance ”,“ Other Properties ”,“ Release Date ”, and“ On Sale ”and It has an item "Pop-up comment".

  Here, the “oil content ratio” is a numerical value representing the weight ratio of the fat component contained in the cosmetic as a percentage. “Ingredient” includes one or more names of ingredients contained in the cosmetic. The “symptom” is a name of a symptom that may be caused by a component indicated by a name included in the “component”.

  “Hue” represents the hue of a cosmetic by a numerical value from 0 to 100. In this embodiment, 0 indicates red, and changes from red → yellow → green → blue → purple → red as the value increases. “Saturation” is a numerical value from 0 to 100 indicating the vividness of the color of cosmetics. The greater the “saturation”, the brighter the color. “Lightness” is a numerical value from 0 to 100 indicating the brightness of the color of the cosmetic. The larger the “lightness” value, the brighter the color.

  “Transparency” is a numerical value from 0 to 100 that indicates the percentage of light that is transmitted when the cosmetic is spread to a certain thickness. In the present embodiment, the constant thinness is 1 mg / pixel. In this case, “pixel” is the area of one pixel in the face still image data. “Viscosity” is a numerical value indicating the viscosity of a cosmetic. The unit of “viscosity” is Pa · s. “Internal capacity” indicates the internal capacity of one sales unit of cosmetics, and “Price” indicates the manufacturer's suggested retail price of one sales unit of cosmetics.

  The “ultraviolet ray transmittance” is a numerical value from 0 to 100, which indicates the percentage of ultraviolet rays that are transmitted as a percentage when the cosmetic is spread to a thickness of 1 mg / pixel. “Water resistance” is a numerical value indicating the percentage by weight of the cosmetics remaining on the skin after a certain period of time after the cosmetics are spread on the skin in a state of being spread to a thickness of 1 mg / pixel. As “other characteristics”, for example, when “cosmetics classification” is mascara, data on characteristics that differ depending on cosmetics, such as a ratio of the eyelash volume increased by the mascara, and the like can be considered.

  “On sale” is data indicating whether the cosmetic is currently on sale, and is “Yes” or “No”. “Pop-up comment” indicates a message for PR of the cosmetic. This “pop-up comment” is blank except for cosmetic data relating to specific cosmetics requiring PR.

  The face still image data group 23 is a three-dimensional object imitating the shape of a head used as a model in the guidance system 1, that is, a head mannequin (hereinafter referred to as a “model mannequin”) and a customer's face taken with a digital camera. It is a collection of still image data obtained by doing. Hereinafter, the still image data included in the face still image data group 23 is referred to as “face still image data”, the model mannequin face still image data is “model face still image data”, and the customer face still image data is “customer face data”. This is called “still image data”. In addition, the resolution of the face still image data is assumed to be 320 × 240 = 76,800 pixels as an example. The model face still image data is acquired by photographing by the staff of the service provider of the guidance system 1 and stored in the HD of the server 13 in advance. The customer face still image data is transmitted from the customer terminal 12 to the server 13 and stored in the HD of the server 13.

  FIG. 6 shows a still image represented by the model face still image data. However, the coordinate axes and ● marks 1 attached to the still image in FIG. 6 are shown for convenience in the following description and are not included in the actual still image.

  The model face feature point data 24 is data relating to feature points in the model face still image data. A feature point is a point whose position can be uniquely specified in face still image data. In addition, “point” is originally a concept that indicates only a position without an area, but in the following description, “point” indicates one pixel in face still image data.

  FIG. 7 is a diagram illustrating the contents of the model face feature point data 24. The model face feature point data 24 corresponding to one feature point has items of “identification number”, “name”, “position”, and “necessity of matching” of the feature point. As the “name” of the feature point, various things such as the intersection of the center line of the face and the face outline can be considered in addition to the left and right corners of the eyes and the left and right mouth edges.

  “Position” represents the position of the feature point in the XY coordinate system shown in FIG. More specifically, in this XY coordinate system, the horizontal line in contact with the still image indicated by the face still image data from the top is the X axis, the vertical line in contact with the left of the still image is the Y axis, The downward direction is the positive direction with respect to the Y axis. The mark 1 in FIG. 6 illustrates the coordinates (25, 40) of the right eye corner.

  “Necessity of matching” means “necessary” or “unnecessary” of matching. In the guidance system 1, as will be described later, matching processing of corresponding feature point positions is performed between the customer face still image data and the model face still image data. “Necessary” indicates that the feature point needs to be matched. On the other hand, “unnecessary” indicates that the feature point does not need to be matched. The model face feature point data 24 is created by the service provider staff of the guidance system 1 and stored in advance in the HD of the server 13.

  The model face shadow data 25 is data indicating shadows at each position of the model face. FIG. 8 is a diagram illustrating the contents of the model face shadow data 25. The model face shadow data 25 corresponding to one position has items of “position” and “shade degree”. “Position” indicates the coordinates of each point in the model face still image data. “Shading degree” is a numerical value from 0 to 100 indicating the degree of shading at the point indicated by “position”. For example, if the brightness of a point when there is no shadow is 30 and the shadow degree corresponding to the position of the point in the model face shadow data 25 is 40, the brightness of the point is 30 × 40 / 100 = 12.

  As described above, the model face shadow data 25 is obtained by photographing the model mannequin under normal light and by photographing the model mannequin at the same angle in a state where the model mannequin is irradiated with light from all sides. The ratio of the brightness of each point of the obtained face still image data is expressed as a percentage, created by the service provider staff of the guidance system 1, and stored in the HD of the server 13 in advance.

  The process database 26 includes a plurality of process data relating to the makeup process. The process data is created by a makeup artist and stored in advance in the HD of the server 13 by the staff of the service provider of the guidance system 1. A method for creating process data by a makeup artist will be described later.

  FIG. 9 is a diagram showing the configuration of the process database 26. The process data corresponding to one process includes “name”, “classification” of the process, “comment” regarding the process, and one or more “steps” that describe the contents of the process, that is, “step 1” to “step 1” to “step 1” to “step 1”. Step k "is included.

  “Category” includes one or a plurality of key data specifying the classification to which the process belongs. As this key data, for example, “Artist: Artist B”, “Image: Cute”, “Talent: Talent M”, and the like can be considered. “Artist: Artist B” indicates that the process is designed by Artist B. “Image: cute” indicates that this process is a cosmetic process for obtaining a cute image. “Talent: Talent M” indicates that this process is a cosmetic process for obtaining a result similar to that frequently applied to the talent M.

  “Comment” is a message introducing the contents of this process. Each of “Step 1” to “Step k” is data relating to each process (hereinafter referred to as “step”) when a series of makeup processes is divided into a single process. Each of “Step 1” to “Step k” includes “guidance”, “cosmetic classification” of cosmetics used in the step, “tool” used in the step, “operation” and “processing data” in the step. Has sub-items. As the “operation”, various things such as “painting”, “stroking”, “happing”, “cutting”, “pulling”, “pressing”, “putting”, and the like can be considered.

  “Processing data” is data such as the position of the center point of the tool and the pressure between the tool and the skin when the operation indicated by “Action” is performed using the tool indicated by “Tool”. . The format of data included in “process data” varies as follows depending on the content of “operation”.

When the “operation” is “painting”, “patting”, etc., the “processing data” is a group of a plurality of time-series three-dimensional vectors as follows.
(20, 25, 10), (21, 23, 11), (23, 19, 15), ...
(35, 48, 25), (33, 46, 28), (32, 43, 31), ...
(47, 53, 41), (48, 53, 38), (51, 50, 27), ...

  In the above example, in the first row, the first time-series three-dimensional vector group representing the first painting operation, and in the second row, the second time-series three-dimensional vector group representing the second painting operation, In the third row, a third time-series three-dimensional vector group representing the third painting operation is shown. However, the number of vector groups is not limited to three, and it is sufficient to provide as many operations as necessary.

  The first component and the second component of each vector are respectively the X and Y coordinates of the center point of the tool in the model face still image data shown in FIG. 6, and the third component is the pressure between the tool and the skin. (Pa). In each time series vector group, each vector indicates the position and pressure of the tool every 1 ms sequentially from the top. That is, the processing data includes time-series position data indicating temporal changes in the position of the tool and time-series pressure data indicating pressure at each position.

When “operation” includes “slap”, “press”, “place”, etc., “process data” is a group of five-dimensional vectors as follows.
(25, 30, 115, 30, 0), (43, 56, 102, 38, 12), ...

  In each of the vectors, the first component and the second component are the X coordinate and Y coordinate of the center point of the tool, respectively, and the third component is the maximum pressure (Pa) between the tool and the skin at the center point of the tool. The fourth component is the time (ms) from the start of contact between the tool and the skin until the end of contact. The fifth component is the start of contact between the tool and the skin indicated by the preceding vector, and the contact start of the tool and the skin indicated by the vector. Time (ms) is shown. That is, the processing data includes position data indicating the position of the tool in time series, pressure data indicating the pressure at each position, and time data indicating how to apply the pressure and an interval between the pressure data. For example, each vector represents 1 hit in the operation of “happing”.

When “Cut”, “Pull”, etc. are included in “Operation”, “Processing data” is the following two-dimensional vector group.
(25, 30), (26, 32), (27, 35), ...

  In the above vector, the first component and the second component respectively indicate the X coordinate and the Y coordinate on the still image indicated by the model face still image data, and the above vector group as a whole is a certain region on the still image. The outer edge is shown.

Further, when “Buhler” is included in “Tool”, “Curl” is included in “Operation”, and “Processing data” is a two-dimensional vector as follows.
(3,4)

  In the above vector, the first component is a numerical value from 1 to 10 indicating the strength of curling of the eyelashes, and the second component is a numerical value from 1 to 10 indicating the shape of the curl. For example, the larger the value of the first component, the stronger the curl, the smaller the value of the second component, the relatively strong curl near the root, and the larger the value of the second component, the closer to the hair tip. Represents a shape with relatively strong curl.

  In addition, when “mascara comb” is included in “tool”, “operation” includes “toku”, and “processing data” is blank. This is because the position where the mascara comb is used is determined by the feature points, and the makeup result does not change depending on the strength of the mascara comb used. Note that the processing data described above is an example, and it goes without saying that processing data relating to various other operations and processing data in various other formats are also applicable to the present invention.

  The coloring characteristic database 27 includes a plurality of coloring characteristic data for changing the color of the still image indicated by the customer face still image data based on the data included in “Step 1” to “Step k” of the process database 26. Yes. The coloring characteristic data is created by the service provider staff of the guidance system 1 and stored in advance in the HD of the server 13.

  FIG. 10 is a diagram showing the configuration of the coloring characteristic database 27. The coloring characteristic data includes the items “tool”, “motion”, “vector from the center point”, “tool to skin”, “skin to tool”, and “from current position to next position”. . “Tool” is a name of a tool used for makeup. “Operation” is a name of an operation performed using the tool shown in “Tool”.

  The “vector from the center point” is a position vector from the center point of the tool. When the “motion” indicates a motion with movement such as “painting”, the “vector from the center point” indicates coordinates in a coordinate system having the center point of the tool as the origin and the direction of movement as the X axis. When the “motion” indicates a motion that does not involve movement such as “happing”, the “vector from the center point” indicates coordinates in a coordinate system in which the center point of the tool is the origin and the vertical direction is the X axis. Further, when the “motion” is “curl” or the like, and the position to be the target of the motion is determined in advance by the feature point, the “vector from the center point” is blank.

  “From tool to skin”, “from skin to tool”, and “from current position to next position” include functional expressions for calculating the amount of movement of the cosmetic product at the position indicated by “vector from center point”. Therefore, when information indicating an operation that does not involve the movement of cosmetics such as “cut” is included in “motion”, “from tool to skin”, “from skin to tool”, and “from current position to next position” All are blank.

  “From tool to skin” includes a functional expression for calculating the amount of cosmetic movement from the tool to the skin. Similarly, “from skin to tool” includes a functional expression for calculating the amount of cosmetics transferred from the skin to the tool. “From current position to next position” is a function for calculating the amount of movement of the cosmetic product from the position before the movement to the position after the movement when the “motion” indicates a movement involving movement such as “painting”. Contains an expression. When “motion” includes information indicating a motion that does not involve movement, such as “happing”, “from current position to next position” is blank.

  For example, when “painting” is included in “motion”, “tool to skin”, “skin to tool”, and “skin to tool” regarding the point indicated by the vector (x, y) included in “vector from the center point” “From the current position to the next position” generally includes a function expression represented by the following expression (1).

ただし、関数ｆの変数は、それぞれ以下のとおりである。
Ｖ_t：用具側の化粧品の粘度（Ｐａ・ｓ）、
Ｖ_s：肌側の化粧品の粘度（Ｐａ・ｓ）、
ｖ：用具の中心点の移動速度（ｐｉｘｅｌ／ｍｓ）（ただし、「ｐｉｘｅｌ」は顔静止画データにおける隣り合う画素の中心点間の距離）、
ｐ：用具の中心点と肌との間の圧力（Ｐａ）。

However, the variables of the function f are as follows.
V _t: the viscosity of the tool side of the cosmetics (Pa · s),
V _s : the viscosity (Pa · s) of the cosmetic on the skin side,
v: Moving speed of the center point of the tool (pixel / ms) (where “pixel” is the distance between the center points of adjacent pixels in the face still image data),
p: Pressure (Pa) between the center point of the device and the skin.

For example, taking the data in the first row of FIG. 10 as an example, when applying cosmetics using a lip brush, the point first indicated by the vector (12, 20) from the center point of the lip brush is attached to the lip brush. Cosmetics in the ratio indicated by f _(12,20) = (V _s × p) / (V _t × v) × 37 (%) move from the lip brush to the skin and adhere to the skin at the same time Among the cosmetics that are being used, cosmetics having a ratio of g _(12,20) = (V _t × p) / (V _s × v) × 5 (%) move from the skin to the lip brush. After that, among the cosmetics adhering to the point at the current position, the cosmetic having the ratio represented by h _(12,20) = (V _t × p) / (V _s × v) × 22 (%) Move to the point. From these pieces of information, the amount of cosmetics adhering to the skin at each point is determined at each time point, and the color information at each point of the face still image data is determined according to the determined amount of cosmetics. A method for determining the color information will be described later.

  The above-described function formula can be obtained by obtaining the relationship between the measured value of the amount of movement of the cosmetic obtained by experiment and each variable by a statistical method such as regression analysis. It should be noted that the above function expression is merely an example, and it goes without saying that the structure of the function expression, the type of variable, the value of the constant, and the like can be variously changed in the implementation of the present invention.

  The HSB-RGB correspondence table 28 is a combination of hue (Hue), saturation (Saturation) and lightness (Brightness) (hereinafter referred to as “HSB”) and the three primary colors of light, that is, RGB (Red, Green and Blue). It is a table | surface which shows a response | compatibility with a combination. In the present embodiment, it is assumed that each element of RGB takes an integer value from 0 to 255. The HSB-RGB correspondence table 28 is stored in advance in the HD of the server 13 by the staff of the service provider of the guidance system 1. FIG. 11 is a diagram showing the configuration of the HSB-RGB correspondence table 28. For example, according to the data in the second row of the HSB-RGB correspondence table 28, the colors represented by hue = 0, saturation = 0, lightness = 1 are represented by R = 2, G = 2, and B = 2. The

  The manufacturer database 29 is a database that stores the email addresses of cosmetic manufacturers. FIG. 12 is a diagram showing the configuration of the manufacturer database 29. The manufacturer database 29 includes a plurality of manufacturer data related to each manufacturer. The maker data includes items of “maker name” and “maker mail address”. “Manufacturer name” is the name of the manufacturer. “Manufacturer mail address” is the mail address of the manufacturer.

  The store database 30 includes a plurality of store data, which are data related to products handled by cosmetic stores. The store data is created by the server 13 based on data transmitted from the store terminal 15 to the server 13 and stored in advance in the HD. FIG. 13 is a diagram showing the configuration of the store database 30. The dealer data includes items of “store name”, “store mail address”, “handling product”, “delivery date”, “price”, and “shipping cost” of the dealer.

  “Handling product” indicates the name of cosmetics as cosmetic identification data for specifying the cosmetics handled by the dealer. “Delivery date” indicates the number of days required from the time when an order is received until the cosmetic indicated by the name of “handled product” is shipped. “Price” indicates the sales price of one sales unit of cosmetics indicated by the name of “handled product”. “Shipping” indicates the shipping cost of the ordered cosmetic.

  The marketing-related database 31 includes a plurality of marketing-related data, which are data that can be used in the business activities of cosmetic manufacturers and cosmetic stores among the data related to cosmetics transmitted from the customer terminal 12 to the server 13. The marketing related data is created by the server 13 based on data transmitted from the customer terminal 12 to the server 13 while the guidance system 1 is being used by the customer, and sequentially stored in the HD. FIG. 14 is a diagram showing the configuration of the marketing related database 31. The marketing related data includes items of “age”, “sex”, “occupation”, “prefecture”, “cosmetics classification”, “behavior classification”, “content”, and “date”.

  “Age”, “Gender”, “Occupation”, and “Prefecture” are attribute data indicating the attribute of the customer who transmitted the marketing related data, that is, the name of the prefecture of age, gender, occupation, and address. “Cosmetics classification” is a name of cosmetic classification. The “behavior classification” is any one of “extraction condition”, “selection”, and “purchase” indicating the behavior by the customer. The “extraction condition” indicates that the marketing related data relates to the condition data input by the customer for extracting cosmetics. “Selection” indicates that the marketing related data relates to a cosmetic selected by the customer as a cosmetic used for virtual makeup described later. “Purchase” indicates that the marketing related data relates to the cosmetics for which the customer places an order for purchase. “Content” indicates the condition data input by the customer, the name of the cosmetic, and the like. “Date” indicates a date when the marketing related data is acquired.

  The web page data group 32 is a collection of data related to various web pages that the server 13 transmits to the manufacturer terminal 14 and the store terminal 15 according to the web server software, and is created in advance by the staff of the service provider of the guidance system 1. It is stored in the HD of the server 13.

[1.3. Configuration of process data creation terminal]
As described above, the process data included in the process database 26 is created by the makeup artist and stored in the HD of the server 13 by the staff of the service provider of the guidance system 1. FIG. 15 is a diagram showing an appearance of a process data creation terminal 41 used when a makeup artist creates process data. The process data creation terminal 41 includes a model mannequin 411, an A / D converter 412, and a general-purpose computer 413.

  The model mannequin 411 is a virtual standard Japanese head feature obtained by statistically processing information such as the shape of the heads of many Japanese, the location of each part, and the color. It is a mannequin having The surface of the model mannequin 411 is covered with a very thin and soft coating of silicon or the like, and minute and highly sensitive pressure sensors are arranged at a sufficiently dense constant interval under the coating. FIG. 15 shows the model mannequin 411 with the covering removed. Note that the model face still image data shown in FIG. 6 is still image data obtained by photographing the model mannequin 411 with the covering.

  In FIG. 15, the ◯ mark and the ● mark on the model mannequin 411 indicate the positions of the pressure sensors, respectively. In the present embodiment, it is assumed that a total of 100,000 pressure sensors are arranged on the surface of the model mannequin 411. Therefore, almost one pressure sensor can be associated with one pixel of the model face still image data. However, in FIG. 15, the pressure sensors are arranged more coarsely than in actuality for simplification of the drawing.

  The ● marks on the model mannequin 411 are pressure sensors colored in different colors. As described above, when some of the pressure sensors are colored in different colors, the corresponding position of each pressure sensor in the model face still image data can be specified. A method for specifying the corresponding position of each pressure sensor in the model face still image data will be described later.

  Each of the pressure sensors arranged on the model mannequin 411 is connected to the A / D converter 412. The A / D converter 412 is an A / D (Analog to Digital) converter that simultaneously receives a plurality of analog signals and converts the analog signals into digital data. The digital data generated by the A / D converter 412 is transmitted to the general-purpose computer 413, and is recorded and edited by processing according to the process data creation program of the general-purpose computer 413, as will be described later. In the present embodiment, the process data creation terminal 41 is realized by the general-purpose computer 413 connected to the model mannequin 411 via the A / D converter 412 performing processing according to the process data creation program. However, instead of using a general-purpose computer, a dedicated device having the same function may be used. Further, a part or all of the devices that realize the functions of the model mannequin 411, the A / D converter 412, and the general-purpose computer 413 may be arranged in one housing.

  The A / D converter 412 has 100,000 input terminals, and can convert up to 100,000 independent analog signals into digital data at the same time and output them. Identification numbers from 1 to 100,000 are assigned to the input / output systems of the A / D converter 412. The A / D converter 412 converts an analog signal input from each pressure sensor into digital data, for example, every 1 ms, and outputs the obtained digital data together with an input / output system identification number to the general-purpose computer 413. Hereinafter, the pressure-related data converted into digital data by the A / D converter 412 is referred to as “pressure data”.

  The general-purpose computer 413 is a general-purpose computer having the same configuration as the general-purpose computer that functions as the customer terminal 12 illustrated in FIG. However, the general-purpose computer 413 does not need to have a digital camera. The HD of the general-purpose computer 413 includes model face still image data, uncovered model face still image data, and pressure sensor in addition to various programs such as an OS, a pressure sensor position data creation program and a process data creation program described below. Stores position data.

  The model face still image data stored in the HD of the general-purpose computer 413 is the same as the model face still image data stored in the HD of the server 13, and the model mannequin 411 with the cover is attached to the digital camera. This is still image data obtained by photographing. The uncovered model face still image data is still image data obtained by photographing the model mannequin 411 with the cover removed with a digital camera. The model face still image data and the uncovered model face still image data are prepared in advance by the staff of the service provider of the guidance system 1 and stored in the HD of the general-purpose computer 413.

  The pressure sensor position data is data indicating the corresponding position of the pressure sensor on the model mannequin 411 in the model face still image data. FIG. 16 is a diagram illustrating the configuration of the pressure sensor position data. Pressure sensor position data corresponding to a certain pressure sensor has items of “identification number” and “position” of the pressure sensor. Here, “position” indicates coordinates corresponding to the position of the pressure sensor in the model face still image data.

  Based on the model face still image data and the uncovered model face still image data, the CPU of the general-purpose computer 413 performs processing according to the pressure sensor position data creation program stored in the HD. Created. Note that it is usually the service provider staff of the guidance system 1 that gives the general-purpose computer 413 an instruction to create pressure sensor position data.

  FIG. 17 is a diagram illustrating still images represented by model face still image data (left) and uncovered model face still image data (right). Since these still images are still images obtained by photographing the same model mannequin 411, the positions of corresponding parts in each still image are almost the same, but depending on the angle of the digital camera at the time of photographing, etc. There is some deviation. The pressure sensor position data creation program corrects the deviation and creates data indicating the corresponding position of the pressure sensor in the model face still image data from the data indicating the position of the pressure sensor in the uncovered model face still image data. Is a program that causes the CPU to perform

  The pressure sensor position data creation program has data relating to the shape and color of the pressure sensor and the data relating to the shape and color of the part corresponding to each feature point of the face still image data as attached data. By comparing the attached data and the face still image data, the position of the pressure sensor or the feature point in the face still image data can be specified. Hereinafter, the process of specifying the position of the pressure sensor and the feature point in the face still image data by the CPU is referred to as “matching process”. The pressure sensor position data creation program has, for example, data related to the interval between the pressure sensors and the approximate position of each feature point as attached data, and causes the CPU to perform determination processing using those data. The accuracy of the matching process may be improved.

  FIG. 18 is a flowchart of processing performed by the CPU of the general-purpose computer 413 in accordance with the pressure sensor position data creation program. First, the CPU performs pressure sensor matching processing on the uncovered model face still image data, and stores the coordinates of the pressure sensor in the uncovered model face still image data in the HD as pressure sensor position data in the format shown in FIG. Step S11).

  In the process of step S11, the CPU first specifies the position of the pressure sensor indicated by ● in FIG. 17, and specifies the identification number of the pressure sensor based on the color information attached to the pressure sensor. Subsequently, the CPU identifies the position of the pressure sensor indicated by a circle, and sequentially identifies the identification number of the pressure sensor based on the positional relationship with the pressure sensor of the circle mark where the identification number has already been identified. In this way, the contents of “identification number” and “position” in the pressure sensor position data are determined. However, for a pressure sensor that is not included as an image in the uncovered model face still image data because it is disposed on the back of the model mannequin 411, the “position” in the pressure sensor position data is blank.

  Following the processing in step S11, the CPU performs feature point matching processing on the uncovered model face still image data, creates uncovered model face feature point data using the coordinates of the specified feature points, and stores them in the HD. (Step S12). FIG. 19 is a diagram illustrating a configuration of uncovered model face feature point data. The uncovered model face feature point data has almost the same configuration as the model face feature point data 24 shown in FIG. 7 and has items of “identification number”, “name” and “position”. It does not have an item “Necessity”. This is because the feature points of the uncovered model face still image data and the model face still image data all need to be matched, and it is not necessary to distinguish whether or not they need to be matched. If the CPU fails to specify a feature point in the process of step S12, the item “position” is blank for the feature point.

  Subsequent to the process of step S12, the CPU determines whether or not there is a specific feature point failure in the process of step S12, that is, whether or not there is data in which the item “position” of the uncovered model face feature point data is blank (step S13). ). If the determination result in step S13 is “Yes”, the CPU sends a message instructing the user to manually identify the feature points that failed to be identified, along with the still image represented by the uncovered model face still image data. Display on the display. In this case, as an example of the message displayed on the display unit, for example, “The left eye corner could not be specified. In the image displayed on the screen, click the position of the left eye corner with the mouse.” Things can be considered. In response to the message, when the user clicks the designated position with the mouse, the CPU uses the coordinates in the uncovered model face still image data designated by the click operation as the “position” of the uncovered model face feature point data. Store (step S14). The CPU repeats the processes of steps S13 and S14 until the “position” of all data in the uncovered model face still image data is not blank.

  If the determination result in step S13 is “No”, the CPU performs feature point matching processing on the model face still image data, creates model face feature point data using the coordinates of the identified feature points, and stores them in the HD. (Step S15). The model face feature point data created in step S15 is the same as the model face feature point data 24 stored in the HD of the server 13 shown in FIG. Also in step S15, when the CPU fails to specify a feature point, the item “position” is blank for the feature point.

  Subsequent to step S16, the CPU performs the same process as steps S13 and S14 on the model face feature point data (steps S16 and S17). If there is no blank in the “position” of the model face feature point data due to the processing in step S17 and the determination result in step S16 is “No”, the CPU performs correction processing of pressure sensor position data described below (step S18). ).

  FIG. 20 is a view for explaining correction processing of pressure sensor position data. In FIG. 20, point A, point B, and point C indicate arbitrary feature points near each other in the uncovered model face still image data. A point A ′, a point B ′, and a point C ′ indicate arbitrary feature points near each other in the model face still image data. However, the point A and the point A ′, the point B and the point B ′, and the point C and the point C ′ are respectively corresponding feature points.

  Point N is a point indicating the position of the pressure sensor located inside the triangle ABC. In the process of step S18, the CPU acquires the coordinates of the points A, B, and C from the uncovered model face feature point data. In addition, the CPU acquires the coordinates of the points A ′, B ′, and C ′ from the model face feature point data. Further, the CPU acquires the coordinates of the point N from the pressure sensor position data created in step S11. Subsequently, the CPU calculates the coordinates of a point N ′ that satisfies the following formula (2).

ただし、「△」は三角形の面積を示す。

However, “Δ” indicates the area of a triangle.

  The CPU replaces the content of the “position” of the data related to the pressure sensor located at the point N in the pressure sensor position data with the coordinates calculated according to the equation (2). The CPU similarly replaces data for all data in which “position” is not blank in the pressure sensor position data. As a result, the coordinates included in the “position” of the pressure sensor position data are corrected from the coordinates indicating the position of each pressure sensor in the uncovered model face still image data to the coordinates indicating the corresponding position in the model face still image data. The

  Formula (2) is an example of a calculation formula that can be used in the processing of step S18, and an arbitrary point N in the region specified by a certain three points and a point in the region specified by the other three points Any calculation formula may be used as long as N ′ can be associated with each other as a bijective mapping.

[1.4. Create process data]
As described above, when the pressure sensor position data subjected to the correction processing is stored in the HD of the general-purpose computer 413, process data can be created by a makeup artist.

  The makeup artist attaches a cover to the model mannequin 411, and starts the process data creation program with the model mannequin 411, the A / D converter 412 and the general-purpose computer 413 connected as shown in FIG. FIG. 21 is a process data creation screen displayed by the general-purpose computer 413 according to the process data creation program. On the process data creation screen, an input item group 51 for the makeup artist to input various data, pressure data recorded by the makeup manikin 411 measured by the model mannequin 411 and converted into digital data by the A / D converter 412, etc. A command button group 52 for instructing is included.

  The input item group 51 includes input items corresponding to the respective items excluding “processing data” in the process database 26 illustrated in FIG. 9. The makeup artist first inputs appropriate information in the “name”, “classification”, and “comment” fields. Subsequently, the makeup artist enters a number indicating each step in the makeup process in “step number”, and provides information regarding the step of that number as “guidance”, “cosmetic classification”, “tool”, and “motion”. To enter. Hereinafter, the operation when the number “n” is input to the “step number” by the makeup artist and the data relating to “step n” is input and created will be described.

  When the process of “step n” relates to an operation of “painting” “lipstick”, for example, the makeup artist performs an information input operation on the input item group 51, and then performs “recording start” of the command button group 52. Click. In response to the click operation, the CPU starts determining whether or not the value of the pressure data transmitted from the A / D converter 412 exceeds a certain value. The CPU stands by without recording the pressure data while all the values of the pressure data transmitted from the A / D converter 412 are equal to or less than a certain value.

  Subsequently, the makeup artist uses the cosmetics classified in “cosmetics classification” in the input item group 51 and the tools entered in “tools” to the model mannequin 411 for makeup relating to “step n”. Process. Hereinafter, it is assumed that the make-up artist attaches lipstick to the lip brush and colors the lip portion of the model mannequin 411 with the lip brush in the creation of the data of “Step n”.

  When the makeup artist performs a coloring operation on the model mannequin 411 with the lip brush, the pressure sensor disposed on the lip portion of the model mannequin 411 converts the analog signal corresponding to the pressure applied by the lip brush to an A / D converter. 412 is output. The A / D converter 412 converts the analog signal input from the pressure sensor into pressure data and transmits it to the general-purpose computer 413. As a result, since the general-purpose computer 413 receives pressure data exceeding a certain value, the CPU of the general-purpose computer 413 starts recording pressure data received thereafter with the reception of the pressure data as a trigger. That is, the CPU sequentially records a set of 100,000 pressure data transmitted from the A / D converter 412 every 1 ms in the RAM. Hereinafter, a set of pressure data stored in the RAM in time series is referred to as “makeup operation recording data”.

  FIG. 22 is a diagram illustrating a configuration of makeup operation recording data. The makeup operation record data includes items “time” and “1” to “100000”. “Time” is an elapsed time from the start of recording of the makeup operation record data. The item “1” stores pressure data corresponding to the identification number 1 of the input / output system of the A / D converter 412. Similarly, the items from “2” to “100000” store the pressure data corresponding to the identification numbers indicated by the respective item names.

  When the makeup artist finishes the lipstick coloring operation with the lip brush in “Step n”, the makeup artist clicks “Recording Complete” in the command button group 52. In response to the click operation, the CPU completes the recording of the makeup operation record data.

Subsequently, the CPU generates processing data corresponding to the operation of “painting” based on the makeup operation record data. As described in the description of the process database 26, when the “operation” is “paint”, one or more sets of time-series three-dimensional vectors in the following format need to be generated in the “processing data”. However, the first and second components of each vector indicate the coordinates of the center point of the tool, the third component indicates the pressure at the center point, and the vectors are arranged in time series every 1 ms.
(20, 25, 10), (21, 23, 11), (23, 19, 15), ...

  Hereinafter, how the processing data is generated by the CPU will be described. First, regarding each time data of the makeup operation recording data, the CPU plots the pressure recorded in the items from “1” to “100000” at the arrangement position on the two-dimensional plane of the corresponding pressure sensor. The center of gravity is calculated, and the identification number of the pressure sensor corresponding to the center of gravity is obtained. Hereinafter, the pressure sensor corresponding to the center of gravity is referred to as a “center of gravity pressure sensor”.

  Subsequently, the CPU searches the pressure sensor position data shown in FIG. 16 for data in which the value of the “identification number” matches the identification number of the gravity center pressure sensor, and acquires the coordinates indicated by the “position” of the retrieved data. . The coordinates acquired in this way indicate the coordinates of the corresponding point in the model face still image data at the center point when the makeup artist presses the lip brush against the surface of the model mannequin 411.

  Subsequently, the CPU calculates an average value of the pressure data measured by the barycentric pressure sensor and a certain number of pressure sensors arranged around it. The CPU calculates the average value of the pressure data measured by the plurality of pressure sensors in this way to alleviate the inhomogeneity of the pressure applied to the model mannequin 411 by the lip brush, and at the position of the barycentric pressure sensor. This is to obtain a more accurate pressure value.

  The CPU creates a vector in which the coordinates thus obtained are the first component and the second component, and the average value of the pressure data is the third component. The CPU similarly creates vectors for each time of the makeup operation record data, and arranges them in time series. As a result, a time-series three-dimensional vector group corresponding to the operation of “painting” is created as processing data.

  As described above, when the makeup artist clicks “Recording Complete” on the process data creation screen and the processing data is created by the CPU, when the makeup artist clicks “Next”, the CPU inputs the input item group. The information input to 51 and the processing data created as described above are temporarily stored in the RAM. Hereinafter, data temporarily stored in the RAM in this way is referred to as “temporary process data”. The temporary process data is data in the same format as the process data shown in FIG.

  Subsequently, the makeup artist sequentially performs operations similar to the data input and creation operations related to “step n” with respect to “step n + 1”, “step +2”,. As a result, the data is sequentially stored in sub-items corresponding to “step n + 1”, “step n + 2”,.

  When the process of “step n” is a process related to an operation such as “stroking”, the makeup artist clicks “recording start” and then applies to the model mannequin 411 in the same manner as in the case of the “painting” operation described above. By performing makeup processing, processing data in the form of a time-series three-dimensional vector group is recorded in the CPU.

  When the process of “step n” is a process related to operations such as “slap”, “press”, “put”, etc., the makeup artist performs “recording start” as in the case of the “paint” operation described above. After clicking, makeup processing is performed on the model mannequin 411 to cause the CPU to create processing data in the form of the five-dimensional vector group described in the description of the process database 26. The CPU creates processing data in the form of a five-dimensional vector group using the makeup operation recording data, as in the case of creating processing data in the form of a set of time-series three-dimensional vector groups described in “Paint”.

  When the process of “Step n” is a process related to operations such as “Cut” and “Pull”, the makeup artist clicks “Start Recording” and then selects “Cut” on the model mannequin 411. By tracing the outer edge of the region with a pencil or the like, the CPU generates processing data in the form of the two-dimensional vector group described in the description of the process database 26. The CPU creates processing data in the form of a two-dimensional vector group using the makeup operation recording data, as in the case of creating processing data in the form of a set of time-series three-dimensional vector groups described in “Paint”.

  If the process of “Step n” is related to an operation such as “Curl”, the makeup artist directly clicks “Manual Input” instead of “Record Start” and “Record Complete”. Input processing data. For example, when the “motion” is “curl”, as described in the explanation of the process database 26, a two-dimensional vector needs to be created as the processing data. When the makeup artist clicks “manual input”, the CPU displays a processing data input screen on the display unit. The makeup artist can specify arbitrary data as processing data by inputting an arbitrary two-dimensional vector on the input screen and clicking a button such as “OK”. If the process of “step n” is a process that does not involve a change in facial color, such as a face wash, the makeup artist clicks “next” without performing an operation using the command button group 52. As a result, the “process data” of the temporary process data related to the step is blank.

  When the makeup artist finishes inputting and creating data regarding all the steps included in the makeup process by the above operation, the makeup artist clicks “Finish” on the process data creation screen. In response to the click operation, the CPU stores temporary process data as process data in the HD.

  After the process data is stored in the HD of the general-purpose computer 413 by operating the general-purpose computer 413 of the makeup artist, the staff of the service provider of the guidance system 1 transfers the process data from the HD of the general-purpose computer 413 to the FD (Floppy). (Registered trademark) Disk) and the like, and the process data is added to the process database 26.

[1.5. Operation of guidance system]
Next, the operation when the information providing service is performed from the server 13 to the customer terminal 12 or the like in the guidance system 1 will be described. The server 13 creates Web page data to be provided to the customer terminal 12 by performing processing according to the guidance program stored in the HD, and the Web page created by performing processing according to the Web server program By transmitting data to the customer terminal 12 or the like, the information providing service described below is performed for the customer terminal 12 or the like.

  A customer who wants to purchase cosmetics first transmits customer information to the server 13 when using the guidance system 1. Hereinafter, an operation when the customer A uses the guidance system 1 will be described as an example. The customer A operates the customer terminal 12 in advance, takes a picture of his / her face with the digital camera of the customer terminal 12, and stores face still image data obtained by the photography in the HD or the like of the customer terminal 12. The customer A may store face still image data obtained by photographing using a single digital camera in the HD of the customer terminal 12 via a storage medium such as an FD or a cable.

  Subsequently, the customer A inputs the URL (Uniform Resource Locator) of the customer information registration page related to the guidance system 1 on the screen of the Web browser software of the customer terminal 12, whereby the data of the customer information registration page is transmitted to the server 13. Send a send request. In response to the transmission request, the server 13 transmits the customer information registration page data included in the Web page data group 32 to the customer terminal 12. As a result, the customer information registration page shown in FIG. 23 is displayed on the display unit of the customer terminal 12.

  The customer information registration page includes input fields corresponding to the items “customer basic information”, “customer skin quality information”, and “face still image data file name” of the customer data included in the customer database 21 shown in FIG. It is out. Customer A enters appropriate data in these input fields and clicks “Send”. In response to the click operation, the customer terminal 12 transmits to the server 13 the data input by the customer A and the face still image data specified by the file name input in the “face still image data file name”. Of the information entered on the customer information registration page, information relating to skin quality and repellent cosmetics is a condition indicating conditions relating to cosmetic attributes for extracting cosmetics suitable for the constitution of customer A, as will be described later. Used as data.

  When the server 13 receives the data and face still image data input from the customer A from the customer terminal 12, the server 13 first stores the received face still image data in the face still image data group 23 as customer face still image data. Subsequently, the server 13 determines whether there is customer data in the customer database 21 that includes a set of a customer ID and a customer password included in the received information. When the server 13 does not find the customer data including the received customer ID and customer password set in the customer database 21, the server 13 adds the received data to the customer database 21 as new customer data. On the other hand, when the server 13 finds customer data including the received customer ID and customer password set in the customer database 21, the server 13 updates the contents of the customer data with the received data.

  When the customer A completes the above-described transmission of the customer information, the customer A sends the information provision service page data to the server 13 by inputting the URL of the information provision service page on the Web browser software screen of the customer terminal 12. Send a request. In response to the transmission request, the server 13 transmits the cosmetic condition information input page data included in the web page data group 32 to the customer terminal 12 as the first page data of the information providing service page. As a result, the cosmetic condition information input page shown in FIG. 24 is displayed on the display unit of the customer terminal 12.

  In the cosmetic condition information input page, customer A is interested in the customer ID and customer password entered on the customer information registration page and the extraction conditions related to the color, price, etc. of the cosmetics belonging to the cosmetic classification, that is, customer A is interested. Enter sequentially the condition data indicating the conditions related to the attributes of the cosmetics you have and click “Send”. The customer terminal 12 transmits the data input by the customer A to the server 13 in response to the click operation. When the server 13 receives the data input by the customer A from the customer terminal 12, the sub-item corresponding to the “cosmetic condition information” of the customer data including the set of the customer ID and the customer password received from the customer database 21. The received data is stored. Further, the server 13 converts each of the cosmetic condition information received from the customer terminal 12 together with the attribute data such as “age” and “gender” included in the customer data of the customer A and the date of the day to the marketing shown in FIG. Add to related database 31. At that time, “extraction condition” is stored in “behavior classification” of the marketing related data.

  Subsequently, based on the “customer skin quality information” and the “cosmetic condition information” of the customer data relating to the customer A, the server 13 sends the customer A to the customer A from the cosmetic data whose “sale” is “Yes” included in the cosmetic database 22. Extract cosmetic data on suitable cosmetics.

  For example, when “skin quality” of “customer skin quality information” in the customer data is “greasy skin”, the server 13 has cosmetic data with a lower “oil content ratio”, for example, cosmetic data with an “oil content ratio” of 5% or less. To extract.

  Further, for example, when “lipstick K” is displayed in the “repellent cosmetic name” of “customer skin quality information” in the customer data and the “symptom” corresponding thereto is “itch”, the server 13 first reads from the cosmetic database 22. The cosmetic data whose “cosmetic name” is “cosmetic K” is searched, and the content of “component” whose “symptom” is “itch” in the searched cosmetic data is acquired. For example, when it is determined from cosmetic data that “component g” of “lipstick K” may cause “itch”, the server 13 uses “component g” as information indicating a component that is not suitable for customer A. get. The server 13 excludes the cosmetic data including the acquired “component g” in the “component” from the cosmetic data regarding the cosmetic suitable for the customer A.

  Further, for example, when “Price: High or Medium” is stored in “Lipstick” of “Cosmetic Condition Information” in the customer data, the server 13 indicates that “Cosmetic Classification” of the cosmetic data is “Lipstick” and “ A value obtained by dividing the content of “price” by the content of “content” is large or medium, for example, a value of 500 yen or more per gram.

  For example, when “similar color cosmetic name: lipstick H” is stored in “lipstick” of “cosmetic condition information” in the customer data, the server 13 selects “cosmetic name” from all cosmetic data in the cosmetic database 22. ”Is“ lipstick H ”, and for each of“ hue ”,“ saturation ”, and“ lightness ”of the cosmetic data of the searched lipstick H, for example, a range having a width of 5 before and after is obtained. Then, from among the cosmetic data in the cosmetic database 22, “sale” is “Yes” and “cosmetics classification” is “lipstick”, so “hue”, “saturation”, and “lightness” The cosmetic data belonging to the range set by is extracted. The cosmetic data extracted in this way is cosmetic data relating to a lipstick having a color similar to that of the lipstick H.

  After completing the extraction of the cosmetic data as exemplified above, the server 13 classifies the extracted cosmetic data according to the content of “cosmetic classification”. Subsequently, the server 13 reads the template data of the cosmetic selection page from the web page data group 32, and creates cosmetic selection page data corresponding to each cosmetic category using the extracted cosmetic data for each cosmetic category. The server 13 transmits the data of the cosmetic selection page corresponding to the cosmetic classification to the customer terminal 12 in the order of “cleansing agent”, “face cleansing agent”, “skin lotion”,. Upon receiving the cosmetic selection page data, the customer terminal 12 displays the cosmetic selection page on the display unit.

  FIG. 25 is a diagram showing a cosmetics selection page related to lipstick. On the cosmetic selection page, the customer A can obtain information on various cosmetics currently on sale suitable for him / her. The cosmetic selection page includes a cosmetic list in which data that is considered useful when the customer selects cosmetics among the data included in the cosmetic data extracted by the server 13 is included. In the cosmetic list, the actual color is displayed in the “color sample” column instead of “hue”, “saturation”, and “lightness” included in the cosmetic data.

  Of the cosmetic data related to cosmetics included in the cosmetic list, those for which message data indicating a message for PR is included in the “pop-up comment” column are displayed in a pop-up window as shown in the upper right of the cosmetic list in FIG. A message for PR indicated by the message data is displayed, and information in the cosmetic list corresponding to the message is highlighted. When there is a PR message regarding a plurality of cosmetics shown in the cosmetic list, when the customer A closes a pop-up window related to a certain cosmetic, a message related to the next cosmetic is displayed in a new pop-up window. The customer A can know the selling point of each cosmetic product by the message shown in this pop-up window.

  When customer A clicks on characters such as “manufacturer name”, “color sample”, “transparency”, etc. shown in the cosmetics list title on the cosmetic selection page, the customer terminal 12 displays cosmetic condition information corresponding to the clicked character. Displays the input screen. On the input screen, the customer A can input new cosmetic condition information or change the cosmetic condition information already stored in the customer data. When customer A clicks “send” on the cosmetic condition information input screen, the information entered by customer A is the customer terminal 12 in the same manner as when “send” is clicked on the cosmetic condition information input page shown in FIG. To the server 13, the contents of the customer database 21 are updated, the marketing-related data is added to the marketing-related database 31, the cosmetic data is re-extracted, and the contents of the cosmetic list included in the cosmetic selection page are updated.

  On the cosmetics selection page, customer A selects one piece of data relating to cosmetics that he / she wants to try from the cosmetics list and clicks “Select”. In response to the click operation, the customer terminal 12 transmits the cosmetic name indicating the cosmetic selected by the customer A to the server 13 as cosmetic identification data for specifying cosmetic data corresponding to the cosmetic. When the server 13 receives the cosmetic name, the server 13 stores the received cosmetic name in the corresponding cosmetic category sub-item included in the “selected cosmetic name” of the customer data regarding the customer A. Subsequently, the server 13 adds the cosmetic name stored in the customer data to the marketing related database 31 together with the attribute data such as “age” and “sex” included in the customer A customer data and the date of the day. At that time, “selection” is stored in the “behavior classification” of the marketing related data. When the server 13 finishes adding the marketing-related data, the server 13 transmits the data of the cosmetic selection page regarding the next cosmetic classification to the customer terminal 12.

  When the customer A finishes selecting cosmetics for all cosmetics selection pages, and the server 13 finishes adding marketing-related data related to the cosmetics selected by the customer A to the marketing-related database 31, the server 13 starts from the Web page data group 32. The template data of the process selection page is read out, and the data of the process selection page is created using the contents of “name”, “classification”, and “comment” regarding all the process data included in the process database 26. The server 13 transmits the process selection page data to the customer terminal 12. As a result, the process selection page shown in FIG. 26 is displayed on the display unit of the customer terminal 12.

  The process selection page includes a process list that lists data relating to the makeup process. When the customer A clicks the character “category” indicated in the title of the process list, the customer terminal 12 displays an extraction condition information input screen. On the input screen, the customer A can narrow down the data related to the process of interest from the data included in the process list by inputting the condition information related to the process of interest.

  Customer A selects one piece of data related to the process he / she wants to use from the process list and clicks “Select”. In response to the click operation, the customer terminal 12 transmits the name of the process selected by the customer A to the server 13 as process identification data for specifying process data related to the process.

  When the server 13 receives the name of the process selected by the customer A from the customer terminal 12, the data specified by the “face still image data file name” of the customer data relating to the customer A, that is, the customer face still image data of the customer A The feature point extraction process is performed. The feature point extraction processing by the server 13 is the same as the feature point extraction processing in the matching processing by the general-purpose computer 413, that is, the processing in steps S12 to S14 or steps S15 to S17 in FIG. The customer face feature point data is stored in the RAM of the server 13 by the feature point extraction process of the server 13. FIG. 27 is a diagram showing the structure of customer face feature point data. The structure of the customer face feature point data is the same as that of the model face feature point data 24 shown in FIG.

  Subsequently, the server 13 searches the process database 26 for process data including the name of the process selected by the customer A in “name”, and copies the searched process data to the RAM. Hereinafter, the process data copied to the RAM is referred to as “work process data”. For each of “Step 1” to “Step k” of the work process data, the server 13 corrects the coordinates when the vector stored in “Processing data” includes data related to the coordinates. For example, regarding “step n” whose “motion” is “paint”, as described above, the “process data” is a time-series three-dimensional vector group, and the first component and the second component of those three-dimensional vectors are The coordinates in the model face still image data are shown. In order to use this processing data in the customer face still image data, it is necessary to correct the data of the first component and the second component of the three-dimensional vector to data indicating the corresponding position of the customer face still image data. Accordingly, the server 13 performs correction processing similar to that in step S18 in FIG. 18 with respect to the coordinates included in the model face feature point data, the customer face feature point data, and the processing data.

Since the correction processing of data indicating coordinates included in the processing data is the same as step S18 in FIG. 18, the description thereof is omitted. However, in the correction processing of the data indicating the coordinates included in the processing data, the feature points in the model face feature point data, the feature points in the customer face feature point data, and the points indicated by the coordinates included in the processing data are the following in FIG. Corresponds to the point.
Arbitrary arbitrary feature points in the model face still image data: points A, B and C,
Arbitrary arbitrary feature points in the customer face still image data: point A ′, point B ′ and point C ′,
Points indicated by coordinates included in the processing data before correction: point N,
A point indicated by coordinates included in the corrected processing data: point N ′.

  Further, in the correction processing of the data indicating the coordinates included in the processing data, in the model face feature point data 24 and the customer face still image data, “matching necessity” is “unnecessary”. B, point C and point A ′, point B ′ and point C ′ are not included. For example, “necessity of matching” of the feature point related to “right eyebrow” is “unnecessary” and is excluded from correction processing targets. This is because the positions of the eyebrows and the like can be changed by cutting or coloring with an eyebrow pencil, and it is not necessary to match the feature points between the model face feature point data 24 and the customer face still image data.

  As a result of the correction processing of the coordinates included in the processing data as described above, the coordinates included in the processing data are rewritten to coordinates indicating a corresponding point in the customer face still image data.

  The server 13 copies the model face / shadow data 25 to the RAM when the coordinate correction process is completed for all the process data included in the work process data. Hereinafter, the model face / shadow data 25 copied to the RAM is referred to as “working face / shadow data”. The server 13 performs a correction process similar to the correction process of the coordinates included in the processing data described above with respect to the coordinates indicated by the “position” of the work face / shadow data. As a result, the coordinates indicated by the “position” of the work face shadow data are rewritten to the coordinates indicating the corresponding point in the customer face still image data.

  When the server 13 completes the coordinate correction process in the work face / shadow data, the server 13 performs a process of removing the shadow indicated by the work face / shadow data from the customer face still image data. More specifically, the server 13 first converts color information indicated by RGB with respect to each point of customer face still image data, that is, each pixel, into color information indicated by HSB using the HSB-RGB correspondence table 28. . Subsequently, the server 13 subtracts the value indicated by the “shade degree” of the work face shadow data from the brightness obtained by the conversion. The server 13 converts the color information indicated by HSB including the brightness obtained by the subtraction into color information indicated by RGB using the HSB-RGB correspondence table 28. The server 13 can obtain the customer face still image data from which the shadow has been removed by performing the above-described processing for all points included in the customer face still image data. In the following description, the process of removing shadows from the customer face still image data described above is referred to as “shadow removal process”.

  When the server 13 finishes the shadow removal process, the server 13 changes the color information of the customer face still image data. FIG. 28 is a flowchart showing the color information changing process performed by the server 13 with respect to one step in the process data, for example, “step n”.

  First, based on the contents of “cosmetics classification”, “tool”, and “motion” in “step n” of the work process data, the server 13 performs the process indicated by “step n” on the color information for the customer face still image data. It is determined whether or not the process involves a change (step S21). An example of processing that does not involve changing color information for customer face still image data is face washing with a face washing agent, for example. When the determination result in step S21 is “No”, the server 13 proceeds to the process related to “step n” without performing the color information change process for the customer face still image data.

  When the determination in step S21 is “Yes”, the server 13 is indicated by “cosmetic classification” of “step n” of the work process data among the sub-items of “selected cosmetic name” of the customer data regarding the customer A. The cosmetic name stored in the sub-item corresponding to the cosmetic classification is acquired (step S22), and “hue”, “saturation”, “lightness” of the cosmetic data including the cosmetic name acquired in step S22 from the cosmetic database 22 "," Transparency "," viscosity ", and" other characteristics "are acquired (step S23). The server 13 temporarily stores the data acquired in step S23 in the RAM.

  Subsequently, the server 13 searches the coloring characteristic database 27 for coloring characteristic data including the contents of “tool” and “operation” of “step n” of the work process data, and “from the tool” of the searched coloring characteristic data. It is determined whether a functional expression is included in any of “to skin”, “from skin to tool”, and “from current position to next position” (step S24).

  When the determination result of step S24 is “Yes”, the server 13 calculates the function expressions included in “Tool to skin”, “Skin to tool”, and “From current position to next position” of the searched coloring characteristic data. These are acquired and temporarily stored in the RAM (step S25). Thereafter, the server 13 performs a cosmetic amount calculation process for each pixel of the customer face still image data (step S26).

  Hereinafter, the process of step S26 will be described. In the following description, as an example, it is assumed that the data included in “step n” of the work process data and the information acquired in steps S23 and S25 are as follows. However, for the functions included in “from tool to skin”, “from skin to tool”, and “from current position to next position”, “vector from center point” is (0,0) and (3,4) Only what is is illustrated.

"Cosmetics classification": Lipstick,
"Tool": Lip brush,
"Operation": Paint,
“Processing data”: (20, 25, 10), (21, 27, 11), (23, 30, 15),...
“From tool to skin” (0,0): f _(0,0) = (V _s × p) / (V _t × v) × 35,
“From skin to tool” (0,0): g _(0,0) = (V _t × p) / (V _s × v) × 15,
“From current position to next position” (0,0): h _(0,0) = (V _t × p) / (V _s × v) × 7,
“From tool to skin” (3, 4): f _(3,4) = (V _s × p) / (V _t × v) × 25,
“From skin to device” (3,4): g _(3,4) = (V _t × p) / (V _s × v) × 12,
“From current position to next position” (3, 4): h _(3,4) = (V _t × p) / (V _s × v) × 6,
"Hue": 90,
“Saturation”: 85,
“Brightness”: 80,
“Transparency”: 25,
“Viscosity”: 35,
“Other characteristics”: (none).

  The process performed by the server 13 in step S26 is a process of simulating the amount of cosmetics attached at each point on the skin when the cosmetics are applied to the skin. In this case, the server 13 calculates the adhesion amount of the lipstick at each point of the lip portion when the lipstick is applied to the skin of the lip portion by the lip brush. In the simulation, the surface where the lip brush touches the skin is assumed to be a set of points having a minute area of the same size as the pixel in the customer face still image data. Each point on the lip brush is specified by coordinates on a coordinate system in which the center point of the lip brush is the origin (0, 0) and the moving direction of the lip brush is the X axis positive direction. That is, the coordinates of each point on the lip brush are in the same coordinate system as the coordinates indicated by “vector from the center point” in the coloring characteristic data.

The coordinate system (hereinafter referred to as “coordinate system a”) in the customer face still image data and the coordinate system (hereinafter referred to as “coordinate system b”) relating to the coordinates indicated by “vector from the center point” in the coloring characteristic data are different. Coordinate system. FIG. 29 is a diagram illustrating the relationship between the coordinate system a and the coordinate system b. In the following description, the coordinates and vectors of the coordinate system a are distinguished by subscripts such as (x, y) _a , and the coordinates and vectors of the coordinate system b are (x, y) _b .

In FIG. 29, the horizontal right direction is the X-axis direction of the coordinate system a in the customer face still image data, and the vertical downward direction is the Y-axis direction of the coordinate system in the customer face still image data. The squares surrounded by the horizontal and vertical parallel lines in FIG. 29 indicate pixels in the customer face still image data. For example, the point K is a point indicated by (21, 21) _a , and a square including the point is a pixel corresponding to the point K.

In the processing based on the first vector and the second vector of the processing data, the origin of the coordinate system b is a point P (20, 25) _a indicated by the first vector of the processing data. That is, (20, 25) _a and (0, 0) _b are coordinates indicating the same point P. Further, the positive X-axis direction of the coordinate system b is the direction from the point P (20, 25) _a indicated by the first vector of the processing data to the point Q (21, 27) _a indicated by the second vector, that is, is a direction indicated by a vector (1, 2) _a.

For example, R a point represented in the coordinate system b with (3, 4) _b, in the coordinate system a, is approximately (25.0,24.4) _a. Since these coordinate conversion methods are based on known formulas relating to the origin movement and rotation around the origin, description thereof will be omitted. Note that the origin and coordinate axes of the coordinate system b change according to the set of vectors in the processing data used for processing. For example, when the processing based on the second vector and the third vector of the processing data is performed, the origin of the coordinate system b is (21, 27) _a , and the X axis positive direction of the coordinate system b is the vector (2, 3 ) The direction indicated by _a .

In the simulation in step S26, a predetermined amount of lipstick, for example, 10 (mg) is attached to each point of the lip brush as an initial value. This initial value is predetermined by the cosmetic classification and the tool. The viscosity V _t of the lipstick attached to the lip brush is 35 (Pa · s) as assumed above. No cosmetics are initially attached to the skin. In this case, a predetermined value, for example, 65 (Pa · s) is set as the skin-side viscosity V _s .

  The pressure p applied between the lip brush and the skin at the point P which is the current position of the center point of the lip brush is 10 (Pa) from the third component of the first vector of the processing data. The center point of the lip brush moves from point P to point Q in 1 ms. The moving speed v at that time is calculated as about 2.24 (pixel / ms) from the coordinates of the points P and Q.

Accordingly, the values of the variables in the above function formula are determined as follows.
V _t : 35 (Pa · s),
V _s : 65 (Pa · s),
v: about 2.24 (pixel / ms),
p: 10 (Pa).
Using the values of these variables, f _(0,0) = about 84.5 (%), g _(0,0) = about 36.2 (%) and h _(0,0) = about 16.9 (%), F _(3,4) = about 60.4 (%), g _(3,4) = about 29.0 (%) and h _(3,4) = about 14.5 (%) Is done.

The above f _(0,0) is obtained from the point (0,0) _b on the lip brush on the customer face still image data in the movement operation of the lip brush indicated by the first vector and the second vector of the processing data. The ratio of the moving amount of the cosmetics moving to the point P is represented. Thus, a point on the lip brush (0,0) _b about 84.5% of the cosmetic 10mg currently attached, or about 8.45mg the point on lip brush (0,0) _b from the customer face still image Move to the pixel corresponding to the point P on the data. Therefore, the cosmetics adhering to the point (0, 0) _b on the lip brush is about 1.55 mg.

Similarly, g _(0,0) is a point _(0,0) on the lip brush from the point P on the customer face still image data in the movement operation of the lip brush indicated by the first vector and the second vector of the processing data. 0) _Expresses the ratio of the amount of cosmetics moving to _b . Therefore, about 36.2% of the cosmetic 0 mg currently attached to the point P on the customer face still image data, that is, 0 mg is the point (0, 0) on the lip brush from the point P on the customer face still image data. Move to _b .

h _(0,0) is the movement of the cosmetic product from the pixel corresponding to the point P on the customer face still image data to the adjacent pixel in the movement operation of the lip brush indicated by the first vector and the second vector of the processing data. It represents the ratio of quantity. In FIG. 29, the point (0, 0) _b on the lip brush moves the cosmetics attached to the region indicated by the rectangle S in the vector PQ direction by moving from the point P to the point Q. Currently, the amount of cosmetics attached to the pixel corresponding to the point P is about 8.45 mg obtained as described above. Accordingly, about 16.9% of the approximately 8.45 mg of cosmetics adhering to the point P, that is, about 1.43 mg, is applied onto the rectangle S. In the present embodiment, for simplification of the arithmetic processing, about 1.43 mg of the cosmetic to be moved is moved to each pixel in proportion to the area of the overlapping portion of the rectangle S and each pixel. .

The above is the calculation method of the movement amount of the cosmetic product between the lip brush and the skin at the point P (0, 0) _b and the movement amount of the cosmetic product between adjacent pixels. The calculation similar to the calculation of the amount of movement of the cosmetic at the point P is simultaneously performed for all points indicated by “vector from the center point” included in the coloring characteristic data. For example, the amount of cosmetic movement between the lip brush and the skin at the point R (3,4) _b and the distance between adjacent pixels by f _(3,4) , g _(3,4) and h _(3,4) The amount of cosmetic movement is calculated.

  The above is the method for calculating the amount of cosmetic movement performed by the server 13 based on the first vector and the second vector of the processing data. As a result of the calculation, the amount of cosmetics attached to each pixel of the customer face still image data at the time corresponding to the second vector, that is, when 1 ms has elapsed from the recording start time of the processing data is determined. Hereinafter, the data regarding the cosmetic amount attached to each pixel of the customer face still image data at each time point is referred to as “attached cosmetic amount data”. The server 13 temporarily stores the attached cosmetic amount data in the RAM.

  The server 13 continues to perform the same calculation based on the second vector and the third vector of the processing data, the third vector and the fourth vector,..., 2 ms, 3 ms,. The amount data of the attached cosmetics corresponding to the point of time is created. For example, the attached cosmetic amount data corresponding to 2 ms is calculated by adding or subtracting the newly calculated cosmetic movement amount to the attached cosmetic amount data corresponding to the previous 1 ms. The server 13 temporarily stores the attached cosmetic amount data corresponding to each time point thus created in the RAM for use in the creation of guidance video data to be described later. Then, when the server 13 completes the cosmetic amount calculation processing for all the vectors included in the processing data, the attached cosmetic amount data corresponding to the final time point of “Step n” is created. The above is the description of the cosmetic amount calculation process in step S26 of FIG.

  When the process of step S26 is completed, the server 13 updates the color information of each pixel based on the attached cosmetic amount data corresponding to the final time point of “step n” (step S27). Hereinafter, the update process of color information will be described using an example.

  For example, the RGB values of the pixels corresponding to the point L on the customer face still image data are R = 237, G = 222, B = 168, and the amount of attached cosmetics calculated for the pixel corresponding to the point L Is 0.85 mg. In this case, the server 13 first converts the HSB value of the cosmetic data used in “Step n” into an RGB value using the HSB-RGB correspondence table 28. The HSB values of the cosmetic data used in “Step n” are H = 90, S = 85, and B = 80, as assumed above. In this case, for example, the RGB values of the cosmetic are R = 204, G = 30, and B = 134. The transparency of this cosmetic is 25. The transparency 25 indicates that 25% of the background color is transmitted when the cosmetic is applied at 1 mg / pixel (where “pixel” indicates the area of one pixel). Therefore, when the amount of cosmetics adhering to the point L is 0.85 mg, the color of the base is transmitted by 25% / 0.85 = about 29.4%.

Using the above numerical values, the server 13 calculates the following numerical values as color information reflecting the color of cosmetics in each pixel.
R: 237 × 29.4 / 100 + 204 × (1-29.4 / 100) = about 214
G: 222 × 29.4 / 100 + 30 × (1-29.4 / 100) = about 86
B: 168 × 29.4 / 100 + 134 × (1-29.4 / 100) = 144.
The server 13 rewrites the information regarding the color of the point L by R = 214, G = 86, and B = 144 obtained in this way.

  If the determination result in step S24 is “No”, the server 13 updates the color information of each pixel of the customer face still image data by a method different from that in steps S25 to S27 described above (step S28).

  For example, when the operation indicated by “step n” is “cut” by “eyebrow shaver”, the processing data is a two-dimensional vector group indicating an area on the customer face still image data. In that case, the server 13 rewrites the color information relating to the pixels of the black portion in the region indicated by the two-dimensional vector with the surrounding non-black color information. The color information update process in step S28 is performed in a combination of “tool” and “motion” such as “curl” by “Buhler”, “attach” by “mascara”, “toku” by “mascara comb”, and the like. .

  The above is the description of the color information changing process regarding “step n” of the working process data. The server 13 repeats the processes of steps S21 to S28 for all steps included in the work process data, that is, “step 1” to “step k”.

  When the server 13 finishes the color information change process for “step k”, the server 13 performs the reverse process of the above-described shadow removal process. That is, the server 13 converts the RGB value into the HSB value using the HSB-RGB correspondence table 28 for each coordinate point of the customer face still image data for which the color information change processing has been completed. Is added to the brightness obtained by the conversion, and the resulting HSB value is converted into an RGB value using the HSB-RGB correspondence table 28. The server 13 rewrites the color information of the customer face still image data with the RGB values thus obtained. As a result, the customer face still image data is data indicating a three-dimensional image having a shadow. Hereinafter, the process of adding the above-described shadow to the customer face still image data is referred to as “shadow addition process”.

  Subsequently, the server 13 reads the template data of the makeup result display page from the Web page data group 32, and creates the data of the makeup result display page using the customer face still image data changed as described above. The server 13 transmits the data of the makeup result display page to the customer terminal 12. As a result, the makeup result display page shown in FIG. 30 is displayed on the display unit of the customer terminal 12.

  The makeup result display page includes an image showing the face of customer A when makeup is applied to the face of customer A using the cosmetics selected by customer A according to the process data selected by customer A. . Customer A clicks “Return” on the makeup result display page, displays a web page such as the cosmetic condition input page shown in FIGS. 24 to 26 on the display unit of the customer terminal 12, and selects another cosmetic or process. Thus, a makeup result display page based on customer face still image data created based on different cosmetic data and process data can be displayed on the display unit of the customer terminal 12.

  When customer A wants to obtain guidance on a makeup method for obtaining the makeup result displayed on the makeup result display page, the customer A clicks “replay” on the makeup result display page. In response to the click operation, the customer terminal 12 transmits a data transmission request for the makeup guidance page to the server 13.

  When the server 13 receives the cosmetic guidance page data transmission request, the server 13 creates guidance video data based on the cosmetic data and the process data corresponding to the cosmetic and the process selected by the customer A at that time. Guidance video data is the image data of the tool indicated by “tool” at the position indicated by the coordinates included in “processing data” or a predetermined position with respect to the customer face still image data that is sequentially updated according to the work process data. This is data obtained by arranging still image data obtained by interposing and interposing messages included in “guidance” in time series.

  Still image data used for the guidance moving image data is created by the server 13 as follows. If the process indicated by “step n” is not a process involving a face color, the server 13 applies a predetermined image corresponding to the contents of “tool” and “motion” to a predetermined position for the customer face still image data that does not change. And interpose messages included in “guidance” are sequentially generated as still image data corresponding to each time point of “step n”. As a result, the manner in which a hand with a facial cleanser applies massage to the customer's face still image data changes over time, with a message such as “Please wash your face thoroughly with facial cleanser”. Still image data is created.

  The process indicated by “step n” is a process involving the color of the face, and the “process data” does not include time-series data, that is, the “motion” is a process such as “curl” or “cut”. In some cases, the server 13 adds a change in predetermined color information corresponding to the processing data to the customer face still image data in time series, and adds a predetermined image corresponding to the contents of “tool” and “motion” to a predetermined position. And interpose messages included in “guidance” are sequentially generated as still image data corresponding to each time point of “step n”. As a result, time-series still image data is displayed with a message such as “Please hold the root of the eyelashes firmly with the viewer,” for example, when the curler is added to the eyelashes by the viewer for the customer face still image data. Is created.

  When the process indicated by “Step n” is a process involving the color of the face and “Processing data” includes time-series data, that is, when “Operation” is a process such as “Paint”, “Hataku”, etc. The server 13 creates still image data corresponding to each time point using the attached cosmetic amount data corresponding to each time point generated in step S28 of FIG.

  That is, the server 13 performs a shadow addition process on the customer face still image data at the final time point of the previous step, after updating the color information based on the attached cosmetic amount data corresponding to each time point. The server 13 interposes the image of the tool indicated by “tool” to the position indicated by the vector corresponding to each time point of the processing data with respect to the customer face still image data thus obtained, and further “guidance”. Interpose messages contained in. The server 13 sequentially creates the customer face still image data in which the tool image and the message are imposed in this manner as still image data corresponding to each time point of “Step n”.

  As a result, for example, a blusher was placed on the cheek with a cheek brush and gradually spread by the cheek brush against the customer face still image data. Then, gradually add gradation while reciprocating the cheek brush several times. "

  The server 13 temporarily stores the data in which the still image data changing as described above is arranged in time series as each frame as guidance moving image data in a format such as JPEG (Joint Photographic Experts Group), for example.

  When the creation of the guidance video data is completed, the server 13 reads the template data of the makeup guidance page from the web page data group 32, and creates the makeup guidance page data using the guidance video data. The server 13 transmits the makeup guidance page data to the customer terminal 12. As a result, the makeup guidance page shown in FIG. 31 is displayed on the display unit of the customer terminal 12.

  The makeup guidance page includes a moving image display window 61 and a command button group 62 for reproducing guidance moving image data. When the customer A clicks “play” included in the command button group 62, the customer terminal 12 displays a moving image based on the guidance moving image data in the moving image display window 61. The moving image includes a message that explains the operation in each step of makeup in characters and an image that shows in real time how makeup is applied to the face of customer A. That is, the customer A can watch the process of applying makeup on the customer A's face as a moving image using the cosmetics selected by the customer A with a suitable tool. For example, the blusher once placed on the face You can see how it gradually spreads with a cheek brush.

  The customer A can move between steps included in the makeup process by clicking “previous step” and “next step” included in the command button group 62. Further, the customer A can alternately display the image at the beginning of the step being reproduced and the image at the current time by clicking “Compare”. Also, the customer A can save the guidance moving image data in the HD of the customer terminal 12 by clicking “Save”.

  When the customer A finishes reproducing the guidance video data on the makeup guidance page and wants to actually purchase the cosmetics corresponding to the cosmetic data used in the guidance video data, the customer A clicks “Purchase”. In response to the click operation, the customer terminal 12 transmits a data transmission request for the cosmetics purchase page to the server 13.

  When the server 13 receives the request for transmitting the cosmetic purchase page data, the server 13 reads the template data of the cosmetic purchase page from the Web page data group 32 and purchases the cosmetic using the “selected cosmetic name” of the customer data of the customer A. Create page data. The server 13 transmits data on the cosmetic purchase page to the customer terminal 12. As a result, the cosmetic purchase page shown in FIG. 32 is displayed on the display unit of the customer terminal 12.

  The cosmetic purchase page includes a cosmetic list that lists cosmetic names selected by the customer A and indicating the cosmetic data used to create the guidance video data. In the cosmetic list, customer A enters the desired quantity in the “quantity” column to the right of the name of the cosmetic product he / she wishes to purchase, and clicks “Send”. In response to the click operation, the customer terminal 12 transmits the cosmetic name as cosmetic identification data for specifying the cosmetic, together with the quantity data indicating the quantity of the cosmetic, to the server 13 as purchase application data.

  When the server 13 receives the purchase application data from the customer terminal 12, the store 13 includes the cosmetic name in the “handling product” for each of the cosmetic names included in the purchase application data from the store database 30 shown in FIG. Extract data. Subsequently, the server 13 charges the order for the specified amount of cosmetics specified by the purchase application data based on the “store name”, “delivery date”, “price”, and “shipping cost” of the extracted store data. And the calculation result is listed as cost data together with data indicating the delivery date. In this case, as the data indicating the delivery date, the latest delivery date among the delivery dates corresponding to the designated cosmetics in each store data is selected.

  Subsequently, the server 13 reads out the template data of the expense list display page from the Web page data group 32, and creates the data of the expense list display page using the created list. The server 13 transmits the cost list display page data to the customer terminal 12. As a result, the expense list display page shown in FIG. 33 is displayed on the display unit of the customer terminal 12.

  The cost list display page includes a cost list that lists combinations of cosmetics, dealers, and quantities in ascending order of total cost for cosmetics that the customer A desires to purchase. The cost list includes a “number” for identifying a combination of cosmetics, a dealer, and a quantity, a “store name” that indicates the name of the dealer, a “cosmetic name” that indicates the name of the cosmetic, and a “quantity” that indicates the number of cosmetics ordered. ”,“ Cost subtotal ”indicating the total price and shipping cost of cosmetics to be paid to each dealer,“ Delivery date ”indicating the delivery date for each dealer, and“ Total cost ”indicating the sum of the cost subtotal for each dealer have. Customer A considers the contents of “total cost”, “delivery date”, etc., and selects a combination indicated by any number, and then clicks “order”. In response to the click operation, the customer terminal 12 transmits data indicating a combination of the store name, cosmetic name, and quantity selected by the customer A to the server 13 as order data.

  Upon receiving the order data, the server 13 searches the store database 30 for store data including the store name for each store name included in the order data, and “store mail address” of the searched store data. Send an e-mail to the e-mail address indicated in. The e-mail includes data on the cosmetic name and quantity included in the order data, corresponding to the e-mail destination store, and data such as the address included in the customer data of customer A. . The e-mail transmitted from the server 13 is received by the store terminal 15, and the cosmetics order details are notified to the staff of the cosmetic store. Thereafter, the cosmetics are delivered and paid for between the cosmetic store and the customer A.

  Subsequently, the server 13 shows the data regarding the name and quantity of the cosmetics ordered by the customer A together with the attribute data such as “age” and “gender” included in the customer data of the customer A and the date of the day in FIG. Added to the marketing related database 31. At that time, “purchase” is stored in the “behavior classification” of the marketing related data.

  The above is the description of the operation of the information providing service that the server 13 provides to the customer terminal 12 in the guidance system 1. In addition to the information providing service for the customer terminal 12, the server 13 also provides a service for providing data stored in the marketing related database 31 to the manufacturer terminal 14 and the store terminal 15. That is, the server 13 periodically sends an e-mail including the contents of the marketing related database 31 to e-mail addresses included in the “maker e-mail address” of the maker database 29 and the “dealer e-mail address” of the store database 30 To do. The e-mail thus transmitted is received by the manufacturer terminal 14 and the store terminal 15, and the staff of the cosmetic manufacturer and the staff of the cosmetic store store information on the consumer's cosmetic propensity related to the cosmetics contained in the marketing related database 31. I can know.

[2. Second Embodiment]
The second embodiment differs from the first embodiment described above only in the configuration of the process data creation terminal and the operation when creating the process data. Therefore, hereinafter, only differences between the second embodiment and the first embodiment will be described.

[2.1. Configuration of process data creation terminal]
In the second embodiment, a makeup artist uses a process data creation terminal 71 instead of the process data creation terminal 41 when creating process data. FIG. 34 is a diagram showing an overview of the process data creation terminal 71. The process data creation terminal 71 includes a pressure sensor panel 711, an A / D converter 712, and a general-purpose computer 713. In the following description, the process data creation terminal 71 is configured by connecting the pressure sensor panel 711, the A / D converter 712, and the general-purpose computer 713 with a cable. For example, the A / D converter 712 may be integrated with the pressure sensor panel 711 or may be integrated with the general-purpose computer 713.

  The pressure sensor panel 711 has a fine and highly sensitive pressure sensor densely arranged on the surface of a transparent plate such as an acrylic board, and is covered with a very thin and soft transparent silicon coating. . The density of the pressure sensor is, for example, equal to the resolution of the display unit of the general-purpose computer 713, and each pressure sensor corresponds to each pixel of the display unit of the general-purpose computer 713 almost on a one-to-one basis. The pressure measured by the pressure sensor is output to the A / D converter 712.

  The A / D converter 712 has the same function as the A / D converter 412 in the first embodiment, converts an analog signal indicating the pressure output from the pressure sensor on the pressure sensor panel 711 into digital data, Along with data indicating the position of the pressure sensor, the data is transmitted to the general-purpose computer 713.

  The general-purpose computer 713 is a general-purpose computer having the same configuration as the general-purpose computer 413 in the first embodiment. In addition to the OS, the HD of the general-purpose computer 713 stores a process data creation program and model face still image data. The model face still image data in the second embodiment may be digital data obtained by photographing a mannequin as a tangible object as in the first embodiment, or may be image data created virtually by a computer. There may be.

[2.2. Create process data]
In the second embodiment, the position of each pressure sensor on the pressure sensor panel 711 and the position of each pixel when the model face still image data is displayed on the display unit of the general-purpose computer 713 correspond approximately one to one. is doing. Therefore, unlike the first embodiment, pressure sensor position data indicating the correspondence between the pressure sensor and each pixel of the model face still image data is not necessary in the second embodiment.

  The makeup artist activates a process data creation program in the general-purpose computer 713 when creating process data. As a result, a process data creation screen similar to that shown in FIG. 21 is displayed on the display unit of the general-purpose computer 713. As in the case of the first embodiment, the makeup artist inputs necessary data on the process data creation screen, and clicks “start recording” and “manual input” as necessary, and is included in the process data. Record or input process data.

  When the makeup artist clicks “start recording”, a still image of the model face indicated by the model face still image data is displayed on the display unit of the general-purpose computer 713. The makeup artist performs a makeup operation similar to the actual makeup on the cover of the pressure sensor panel 711 on the still image of the model face that can be seen through the pressure sensor panel 711. As a result, the makeup operation by the makeup artist is sequentially stored in the RAM of the general-purpose computer 713 as time-series pressure data at the position corresponding to each pixel of the model face still image data, and the processing data is based on the data stored in the RAM. Is generated.

  The process data including the processing data generated as described above is added to the process database 26 in the HD of the server 13 by the staff of the service provider of the guidance system 1 and used in the guidance system 1 as in the first embodiment. Is done.

  In the second embodiment, the makeup artist uses a makeup tool such as a cheek brush without any cosmetics, a stylus imitating the shape of a lipstick, or the like, instead of performing a makeup operation using actual cosmetics. A makeup operation may be performed. In that case, the general-purpose computer 713 sequentially changes the color data of the model face still image data according to the pressure or locus of the makeup tool or stylus, the cosmetic data specified by the makeup artist, etc., and virtually displays the makeup on the display unit. The state of the model face to which is applied is displayed as a video. Since the operation of the general-purpose computer 713 sequentially changing the color data of the model face still image data is the same as the operation of creating the guidance moving image data described in the first embodiment, detailed description thereof is omitted.

  In the second embodiment, a makeup artist can create process data more easily by using a small pressure sensor panel that measures in place of a mannequin that is inconvenient to carry. In the second embodiment, since the makeup artist can also create process data without using actual cosmetics, it is possible to save the trouble of cleaning cosmetics attached to the pressure sensor panel cover. it can.

[3. Third Embodiment]
The third embodiment is different from the second embodiment described above only in the configuration of the process data creation terminal. Therefore, hereinafter, only the differences of the third embodiment from the second embodiment will be described.

[3.1. Configuration of process data creation terminal]
FIG. 35 is a diagram showing an overview of a process data creation terminal 81 used when a makeup artist creates process data in the third embodiment. The process data creation terminal 81 includes a touch panel 811, a makeup tool adapter 812, and a general-purpose computer 813.

  The touch panel 811 has a minute and highly sensitive touch sensor densely arranged on the surface of a transparent plate such as an acrylic board, and is covered with a very thin and soft transparent silicon coating. For each touch sensor, data indicating the position of the touch sensor and data indicating that pressure is applied while a certain pressure or more is applied, for example, “1” is applied. If not, for example, “0” is transmitted to the general-purpose computer 813 as digital data. The density of the touch sensor is equal to, for example, the resolution of the display unit of the general-purpose computer 813, and each touch sensor corresponds to each pixel of the display unit of the general-purpose computer 813 on a one-to-one basis.

  The cosmetic tool adapter 812 is a device to which a cosmetic tool can be attached and capable of transmitting the magnitude of the force applied to the attached cosmetic tool as digital data. There are a plurality of different types of makeup tool adapters 812 according to different shapes of makeup tools. FIG. 36 is a cross-sectional view for illustrating the configuration of the cosmetic tool adapter 812. Note that FIG. 36 shows a lipstick-type cosmetic tool adapter 812 as an example, but the configuration corresponding to other cosmetic tools is also lipstick-type except that the shapes of the inner cylinder 91 and the outer cylinder 93 are different. Is the same as

  The makeup tool adapter 812 includes an inner cylinder 91, a clip 92, an outer cylinder 93, a cushion 94, a pressure sensor 95, and an A / D converter 96. The inner cylinder 91 is a shape that covers a corresponding cosmetic tool from the outside, for example, a cylindrical case with one bottom surface opened. A clip 92 that is elastically expanded in the inner direction is attached to the inner side surface of the inner cylinder 91, and the inner cylinder 91 can hold lipsticks having different diameters by the clip 92. The outer cylinder 93 is a case of the same shape that is slightly larger in size than the inner cylinder 91 and covers the inner cylinder 91 from the outside. The outer cylinder 93 is provided with, for example, a stopper 931 that protrudes inward in the opening, so that the inner cylinder 91 does not fall off the outer cylinder 93.

  A cushion 94 and a pressure sensor 95 are sandwiched between the bottom surface of the inner cylinder 91 and the bottom surface of the outer cylinder 93. The cushion 94 is, for example, an elastic body or a spring such as rubber or silicon molded into a disk shape. The pressure sensor 95 is fixed to the center inside the bottom surface of the outer cylinder 93. The pressure sensor 95 outputs an analog signal corresponding to the applied pressure, for example, to an A / D converter 96 attached outside the bottom surface of the outer cylinder 93. The A / D converter 96 converts the analog signal input from the pressure sensor 95 into digital data indicating a pressure value, and transmits the digital data to the general-purpose computer 813.

  The general-purpose computer 813 is a general-purpose computer having the same configuration as the general-purpose computer 713 in the second embodiment. However, the general-purpose computer receives data indicating the position of each touch sensor and the presence / absence of a certain level of pressure from the touch panel 811 and simultaneously receives digital data indicating the pressure from the cosmetic tool adapter 812.

  When the makeup artist starts the process data creation program on the general-purpose computer 813 and clicks “start recording” on the process data creation screen similar to that shown in FIG. 21, the general-purpose computer 813 displays the model face on the display unit. The image is displayed and the storage of the data indicating the makeup operation by the makeup artist is started. The makeup artist attaches a makeup tool such as lipstick to the makeup tool adapter 812, and performs a makeup operation similar to the actual makeup on the cover of the touch panel 811.

  A touch sensor arranged on the touch panel 811 with a makeup artist's makeup operation that detects a pressure above a certain level is “1”, the others are “0”, and data indicating the respective positions is general-purpose. Send to computer 813. Further, the makeup tool adapter 812 transmits the pressure measured by the pressure sensor 95 by the force applied to the attached makeup tool to the general-purpose computer 813 as digital data by the makeup operation of the makeup artist. The general-purpose computer 813 generates processing data corresponding to each makeup operation based on the data received from the touch panel 811 and the makeup tool adapter 812.

  At this time, the general-purpose computer 813 can acquire data indicating the pressure at the center point of the makeup tool from the makeup tool adapter 812, but can acquire measured data regarding the pressure at each point around the center point. Can not. Therefore, the process data creation program used by the general-purpose computer 813 is preliminarily provided with data indicating the pressure distribution at each position from the center point of the cosmetic tool as attached data according to the type of the cosmetic tool. The general-purpose computer 813 uses the data indicating the pressure distribution instead of the actually measured pressure data to calculate the pressure at each point around the center point of the cosmetic and uses it to generate processing data.

  The process data including the processing data generated as described above is added to the process database 26 in the HD of the server 13 by the staff of the service provider of the guidance system 1 as in the first and second embodiments. Used in the guidance system 1. In the third embodiment, as in the second embodiment, the makeup artist creates process data by virtually applying makeup to the still image of the model face without using actual cosmetics. You can also.

  In the third embodiment, compared with the second embodiment, the present invention can be implemented by using only one pressure sensor for the cosmetic adapter, so that the cost required for the pressure sensor and the A / D converter is reduced. Can do. Note that the cosmetic tool adapter used in the third embodiment can also be used as an input device that inputs data indicating writing pressure in real time. In that case, other writing tools or coloring tools such as pencils may be attached to the adapter instead of the makeup tools. For example, by attaching a stylus to a makeup tool adapter, it is possible to easily input data relating to the writing pressure of the stylus on the touch panel to a tablet personal computer having a liquid crystal screen with a touch panel.

[4. Fourth Embodiment]
The fourth embodiment differs from the first embodiment described above only in the configuration of the process data creation terminal. Therefore, hereinafter, only differences between the fourth embodiment and the first embodiment will be described.

  FIG. 37 is a diagram showing an overview of the process data creation terminal 101 used when the makeup artist creates process data in the fourth embodiment. The process data creation terminal 101 includes a model mannequin 1011, a makeup tool adapter 1012, and a general-purpose computer 1013.

  A model mannequin 1011 according to the fourth embodiment is a mannequin having substantially the same configuration as the model mannequin 411 according to the first embodiment, but a plurality of touch sensors are arranged on the surface instead of a plurality of pressure sensors. This touch sensor is the same as the touch sensor arranged on the touch panel 811 in the third embodiment.

  The makeup tool adapter 1012 in the fourth embodiment is the same as the makeup tool adapter 812 in the third embodiment.

  A general-purpose computer 1013 in the fourth embodiment is substantially the same as the general-purpose computer 713 in the first embodiment. However, the general-purpose computer 1013 simultaneously receives data indicating the presence or absence of pressure detection and data indicating the position of each touch sensor from the model mannequin 1011, and data indicating pressure from the cosmetic tool adapter 1012.

  At that time, the general-purpose computer 1013 can only acquire data indicating the pressure at the center point of the cosmetic tool from the cosmetic tool adapter 1012. Therefore, as in the case of the third embodiment, the general-purpose computer 1013 calculates the pressure at each point around the center point of the cosmetic tool using the data indicating the pressure distribution according to the type of the cosmetic tool, and performs processing. Used for data generation.

  In the fourth embodiment, compared with the first embodiment, the present invention can be implemented by using only one pressure sensor for the cosmetic adapter, so that the cost required for the pressure sensor and the A / D converter is reduced. Can do.

[5. Modified example]
The above-described embodiment is an exemplification of the embodiment of the present invention, and various modifications can be made to the above-described embodiment without departing from the gist of the present invention.

[5.1. First Modification]
In the first modification, still image data processed into three-dimensional shape data based on a plurality of still image data obtained by photographing a customer's face from a plurality of angles is used as the customer face still image data. Similarly, as model face still image data, still image data which is three-dimensional shape data processed based on a plurality of still image data obtained by photographing a model mannequin from a plurality of angles, or 3 created virtually by a computer Still image data which is dimensional shape data is used. Further, in the first modified example, the shadow removal process and the shadow addition process using the model face shadow data are not performed, but instead by calculation based on the three-dimensional shape indicated by the customer face still image data which is three-dimensional shape data. Then, shading information removal and addition processing is performed.

  According to the first modification, the server can provide the customer terminal with guidance on makeup using an image closer to the real product. Moreover, according to the 1st modification, the server can provide the image at the time of seeing the customer's face with which makeup was given from various angles with respect to a customer terminal.

[5.2. Second Modification]
In the second modification, a plurality of model mannequins prepared for various classifications such as sex, skin color, face shape, age, race, etc., and models created according to each of those model mannequins Face still image data is used. Further, in the second modification, makeup data having similar contents is applied according to the characteristics of each model mannequin by the makeup artist, so that process data having the same name corresponding to each model mannequin is created.

  The server determines the gender, age, race, etc. of the customer based on the customer data received from the customer terminal. The server also analyzes the customer face still image data received from the customer terminal and determines the skin color, face shape, and the like. Based on these determinations, the server updates the customer face still image data using model face still image data, model face feature point data, process data, and the like corresponding to the model mannequin suitable for the customer.

  According to the second modification, model face still image data, process data, etc. based on a model mannequin similar to the features of the customer's face are used, so that guidance guidance caused by the difference between the model mannequin face and the customer's face is used. Problems such as positional deviation in image display are reduced. Further, according to the second modification, even when a process having the same name is selected by the customer, guidance according to the facial features of the customer is provided to the customer.

[5.3. Third Modification]
In the third modification, a storefront terminal in which a customer terminal and a server are integrated is used. That is, the customer terminal and the server are not connected via the Internet, and both the customer terminal and the server functions are realized by, for example, one general-purpose computer. The store terminal is usually placed at a store such as a cosmetic store, and is freely used by customers who visit the store. According to the third modification, the customer can receive the information providing service according to the present invention without preparing the customer terminal.

[5.4. Fourth Modification]
In the fourth modified example, the delivery of data between the customer terminal, the server, the manufacturer terminal, and the store terminal is not limited to the one via the Internet, the data delivery via another communication network such as an in-house LAN, Data is also delivered by mailing storage media such as CD-ROM (Compact Disc Read Only Memory) and FD. According to the fourth modified example, the information providing service according to the present invention can be implemented even when the Internet is not available or when data transfer via the Internet is not desirable.

[5.5. Fifth Modification]
In the fifth modification, the model mannequin is not limited to the human head, and mannequins that imitate various parts of the body are used. According to the fifth modification, the information providing service of the present invention can be implemented not only for the makeup for the face but also for the makeup for other parts such as limbs.

[5.6. Sixth Modification]
In the sixth modified example, the number of pressure sensors arranged on the surface of the model mannequin or the pressure sensor panel is as small as, for example, one third compared with the number of pixels of the face still image data. On the other hand, a general-purpose computer connected to a model mannequin or a pressure sensor panel has a function of estimating a pressure at a position corresponding to each pixel of face still image data based on, for example, pressure measured by three adjacent pressure sensors. . That is, when the positions of the face still image data corresponding to the positions of the pressure sensor k, the pressure sensor l, and the pressure sensor m that are adjacent to each other are the point K, the point L, and the point M, respectively, the general-purpose computer enters the triangle KLM. The pressure at the point N is calculated based on the measured pressure values at the points K, L, and M, and the distances between the points K and N, the points L and N, and the points M and N. According to the sixth modification, the creation of the model mannequin or pressure sensor panel is simplified, and the amount of data transmitted from the model mannequin or pressure sensor panel to the general-purpose computer is reduced.

[5.7. Seventh Modification]
In the seventh modified example, the server, from the customer terminal, the main customer face still image data indicating the still image with the customer raising his hair and the sub customer face still image indicating the normal still image with the customer lowered his hair. Receive data. The server generates customer hair still image data by extracting data indicating the hair portion from the sub customer face still image data. The server performs the color information change processing shown in the embodiment for the regular customer face still image data, and then interposes the customer hair still image data, thereby causing the customer to drop a normal still image whose hair has been lowered. To create still image data.

  According to the seventh modified example, the accuracy of the matching process between the model face feature point data related to the model mannequin having no hair and the customer face feature point data related to the customer who usually has hair is improved and transmitted to the customer terminal. The still image data and guidance moving image data for displaying the makeup result include images of customers who have hair.

It is a figure which shows the whole structure of the guidance system concerning embodiment of this invention. It is a figure which shows the structure of the computer which functions as a customer terminal concerning embodiment of this invention. It is a figure which shows the structure of the data memorize | stored in HD of the server concerning embodiment of this invention. It is a figure which shows the structure of the customer database concerning embodiment of this invention. It is a figure which shows the structure of the cosmetics database concerning embodiment of this invention. It is a figure which shows the still image represented by the model face still image data concerning embodiment of this invention. It is a figure which shows the content of the model face feature point data concerning embodiment of this invention. It is a figure which shows the content of the model face shadow data concerning embodiment of this invention. It is a figure which shows the structure of the process database concerning embodiment of this invention. It is a figure which shows the structure of the coloring characteristic database concerning embodiment of this invention. It is a figure which shows the structure of the HSB-RGB correspondence table concerning embodiment of this invention. It is a figure which shows the structure of the manufacturer database concerning embodiment of this invention. It is a figure which shows the structure of the store database concerning embodiment of this invention. It is a figure which shows the structure of the marketing related database concerning embodiment of this invention. It is a figure which shows the external appearance of the terminal for process data creation concerning 1st Embodiment of this invention. It is a figure which shows the structure of the pressure sensor position data concerning 1st Embodiment of this invention. It is a figure which shows the still image represented by the model face still image data and uncovered model face still image data concerning 1st Embodiment of this invention. It is a flowchart which shows the procedure of preparation of the pressure sensor position data concerning 1st Embodiment of this invention. It is a figure which shows the structure of the uncovered model face feature point data concerning 1st Embodiment of this invention. It is a figure for demonstrating the correction process of the pressure sensor position data concerning 1st Embodiment of this invention. It is a figure which shows the process data creation screen concerning embodiment of this invention. It is a figure which shows the structure of the makeup | decoration operation | movement recording data concerning embodiment of this invention. It is a figure which shows the customer information registration page concerning embodiment of this invention. It is a figure which shows the cosmetics condition information input page concerning embodiment of this invention. It is a figure which shows the cosmetics selection page concerning embodiment of this invention. It is a figure which shows the process selection page concerning embodiment of this invention. It is a figure which shows the structure of the customer face feature point data concerning embodiment of this invention. It is a flowchart which shows the procedure of the change process of the color information concerning embodiment of this invention. It is a figure which shows the relationship between the coordinate system a and the coordinate system b concerning embodiment of this invention. It is a figure which shows the makeup | decoration result display page concerning embodiment of this invention. It is a figure which shows the makeup | decoration guidance page concerning embodiment of this invention. It is a figure which shows the cosmetics purchase page concerning embodiment of this invention. It is a figure which shows the expense list display page concerning embodiment of this invention. It is a figure which shows the external appearance of the terminal for process data creation concerning 2nd Embodiment of this invention. It is a figure which shows the external appearance of the terminal for process data creation concerning 3rd Embodiment of this invention. It is sectional drawing which shows the structure of the makeup | decoration tool adapter concerning 3rd Embodiment of this invention. It is a figure which shows the structure of the makeup | decoration tool adapter concerning 4th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Guidance system, 11 ... Internet, 12 ... Customer terminal, 13 ... Server, 14 ... Manufacturer terminal, 15 ... Store terminal, 21 ... Customer database, 22 ... Cosmetics database, 23 ... Face still image data group, 24 ... Model Facial feature point data, 25: Model face / shadow data, 26: Process database, 27: Coloring property database, 28: HSB-RGB correspondence table, 29: Manufacturer database, 30: Dealer database, 31: Marketing related database, 32 ... Web page data group, 411, 1011 ... model mannequin, 41, 71, 81.101 ... Process data creation terminal, 51 ... input item group, 52, 62 ... command button group, 412, 712, 96 ... A / D converter 413, 713, 813, 1013 ... General-purpose computer 61 ... video display window, 711 ... pressure sensor panel, 811 ... touch panel, 812, 1012 ... makeup tool adapter, 91 ... inner cylinder, 92 ... clip, 93 ... outer cylinder, 94 ... cushion, 95 ... pressure sensor.

Claims (16)

A terminal device and a server device,
The terminal device
Means for transmitting and receiving image data via a communication network;
Means for displaying image data received via the communication network;
The server device
Cosmetic data storage means for storing cosmetic data including at least information on the color of the cosmetic;
Process data storage means for storing process data indicating a process in which makeup is applied to the body;
Still image data receiving means for receiving still image data indicating a customer's body image from the terminal device via the communication network;
Based on cosmetic data stored in the cosmetic data storage means, process data stored in the process data storage means, and still image data indicating an image of the customer's body received by the still image data receiving means. Moving image data creating means for creating moving image data representing a temporal change in the surface of the customer's body when applying makeup to the customer;
A cosmetic guidance system comprising: moving image data transmitting means for transmitting the moving image data generated by the moving image data generating means to the terminal device via the communication network. Cosmetic data storage means for storing cosmetic data including at least information on the color of the cosmetic;
Process data storage means for storing process data indicating a process in which makeup is applied to the body;
Still image data receiving means for receiving still image data indicating a customer's body image from a terminal device via a communication network;
Based on cosmetic data stored in the cosmetic data storage means, process data stored in the process data storage means, and still image data indicating an image of the customer's body received by the still image data receiving means. Moving image data creating means for creating moving image data representing a temporal change in color on the surface of the customer's body when applying makeup to the customer;
A server device comprising: moving image data transmitting means for transmitting the moving image data generated by the moving image data generating means to the terminal device via the communication network. The process data stored in the process data storage means includes time-series data indicating the operation of the cosmetic device in the process of applying makeup on the body in time series,
The moving image data creating means obtains a temporal change in color on the surface of the customer's body based on time series data included in the process data stored in the process data storage means, and calculates the temporal change in the color. The server apparatus according to claim 2, wherein moving image data to be expressed is created. Process identification data receiving means for receiving process identification data capable of specifying one process data from the terminal device via the communication network,
The server apparatus according to claim 2, wherein the moving image data creating unit creates moving image data based on the process data specified by the process identification data received by the process identification data receiving unit. A cosmetic identification data receiving means for receiving one or more cosmetic identification data capable of specifying one cosmetic data from the terminal device via the communication network;
The server apparatus according to claim 2, wherein the moving image data creating unit creates moving image data based on cosmetic data specified by the cosmetic identification data received by the cosmetic identification data receiving unit. Condition data receiving means for receiving condition data indicating conditions relating to cosmetic attributes from the terminal device via the communication network;
Cosmetic data extraction means for extracting cosmetic data relating to cosmetics that satisfy the condition indicated by the condition data received by the condition data receiving means from one or more cosmetic data stored by the cosmetic data storage means;
A cosmetic list transmitting means for transmitting a cosmetic list including at least one cosmetic identification data capable of specifying each of the cosmetic data extracted by the cosmetic data extracting means to the terminal device via the communication network; The server device according to claim 5, wherein: The terminal device is associated with one or more cosmetic identification data capable of specifying one cosmetic data stored in the cosmetic data storage means via the communication network and any one of the one or more cosmetic identification data. The server apparatus according to claim 5, further comprising: a cosmetic list transmission unit that transmits a cosmetic list including at least one message data indicating the received message. Attribute data receiving means for receiving attribute data indicating customer attributes from the terminal device via the communication network;
Marketing related data storage means for storing data relating to cosmetic data specified by the cosmetic identification data received by the cosmetic identification data receiving means in association with attribute data received by the attribute data receiving means and storing it as marketing related data The server apparatus according to claim 5, further comprising: Store one or more store data including store identification data that can specify a store for cosmetics, cosmetic identification data that can specify cosmetics, and price data that indicates a price associated with the cosmetic identification data Store data storage means,
Purchase application data including at least one set of cosmetic identification data capable of specifying one cosmetic data and quantity data indicating a quantity associated with the cosmetic identification data from the terminal device via the communication network Purchase application data receiving means for receiving,
Price data included in the store data including the same contents as the cosmetic identification data included in the purchase application data received by the purchase application data receiving unit and stored in the store data storage unit, and included in the purchase application data Calculation means for calculating cost data related to costs based on
The server apparatus according to claim 2, further comprising: cost data transmission means for transmitting the cost data calculated by the calculation means to the terminal device via the communication network. A sensor unit for detecting an operation performed using a cosmetic tool;
Processing data generating means for generating processing data that is data representing the contact position and the contact pressure of the cosmetic tool on the surface of the virtual human body based on the content of the operation of the cosmetic tool detected by the sensor unit;
Storage means for storing processing data generated by the processing data generation means. The sensor unit has a plurality of pressure sensors for measuring pressure,
11. The terminal device according to claim 10, wherein each of the plurality of pressure sensors is arranged on a surface of a three-dimensional object imitating the virtual human body, and measures a pressure at the arranged position. The sensor unit includes a plurality of touch sensors that detect a pressure equal to or higher than a certain value, and a pressure sensor that measures the pressure applied to the cosmetic device,
11. The terminal device according to claim 10, wherein each of the plurality of touch sensors is arranged on a surface of a three-dimensional object imitating the virtual human body, and detects a pressure at the arranged position. Display means for displaying a two-dimensional or three-dimensional image representing the virtual human body,
The sensor unit has a plurality of pressure sensors for measuring pressure,
Each of these pressure sensors is arrange | positioned on the surface of the panel which covers the display screen of the said display means, and measures the pressure in the arrange | positioned position. The terminal device of Claim 10 characterized by the above-mentioned. Display means for displaying a two-dimensional or three-dimensional image representing the virtual human body,
The sensor unit includes a plurality of touch sensors that detect a pressure equal to or higher than a certain value, and a pressure sensor that measures the pressure applied to the cosmetic device,
Each of these touch sensors is arrange | positioned on the surface of the panel which covers the display screen of the said display means, and detects the pressure in the arrange | positioned position. The terminal device of Claim 10 characterized by the above-mentioned. The said display means produces | generates the image showing the locus | trajectory of the operation | movement of the said cosmetics based on the process data memorize | stored in the said memory | storage means, It displays with the image showing the said virtual human body. The terminal device according to 13 or 14. Cosmetic data storage processing for storing cosmetic data including at least information on the color of the cosmetic;
A process data storage process for storing process data indicating a process in which makeup is applied to the body;
Still image data reception processing for receiving still image data indicating a customer's body image from a terminal device via a communication network;
Based on cosmetic data stored in the cosmetic data storage process, process data stored in the process data storage process, and still image data indicating an image of the customer's body received in the still image data reception process Video data creation processing for creating video data representing temporal changes in color on the surface of the customer's body when applying makeup to the customer;
A program that causes a computer to execute moving image data transmission processing for transmitting moving image data created in the moving image data creation processing to the terminal device via the communication network.

Images