JP5041472B2 - Makeup support system - Google Patents

Description

  The present invention relates to a makeup support system that can be used for advice on makeup methods and advice for selecting cosmetics.

  2. Description of the Related Art Conventionally, as a makeup support system, a system that displays an image of a subject's face on a display device and performs a simulation of makeup using a computer image processing technique is known.

  For example, in Patent Document 1 (Japanese Patent Laid-Open No. 2000-194835), a color image of a face of a subject is photographed by a camera, and makeup data representing a change in face by a type of makeup desired by the subject is included in the photographed image. A makeup advice system is disclosed that displays the appearance of the face when the subject is made up by deforming and synthesizing to fit the face image of the subject.

In addition, Patent Document 2 (Japanese Patent Laid-Open No. 2003-57017) obtains information related to a three-dimensional shape in addition to the color information of a subject's face by using a plurality of imaging devices at the same time. In addition, a makeup simulation apparatus that displays a realistic image is disclosed.
JP 2000-194835 A JP 2003-57017 A

  The makeup advice system described in Patent Document 1 is a system that synthesizes makeup data with a face image of a subject acquired by a camera, and does not take into account depth information of the subject's face. Although the information related to the depth of the face of the subject is very important information from an aesthetic point of view, in the system described in Patent Document 1, the information is taken by a single camera in principle. It cannot handle information. On the other hand, since the makeup simulation apparatus described in Patent Document 2 uses a plurality of photographing apparatuses, it can handle three-dimensional information related to the depth of the face of the subject.

  However, although the makeup simulation apparatus described in Patent Document 2 proposes to extract information on the color and shape of a three-dimensional object using a plurality of photographing devices, the color of the three-dimensional object to be measured is proposed. Since information, shape information, and a displacement amount of the shape are required, it is not always a simple system, and it is difficult to reduce the cost and installation space of the system.

  Many makeup support systems are installed in a relatively small space such as a department store's cosmetics department, and a large number of makeup support systems must be installed when considering sales activities nationwide. The system is small and simple. It is important that the cost is low. In addition, when performing a makeup simulation at a cosmetics department store in a department store or the like, it is necessary to perform it in a dialogue with a customer, and it is important that face image processing is performed in real time.

  In addition, neither one described in Patent Document 1 nor Patent Document 2 objectively determines the finish of the subject's makeup. In other words, the systems described in both Patent Document 1 and Patent Document 2 are systems that synthesize makeup data with the face data of the subject. It cannot be determined whether or not appropriate makeup is applied.

In order to solve the various problems as described above, the invention according to claim 1 of the present invention includes face image data acquisition means for acquiring face image data, and face image data acquired by the face image data acquisition means. A front face image data generating means for generating front face image data from the image, a pseudo depth information calculating means for calculating pseudo depth information on a predetermined line of the front face image data obtained by the front face image data generating means, and an ideal Ideal pseudo depth distribution information storage means for storing typical pseudo depth distribution information, pseudo depth information calculated by the pseudo depth information calculation means, and ideal pseudo depth stored in the ideal pseudo depth distribution information storage means Pseudo depth information comparing means for comparing distribution information, and a facial image for determining the suitability of face image data acquired based on the comparison result of the pseudo depth information comparing means A data appropriateness determination means consist, characterized in that the predetermined line of the front face image data is a line passing through both cheeks and nose.

  According to a second aspect of the present invention, in the makeup support system according to the first aspect, the face image data acquisition means is any one of a camera, a scanner, and a storage medium reading device.

  According to a third aspect of the present invention, in the makeup support system according to the first or second aspect, the front face image data generating means uses the face image data based on the positions of both eyes and the nostrils of the face image data. From the above, front face image data is generated.

  According to a fourth aspect of the present invention, in the makeup support system according to any one of the first to third aspects, the pseudo depth information calculating means calculates the pseudo depth information by a shape restoration method from shades (shadows). It is characterized by doing.

  According to the makeup support system of the present invention, information relating to the depth of the face, which is very important information from an aesthetic point of view, can be handled with a simple system configuration that does not require cost and installation space. In addition, it is possible to objectively determine the degree of makeup of the subject based on information related to the depth of the face.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a system configuration of a makeup support system according to an embodiment of the present invention. In FIG. 1, 100 is a makeup support system, 101 is a main calculation unit, 102 is a face image data acquisition unit, 103 is a determination result output unit, 104 is an ideal pseudo depth distribution information storage unit, 111 is a front face image data generation unit, Reference numeral 112 denotes a pseudo depth information calculation unit, 113 denotes a pseudo depth information comparison unit, and 114 denotes a face image data suitability determination unit.

  The makeup support system 100 outlines a main calculation unit 101 that performs main calculation, a face image data acquisition unit 102 that inputs face image data to the makeup support system 100, and a determination result by the makeup support system 100 to the user. The determination result output unit 103 to output and an ideal pseudo depth distribution information storage unit 104 which is a database serving as a determination criterion.

  The information processing mechanism of the main arithmetic unit 101 is configured by, for example, a general-purpose computer system, and each component or function is realized by, for example, executing a computer program written in a storage unit (not shown).

  The face image data acquisition unit 102 is a configuration for inputting, for example, face image data after applying makeup to the makeup support system 100, and any configuration can be used so that the face image data can be input to the system side. In general, a digital camera, a scanner, and a storage medium reading device can be used. Here, the storage medium reading device is a device that reads contents stored in a storage medium such as a flexible disk, a CD-ROM, a DVD-ROM, or a flash memory.

  The determination result output unit 103 is an output user interface of the makeup support system 100, and specifically includes a monitor, a liquid crystal display panel, a printer, and the like. The determination result output unit 103 mainly outputs the determination result by the face image data suitability determination unit 114. Although not shown, an input user interface unit may be provided for setting the makeup support system 100 and the like.

  The ideal pseudo depth distribution information storage unit 104 stores data constructed by a plurality of post-makeup face image data having an ideal pseudo depth distribution, and the pseudo depth information comparison unit 113 receives the input data. The pseudo depth information (data) calculated based on the face image data is compared with the ideal pseudo depth distribution data stored in the ideal pseudo depth distribution information storage unit 104.

  The face image data input by the face image data acquisition unit 102 is not necessarily an optimal image for calculating pseudo depth information described later. Therefore, the front face image data generation unit 111 converts (normalizes) the face image data acquired by the face image data acquisition unit 102 into front face image data. Note that affine transformation is used as the image transformation method at this time.

  The pseudo depth information calculation unit 112 is a calculation unit that calculates depth information from a part of front face image data that is two-dimensional image data. Here, since such depth information is not actual depth information but depth information estimated by an image analysis technique, information obtained by calculation by the pseudo depth information calculation unit 112 is referred to as “ It is expressed as “pseudo depth information”. In addition, as a method for reconstructing a three-dimensional shape from two-dimensional image data, there is shape from X (X is shading, stereo, silhouette, motion, texture), and various researches have been made. The technique used in the pseudo depth information calculation unit 112 of the present invention is a “shape restoration method from shades (shades)” (shape from shading method). As described above, the pseudo depth information calculation unit 112 calculates the pseudo depth data of the face image data acquired by the face image data acquisition unit 102.

  The pseudo depth information comparison unit 113 compares the pseudo depth data calculated by the pseudo depth information calculation unit 112 with the ideal pseudo depth distribution data stored in the ideal pseudo depth distribution information storage unit 104, and the difference data Is output.

  The face image data suitability determination unit 114 determines whether the face image data acquired by the face image data acquisition unit 102 is appropriate or inappropriate from the viewpoint of pseudo depth according to the difference data. .

  Here, the reason why the present invention determines the suitability of the makeup from the pseudo depth data of the face image data as described above will be described. Applying makeup to the face can be said to change the three-dimensional appearance of the face by applying makeup. In other words, makeup can be said to be a technique that changes the shadow and changes the sense of depth (three-dimensional effect) perceived by humans. In fact, professional makeup artists change the depth information of appearance by changing the shade of lightness by makeup.

  The present invention determines the suitability of the makeup from the effects of makeup as described above. That is, the makeup support system of the present invention obtains pseudo depth data of face image data input to the system, and determines how much the pseudo depth data differs from the ideal pseudo depth distribution data.

  Next, more specific processing of the makeup support system of the present invention will be described with reference to FIGS. FIG. 2 is a diagram showing a flow of processing in the makeup support system according to the embodiment of the present invention. 3 is a diagram showing face image data acquired by the face image data acquisition unit 102, and FIG. 4 is a diagram showing facial feature points used when the front face image data generation unit 111 performs affine transformation. FIG. 5 is a diagram showing the front face image data generated by the front face image data generation unit 111, and FIG. 6 is a diagram showing the pseudo depth data calculated by the pseudo depth information calculation unit 112 by shading.

  In step S <b> 101, the makeup support system 100 acquires the face image data of the subject by the face image data acquisition unit 102. Here, it is assumed that the face image data acquired by the face image data acquisition unit 102 in step S101 is, for example, as shown in FIG.

  In step S102, the front face image data generation unit 111 generates front face image data from the face image data obtained by the face image data acquisition unit 102 using affine transformation. Here, in the case of performing affine transformation, affine transformation is performed on the basis of a total of three facial feature points, which are the two points of the eyes in the face image data and the midpoint of the nostril. For example, FIG. 4 shows a state in which two points of eyes in the face image data and three points in the middle of the nostril are feature points.

  It should be noted that when extracting such feature points of face image data, this may be performed manually or automatically by the system. When performing manually, the makeup support system 100 is provided with an appropriate input interface. In addition, a well-known image analysis technique can be used in order to automatically capture a total of three points of the face image data, that is, the two points of the eye in the face image data and the midpoint of the nostril. In step S102, the generated front image data is as shown in FIG.

  A well-known technique can be used as the technique related to the affine transformation in the front face image data generation unit 111. Examples of such well-known techniques include Japanese Patent Application Laid-Open Nos. 2003-263639, 2000-163592 and 2000-268172.

  In step S <b> 103, the pseudo depth information calculation unit 112 uses the shape restoration method (shape from shading) based on the front image data generated by the front face image data generation unit 111 to use the front image data. The pseudo depth data is calculated. The pseudo depth information calculating unit 112 can calculate the pseudo depth of the entire face, and the pseudo depth data calculated by the pseudo depth information calculating unit 112 is represented in the drawing by shading. 6. On the other hand, in the present invention, the pseudo depth data of the entire face calculated by the pseudo depth information calculation unit 112 is not used, and attention is paid to a line indicated by A-A ′ passing through both cheeks and nose. That is, in the present invention, the line used as a criterion for determining the suitability of makeup is an AA ′ line passing through both cheeks and nose, and the pseudo depth data on this AA ′ line is the range of the ideal pseudo depth distribution data. The suitability of makeup is determined by how much is contained in the eye.

Here, regarding a shape restoration method from shades (shades), “Numerical shape from shading and occlusive”.
ing boundaries ", K. Ikeuchi and B.K.P. Horn,
Artificial Intelligence, vol. 17, pp. 141-184, 1981; "One Model for Estimating Three-Dimensional Shapes from Human Gray Images", Manas Sanwaraj et al., Theory of Science (D-II), J-73-D-II, pp. 1789-1791, 1990 etc. can be used.

  In step S104, the pseudo depth information comparison unit 113 compares the pseudo depth data calculated by the pseudo depth information calculation unit 112 with the ideal pseudo depth distribution data stored in the ideal pseudo depth distribution information storage unit 104. The difference data is output.

  Various forms are conceivable as such differential data. As an example, the length of the pseudo depth data on the AA ′ line is within the range of the ideal pseudo depth distribution data, and on the AA ′ line. A set of lengths in which the pseudo depth data is outside the range of the ideal pseudo depth distribution data may be mentioned.

  In step S105, the face image data suitability determination unit 114 determines whether the makeup finish in the face image data is appropriate or inappropriate based on the difference data as described above. Various criteria can be considered as the determination criteria in the face image data suitability determination unit 114, for example, the length that the pseudo depth data on the AA ′ line is within the range of the ideal pseudo depth distribution data, and the pseudo depth on the AA ′ line. It may be possible to determine whether or not the finish of the makeup is appropriate according to the ratio of the data being outside the range of the ideal pseudo depth distribution data. Note that such a determination criterion may be provided with an appropriate input interface in the makeup support system 100 so that it can be adjusted and changed as appropriate.

  In step S106, the result of determination by the face image data suitability determination unit 114 is output to the display panel of the determination result output unit 103 or the like. The above is a series of flows from the acquisition of the face image data to the determination of the suitability of the makeup finish, which is executed by the makeup support system 100 of the present invention.

  Next, verification for determining the makeup finish based on the pseudo depth data used by the makeup support system 100 according to the embodiment of the present invention will be described. FIG. 7 is a diagram for explaining a verification result of the makeup support system 100 according to the embodiment of the present invention.

  In FIG. 7, the vertical axis represents pseudo depth (unit is length), and the horizontal axis represents A-A ′ line (unit is pixel) of face image data. An area between two solid lines is the range of ideal pseudo depth distribution data. The range of the ideal pseudo depth distribution data is obtained under the condition that the pseudo depth data of the face image data after makeup made by a professional makeup artist exists in the range with 95% reliability. In FIG. 7, the pseudo depth data indicated by the alternate long and short dash line is pseudo depth data before makeup of a subject. On the other hand, the pseudo depth data indicated by a dotted line is pseudo depth data after makeup of the same person. As can be seen from FIG. 7, the pseudo depth data after makeup has more regions within the range of ideal pseudo depth distribution data than the pseudo depth data before makeup. Thus, it can be understood that it is effective to determine the suitability of the makeup finish based on the pseudo depth data used by the makeup support system 100 according to the embodiment of the present invention.

  As described above, according to the makeup support system of the present invention, information related to the depth of the face, which is very important information from an aesthetic point of view, can be easily calculated without using a plurality of imaging devices or complicated measurement systems. can do. That is, according to the present invention, a makeup support system can be constructed with a simple system configuration that does not require cost and installation space.

  Further, according to the makeup support system of the present invention, it is possible to objectively determine the finish of the subject's makeup based on the information related to the depth of the face, and makeup that has remained in the conventional image composition system Significant improvement can be achieved compared to the support system.

It is a figure which shows the system configuration | structure of the makeup | decoration assistance system which concerns on embodiment of this invention. It is a figure which shows the flow of a process in the makeup | decoration assistance system which concerns on embodiment of this invention. It is a figure which shows the face image data which the face image data acquisition part 102 acquired. It is a figure which shows the feature point of the face used when the front face image data generation part 111 performs an affine transformation. It is a figure which shows the front face image data which the front face image data generation part 111 produces | generates. It is the figure which expressed the pseudo depth data which the pseudo depth information calculation part 112 calculates to the shadow. It is a figure explaining the verification result of the makeup | decoration assistance system 100 which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Makeup support system, 101 ... Main operation part, 102 ... Face image data acquisition part, 103 ... Determination result output part, 104 ... Ideal pseudo depth distribution information storage part, 111 ... Front face image data generation unit, 112 ... pseudo depth information calculation unit, 113 ... pseudo depth information comparison unit, 114 ... face image data suitability determination unit

Claims (4)

Face image data acquisition means for acquiring face image data;
Front face image data generation means for generating front face image data from the face image data acquired by the face image data acquisition means;
Pseudo depth information calculating means for calculating pseudo depth information on a predetermined line of the front face image data obtained by the front face image data generating means;
Ideal pseudo depth distribution information storage means for storing ideal pseudo depth distribution information;
Pseudo depth information comparison means for comparing the pseudo depth information calculated by the pseudo depth information calculation means with the ideal pseudo depth distribution information stored in the ideal pseudo depth distribution information storage means;
And face image data suitability judging means for judging suitability of the face image data acquired based on the comparison result of the pseudo depth information comparing means ,
A makeup support system, wherein the predetermined line of the front face image data is a line passing through both cheeks and nose . The makeup support system according to claim 1, wherein the face image data acquisition means is any one of a camera, a scanner, and a storage medium reading device. 3. The makeup support according to claim 1, wherein the front face image data generating means generates front face image data from the face image data based on the positions of both eyes and the nostrils of the face image data. system. The makeup support system according to any one of claims 1 to 3, wherein the pseudo depth information calculation means calculates the pseudo depth information by a shape restoration method from shading (shading).

Images