JP7335110B2 - EYE MAKEUP PRODUCT RECOMMENDATION PROGRAM, METHOD, APPARATUS AND SYSTEM - Google Patents

Description

The present invention relates to an eye makeup product recommendation program, method, apparatus, and system.

Traditionally, beauty experts such as makeup artists and beauty consultants have found empirical rules for many years (for example, effects due to optical illusions such as eyes appear larger with certain types of makeup. See Non-Patent Document 1), They choose cosmetics that suit the person to whom makeup is applied.

Fragrance Journal March 2013, pp.55-61

However, a user who intends to purchase cosmetics does not always have the opportunity to have cosmetics selected by a beauty expert, and when the user has to select cosmetics himself (for example, when a user purchases cosmetics on an EC site , etc.). Eye makeup products come in a variety of colors and functions, and it is particularly difficult for users to select products that suit them.

Accordingly, an object of one embodiment of the present invention is to recommend an eye makeup product suitable for the user.

According to one aspect of the present invention, an information processing apparatus includes an image acquisition unit that acquires an image of an area around the eye including the eye of a user, a feature extraction unit that extracts a feature from the area around the eye including the eye, the feature and a display unit for displaying the recommended products.

According to the present invention, it is possible to recommend eye makeup products suitable for the user.

1 is an example of an overall configuration diagram (Embodiment 1) according to an embodiment of the present invention; FIG. It is an example (embodiment 1) of a functional block diagram of a user terminal concerning one embodiment of the present invention. FIG. 4 is a diagram for explaining feature extraction (for eyeshadow and open eyes) according to an embodiment of the present invention; FIG. 4 is a diagram for explaining feature extraction (for eyeshadow and closed eyes) according to an embodiment of the present invention; FIG. 5 is a diagram for explaining feature extraction (for mascara and eyeliner) according to an embodiment of the present invention; 6 is a flow chart showing an example of product recommendation processing according to an embodiment of the present invention; 4 is a flow chart showing an example of feature extraction processing (in the case of eyeshadow) according to an embodiment of the present invention; 4 is a flowchart showing an example of feature extraction processing (for mascara and eyeliner) according to an embodiment of the present invention; It is an example (embodiment 2) of the whole lineblock diagram concerning one embodiment of the present invention. 1 is an example of a functional block diagram of an eye makeup product recommendation system according to an embodiment of the present invention (Embodiment 2); FIG. It is a block diagram showing an example of hardware constitutions of a user terminal and a server concerning one embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below based on the drawings. It is assumed that the user 11 receives a recommendation of products for eye makeup that suit the user 11 (hereinafter also referred to as eye makeup products). A case where the user 11 receives product recommendation without connecting the user terminal 10 to the server 20 (Embodiment 1), and a case where the user 11 connects the user terminal 10 to the server 20 and receives product recommendation (Embodiment 2) ) will be described separately.

<Embodiment 1>
Embodiment 1 will be described below.

<Overall configuration diagram>
FIG. 1 is an example (Embodiment 1) of an overall configuration diagram according to one embodiment of the present invention. In <Embodiment 1>, the user 11 uses the user terminal 10 (without connecting the user terminal 10 to the server 20) to receive recommendations for eye makeup products suitable for the user 11. FIG.

A user terminal 10 is a terminal used by a user 11 who receives an eye makeup product recommendation. For example, the user 11 can use a smart phone application installed in the user terminal 10 to receive recommendations for eye makeup products. The user terminal 10 is, for example, any computer (also referred to as an information processing device) such as a smart phone, tablet, or personal computer.

<Functional configuration>
FIG. 2 is an example (embodiment 1) of a functional block diagram of the user terminal 10 according to one embodiment of the present invention. The user terminal 10 includes an image acquisition unit 101 , a feature extraction unit 102 , a product recommendation unit 103 , a display unit 104 , and a feature/product correspondence storage unit 105 . The user terminal 10 functions as an image acquisition unit 101, a feature extraction unit 102, a product recommendation unit 103, and a display unit 104 by executing a program (for example, an application for smartphones). Each of these will be described below.

The image acquisition unit 101 acquires an image in which the periphery of the eye including the eye of the user 11 is captured. For example, the image acquisition unit 101 acquires an image captured by a camera built into the user terminal 10 . Note that the image captured around the eyes of the user 11 may be an image captured of the face including the surroundings of the eyes of the user 11, or may be an image captured around the eyes including the eyes of the user 11. An image in which only the periphery is captured may be used.

Here, an image captured around the eyes including the eyes of the user 11 will be described. In this specification, the eye area including the eye of the user 11 refers to the eye of the user 11 and the area around the eye. There are two images taken around the eyes of the user 11: an image taken with the eyes of the user 11 open and an image taken with the eyes closed. can contain one image.

The feature extraction unit 102 extracts features from around the eyes including the eyes of the user 11 . In the following, the case where the recommended eye makeup product is eyeshadow and the case where it is mascara and eyeliner will be explained separately. Note that the following feature points are just an example, and other feature points may be used.

<For eyeshadow>
A method of extracting features from around the eyes including the eyes of the user 11 will be described with reference to FIG. 3 (the user 11's eyes are open) and FIG. .

FIG. 3 is a diagram for explaining feature extraction (in the case of eye shadow and eye opening) according to an embodiment of the present invention. The feature extraction unit 102 calculates the brightness of the iris in the pupil of the user 11 .

For example, the feature extracting unit 102 extracts the feature points of the left eye ((6) to (11) in FIG. 3) and the feature points of the right eye in an image of the user 11 whose eyes are open as shown in FIG. ((17) to (22) in FIG. 3) are extracted. (6) and (17) indicate the inner corner of the eye (the edge closer to the nose). (7), (8), (10), (11) and (18), (19), (21), (22) indicate the boundaries between the iris, the white of the eye and the area around the eye. (9) and (20) indicate the outer corner of the eye (end closer to the ear).

For example, for the left eye, the feature extraction unit 102 draws a line connecting (7) and (8), a line connecting (8) and (10), a line connecting (10) and (11), A rectangle inscribed in a quadrangle formed by a line connecting (11) and (7) is generated. The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains the minimum value of the converted L* value (brightness).

Similarly for the right eye, the feature extraction unit 102 extracts a line connecting (18) and (19), a line connecting (19) and (21), and a line connecting (21) and (22). , and a line connecting (22) and (18). The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains the minimum value of the converted L* value (brightness). The minimum value of L* (lightness) for the left eye and the minimum value of L* (lightness) for the right eye are compared, and the minimum value between them is defined as "L_PPL _GND ".

FIG. 4 is a diagram for explaining feature extraction (for eyeshadow and closed eyes) according to an embodiment of the present invention. The feature extraction unit 102 calculates the brightness of the user's 11 eyelids (inner corner and center).

For example, the feature extracting unit 102 extracts the feature points of the left eye ((6) to (11) in FIG. 4) and the feature points of the right eye in an image of the user 11 whose eyes are closed as shown in FIG. ((17) to (22) in FIG. 4) are extracted. (6) to (11) and (17) to (22) in FIG. 4 are characteristic points corresponding to (6) to (11) and (17) to (22) in FIG.

Further, the feature extraction unit 102 extracts feature points ((1) to (5) in FIG. 4) of the left eyebrow and right Characteristic points of eyebrows ((12) to (16) in FIG. 4) are extracted. (1) and (12) indicate the inner corner of the eye, and (5) and (16) indicate the outer corner of the eye. (2), (3), and (4) are points that divide the curve connecting (1) and (5) into four equal parts. (13), (14), and (15) are points that divide the curve connecting (12) and (16) into four equal parts.

<Brightness of the inner corner of the eye>
For example, for the left eye, the feature extraction unit 102 connects the eyebrow (1) and the eye (6) with a line, and obtains the distance and midpoint between the two points. The feature extraction unit 102 generates a rectangle centered at the found midpoint and having a side length of 1/3 of the distance between the two points (i.e., the distance between (1) and (6)). . The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains an average value of the converted L* values (brightness), and sets the average value of the L* values as "L_EYE _IN ".

Similarly for the right eye, the feature extraction unit 102 connects the eyebrow (12) and the eye (17) with a line, and obtains the distance and midpoint between the two points. The feature extracting unit 102 generates a rectangle centered at the obtained midpoint and having a side length of 1/3 of the distance between the two points (i.e., the distance between (12) and (17)). . The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains an average value of the converted L* values (brightness), and sets the average value of the L* values as "L_EYE _IN ".

<Brightness of central part>
For example, for the left eye, the feature extraction unit 102 connects the eyebrow (4) and the eye (7) with a line, and obtains the distance and midpoint between the two points. The feature extraction unit 102 generates a rectangle centered at the found midpoint and having a side length of 1/3 of the distance between the two points (i.e., the distance between (4) and (7)). . The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains an average value of the converted L* values (brightness), and sets the average value of the L* values as "L_EYE _CNT ".

Similarly for the right eye, the feature extraction unit 102 connects the eyebrow (15) and the eye (18) with a line, and obtains the distance and midpoint between the two points. The feature extraction unit 102 generates a rectangle centered at the found midpoint and having a side length of 1/3 of the distance between the two points (i.e., the distance between (15) and (18)). . The feature extraction unit 102 converts the RGB values of each pixel in the generated rectangular area into L*a*b* values. The feature extraction unit 102 obtains an average value of the converted L* values (brightness), and sets the average value of the L* values as "L_EYE _CNT ".

As described above, the feature extraction unit 102 calculates the brightness of the user's 11 eyelids (center portion and inner corner portion) and the brightness of the iris in the pupil of the user 11 for the right eye and the left eye. The feature extraction unit 102 substitutes the calculated brightness into (Equation 1) to calculate the brightness of the right eyelid and the brightness of the left eyelid. Instead of using a constant ("100" in (Formula 1)) in the denominator as in (Formula 1), a variable (for example, "L_EYE _CNT " instead of "100") may be used. .

・・・（式１）

... (Formula 1)

In this way, based on the average value of the brightness of the central part of the eyelid of the user 11 (that is, the part that can be assumed to be the brightest in the eyelid) and the brightness of the inner corner part (that is, the part that can be assumed to be the darkest in the eyelid) Then, the brightness of each eyelid is calculated.

The feature extraction unit 102 calculates the average value of the brightness of the right eyelid and the brightness of the left eyelid calculated by the above (Equation 1). The average value of the brightness of the right eyelid and the brightness of the left eyelid is used to select products to recommend to the user 11 .

Here, the brightness of the eyelids will be explained. The method for calculating the brightness of the eyelids is not limited to the above method. For example, the feature extraction unit 102 can obtain the minimum V value of the HSV values of each pixel in the rectangular area of the iris in the pupil, and set the minimum V value of the HSV values to "L_PPL _GND ". . Further, the feature extraction unit 102 can obtain the average value of the V values of the HSV values of the pixels in the rectangular region of the inner corner of the eyelid, and set the average value of the V values of the HSV values to "L_EYE _IN ". . In addition, the feature extraction unit 102 can obtain the average value of the V values of the HSV values of the pixels in the rectangular area in the central portion of the eyelid, and set the V value of the HSV values to "L_EYE _CNT ". In addition, it is also possible to use the luminance value of any of the RGB values (for example, the average value of the G value on the G frame or the B value on the B frame).

Note that the feature extraction unit 102 provides the user 11 with the saturation (vividness) of the eyelid, such as the S value of the HSV value, in addition to the brightness of the eyelid, or instead of the brightness of the eyelid. It may be extracted as a feature used for selecting recommended products.

<For mascara and eyeliner>
A method of extracting features from around the eyes including the eyes of the user 11 will be described with reference to FIG.

FIG. 5 is a diagram for explaining feature extraction (for mascara and eyeliner) according to an embodiment of the present invention. The feature extraction unit 102 calculates the ratio of the vertical width to the horizontal width of the eyes of the user 11 .

For example, the feature extraction unit 102 extracts the feature points of the left eye ((6) to (11) in FIG. 5) and the feature points of the right eye in the image of the user 11 with the eyes open as shown in FIG. ((17) to (22) in FIG. 5) are extracted. (6) and (17) indicate the inner corner of the eye (the edge closer to the nose). (7), (8), (10), (11) and (18), (19), (21), (22) indicate the boundaries between the iris, the white of the eye and the area around the eye. (9) and (20) indicate the outer corner of the eye (end closer to the ear).

For example, the feature extraction unit 102 generates a line (line 1) passing through (7) or (8) perpendicular to the median line for the left eye. Also, the feature extraction unit 102 generates a line (line 2) passing through (10) or (11) perpendicular to the median line. The feature extraction unit 102 sets the shortest distance between lines 1 and 2 as the vertical width of the left eye of the user 11, and sets the distance between the two points (6) and (9) as the horizontal width of the left eye of the user 11. FIG.

Similarly for the right eye, the feature extraction unit 102 generates a line (line 1) passing through (18) or (19) perpendicular to the median line. Also, the feature extraction unit 102 generates a line (line 2) passing through (21) or (22) perpendicular to the median line. The feature extraction unit 102 sets the shortest distance between the lines 1 and 2 as the vertical width of the user's 11 right eye, and sets the distance between the two points (17) and (20) as the horizontal width of the user's 11 right eye.

As described above, the feature extraction unit 102 calculates the vertical width and horizontal width of the eyes of the user 11 for the right eye and the left eye, and calculates the ratio between the vertical width and the horizontal width. For example, based on the aspect ratio of the eyes, it is determined whether the user's 11 eyes are round or slit.

The feature extraction unit 102 calculates an average value of the calculated ratio of the vertical width to the horizontal width of the right eye and the ratio of the vertical width to the horizontal width of the left eye. The average value of the ratio of the height to width of the right eye and the ratio of the height to width of the left eye is used to select products to recommend to the user 11 .

The feature/merchandise correspondence relationship storage unit 105 contains information on features extracted from the eye area including the user's eyes, and information on items that are judged by beauty experts such as makeup artists and beauty consultants to be suitable for the user having the features. and information on one or more products are stored in association with each other.

A plurality of beauty experts such as makeup artists and beauty consultants may recommend products. In this case, the user 11 is allowed to designate a desired makeup artist or the like on the user terminal 10, and the product recommendation unit 103 refers to the correspondence relationship by the designated makeup artist or the like.

For example, the characteristic/merchandise correspondence storage unit 105 stores a plurality of colors suitable for the user 11 whose eyelids are bright (for example, the average value of the brightness of the right eyelid and the brightness of the left eyelid is equal to or greater than a threshold). and eyeshadow information of a plurality of colors suitable for the user 11 whose eyelid brightness is dark (for example, the average value of the brightness of the right eyelid and the brightness of the left eyelid is smaller than a threshold value). stored.

For example, in the feature/merchandise correspondence storage unit 105, for the user 11 who has round eyes (for example, the average value of the ratio of the vertical width to the horizontal width of the right eye and the ratio of the vertical width to the horizontal width of the left eye is smaller than a threshold value), Information on multiple types of mascara and eyeliner that match, and a user 11 who is a break (for example, the average value of the ratio of the height to width of the right eye and the ratio of the height to width of the left eye is greater than or equal to a threshold). It stores information on multiple types of mascara and eyeliner that suit the user.

In addition, information on the characteristics of a plurality of users is collected in advance, and those users are divided into four equal parts (that is, round eyes with bright eyelids, split eyes with bright eyelids, round eyes with dark eyelids, A value that can be used as a threshold value (that is, a threshold value for determining whether the brightness of the eyelid is light or dark, a threshold value for determining whether the eyelid is round or a discontinuity) may be used.

The product recommendation unit 103 recommends products for eye makeup based on the features extracted by the feature extraction unit 102 . Specifically, the product recommendation unit 103 refers to the correspondence stored in the feature/product correspondence storage unit 105, and refers to the product corresponding to the feature extracted by the feature extraction unit 102 (for example, the product matching the user 11). The user 11 selects an eyeshadow of a different color and a type of mascara that suits the user 11 (eg, a mascara that makes the eyelashes look longer, a mascara that makes the eyelashes look thicker, a mascara that makes the eyelashes look thicker, etc.).

<User Preferences>
In one embodiment of the present invention, suitable products can be recommended to the user based on preferences specified by the user 11 . Specifically, the image acquisition unit 101 acquires preferences specified by the user (for example, selection of an image that evokes an impression after applying makeup, answers to a questionnaire) along with the image. The product recommendation unit 103 refers to the correspondence stored in the feature/product correspondence storage unit 105 and selects products corresponding to the user's preference and the features extracted by the feature extraction unit 102 . In this case, the feature/merchandise correspondence storage unit 105 stores the user's preferences, features extracted from the eye area including the user's eyes, and beauty products such as makeup artists and beauty consultants that match the user who has the features. are stored in association with the goods judged by the experts.

<Combination of eyeshadow and mascara>
In one embodiment of the present invention, products can be recommended considering the combination of eyeshadow and mascara. Specifically, eye shadows of a plurality of colors suitable for the user 11 with bright eyelids and a plurality of types of mascara suitable for the user 11 with round eyes are recommended by a makeup artist or the like. The products recommended to the user 11 are shadows, mascara, and application methods. Among the combinations of eye shadows of multiple colors suitable for the user 11 with bright eyelids and multiple types of mascaras suitable for the user 11 having gaps, the combination of eye shadows and mascara recommended by a makeup artist or the like. and the application method are products recommended to the user 11 . Among the combinations of eye shadows of multiple colors suitable for the user 11 with dark eyelids and multiple types of mascara suitable for the user 11 with round eyes, eye shadows and mascaras recommended by makeup artists. and the application method are products recommended to the user 11 . Also, among combinations of eye shadows of multiple colors suitable for the user 11 whose eyelids are dark and multiple types of mascara suitable for the user 11 with gaps, a combination of eye shadows and mascara recommended by a makeup artist or the like. and the application method are products recommended to the user 11 . In addition, you may combine an eye shadow, a mascara, and an eyeliner.

The display unit 104 presents the product recommended by the product recommendation unit 103 to the user 11 . Specifically, the display unit 104 displays information such as the product name, description, price, and product image of the product recommended by the product recommendation unit 103 on the screen of the user terminal 10 . In one embodiment of the present invention, the user 11 may purchase the displayed product on the Internet.

FIG. 6 is a flow chart showing an example of product recommendation processing according to one embodiment of the present invention.

In step 11 ( S<b>11 ), the image acquisition unit 101 acquires an image of the eye area including the eyes of the user 11 .

In step 12 (S12), the feature extraction unit 102 extracts features from the periphery of the eyes including the eyes of the image acquired in S11.

At step 13 (S13), the product recommendation unit 103 recommends products for eye makeup based on the features extracted at S12.

At step 14 (S14), the display unit 104 displays the product recommended at S13.

FIG. 7 is a flowchart showing an example of feature extraction processing (in the case of eyeshadow) according to one embodiment of the present invention.

In step 21 (S21), the feature extraction unit 102 extracts feature points (left eye (6) to (11) of the right eye, (17) to (22) of the right eye, (1) to (5) of the left eyebrow, and (12) to (16) of the right eyebrow.

In step 22 (S22), the feature extraction unit 102 calculates the brightness of the central portion of the eyelid of the user 11, the brightness of the inner corner of the eyelid of the user 11, and the brightness of the pupil of the user 11 based on the feature points extracted in S21. Calculate the lightness of the iris.

In step 23 (S23), the feature extraction unit 102 substitutes the brightness calculated in S22 into the above equation (1) to calculate the brightness of the right eyelid and the brightness of the left eyelid.

At step 24 (S24), the feature extraction unit 102 calculates the average value of the brightness of the right eyelid and the brightness of the left eyelid calculated at S23. The average value of the brightness of the right eyelid and the brightness of the left eyelid is used to select products to recommend to the user 11 .

FIG. 8 is a flow chart showing an example of feature extraction processing (for mascara and eyeliner) according to an embodiment of the present invention.

In step 31 (S31), the feature extraction unit 102 extracts feature points (left eye (6) to (11), right eye (17) to (22 )) to extract.

At step 32 (S32), the feature extraction unit 102 calculates the vertical width and horizontal width of the eye based on the feature points extracted at S31.

At step 33 (S33), the feature extraction unit 102 calculates the ratio of the vertical width to the horizontal width of the eye calculated at S32.

At step 34 (S34), the feature extraction unit 102 calculates the average value of the ratio of the height to width of the right eye and the ratio of the height to width of the left eye calculated at S33. The average value of the ratio of the height to width of the right eye and the ratio of the height to width of the left eye is used to select products to recommend to the user 11 .

<Embodiment 2>
Embodiment 2 will be described below.

<Overall configuration diagram>
FIG. 9 is an example of an overall configuration diagram (Embodiment 2) according to one embodiment of the present invention. In <Embodiment 2>, the user 11 connects the user terminal 10 to the server 20 and receives recommendations for eye makeup products suitable for the user 11 .

As shown in FIG. 9 , an eye makeup product recommendation system (also referred to as an information processing system) 1 includes a user terminal 10 and a server 20 . The user terminal 10 and the server 20 are communicably connected via an arbitrary network 30 . Each of these will be described below.

A user terminal 10 is a terminal used by a user 11 who receives an eye makeup product recommendation. For example, the user 11 can connect to the server 20 using a smartphone application installed in the user terminal 10 and receive recommendations for eye makeup products. The user terminal 10 is, for example, any computer (also referred to as an information processing device) such as a smart phone, tablet, or personal computer.

The server 20 executes processing for recommending eye makeup products suitable for the user 11 in response to a request from the user terminal 10 . The server 20 consists of one or more computers (also called information processing devices).

<Functional configuration>
FIG. 10 is an example (embodiment 2) of a functional block diagram of an eye makeup product recommendation system according to an embodiment of the present invention. In one embodiment of the present invention, at least one of the image acquisition unit 101, the feature extraction unit 102, the product recommendation unit 103, the display unit 104, and the feature/product correspondence relationship storage unit 105 described in <Embodiment 1> is A server 20 may comprise. In <Embodiment 2>, the user terminal 10 includes an image acquisition unit 101 and a display unit 104 , and the server 20 includes a feature extraction unit 102 , a product recommendation unit 103 , and a feature/product correspondence storage unit 105 . A detailed description will be given below.

The user terminal 10 includes an image acquisition section 101 , an image transmission section 106 , a product information reception section 109 and a display section 104 . Each of these will be described below.

The image acquisition unit 101 is the same as that of <Embodiment 1>, so description thereof is omitted.

The image transmission unit 106 transmits the image acquired by the image acquisition unit 101 to the server 20 .

The product information receiving unit 109 receives information on products recommended by the server 20 from the server 20 .

The display unit 104 displays the product information received by the product information receiving unit 109 on the screen of the user terminal 10 .

The server 20 includes an image reception unit 107 , a feature extraction unit 102 , a product recommendation unit 103 , a product information transmission unit 108 , and a feature/product correspondence storage unit 105 . Each of these will be described below.

The image receiving unit 107 receives the image acquired by the image acquiring unit 101 of the user terminal 10 from the user terminal 10 .

The feature extraction unit 102 is the same as that of the <Embodiment 1>, so description thereof is omitted.

The product recommendation unit 103 is the same as that of <Embodiment 1>, so description thereof will be omitted.

The feature/merchandise correspondence relationship storage unit 105 is the same as that of the <first embodiment>, so the description thereof is omitted.

The product information transmission unit 108 transmits information on products recommended by the product recommendation unit 103 to the user terminal 10 .

As described above, in one embodiment of the present invention, the image acquisition unit 101, the feature extraction unit 102, the product recommendation unit 103, the display unit 104, and the feature/product correspondence relationship storage unit 105 described in <Embodiment 1> At least one can be provided by server 20 .

<effect>
Thus, in one embodiment of the present invention, based on features extracted from the eye area, including the user's eyes, a beauty professional, such as a makeup artist or beauty consultant, may be fitted with a user having such features. Can recommend products for eye makeup as judged by For example, an appropriate color based on the brightness of the eyelid (specifically, the average value of the brightness of the central part of the eyelid and the brightness of the inner corner of the eyelid and the brightness of the darkest part of the black eye in the pupil) It can recommend eye shadows, or recommend the appropriate type of mascara or eyeliner and how to apply it depending on whether the eye is round or not (specifically, the aspect ratio of the eye).

<Hardware configuration>
FIG. 11 is a block diagram showing an example of the hardware configuration of an information processing device (that is, user terminal 10 and server 20) according to an embodiment of the present invention. The user terminal 10 and the server 20 have a CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 . The CPU 1001, ROM 1002, and RAM 1003 form a so-called computer.

The user terminal 10 and the server 20 also have an auxiliary storage device 1004 , a display device 1005 , an operation device 1006 , an I/F (Interface) device 1007 and a drive device 1008 . The hardware of the user terminal 10 and the server 20 are interconnected via a bus B. FIG.

A CPU 1001 is an arithmetic device that executes various programs installed in an auxiliary storage device 1004 .

ROM 1002 is a non-volatile memory. The ROM 1002 functions as a main storage device that stores various programs, data, etc. necessary for the CPU 1001 to execute various programs installed in the auxiliary storage device 1004 . Specifically, the ROM 1002 functions as a main storage device that stores boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface).

A RAM 1003 is a volatile memory such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). The RAM 1003 functions as a main storage device that provides a work area that is developed when various programs installed in the auxiliary storage device 1004 are executed by the CPU 1001 .

The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when various programs are executed.

The display device 1005 is a display device that displays the internal states of the user terminal 10 and the server 20, and the like.

The operation device 1006 is an input device through which an administrator of the user terminal 10 and the server 20 inputs various instructions to the user terminal 10 and the server 20 .

The I/F device 1007 is a communication device for connecting to the network 30 and communicating with the user terminal 10 and the server 20 .

A drive device 1008 is a device for setting a storage medium 1009 . The storage medium 1009 here includes media for optically, electrically, or magnetically recording information such as CD-ROMs, flexible disks, and magneto-optical disks. The storage medium 1009 may also include a semiconductor memory that electrically records information such as an EPROM (Erasable Programmable Read Only Memory), a flash memory, or the like.

Various programs to be installed in the auxiliary storage device 1004 are installed by, for example, setting the distributed storage medium 1009 in the drive device 1008 and reading the various programs recorded in the storage medium 1009 by the drive device 1008. be done. Alternatively, various programs installed in the auxiliary storage device 1004 may be installed by being downloaded from another network different from the network 30 via the I/F device 1007 .

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications can be made within the scope of the gist of the present invention described in the claims.・Changes are possible.

1 eye makeup product recommendation system 10 user terminal 11 user 20 server 30 network 101 image acquisition unit 102 feature extraction unit 103 product recommendation unit 104 display unit 105 feature/product correspondence storage unit 106 image transmission unit 107 image reception unit 108 product information transmission Unit 109 Merchandise information receiving unit

Claims (11)

information processing equipment,
An image of the vicinity of the user's eyes including the eyes is acquired, and the images are an image captured with the user's eyes open and a image captured with the user's eyes closed. an image acquisition unit, which is an image obtained by
A feature extraction unit that extracts the brightness of the user's eyelids and the brightness of the iris in the user's pupil from the periphery of the eye including the eye,
A product recommendation unit that recommends an eye shadow suitable for the user based on the average value of the brightness of the central portion of the eyelid and the brightness of the inner corner of the eyelid, and the brightness of the black eye;
A program for functioning as a display for displaying the recommended eyeshadow . information processing equipment,
an image acquisition unit that acquires an image of a vicinity of the user's eyes including the eyes, wherein the image is an image of the user with the eyes open;
a feature extracting unit that extracts the ratio of the vertical width to the horizontal width of the user's eyes from around the eyes, including the eyes;
a product recommendation unit that recommends mascara and eyeliner suitable for the user based on the ratio of the vertical width to the horizontal width of the eye;
A program for acting as a display for displaying said recommended mascara and eyeliner. The feature extraction unit further extracts the saturation of the user's eyelids,
2. The program according to claim 1 , wherein said product recommendation unit recommends eye shadow suitable for said user based on said eyelid lightness and said eyelid saturation. The product recommendation unit recommends products suitable for the user based on a correspondence relationship between the features extracted by the feature extraction unit and products suitable for the user having the features extracted by the feature extraction unit. 4. The program according to any one of 1 to 3 . 5. The product recommendation unit according to any one of claims 1 to 4 , wherein the product recommendation unit recommends products suitable for the user based on the features extracted by the feature extraction unit and preferences specified by the user. program. A method executed by an information processing device, comprising:
An image of the vicinity of the user's eyes including the eyes is acquired, and the images are an image captured with the user's eyes open and a image captured with the user's eyes closed. Steps and
extracting the lightness of the user's eyelids and the lightness of the iris in the user's pupil from the periphery of the eye, including the eye;
recommending an eye shadow suitable for the user based on the average value of the brightness of the central portion of the eyelid and the brightness of the inner corner of the eyelid, and the brightness of the iris;
and displaying said recommended eyeshadow . A method executed by an information processing device, comprising:
a step of obtaining an image of a vicinity of the user's eyes including the eyes, wherein the image is an image of the user with the eyes open;
extracting the ratio of the vertical width to the horizontal width of the eye of the user from the vicinity of the eye including the eye;
recommending suitable mascara and eyeliner for the user based on the length-to-width ratio of the eye;
displaying said recommended mascara and eyeliner;
method including. An image of the vicinity of the user's eyes including the eyes is acquired, and the images are an image captured with the user's eyes open and a image captured with the user's eyes closed. an image acquisition unit, which is an image obtained by
A feature extraction unit that extracts the brightness of the user's eyelids and the brightness of the iris in the user's pupil from the periphery of the eye including the eye;
a product recommendation unit that recommends an eye shadow suitable for the user based on the average value of the brightness of the central portion of the eyelid and the brightness of the inner corner of the eyelid, and the brightness of the iris;
and a display unit that displays the recommended eyeshadow . an image acquisition unit that acquires an image of the vicinity of the user's eyes including the eyes, wherein the image is an image of the user with the eyes open;
a feature extracting unit that extracts the ratio of the vertical width to the horizontal width of the user's eyes from around the eyes, including the eyes;
a product recommendation unit that recommends mascara and eyeliner suitable for the user based on the ratio of the vertical width to the horizontal width of the eye;
a display displaying said recommended mascara and eyeliner;
Information processing device with An information processing system including a user terminal and a server,
An image of the user's eye area including the eye is acquired, and the image is an image taken with the user's eyes open and an image taken with the user's eyes closed. an image acquisition unit, which is an image obtained by
A feature extraction unit that extracts the brightness of the user's eyelids and the brightness of the iris in the user's pupil from the periphery of the eye including the eye;
a product recommendation unit that recommends an eye shadow suitable for the user based on the average value of the brightness of the central portion of the eyelid and the brightness of the inner corner of the eyelid, and the brightness of the iris;
an information processing system comprising: a display unit that displays the recommended eye shadow . An information processing system including a user terminal and a server,
an image acquisition unit that acquires an image of the vicinity of the user's eyes including the eyes, wherein the image is an image of the user with the eyes open;
a feature extracting unit that extracts the ratio of the vertical width to the horizontal width of the user's eyes from around the eyes, including the eyes;
a product recommendation unit that recommends mascara and eyeliner suitable for the user based on the ratio of the vertical width to the horizontal width of the eye;
a display displaying said recommended mascara and eyeliner;
Information processing system with

Images