HK40112944A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

The present invention promotes improved usability for a user who is referring to another person's makeup. This information processing device (100) comprises a generation unit (132) and a determination unit (135). The generation unit (132) generates, from a facial image of a poster who has applied prescribed makeup, makeup information for generating a facial image in which an equivalent makeup to the prescribed makeup has been applied to a facial image. The determination unit (135) determines a remuneration to be provided to the poster who posted the facial image serving as the source from which the makeup information was generated, said remuneration being provided in accordance with the usage status of a facial image in which the makeup information has been applied to a facial image of the user.

Description

Information processing device, information processing method and information processing program

Technical Field

This invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Technology

When researching cosmetics purchases, the following situation often occurs: even when watching makeup applied to the faces of models or other people, one cannot imagine the image of oneself in such a situation, and therefore does not know whether it is suitable for oneself.

Conventionally, as a technique for fulfilling needs such as confirming whether makeup applied to another person's face is suitable for oneself, techniques that use other people's makeup as a reference to assist in makeup application are known. For example, techniques that present other people's makeup information corresponding to areas of the user's facial features are known.

Existing technical documents

Patent documents

Patent Document 1: Japanese Patent No. 5991536

Non-Patent Literature 1: W. Jiang, S. Liu, C. Gao, J. Cao, R. He, J. Feng, S. Yan, “PSGAN: Pose and Expression Robust Spatial-Aware CAN for Customizable Makeup Transfer,” [online], [searched January 24, 2022], Internet <https://openaccess.thecvf.com/content_CVPR_2020/html/Jiang_PSGAN_Pose_and_Expression_Robust_Spatial-Aware_GAN_for_Customizable_Makeup_CVPR_2020_paper.html>

Summary of the Invention

The problem the invention aims to solve

However, current technology lacks features such as the ability to guide customers to sales websites after they've confirmed whether other people's makeup looks suit them, and the inability to use performance-based rewards to incentivize makeup creators to actively guide customers. Therefore, there is room for improvement in usability for users who use others' makeup looks as a reference.

This application was made in view of the above circumstances, and its purpose is to further improve usability for users who use other people's makeup as a reference.

Solution for solving the problem

The information processing apparatus of this application is characterized by comprising: a generation unit that generates makeup information based on a facial image of a publisher who has applied prescribed makeup, the makeup information being used to generate a facial image after applying the same makeup as the prescribed makeup to the facial image; and a determination unit that determines the reward to be given to the publisher who published the facial image that is the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

The effects of the invention

According to one aspect of the implementation, this achieves the effect of further improving usability for users who can use other people's makeup as a reference.

Attached Figure Description

Figure 1 is a diagram illustrating a structural example of an information processing system according to an embodiment.

Figure 2 is a diagram illustrating an example of information processing involved in the implementation method.

Figure 3 is an explanatory diagram illustrating the information processing in the case of increasing the priority of facial images according to the embodiment.

Figure 4 is a diagram showing an example of the structure of a user terminal according to the embodiment.

Figure 5 is a diagram illustrating an example of the structure of the publisher terminal involved in the implementation method.

Figure 6 is a diagram showing a structural example of the information processing apparatus according to the embodiment.

Figure 7 is a diagram illustrating an example of the publisher information storage unit involved in the embodiment.

Figure 8 is a diagram illustrating an example of the cosmetic information storage unit involved in the embodiment.

Figure 9 is a flowchart illustrating an example of information processing involved in the implementation method.

Figure 10 is a hardware structure diagram illustrating an example of a computer that implements the functions of an information processing device.

Detailed Implementation

Hereinafter, with reference to the accompanying drawings, we will describe in detail the methods (hereinafter referred to as "implementations") for carrying out the information processing apparatus, information processing method, and information processing program involved in this application. However, the information processing apparatus, information processing method, and information processing program involved in this application are not limited to these embodiments. Furthermore, in the following embodiments, the same reference numerals will be used to label the same parts, and repeated descriptions will be omitted.

(Implementation Method)

(1. The structure of an information processing system)

The information processing system 1 shown in FIG1 will be described. As shown in FIG1, the information processing system 1 includes a user terminal 10, a publisher terminal 20, and an information processing device 100. The user terminal 10, the publisher terminal 20, and the information processing device 100 are connected via a predetermined communication network (network N) in a wired or wireless communication manner. FIG1 is a diagram showing an example of the structure of the information processing system 1 according to the embodiment.

User terminal 10 is an information processing device used by a user who wants to confirm whether makeup applied to another person's face is suitable for them. Additionally, the user using user terminal 10 may be researching and purchasing cosmetics. User terminal 10 can be any device as long as it can perform the processing described in the embodiment. Furthermore, user terminal 10 can also be a smartphone, tablet, laptop PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), etc. Figure 2 shows the case where user terminal 10 is a smartphone.

User terminal 10 is a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G (Generation) and LTE (Long Term Evolution). Additionally, user terminal 10 may have a screen such as an LCD display with touch panel functionality, allowing the user to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or a stylus. In Figure 2, user terminal 10 is used by user U11.

The publisher terminal 20 is an information processing device used by a publisher who provides a facial image after makeup. Alternatively, a publisher using the publisher terminal 20 may publish, for example, a facial image after makeup is created by applying makeup from a bare face before makeup application. The publisher terminal 20 can be any device as long as it can perform the processing described in the embodiment. Furthermore, the publisher terminal 20 can also be a smartphone, tablet, laptop PC, desktop PC, mobile phone, PDA, or other similar device. Figure 2 shows the case where the publisher terminal 20 is a smartphone.

The publisher terminal 20 is, for example, a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G and LTE. Additionally, the publisher terminal 20 may also have a screen such as an LCD display with touch panel functionality, allowing the publisher to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or styluses. In Figure 2, the publisher terminal 20 is used by publishers P11 and P12. Publisher terminal 20 ₁ and Publisher terminal 20 ₂ are the publisher terminals 20 for publishers P11 and P12, respectively.

Information processing device 100 is an information processing device aimed at further improving usability for users who use other people's makeup as a reference. Any device is acceptable as long as it can perform the processing described in the embodiment. Information processing device 100 is implemented, for example, by a server device, cloud system, etc. For example, information processing device 100 is implemented by a server device, cloud system, etc., that provides a prescribed electronic mall for selling cosmetics.

(2. An example of information processing)

Figure 2 is a diagram illustrating an example of information processing in the information processing system 1 according to the embodiment. Hereinafter, in the embodiment, the processing for a user U11 who wants to confirm whether the publisher P11's makeup suits them will be described as an example.

Information processing device 100 acquires information about the facial image selected by user U11 from facial images published by a publisher (step S101). This publisher is, for example, a celebrity or influencer whose number of followers on an SNS (Social Networking Service) exceeds a predetermined threshold. Furthermore, the facial images that user U11 can select may include multiple facial images published by a single publisher. In Figure 2, this is assumed to include facial image IM11 published by publisher P11, facial image IM12 published by publisher P12, etc. The following explanation assumes that user U11 selected facial image IM11 from content CT11. Furthermore, the facial images displayed in content CT11 can be selected based on any criterion. For example, facial images published by publishers followed by user U11 can be preferentially selected. Additionally, content CT11 is content within a specified service that manages the facial images of publishers.

Facial image IM11 is a facial image of publisher P11 after applying prescribed makeup. Furthermore, facial image IM21 is a facial image of publisher P11 before makeup, corresponding to the makeup applied to facial image IM11. Facial image IM21 can be displayed together with facial image IM11 on content CT11, or it can be displayed through operations on facial image IM11 (e.g., clicking or tapping). Alternatively, facial image IM21 may not be displayed to the user. Therefore, the publisher can avoid providing a bare-faced image to the user, thus improving usability for the publisher. Additionally, facial image IM12 is a facial image of publisher P12 after applying prescribed makeup. Furthermore, facial image IM22 is a facial image of publisher P12 before makeup, corresponding to the makeup applied to facial image IM12. Facial image IM22 can be displayed together with facial image IM12 on content CT11, or it can be displayed through operations on facial image IM12. Alternatively, facial image IM22 may not be displayed to the user.

Here, as an example of a method for generating makeup information, we cite, for instance, the method disclosed in Non-Patent Document 1, known as PSGAN (Pose and Expression Robust Spatial-Aware Generative Adversarial Network), which simply moves makeup information from one face image to another (Non-Patent Document 1). This method uses deep learning and is able to move makeup information to another face regardless of expression or pose. The information processing device 100 uses a method such as PSGAN to generate makeup information, which is used to generate a face image after applying the same makeup as that used by publisher P11 in the face image IM11 selected by user U11 (step S102). Then, the information processing device 100 uses a method such as PSGAN to apply the makeup information generated in step S102 to the face image IM1 of user U11 (step S103). At this time, the face image IM1 can also be any arbitrarily acquired face image. Facial image IM1 can be, for example, a facial image pre-registered by user U11 and stored in an external information processing device or within this device, or a facial image sent from user terminal 10. Furthermore, facial image IM1 can be a facial image selected as the application target by user U11, or a facial image automatically selected as the application target. Additionally, facial image IM2 is the applied facial image corresponding to facial image IM1.

The information processing device 100 sends information to the user terminal 10 for displaying a facial image IM2 of the user U11 after the makeup information was applied in step S103, and a description of the cosmetics used in the makeup (corresponding to the item) (step S104). Then, the user terminal 10 displays content CT1 based on the information sent from the information processing device 100.

Here, content CT1 includes descriptions of the cosmetics used by publisher P11 in the makeup application related to facial image IM11. For example, it includes information such as the product name and price of the cosmetics. Figure 2 shows the case where two cosmetics (PR1 and PR2) are included, namely lipstick and foundation, but there is no particular limitation on the number of cosmetics. For example, content CT1 may also include descriptions of the complete set of cosmetics used by publisher P11 to create the prescribed post-makeup state of facial image IM11 from the pre-makeup state of facial image IM21. Additionally, the descriptions may include operation acceptance information B1 for accessing a prescribed online store where users can purchase the complete set of cosmetics used in the makeup application. When operation acceptance information B1 is accessed, users can access either the product page introducing the complete set of cosmetics or the purchase page after adding the complete set to their cart. Furthermore, operation acceptance information B1 can be integrated with facial image IM2 and other content in the descriptions, allowing access to the product page and purchase page when operations are performed on facial image IM2 and other content in the descriptions.

Information processing device 100 determines whether user U11 has purchased cosmetics associated with the description content. For example, information processing device 100 determines whether user U11 purchased a complete set of cosmetics, including cosmetics PR1 and cosmetics PR2, or a portion of cosmetics from a designated online store accessed through operation acceptance information B1. If information processing device 100 determines that user U11 has made a purchase, it decides to award the corresponding reward to publisher P11, who is the publisher who published the facial image after makeup using the makeup information corresponding to image IM2. Then, information processing device 100 performs processing to award the corresponding reward to publisher P11. Alternatively, information processing device 100 may decide to send information for publisher P11 to receive the corresponding reward for user U11's purchase. In this case, information processing device 100 sends information for publisher P11 to receive the corresponding reward for user U11's purchase. Alternatively, the information processing device 100 may decide to transfer the payment to the publisher P11's account. In this case, the information processing device 100 performs the processing for transferring the payment to the publisher P11's account.

(Version 1: Display using a pre-makeup facial image)

In the above embodiment, a case is shown where pre-defined post-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, but this is not limited to this example. For example, facial images such as facial image IM21 and facial image IM22 before makeup can also be displayed in content CT11, and the post-makeup facial image can be displayed by manipulating the pre-makeup facial image. Furthermore, the information processing device 100 can also generate makeup information applied to the user's facial image by allowing the user U11 to view and select the post-makeup facial image corresponding to the pre-makeup facial image.

(Processed Variation 2: Display of dynamic makeup images)

The above embodiment illustrates a case where pre-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, and a pre-makeup facial image is displayed by manipulating the pre-makeup facial images, but this is not limited to this example. For instance, a makeup animation image showing the publisher applying makeup to create a finished look from a pre-makeup state can also be displayed by manipulating the pre-makeup facial images. Furthermore, the makeup animation image may include, for example, pre-makeup facial images such as facial image IM21 and facial image IM22, as well as pre-makeup facial images such as facial image IM11 and facial image IM12. Additionally, the makeup animation image may be displayed together with the pre-makeup facial image in content CT11.

(Version 3: Limitations of the facial image before makeup)

In the above embodiment, the case shown is that the facial images before makeup, such as facial image IM21 and facial image IM22, are images of the publisher's bare face, but this is not limited to this example. For example, the facial images before makeup, such as facial image IM21 and facial image IM22, can be any facial images before makeup, such as facial image IM11 and facial image IM12, or they can be images of the face after some degree of makeup, such as lipstick and foundation. In this case, the information processing device 100 can generate makeup information such as makeup information for the user's facial image that does not use the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22, or makeup information for the user's facial image that uses the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22. Furthermore, since the facial images before makeup, such as facial image IM21 and facial image IM22, are used for bone learning after generating makeup information, appropriate makeup information can be generated even if the facial image is not a bare face. Therefore, the publisher's pre-makeup facial image only needs to be registered once in the service that manages the publisher's facial images. Furthermore, the description of the cosmetics used in the pre-makeup facial images such as facial image IM21 and facial image IM22 may or may not be included in content CT1.

(Version 4: Automatic extraction of item information)

In the above embodiments, for example, the cosmetic used by the publisher P11 in the makeup application related to the facial image IM11 can be determined in advance from the facial image IM11 and facial image IM21 through image analysis, and the cosmetic is stored in association with the facial image IM11 and facial image IM21 in an external information processing device or this device. Alternatively, the cosmetic can be determined based on the publisher P11's purchase history of cosmetics purchased from a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Alternatively, the publisher P11 can select from a list of candidates based on the cosmetic purchase history to determine the cosmetic, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Furthermore, the cosmetic can be determined by the publisher P11's registration of cosmetics not sold in a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21.

(Version 5: Reflecting the order of makeup application)

In the above embodiment, the information processing device 100 can also generate makeup information reflecting the makeup order by determining the makeup order performed by the publisher P11 in relation to the facial image IM11. In this case, the information processing device 100 can determine the makeup order by receiving the makeup order input from the publisher P11, and in the case of a makeup animation image related to the facial image IM11, it can also determine the makeup order based on the makeup animation image. Alternatively, the information processing device 100 can determine the makeup order by learning from multiple makeup animation images. Furthermore, in the case of a makeup animation image, the information processing device 100 can generate makeup information according to each category of cosmetics, thus allowing for the selection of applicable facial images. Additionally, explanatory content reflecting the makeup order can also be included in the content CT1. Therefore, the information processing device 100 can provide appropriate suggestions, including the makeup order.

(Processing Variation 6: Prioritized Display of Facial Images 1)

In the above embodiments, the facial images displayed in content CT1 can be selected based on any criterion or according to any priority. For example, facial images published by publishers followed by user U11 can be prioritized, or the more followers a publisher has, the higher the priority of displaying the publisher's facial images. Alternatively, the selection or display can be prioritized based on user U11's facial information or other body information.

Here, examples of facial information include the shape, size, and arrangement of the components that make up the face; the color and contour of the face; and the color and size of the pupils. Additionally, facial color includes, for example, the color of a bare face. Examples include, for instance, the average color of the entire measurement area excluding the lips; the average color of the forehead area; the average color of the area under the left eye; the average color of the area under the right eye; the average color of the left cheek area; the average color of the right cheek area; the average color of the nose area; the average color of the upper lip; the average color of the lower lip; and the average color of the chin area. Furthermore, examples of other body information besides facial information include, for instance, hair (hairstyle, hair texture, hair color, accessories, etc.); skin (skin tone, skin texture, wrinkles, etc.); and body shape (the size of the components, overall balance).

In the above embodiments, priority can be increased by prioritizing the publisher's facial image in order to be selected or displayed more preferentially, based on the similarity of the publisher's body information to that of user U11. For example, the more similar or identical the publisher's facial and other body information is to user U11, the higher the priority of the publisher's facial image. Alternatively, the more similar or identical the publisher's age, skin age, and skin concerns are to user U11, the higher the priority of the publisher's facial image. Similarly, the more similar or identical the publisher's concerns about makeup are to user U11, the higher the priority of the publisher's facial image. Furthermore, the more similar or identical the publisher's hairstyle, hair color, and hair preferences are to user U11, the higher the priority of the publisher's facial image. Conversely, priority can be decreased by prioritizing the publisher's facial image in order to be less preferentially selected or displayed, based on the disparity between the publisher's body information and that of user U11. For example, the more different the balance of facial features is from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the facial contours are from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the bare-faced facial color is from user U11, the lower the priority of the publisher's facial image.

Alternatively, priority can be given to selecting or displaying facial images based on indicators of their popularity, such as user browsing history, purchase history, and review history. Specifically, priority can be given based on user browsing history of facial images selected by multiple users, user purchase history generated through customer guidance provided by content offered via facial image selection, user review history when applying makeup using facial images, and user review history of facial images or purchased cosmetics after purchase. For example, higher user browsing history, purchase history, and review history, indicating the popularity of a facial image, can result in higher priority for that image. Alternatively, priority can also be given based on the total price of the complete set of cosmetics used by the user in the makeup application. For example, a higher total price of the complete set of cosmetics used by the user in the makeup application can either result in higher or lower priority for the facial image.

(Processing Variation 7: Prioritized Display of Facial Images 2)

In the above embodiments, the selection or display of cosmetics can also be prioritized based on the user U11's purchase history. For example, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the publisher's facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are included in the publisher's purchase history, the higher the priority of the publisher's facial image. For example, the priority of a publisher's facial image that includes both cosmetics PR2 and PR3 in their purchase history may be higher than that of a publisher whose purchase history only includes cosmetic PR1. Furthermore, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are used in makeup, the higher the priority of the facial image after using the cosmetics.

Figure 3 is an illustration of information processing where the higher the priority of a facial image, the more identical or similar it is to the cosmetics available to user U11. This priority is based on user U11's cosmetic purchase history. Facial images IM11 and IM12 are facial images published by publisher P11, while IM12 is a facial image published by publisher P12. Furthermore, the cosmetics used in the makeup associated with facial image IM11 are cosmetics PR1, PR2, and PR3; those used in the makeup associated with facial image IM11 are cosmetics PR1, PR4, and PR5; and those used in the makeup associated with facial image IM12 are cosmetics PR2, PR6, PR7, and PR8. In the user U11's purchase history, which includes cosmetics PR1, PR2, PR3, and PR6, the facial image IM11, which uses the common cosmetics PR1, PR2, and PR3, has the highest priority; the facial image IM111, which uses the common cosmetics PR2 and PR6, has a medium priority; and the facial image IM12, which uses the common cosmetics PR1, has the lowest priority.

(Processing Variation 8: Acquisition of Color Information)

In the above embodiment, skin color, such as facial color, can be measured by measuring equipment such as ZOZOGLASS (registered trademark), which can be used by the user U11 for measurement. By accurately measuring skin color after light correction, the accuracy of the makeup information generated by the information processing device 100 can be improved.

[3. Structure of the user terminal]

Next, the structure of the user terminal 10 according to the embodiment will be described using FIG4. FIG4 is a diagram showing an example of the structure of the user terminal 10 according to the embodiment. As shown in FIG4, the user terminal 10 includes a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

(Communications Department 11)

The communication unit 11 is implemented, for example, by a NIC (Network Interface Card). Furthermore, the communication unit 11 establishes a wired or wireless connection with a designated network N, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 12)

The input unit 12 accepts various operations from the user. In Figure 2, it accepts various operations from the user U11. For example, the input unit 12 can also accept various operations from the user via the display surface through the touch panel function. In addition, the input unit 12 can also accept various operations from buttons provided on the user terminal 10, keyboards connected to the user terminal 10, and mice.

(Output Section 13)

The output unit 13 is, for example, a display screen of a flat panel terminal implemented with a liquid crystal display, an organic EL (electro-Luminescence) display, etc., and is a display device used to display various information. For example, the output unit 13 displays information sent from the information processing device 100.

(Control Unit 14)

The control unit 14 is, for example, a controller, implemented by a CPU (Central Processing Unit), MPU (Micro Processing Unit), or similar device that uses RAM (Random Access Memory) as its operating area to execute various programs stored in the internal storage device of the user terminal 10. These programs include, for example, programs for applications installed on the user terminal 10, such as programs for displaying content containing facial images sent from the information processing device 100. Furthermore, the control unit 14 is implemented using integrated circuits such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

As shown in Figure 4, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and performs the information processing described below.

(Receiving Unit 141)

The receiving unit 141 receives various types of information from other information processing devices such as the information processing device 100. For example, the receiving unit 141 receives information for displaying a user's facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user, as well as instructions on the cosmetics used in the makeup.

(Sending Department 142)

The sending unit 142 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends information about a facial image selected by the user to confirm whether the publisher's makeup suits them.

[4. Structure of the publisher's terminal]

Next, the structure of the publisher terminal 20 according to the embodiment will be described using FIG5. FIG5 is a diagram showing an example of the structure of the publisher terminal 20 according to the embodiment. As shown in FIG5, the publisher terminal 20 includes a communication unit 21, an input unit 22, an output unit 23, and a control unit 24.

(Communications Department 21)

The communication unit 21 is implemented, for example, by a NIC. Furthermore, the communication unit 21 is connected to a designated network N via wired or wireless means, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 22)

The input unit 22 accepts various operations from the publisher. In Figure 2, it accepts various operations from publishers P11 and P12. For example, the input unit 22 can also accept various operations from the publisher via the display surface through the touch panel function. In addition, the input unit 22 can also accept various operations from buttons provided on the publisher terminal 20, keyboards connected to the publisher terminal 20, and mice.

(Output Section 23)

The output unit 23 is, for example, a display screen of a flat panel terminal implemented by a liquid crystal display, an organic EL display, or the like, and is a display device used to display various information. For example, the output unit 23 displays information sent from the information processing device 100.

(Control Department 24)

The control unit 24 is, for example, a controller, implemented by executing various programs stored in the internal storage device of the publisher terminal 20 using RAM as the operating area, such as a CPU or MPU. These various programs include, for example, programs for applications installed on the publisher terminal 20. For example, these programs include programs for applications that send facial images of the publisher based on the publisher's actions. Furthermore, the control unit 14 is implemented using integrated circuits such as ASICs or FPGAs.

As shown in Figure 5, the control unit 24 has a receiving unit 241 and a transmitting unit 242, and performs the information processing described below.

(Receiving Unit 241)

The receiving unit 241 receives various types of information from the information processing device 100 and other information processing devices. For example, if a user who has viewed a facial image provided by a publisher and applied makeup to their own facial image purchases cosmetics corresponding to the applied makeup from a designated online store, the receiving unit 241 receives information related to the remuneration that the publisher can collect based on the user's purchase. Furthermore, the receiving unit 241 may also receive information for the publisher to collect remuneration corresponding to the user's purchase.

(Department 242)

The sending unit 242 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends a facial image of the publisher based on the publisher's operation.

(5. Configuration of information processing devices)

Next, the structure of the information processing apparatus 100 according to the embodiment will be described using FIG6. FIG6 is a diagram showing an example of the structure of the information processing apparatus 100 according to the embodiment. As shown in FIG6, the information processing apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. In addition, the information processing apparatus 100 may also include an input unit (e.g., a keyboard, mouse, etc.) for receiving various operations from the administrator of the information processing apparatus 100, and a display unit (e.g., a liquid crystal display, etc.) for displaying various information.

(Communication Department 110)

The communication unit 110 is implemented, for example, by a NIC. Furthermore, the communication unit 110 is connected to the network N via a wired or wireless connection, and transmits and receives information with the user terminal 10 via the network N.

(Storage Department 120)

The storage unit 120 is implemented, for example, by semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks and optical disks. As shown in FIG6, the storage unit 120 includes a publisher information storage unit 121 and a cosmetics information storage unit 122.

The publisher information storage unit 121 stores the publisher's facial image. Here, an example of the publisher information storage unit 121 according to the embodiment is shown in FIG7. As shown in FIG7, the publisher information storage unit 121 has items such as "publisher ID" and "facial image".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a bare-faced facial image of the publisher registered upon initial use. The example shown in Figure 7 illustrates a case where conceptual information such as "Facial Image #11" and "Facial Image #12" is stored for "Facial Image," but in reality, image data is stored. Additionally, for "Facial Image," the URL containing the image data and the file path indicating the storage location may also be stored.

That is, Figure 7 shows an example of a bare-faced facial image of the publisher identified according to the publisher "P11" as "Face Image #11".

The cosmetics information storage unit 122 stores information about the cosmetics used by the publisher in applying makeup. As shown in Figure 8, the cosmetics information storage unit 122 has items such as "Publisher ID", "Facial Image", and "Complete Set of Cosmetics".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a facial image published by the publisher. The example shown in Figure 8 illustrates a conceptual information storage term such as "Facial Image #21" and "Facial Image #22" for "Facial Image," but in reality, it stores image data. Additionally, "Facial Image" could also store, for example, the URL containing the image data, the file path indicating the storage location, etc. "Full Makeup Set" represents the complete set of cosmetics used by the publisher in their makeup routine.

That is, in Figure 8, the following example is shown: the facial image posted by the publisher identified according to the publisher "P11" is "Facial Image #21", and the full set of cosmetics used by the publisher in applying makeup is "Brand 〇〇 Product A1 (lipstick), Brand ×× Product A2 (foundation), ...". In addition, the categories of cosmetics are indicated in parentheses.

(Control Department 130)

The control unit 130 is a controller, which is implemented, for example, by executing various programs stored in the storage device inside the information processing device 100 using RAM as the operating area by a CPU, MPU, or the like. Alternatively, the control unit 130 is a controller, which is implemented, for example, by an integrated circuit such as an ASIC or FPGA.

As shown in Figure 6, the control unit 130 includes an acquisition unit 131, a generation unit 132, an application unit 133, a provision unit 134, and a decision unit 135, and performs the information processing described below. Furthermore, the internal structure of the control unit 130 is not limited to the structure shown in Figure 6; it can be any structure that performs the information processing described later, or any other structure.

(Acquisition Department 131)

The acquisition unit 131 acquires various information from external information processing devices. The acquisition unit 131 also acquires various information from other information processing devices such as the user terminal 10.

The acquisition unit 131 acquires various information from the storage unit 120. Furthermore, the acquisition unit 131 stores the acquired information in the storage unit 120.

The acquisition unit 131 acquires information about facial images selected by users from facial images published by publishers in the prescribed service.

The acquisition unit 131 acquires information about the cosmetics used by the publisher in relation to the facial image selected by the user.

(Generation Department 132)

Based on the information acquired by the acquisition unit 131, the generation unit 132 generates makeup information using methods such as PSGAN. This makeup information is used to generate a facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user. Alternatively, if the makeup order in relation to the facial image selected by the user and the order in which the publisher applied the makeup is determined, the generation unit 132 generates makeup information reflecting the makeup order.

The generation unit 132 generates information for displaying a facial image containing the makeup information applied by the application unit 133 (described later) and a description of the cosmetics determined based on the information obtained by the acquisition unit 131.

(Application Department 133)

The application unit 133 applies makeup information generated by the generation unit 132 to the facial image of the user selected as the application object, thereby generating a facial image that is displayed on the user's terminal 10.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed according to priority. The priority is determined based on the user's facial information, body information, cosmetic purchase history, etc. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed with priority. The priority is determined based on indicators representing the popularity of facial images, such as access history, purchase history, and evaluation history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected from those selected or displayed with priority. The priority is determined based on the total price of the cosmetics used by the publisher in the makeup. The higher the priority of the facial image, the more preferentially it is selected or displayed.

(Provide Department 134)

The providing unit 134 sends information generated by the generating unit 132 for displaying content to the user terminal 10. When the user terminal 10 receives the information sent by the providing unit 134, it displays the content corresponding to the information generated by the generating unit 132.

(Decision Section 135)

The decision unit 135 determines whether the user has purchased cosmetics associated with the content provided by the provision unit 134. If it determines that the user has purchased cosmetics, it decides to award the corresponding reward to the publisher who published the facial image selected by the user. Then, the provision unit 134 performs the processing for awarding the corresponding reward to the publisher. Alternatively, the decision unit 135 may also decide to send information for the publisher to receive the corresponding reward. In this case, the provision unit 134 sends the information for the publisher to receive the corresponding reward. Alternatively, the decision unit 135 may also decide to transfer the reward to the publisher's account. In this case, the provision unit 134 performs the processing for transferring the reward to the publisher's account.

(6. Information processing flow)

Next, FIG9 will be used to explain the information processing process performed by the information processing system 1 according to the embodiment. FIG9 is a flowchart showing the information processing process performed by the information processing system 1 according to the embodiment.

As shown in Figure 9, the information processing device 100 acquires information about the facial image selected by the user from facial images published by the publisher in the specified service (step S201).

The information processing device 100 generates makeup information based on the acquired information, which is used to generate makeup that is the same as the publisher's makeup used in the facial image selected by the user (step S202).

The information processing device 100 generates a facial image to be provided to the user by applying the generated makeup information to the facial image of the user selected as the application object (step S203).

The information processing device 100 sends information for displaying content including the generated facial image and descriptions of the cosmetics used by the publisher in the makeup (step S204).

[7. Variations]

The information processing system 1 described above can be implemented in various other ways besides the embodiments described above. Therefore, other embodiments of the information processing system 1 will be described below.

In the above embodiment, a case is shown where the user's facial image after applying makeup information is displayed on the user's terminal 10, but this is not limited to this example. For example, it could also be displayed on the terminal devices of other people communicating online with the user. For example, when a user selects a publisher's facial image to participate in an online meeting, makeup related to the selected facial image could be virtually applied to the user's face, and the facial image after applying the makeup related to the facial image selected by the user could be displayed on the terminal devices of other people. Furthermore, when a user uses the publisher's makeup to participate in a meeting, the user could be charged based on usage time, frequency, and other usage conditions. Moreover, a process could be implemented to give a portion of the fee charged to the user as compensation to the publisher. Thus, since the user's made-up face is displayed to others online, the user can participate in meetings with a natural face.

In this case, the application unit 133 applies makeup information generated by the generation unit 132 to the user's facial image captured by the camera in real time, thereby generating a facial image to be displayed on other people's terminal devices in real time. The generation unit 132 uses the facial image generated by the application unit 133 to generate information in real time for displaying the facial image after applying makeup information to the user's facial image on other people's terminal devices. The providing unit 134 transmits the facial image after applying makeup information via the application unit 133 to other people communicating online in real time with the user by sending the information generated by the generation unit 132. The decision unit 135 determines the reward given to the publisher of the facial image selected by the user based on the online usage of the facial image generated by the application unit 133 (e.g., usage time, usage frequency). In other words, the decision unit 135 determines the reward given to the publisher of the facial image that serves as the source of the makeup information based on the usage of the facial image after applying makeup information to the user's facial image. Furthermore, applications can also be performed on the user's cloud, in which case the providing unit 134 may, for example, only provide the right to use the makeup information. Moreover, the user can also exercise the provided right to use the information to display the facial image of the user after applying the makeup information to their facial image on other people's terminal devices.

[8. Effects]

As described above, the information processing apparatus 100 according to the embodiment includes a generation unit 132 and a determination unit 135. The generation unit 132 generates makeup information based on a facial image of a publisher who has applied prescribed makeup. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image. The determination unit 135 determines the reward to be given to the publisher who published the facial image that serves as the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

Therefore, the information processing device 100 according to the embodiment can appropriately meet the user's need to participate in online meetings, etc., by using a facial image of themselves with other people's makeup applied.

In addition, the decision unit 135 determines the compensation based on the usage time or frequency of the user's facial image after applying makeup information to it.

Therefore, the information processing apparatus 100 according to the embodiment can appropriately encourage the publisher to publish facial images.

In addition, the application unit 133 applies makeup information generated based on a facial image selected from facial images that are preferentially selected or displayed based on prescribed criteria.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on a predetermined benchmark, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority. The priority is determined based on at least one of the user's facial information, body information, and item purchase history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing device 100 according to the embodiment can provide appropriate information to the user based on facial information, body information, purchase history of items, etc., and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, wherein the priority is determined based on at least one of the facial image's access history, purchase history, and review history, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on access history, purchase history, evaluation history, etc. of facial images, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, the priority being determined according to the total price of the item, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on the total price of the item, and thus can more appropriately meet the user's needs.

[9. Hardware Structure]

Furthermore, the user terminal 10, publisher terminal 20, and information processing device 100 involved in the above-described embodiments can be implemented, for example, by a computer 1000 with the structure shown in FIG10. FIG10 is a hardware structure diagram showing an example of a computer that implements the functions of the user terminal 10, publisher terminal 20, and information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

The CPU 1100 operates based on the programs stored in the ROM 1300 or HDD 1400 to control various components. The ROM 1300 stores the startup program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 stores the program executed by CPU 1100 and the data used by that program. Communication interface 1500 obtains data from other devices and sends it to CPU 1100 via a specified communication network, and sends data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as monitors and printers, as well as input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. In addition, the CPU 1100 outputs the generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads the program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 into the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as DVD (Digital Versatile Disc), PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical Disk), magnetic tape, magnetic recording media, or semiconductor memory.

For example, when the computer 1000 functions as the terminal device 10, publisher terminal 20, and information processing device 100 in the implementation, the CPU 1100 of the computer 1000 implements the functions of the control units 14, 24, and 130 by executing programs loaded into the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800; as another example, these programs can also be obtained from other devices via a defined communication network.

[10. Others]

Furthermore, all or part of the processes described in the above embodiments that are intended to be performed automatically can also be performed manually, or all or part of the processes described as intended to be performed manually can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, various data, and parameters shown in the foregoing and accompanying drawings can be arbitrarily changed unless otherwise stated. For example, the various information shown in the figures is not limited to the information illustrated.

Furthermore, the constituent elements of each device illustrated are functional concepts and do not need to be physically configured as shown in the illustrations. That is, the specific way in which the devices are distributed/integrated is not limited to the way illustrated, and they can be configured to be distributed/integrated functionally or physically in any unit according to various loads, usage conditions, etc.

Furthermore, the above-described implementation methods can be appropriately combined within the scope of not contradicting the processing content.

Several embodiments of this application have been described in detail above with reference to the accompanying drawings. However, these are illustrative, and the invention can be implemented in other ways by means of various modifications and alterations based on the knowledge of those skilled in the art, based on the manner described in the Summary of the Invention section.

Furthermore, the terms "section, module, unit" mentioned above can be replaced with "unit," "circuit," etc. For example, "acquisition section" can be replaced with "acquisition unit" or "acquisition circuit."

Explanation of reference numerals in the attached figures

1: Information processing system; 10: User terminal; 11: Communication unit; 12: Input unit; 13: Output unit; 14: Control unit; 20: Publisher terminal; 21: Communication unit; 22: Input unit; 23: Output unit; 24: Control unit; 100: Information processing device; 110: Communication unit; 120: Storage unit; 121: Publisher information storage unit; 122: Cosmetic information storage unit; 130: Control unit; 131: Acquisition unit; 132: Generation unit; 133: Application unit; 134: Provision unit; 135: Decision unit; 141: Receiving unit; 142: Transmitting unit; 241: Receiving unit; 242: Transmitting unit; N: Network.

Technical Field

This invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Technology

When researching cosmetics purchases, the following situation often occurs: even when watching makeup applied to the faces of models or other people, one cannot imagine the image of oneself in such a situation, and therefore does not know whether it is suitable for oneself.

Conventionally, as a technique for fulfilling needs such as confirming whether makeup applied to another person's face is suitable for oneself, techniques that use other people's makeup as a reference to assist in makeup application are known. For example, techniques that present other people's makeup information corresponding to areas of the user's facial features are known.

Existing technical documents

Patent documents

Patent Document 1: Japanese Patent No. 5991536

Non-Patent Literature 1: W. Jiang, S. Liu, C. Gao, J. Cao, R. He, J. Feng, S. Yan, “PSGAN: Pose and Expression Robust Spatial-Aware CAN for Customizable Makeup Transfer,” [online], [searched January 24, 2022], Internet <https://openaccess.thecvf.com/content_CVPR_2020/html/Jiang_PSGAN_Pose_and_Expression_Robust_Spatial-Aware_GAN_for_Customizable_Makeup_CVPR_2020_paper.html>

Summary of the Invention

The problem the invention aims to solve

However, current technology lacks features such as the ability to guide customers to sales websites after they've confirmed whether other people's makeup looks suit them, and the inability to use performance-based rewards to incentivize makeup creators to actively guide customers. Therefore, there is room for improvement in usability for users who use others' makeup looks as a reference.

This application was made in view of the above circumstances, and its purpose is to further improve usability for users who use other people's makeup as a reference.

Solution for solving the problem

The information processing apparatus of this application is characterized by comprising: a generation unit that generates makeup information based on a facial image of a publisher who has applied prescribed makeup, the makeup information being used to generate a facial image after applying the same makeup as the prescribed makeup to the facial image; and a determination unit that determines the reward to be given to the publisher who published the facial image that is the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

The effects of the invention

According to one aspect of the implementation, this achieves the effect of further improving usability for users who can use other people's makeup as a reference.

Attached Figure Description

Figure 1 is a diagram illustrating a structural example of an information processing system according to an embodiment.

Figure 2 is a diagram illustrating an example of information processing involved in the implementation method.

Figure 3 is an explanatory diagram illustrating the information processing in the case of increasing the priority of facial images according to the embodiment.

Figure 4 is a diagram showing an example of the structure of a user terminal according to the embodiment.

Figure 5 is a diagram illustrating an example of the structure of the publisher terminal involved in the implementation method.

Figure 6 is a diagram showing a structural example of the information processing apparatus according to the embodiment.

Figure 7 is a diagram illustrating an example of the publisher information storage unit involved in the embodiment.

Figure 8 is a diagram illustrating an example of the cosmetic information storage unit involved in the embodiment.

Figure 9 is a flowchart illustrating an example of information processing involved in the implementation method.

Figure 10 is a hardware structure diagram illustrating an example of a computer that implements the functions of an information processing device.

Detailed Implementation

Hereinafter, with reference to the accompanying drawings, we will describe in detail the methods (hereinafter referred to as "implementations") for carrying out the information processing apparatus, information processing method, and information processing program involved in this application. However, the information processing apparatus, information processing method, and information processing program involved in this application are not limited to these embodiments. Furthermore, in the following embodiments, the same reference numerals will be used to label the same parts, and repeated descriptions will be omitted.

(Implementation Method)

(1. The structure of an information processing system)

The information processing system 1 shown in FIG1 will be described. As shown in FIG1, the information processing system 1 includes a user terminal 10, a publisher terminal 20, and an information processing device 100. The user terminal 10, the publisher terminal 20, and the information processing device 100 are connected via a predetermined communication network (network N) in a wired or wireless communication manner. FIG1 is a diagram showing an example of the structure of the information processing system 1 according to the embodiment.

User terminal 10 is an information processing device used by a user who wants to confirm whether makeup applied to another person's face is suitable for them. Additionally, the user using user terminal 10 may be researching and purchasing cosmetics. User terminal 10 can be any device as long as it can perform the processing described in the embodiment. Furthermore, user terminal 10 can also be a smartphone, tablet, laptop PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), etc. Figure 2 shows the case where user terminal 10 is a smartphone.

User terminal 10 is a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G (Generation) and LTE (Long Term Evolution). Additionally, user terminal 10 may have a screen such as an LCD display with touch panel functionality, allowing the user to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or a stylus. In Figure 2, user terminal 10 is used by user U11.

The publisher terminal 20 is an information processing device used by a publisher who provides a facial image after makeup. Alternatively, a publisher using the publisher terminal 20 may publish, for example, a facial image after makeup is created by applying makeup from a bare face before makeup application. The publisher terminal 20 can be any device as long as it can perform the processing described in the embodiment. Furthermore, the publisher terminal 20 can also be a smartphone, tablet, laptop PC, desktop PC, mobile phone, PDA, or other similar device. Figure 2 shows the case where the publisher terminal 20 is a smartphone.

The publisher terminal 20 is, for example, a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G and LTE. Additionally, the publisher terminal 20 may also have a screen such as an LCD display with touch panel functionality, allowing the publisher to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or styluses. In Figure 2, the publisher terminal 20 is used by publishers P11 and P12. Publisher terminal 20 ₁ and Publisher terminal 20 ₂ are the publisher terminals 20 for publishers P11 and P12, respectively.

Information processing device 100 is an information processing device aimed at further improving usability for users who use other people's makeup as a reference. Any device is acceptable as long as it can perform the processing described in the embodiment. Information processing device 100 is implemented, for example, by a server device, cloud system, etc. For example, information processing device 100 is implemented by a server device, cloud system, etc., that provides a prescribed electronic mall for selling cosmetics.

(2. An example of information processing)

Figure 2 is a diagram illustrating an example of information processing in the information processing system 1 according to the embodiment. Hereinafter, in the embodiment, the processing for a user U11 who wants to confirm whether the publisher P11's makeup suits them will be described as an example.

Information processing device 100 acquires information about the facial image selected by user U11 from facial images published by a publisher (step S101). This publisher is, for example, a celebrity or influencer whose number of followers on an SNS (Social Networking Service) exceeds a predetermined threshold. Furthermore, the facial images that user U11 can select may include multiple facial images published by a single publisher. In Figure 2, this is assumed to include facial image IM11 published by publisher P11, facial image IM12 published by publisher P12, etc. The following explanation assumes that user U11 selected facial image IM11 from content CT11. Furthermore, the facial images displayed in content CT11 can be selected based on any criterion. For example, facial images published by publishers followed by user U11 can be preferentially selected. Additionally, content CT11 is content within a specified service that manages the facial images of publishers.

Facial image IM11 is a facial image of publisher P11 after applying prescribed makeup. Furthermore, facial image IM21 is a facial image of publisher P11 before makeup, corresponding to the makeup applied to facial image IM11. Facial image IM21 can be displayed together with facial image IM11 on content CT11, or it can be displayed through operations on facial image IM11 (e.g., clicking or tapping). Alternatively, facial image IM21 may not be displayed to the user. Therefore, the publisher can avoid providing a bare-faced image to the user, thus improving usability for the publisher. Additionally, facial image IM12 is a facial image of publisher P12 after applying prescribed makeup. Furthermore, facial image IM22 is a facial image of publisher P12 before makeup, corresponding to the makeup applied to facial image IM12. Facial image IM22 can be displayed together with facial image IM12 on content CT11, or it can be displayed through operations on facial image IM12. Alternatively, facial image IM22 may not be displayed to the user.

Here, as an example of a method for generating makeup information, we cite, for instance, the method disclosed in Non-Patent Document 1, known as PSGAN (Pose and Expression Robust Spatial-Aware Generative Adversarial Network), which simply moves makeup information from one face image to another (Non-Patent Document 1). This method uses deep learning and is able to move makeup information to another face regardless of expression or pose. The information processing device 100 uses a method such as PSGAN to generate makeup information, which is used to generate a face image after applying the same makeup as that used by publisher P11 in the face image IM11 selected by user U11 (step S102). Then, the information processing device 100 uses a method such as PSGAN to apply the makeup information generated in step S102 to the face image IM1 of user U11 (step S103). At this time, the face image IM1 can also be any arbitrarily acquired face image. Facial image IM1 can be, for example, a facial image pre-registered by user U11 and stored in an external information processing device or within this device, or a facial image sent from user terminal 10. Furthermore, facial image IM1 can be a facial image selected as the application target by user U11, or a facial image automatically selected as the application target. Additionally, facial image IM2 is the applied facial image corresponding to facial image IM1.

The information processing device 100 sends information to the user terminal 10 for displaying a facial image IM2 of the user U11 after the makeup information was applied in step S103, and a description of the cosmetics used in the makeup (corresponding to the item) (step S104). Then, the user terminal 10 displays content CT1 based on the information sent from the information processing device 100.

Here, content CT1 includes descriptions of the cosmetics used by publisher P11 in the makeup application related to facial image IM11. For example, it includes information such as the product name and price of the cosmetics. Figure 2 shows the case where two cosmetics (PR1 and PR2) are included, namely lipstick and foundation, but there is no particular limitation on the number of cosmetics. For example, content CT1 may also include descriptions of the complete set of cosmetics used by publisher P11 to create the prescribed post-makeup state of facial image IM11 from the pre-makeup state of facial image IM21. Additionally, the descriptions may include operation acceptance information B1 for accessing a prescribed online store where users can purchase the complete set of cosmetics used in the makeup application. When operation acceptance information B1 is accessed, users can access either the product page introducing the complete set of cosmetics or the purchase page after adding the complete set to their cart. Furthermore, operation acceptance information B1 can be integrated with facial image IM2 and other content in the descriptions, allowing access to the product page and purchase page when operations are performed on facial image IM2 and other content in the descriptions.

Information processing device 100 determines whether user U11 has purchased cosmetics associated with the description content. For example, information processing device 100 determines whether user U11 purchased a complete set of cosmetics, including cosmetics PR1 and cosmetics PR2, or a portion of cosmetics from a designated online store accessed through operation acceptance information B1. If information processing device 100 determines that user U11 has made a purchase, it decides to award the corresponding reward to publisher P11, who is the publisher who published the facial image after makeup using the makeup information corresponding to image IM2. Then, information processing device 100 performs processing to award the corresponding reward to publisher P11. Alternatively, information processing device 100 may decide to send information for publisher P11 to receive the corresponding reward for user U11's purchase. In this case, information processing device 100 sends information for publisher P11 to receive the corresponding reward for user U11's purchase. Alternatively, the information processing device 100 may decide to transfer the payment to the publisher P11's account. In this case, the information processing device 100 performs the processing for transferring the payment to the publisher P11's account.

(Version 1: Display using a pre-makeup facial image)

In the above embodiment, a case is shown where pre-defined post-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, but this is not limited to this example. For example, facial images such as facial image IM21 and facial image IM22 before makeup can also be displayed in content CT11, and the post-makeup facial image can be displayed by manipulating the pre-makeup facial image. Furthermore, the information processing device 100 can also generate makeup information applied to the user's facial image by allowing the user U11 to view and select the post-makeup facial image corresponding to the pre-makeup facial image.

(Processed Variation 2: Display of dynamic makeup images)

The above embodiment illustrates a case where pre-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, and a pre-makeup facial image is displayed by manipulating the pre-makeup facial images, but this is not limited to this example. For instance, a makeup animation image showing the publisher applying makeup to create a finished look from a pre-makeup state can also be displayed by manipulating the pre-makeup facial images. Furthermore, the makeup animation image may include, for example, pre-makeup facial images such as facial image IM21 and facial image IM22, as well as pre-makeup facial images such as facial image IM11 and facial image IM12. Additionally, the makeup animation image may be displayed together with the pre-makeup facial image in content CT11.

(Version 3: Limitations of the facial image before makeup)

In the above embodiment, the case shown is that the facial images before makeup, such as facial image IM21 and facial image IM22, are images of the publisher's bare face, but this is not limited to this example. For example, the facial images before makeup, such as facial image IM21 and facial image IM22, can be any facial images before makeup, such as facial image IM11 and facial image IM12, or they can be images of the face after some degree of makeup, such as lipstick and foundation. In this case, the information processing device 100 can generate makeup information such as makeup information for the user's facial image that does not use the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22, or makeup information for the user's facial image that uses the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22. Furthermore, since the facial images before makeup, such as facial image IM21 and facial image IM22, are used for bone learning after generating makeup information, appropriate makeup information can be generated even if the facial image is not a bare face. Therefore, the publisher's pre-makeup facial image only needs to be registered once in the service that manages the publisher's facial images. Furthermore, the description of the cosmetics used in the pre-makeup facial images such as facial image IM21 and facial image IM22 may or may not be included in content CT1.

(Version 4: Automatic extraction of item information)

In the above embodiments, for example, the cosmetic used by the publisher P11 in the makeup application related to the facial image IM11 can be determined in advance from the facial image IM11 and facial image IM21 through image analysis, and the cosmetic is stored in association with the facial image IM11 and facial image IM21 in an external information processing device or this device. Alternatively, the cosmetic can be determined based on the publisher P11's purchase history of cosmetics purchased from a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Alternatively, the publisher P11 can select from a list of candidates based on the cosmetic purchase history to determine the cosmetic, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Furthermore, the cosmetic can be determined by the publisher P11's registration of cosmetics not sold in a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21.

(Version 5: Reflecting the order of makeup application)

In the above embodiment, the information processing device 100 can also generate makeup information reflecting the makeup order by determining the makeup order performed by the publisher P11 in relation to the facial image IM11. In this case, the information processing device 100 can determine the makeup order by receiving the makeup order input from the publisher P11, and in the case of a makeup animation image related to the facial image IM11, it can also determine the makeup order based on the makeup animation image. Alternatively, the information processing device 100 can determine the makeup order by learning from multiple makeup animation images. Furthermore, in the case of a makeup animation image, the information processing device 100 can generate makeup information according to each category of cosmetics, thus allowing for the selection of applicable facial images. Additionally, explanatory content reflecting the makeup order can also be included in the content CT1. Therefore, the information processing device 100 can provide appropriate suggestions, including the makeup order.

(Processing Variation 6: Prioritized Display of Facial Images 1)

In the above embodiments, the facial images displayed in content CT1 can be selected based on any criterion or according to any priority. For example, facial images published by publishers followed by user U11 can be prioritized, or the more followers a publisher has, the higher the priority of displaying the publisher's facial images. Alternatively, the selection or display can be prioritized based on user U11's facial information or other body information.

Here, examples of facial information include the shape, size, and arrangement of the components that make up the face; the color and contour of the face; and the color and size of the pupils. Additionally, facial color includes, for example, the color of a bare face. Examples include, for instance, the average color of the entire measurement area excluding the lips; the average color of the forehead area; the average color of the area under the left eye; the average color of the area under the right eye; the average color of the left cheek area; the average color of the right cheek area; the average color of the nose area; the average color of the upper lip; the average color of the lower lip; and the average color of the chin area. Furthermore, examples of other body information besides facial information include, for instance, hair (hairstyle, hair texture, hair color, accessories, etc.); skin (skin tone, skin texture, wrinkles, etc.); and body shape (the size of the components, overall balance).

In the above embodiments, priority can be increased by prioritizing the publisher's facial image in order to be selected or displayed more preferentially, based on the similarity of the publisher's body information to that of user U11. For example, the more similar or identical the publisher's facial and other body information is to user U11, the higher the priority of the publisher's facial image. Alternatively, the more similar or identical the publisher's age, skin age, and skin concerns are to user U11, the higher the priority of the publisher's facial image. Similarly, the more similar or identical the publisher's concerns about makeup are to user U11, the higher the priority of the publisher's facial image. Furthermore, the more similar or identical the publisher's hairstyle, hair color, and hair preferences are to user U11, the higher the priority of the publisher's facial image. Conversely, priority can be decreased by prioritizing the publisher's facial image in order to be less preferentially selected or displayed, based on the disparity between the publisher's body information and that of user U11. For example, the more different the balance of facial features is from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the facial contours are from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the bare-faced facial color is from user U11, the lower the priority of the publisher's facial image.

Alternatively, priority can be given to selecting or displaying facial images based on indicators of their popularity, such as user browsing history, purchase history, and review history. Specifically, priority can be given based on user browsing history of facial images selected by multiple users, user purchase history generated through customer guidance provided by content offered via facial image selection, user review history when applying makeup using facial images, and user review history of facial images or purchased cosmetics after purchase. For example, higher user browsing history, purchase history, and review history, indicating the popularity of a facial image, can result in higher priority for that image. Alternatively, priority can also be given based on the total price of the complete set of cosmetics used by the user in the makeup application. For example, a higher total price of the complete set of cosmetics used by the user in the makeup application can either result in higher or lower priority for the facial image.

(Processing Variation 7: Prioritized Display of Facial Images 2)

In the above embodiments, the selection or display of cosmetics can also be prioritized based on the user U11's purchase history. For example, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the publisher's facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are included in the publisher's purchase history, the higher the priority of the publisher's facial image. For example, the priority of a publisher's facial image that includes both cosmetics PR2 and PR3 in their purchase history may be higher than that of a publisher whose purchase history only includes cosmetic PR1. Furthermore, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are used in makeup, the higher the priority of the facial image after using the cosmetics.

Figure 3 is an illustration of information processing where the higher the priority of a facial image, the more identical or similar it is to the cosmetics available to user U11. This priority is based on user U11's cosmetic purchase history. Facial images IM11 and IM12 are facial images published by publisher P11, while IM12 is a facial image published by publisher P12. Furthermore, the cosmetics used in the makeup associated with facial image IM11 are cosmetics PR1, PR2, and PR3; those used in the makeup associated with facial image IM11 are cosmetics PR1, PR4, and PR5; and those used in the makeup associated with facial image IM12 are cosmetics PR2, PR6, PR7, and PR8. In the user U11's purchase history, which includes cosmetics PR1, PR2, PR3, and PR6, the facial image IM11, which uses the common cosmetics PR1, PR2, and PR3, has the highest priority; the facial image IM111, which uses the common cosmetics PR2 and PR6, has a medium priority; and the facial image IM12, which uses the common cosmetics PR1, has the lowest priority.

(Processing Variation 8: Acquisition of Color Information)

In the above embodiment, skin color, such as facial color, can be measured by measuring equipment such as ZOZOGLASS (registered trademark), which can be used by the user U11 for measurement. By accurately measuring skin color after light correction, the accuracy of the makeup information generated by the information processing device 100 can be improved.

[3. Structure of the user terminal]

Next, the structure of the user terminal 10 according to the embodiment will be described using FIG4. FIG4 is a diagram showing an example of the structure of the user terminal 10 according to the embodiment. As shown in FIG4, the user terminal 10 includes a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

(Communications Department 11)

The communication unit 11 is implemented, for example, by a NIC (Network Interface Card). Furthermore, the communication unit 11 establishes a wired or wireless connection with a designated network N, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 12)

The input unit 12 accepts various operations from the user. In Figure 2, it accepts various operations from the user U11. For example, the input unit 12 can also accept various operations from the user via the display surface through the touch panel function. In addition, the input unit 12 can also accept various operations from buttons provided on the user terminal 10, keyboards connected to the user terminal 10, and mice.

(Output Section 13)

The output unit 13 is, for example, a display screen of a flat panel terminal implemented with a liquid crystal display, an organic EL (electro-Luminescence) display, etc., and is a display device used to display various information. For example, the output unit 13 displays information sent from the information processing device 100.

(Control Unit 14)

The control unit 14 is, for example, a controller, implemented by a CPU (Central Processing Unit), MPU (Micro Processing Unit), or similar device that uses RAM (Random Access Memory) as its operating area to execute various programs stored in the internal storage device of the user terminal 10. These programs include, for example, programs for applications installed on the user terminal 10, such as programs for displaying content containing facial images sent from the information processing device 100. Furthermore, the control unit 14 is implemented using integrated circuits such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

As shown in Figure 4, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and performs the information processing described below.

(Receiving Unit 141)

The receiving unit 141 receives various types of information from other information processing devices such as the information processing device 100. For example, the receiving unit 141 receives information for displaying a user's facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user, as well as instructions on the cosmetics used in the makeup.

(Sending Department 142)

The sending unit 142 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends information about a facial image selected by the user to confirm whether the publisher's makeup suits them.

[4. Structure of the publisher's terminal]

Next, the structure of the publisher terminal 20 according to the embodiment will be described using FIG5. FIG5 is a diagram showing an example of the structure of the publisher terminal 20 according to the embodiment. As shown in FIG5, the publisher terminal 20 includes a communication unit 21, an input unit 22, an output unit 23, and a control unit 24.

(Communications Department 21)

The communication unit 21 is implemented, for example, by a NIC. Furthermore, the communication unit 21 is connected to a designated network N via wired or wireless means, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 22)

The input unit 22 accepts various operations from the publisher. In Figure 2, it accepts various operations from publishers P11 and P12. For example, the input unit 22 can also accept various operations from the publisher via the display surface through the touch panel function. In addition, the input unit 22 can also accept various operations from buttons provided on the publisher terminal 20, keyboards connected to the publisher terminal 20, and mice.

(Output Section 23)

The output unit 23 is, for example, a display screen of a flat panel terminal implemented by a liquid crystal display, an organic EL display, or the like, and is a display device used to display various information. For example, the output unit 23 displays information sent from the information processing device 100.

(Control Department 24)

The control unit 24 is, for example, a controller, implemented by executing various programs stored in the internal storage device of the publisher terminal 20 using RAM as the operating area, such as a CPU or MPU. These various programs include, for example, programs for applications installed on the publisher terminal 20. For example, these programs include programs for applications that send facial images of the publisher based on the publisher's actions. Furthermore, the control unit 14 is implemented using integrated circuits such as ASICs or FPGAs.

As shown in Figure 5, the control unit 24 has a receiving unit 241 and a transmitting unit 242, and performs the information processing described below.

(Receiving Unit 241)

The receiving unit 241 receives various types of information from the information processing device 100 and other information processing devices. For example, if a user who has viewed a facial image provided by a publisher and applied makeup to their own facial image purchases cosmetics corresponding to the applied makeup from a designated online store, the receiving unit 241 receives information related to the remuneration that the publisher can collect based on the user's purchase. Furthermore, the receiving unit 241 may also receive information for the publisher to collect remuneration corresponding to the user's purchase.

(Department 242)

The sending unit 242 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends a facial image of the publisher based on the publisher's operation.

(5. Configuration of information processing devices)

Next, the structure of the information processing apparatus 100 according to the embodiment will be described using FIG6. FIG6 is a diagram showing an example of the structure of the information processing apparatus 100 according to the embodiment. As shown in FIG6, the information processing apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. In addition, the information processing apparatus 100 may also include an input unit (e.g., a keyboard, mouse, etc.) for receiving various operations from the administrator of the information processing apparatus 100, and a display unit (e.g., a liquid crystal display, etc.) for displaying various information.

(Communication Department 110)

The communication unit 110 is implemented, for example, by a NIC. Furthermore, the communication unit 110 is connected to the network N via a wired or wireless connection, and transmits and receives information with the user terminal 10 via the network N.

(Storage Department 120)

The storage unit 120 is implemented, for example, by semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks and optical disks. As shown in FIG6, the storage unit 120 includes a publisher information storage unit 121 and a cosmetics information storage unit 122.

The publisher information storage unit 121 stores the publisher's facial image. Here, an example of the publisher information storage unit 121 according to the embodiment is shown in FIG7. As shown in FIG7, the publisher information storage unit 121 has items such as "publisher ID" and "facial image".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a bare-faced facial image of the publisher registered upon initial use. The example shown in Figure 7 illustrates a case where conceptual information such as "Facial Image #11" and "Facial Image #12" is stored for "Facial Image," but in reality, image data is stored. Additionally, for "Facial Image," the URL containing the image data and the file path indicating the storage location may also be stored.

That is, Figure 7 shows an example of a bare-faced facial image of the publisher identified according to the publisher "P11" as "Face Image #11".

The cosmetics information storage unit 122 stores information about the cosmetics used by the publisher in applying makeup. As shown in Figure 8, the cosmetics information storage unit 122 has items such as "Publisher ID", "Facial Image", and "Complete Set of Cosmetics".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a facial image published by the publisher. The example shown in Figure 8 illustrates a conceptual information storage term such as "Facial Image #21" and "Facial Image #22" for "Facial Image," but in reality, it stores image data. Additionally, "Facial Image" could also store, for example, the URL containing the image data, the file path indicating the storage location, etc. "Full Makeup Set" represents the complete set of cosmetics used by the publisher in their makeup routine.

That is, in Figure 8, the following example is shown: the facial image posted by the publisher identified according to the publisher "P11" is "Facial Image #21", and the full set of cosmetics used by the publisher in applying makeup is "Brand 〇〇 Product A1 (lipstick), Brand ×× Product A2 (foundation), ...". In addition, the categories of cosmetics are indicated in parentheses.

(Control Department 130)

The control unit 130 is a controller, which is implemented, for example, by executing various programs stored in the storage device inside the information processing device 100 using RAM as the operating area by a CPU, MPU, or the like. Alternatively, the control unit 130 is a controller, which is implemented, for example, by an integrated circuit such as an ASIC or FPGA.

As shown in Figure 6, the control unit 130 includes an acquisition unit 131, a generation unit 132, an application unit 133, a provision unit 134, and a decision unit 135, and performs the information processing described below. Furthermore, the internal structure of the control unit 130 is not limited to the structure shown in Figure 6; it can be any structure that performs the information processing described later, or any other structure.

(Acquisition Department 131)

The acquisition unit 131 acquires various information from external information processing devices. The acquisition unit 131 also acquires various information from other information processing devices such as the user terminal 10.

The acquisition unit 131 acquires various information from the storage unit 120. Furthermore, the acquisition unit 131 stores the acquired information in the storage unit 120.

The acquisition unit 131 acquires information about facial images selected by users from facial images published by publishers in the prescribed service.

The acquisition unit 131 acquires information about the cosmetics used by the publisher in relation to the facial image selected by the user.

(Generation Department 132)

Based on the information acquired by the acquisition unit 131, the generation unit 132 generates makeup information using methods such as PSGAN. This makeup information is used to generate a facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user. Alternatively, if the makeup order in relation to the facial image selected by the user and the order in which the publisher applied the makeup is determined, the generation unit 132 generates makeup information reflecting the makeup order.

The generation unit 132 generates information for displaying a facial image containing the makeup information applied by the application unit 133 (described later) and a description of the cosmetics determined based on the information obtained by the acquisition unit 131.

(Application Department 133)

The application unit 133 applies makeup information generated by the generation unit 132 to the facial image of the user selected as the application object, thereby generating a facial image that is displayed on the user's terminal 10.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed according to priority. The priority is determined based on the user's facial information, body information, cosmetic purchase history, etc. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed with priority. The priority is determined based on indicators representing the popularity of facial images, such as access history, purchase history, and evaluation history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected from those selected or displayed with priority. The priority is determined based on the total price of the cosmetics used by the publisher in the makeup. The higher the priority of the facial image, the more preferentially it is selected or displayed.

(Provide Department 134)

The providing unit 134 sends information generated by the generating unit 132 for displaying content to the user terminal 10. When the user terminal 10 receives the information sent by the providing unit 134, it displays the content corresponding to the information generated by the generating unit 132.

(Decision Section 135)

The decision unit 135 determines whether the user has purchased cosmetics associated with the content provided by the provision unit 134. If it determines that the user has purchased cosmetics, it decides to award the corresponding reward to the publisher who published the facial image selected by the user. Then, the provision unit 134 performs the processing for awarding the corresponding reward to the publisher. Alternatively, the decision unit 135 may also decide to send information for the publisher to receive the corresponding reward. In this case, the provision unit 134 sends the information for the publisher to receive the corresponding reward. Alternatively, the decision unit 135 may also decide to transfer the reward to the publisher's account. In this case, the provision unit 134 performs the processing for transferring the reward to the publisher's account.

(6. Information processing flow)

Next, FIG9 will be used to explain the information processing process performed by the information processing system 1 according to the embodiment. FIG9 is a flowchart showing the information processing process performed by the information processing system 1 according to the embodiment.

As shown in Figure 9, the information processing device 100 acquires information about the facial image selected by the user from facial images published by the publisher in the specified service (step S201).

The information processing device 100 generates makeup information based on the acquired information, which is used to generate makeup that is the same as the publisher's makeup used in the facial image selected by the user (step S202).

The information processing device 100 generates a facial image to be provided to the user by applying the generated makeup information to the facial image of the user selected as the application object (step S203).

The information processing device 100 sends information for displaying content including the generated facial image and descriptions of the cosmetics used by the publisher in the makeup (step S204).

[7. Variations]

The information processing system 1 described above can be implemented in various other ways besides the embodiments described above. Therefore, other embodiments of the information processing system 1 will be described below.

In the above embodiment, a case is shown where the user's facial image after applying makeup information is displayed on the user's terminal 10, but this is not limited to this example. For example, it could also be displayed on the terminal devices of other people communicating online with the user. For example, when a user selects a publisher's facial image to participate in an online meeting, makeup related to the selected facial image could be virtually applied to the user's face, and the facial image after applying the makeup related to the facial image selected by the user could be displayed on the terminal devices of other people. Furthermore, when a user uses the publisher's makeup to participate in a meeting, the user could be charged based on usage time, frequency, and other usage conditions. Moreover, a process could be implemented to give a portion of the fee charged to the user as compensation to the publisher. Thus, since the user's made-up face is displayed to others online, the user can participate in meetings with a natural face.

In this case, the application unit 133 applies makeup information generated by the generation unit 132 to the user's facial image captured by the camera in real time, thereby generating a facial image to be displayed on other people's terminal devices in real time. The generation unit 132 uses the facial image generated by the application unit 133 to generate information in real time for displaying the facial image after applying makeup information to the user's facial image on other people's terminal devices. The providing unit 134 transmits the facial image after applying makeup information via the application unit 133 to other people communicating online in real time with the user by sending the information generated by the generation unit 132. The decision unit 135 determines the reward given to the publisher of the facial image selected by the user based on the online usage of the facial image generated by the application unit 133 (e.g., usage time, usage frequency). In other words, the decision unit 135 determines the reward given to the publisher of the facial image that serves as the source of the makeup information based on the usage of the facial image after applying makeup information to the user's facial image. Furthermore, applications can also be performed on the user's cloud, in which case the providing unit 134 may, for example, only provide the right to use the makeup information. Moreover, the user can also exercise the provided right to use the information to display the facial image of the user after applying the makeup information to their facial image on other people's terminal devices.

[8. Effects]

As described above, the information processing apparatus 100 according to the embodiment includes a generation unit 132 and a determination unit 135. The generation unit 132 generates makeup information based on a facial image of a publisher who has applied prescribed makeup. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image. The determination unit 135 determines the reward to be given to the publisher who published the facial image that serves as the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

Therefore, the information processing device 100 according to the embodiment can appropriately meet the user's need to participate in online meetings, etc., by using a facial image of themselves with other people's makeup applied.

In addition, the decision unit 135 determines the compensation based on the usage time or frequency of the user's facial image after applying makeup information to it.

Therefore, the information processing apparatus 100 according to the embodiment can appropriately encourage the publisher to publish facial images.

In addition, the application unit 133 applies makeup information generated based on a facial image selected from facial images that are preferentially selected or displayed based on prescribed criteria.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on a predetermined benchmark, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority. The priority is determined based on at least one of the user's facial information, body information, and item purchase history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing device 100 according to the embodiment can provide appropriate information to the user based on facial information, body information, purchase history of items, etc., and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, wherein the priority is determined based on at least one of the facial image's access history, purchase history, and review history, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on access history, purchase history, evaluation history, etc. of facial images, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, the priority being determined according to the total price of the item, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on the total price of the item, and thus can more appropriately meet the user's needs.

[9. Hardware Structure]

Furthermore, the user terminal 10, publisher terminal 20, and information processing device 100 involved in the above-described embodiments can be implemented, for example, by a computer 1000 with the structure shown in FIG10. FIG10 is a hardware structure diagram showing an example of a computer that implements the functions of the user terminal 10, publisher terminal 20, and information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

The CPU 1100 operates based on the programs stored in the ROM 1300 or HDD 1400 to control various components. The ROM 1300 stores the startup program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 stores the program executed by CPU 1100 and the data used by that program. Communication interface 1500 obtains data from other devices and sends it to CPU 1100 via a specified communication network, and sends data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as monitors and printers, as well as input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. In addition, the CPU 1100 outputs the generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads the program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 into the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as DVD (Digital Versatile Disc), PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical Disk), magnetic tape, magnetic recording media, or semiconductor memory.

For example, when the computer 1000 functions as the terminal device 10, publisher terminal 20, and information processing device 100 in the implementation, the CPU 1100 of the computer 1000 implements the functions of the control units 14, 24, and 130 by executing programs loaded into the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800; as another example, these programs can also be obtained from other devices via a defined communication network.

[10. Others]

Furthermore, all or part of the processes described in the above embodiments that are intended to be performed automatically can also be performed manually, or all or part of the processes described as intended to be performed manually can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, various data, and parameters shown in the foregoing and accompanying drawings can be arbitrarily changed unless otherwise stated. For example, the various information shown in the figures is not limited to the information illustrated.

Furthermore, the constituent elements of each device illustrated are functional concepts and do not need to be physically configured as shown in the illustrations. That is, the specific way in which the devices are distributed/integrated is not limited to the way illustrated, and they can be configured to be distributed/integrated functionally or physically in any unit according to various loads, usage conditions, etc.

Furthermore, the above-described implementation methods can be appropriately combined within the scope of not contradicting the processing content.

Several embodiments of this application have been described in detail above with reference to the accompanying drawings. However, these are illustrative, and the invention can be implemented in other ways by means of various modifications and alterations based on the knowledge of those skilled in the art, based on the manner described in the Summary of the Invention section.

Furthermore, the terms "section, module, unit" mentioned above can be replaced with "unit," "circuit," etc. For example, "acquisition section" can be replaced with "acquisition unit" or "acquisition circuit."

Explanation of reference numerals in the attached figures

1: Information processing system; 10: User terminal; 11: Communication unit; 12: Input unit; 13: Output unit; 14: Control unit; 20: Publisher terminal; 21: Communication unit; 22: Input unit; 23: Output unit; 24: Control unit; 100: Information processing device; 110: Communication unit; 120: Storage unit; 121: Publisher information storage unit; 122: Cosmetic information storage unit; 130: Control unit; 131: Acquisition unit; 132: Generation unit; 133: Application unit; 134: Provision unit; 135: Decision unit; 141: Receiving unit; 142: Transmitting unit; 241: Receiving unit; 242: Transmitting unit; N: Network.

Technical Field

This invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Technology

When researching cosmetics purchases, the following situation often occurs: even when watching makeup applied to the faces of models or other people, one cannot imagine the image of oneself in such a situation, and therefore does not know whether it is suitable for oneself.

Conventionally, as a technique for fulfilling needs such as confirming whether makeup applied to another person's face is suitable for oneself, techniques that use other people's makeup as a reference to assist in makeup application are known. For example, techniques that present other people's makeup information corresponding to areas of the user's facial features are known.

Existing technical documents

Patent documents

Patent Document 1: Japanese Patent No. 5991536

Non-Patent Literature 1: W. Jiang, S. Liu, C. Gao, J. Cao, R. He, J. Feng, S. Yan, “PSGAN: Pose and Expression Robust Spatial-Aware CAN for Customizable Makeup Transfer,” [online], [searched January 24, 2022], Internet <https://openaccess.thecvf.com/content_CVPR_2020/html/Jiang_PSGAN_Pose_and_Expression_Robust_Spatial-Aware_GAN_for_Customizable_Makeup_CVPR_2020_paper.html>

Summary of the Invention

The problem the invention aims to solve

However, current technology lacks features such as the ability to guide customers to sales websites after they've confirmed whether other people's makeup looks suit them, and the inability to use performance-based rewards to incentivize makeup creators to actively guide customers. Therefore, there is room for improvement in usability for users who use others' makeup looks as a reference.

This application was made in view of the above circumstances, and its purpose is to further improve usability for users who use other people's makeup as a reference.

Solution for solving the problem

The information processing apparatus of this application is characterized by comprising: a generation unit that generates makeup information based on a facial image of a publisher who has applied prescribed makeup, the makeup information being used to generate a facial image after applying the same makeup as the prescribed makeup to the facial image; and a determination unit that determines the reward to be given to the publisher who published the facial image that is the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

The effects of the invention

According to one aspect of the implementation, this achieves the effect of further improving usability for users who can use other people's makeup as a reference.

Attached Figure Description

Figure 1 is a diagram illustrating a structural example of an information processing system according to an embodiment.

Figure 2 is a diagram illustrating an example of information processing involved in the implementation method.

Figure 3 is an explanatory diagram illustrating the information processing in the case of increasing the priority of facial images according to the embodiment.

Figure 4 is a diagram showing an example of the structure of a user terminal according to the embodiment.

Figure 5 is a diagram illustrating an example of the structure of the publisher terminal involved in the implementation method.

Figure 6 is a diagram showing a structural example of the information processing apparatus according to the embodiment.

Figure 7 is a diagram illustrating an example of the publisher information storage unit involved in the embodiment.

Figure 8 is a diagram illustrating an example of the cosmetic information storage unit involved in the embodiment.

Figure 9 is a flowchart illustrating an example of information processing involved in the implementation method.

Figure 10 is a hardware structure diagram illustrating an example of a computer that implements the functions of an information processing device.

Detailed Implementation

Hereinafter, with reference to the accompanying drawings, we will describe in detail the methods (hereinafter referred to as "implementations") for carrying out the information processing apparatus, information processing method, and information processing program involved in this application. However, the information processing apparatus, information processing method, and information processing program involved in this application are not limited to these embodiments. Furthermore, in the following embodiments, the same reference numerals will be used to label the same parts, and repeated descriptions will be omitted.

(Implementation Method)

(1. The structure of an information processing system)

The information processing system 1 shown in FIG1 will be described. As shown in FIG1, the information processing system 1 includes a user terminal 10, a publisher terminal 20, and an information processing device 100. The user terminal 10, the publisher terminal 20, and the information processing device 100 are connected via a predetermined communication network (network N) in a wired or wireless communication manner. FIG1 is a diagram showing an example of the structure of the information processing system 1 according to the embodiment.

User terminal 10 is an information processing device used by a user who wants to confirm whether makeup applied to another person's face is suitable for them. Additionally, the user using user terminal 10 may be researching and purchasing cosmetics. User terminal 10 can be any device as long as it can perform the processing described in the embodiment. Furthermore, user terminal 10 can also be a smartphone, tablet, laptop PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), etc. Figure 2 shows the case where user terminal 10 is a smartphone.

User terminal 10 is a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G (Generation) and LTE (Long Term Evolution). Additionally, user terminal 10 may have a screen such as an LCD display with touch panel functionality, allowing the user to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or a stylus. In Figure 2, user terminal 10 is used by user U11.

The publisher terminal 20 is an information processing device used by a publisher who provides a facial image after makeup. Alternatively, a publisher using the publisher terminal 20 may publish, for example, a facial image after makeup is created by applying makeup from a bare face before makeup application. The publisher terminal 20 can be any device as long as it can perform the processing described in the embodiment. Furthermore, the publisher terminal 20 can also be a smartphone, tablet, laptop PC, desktop PC, mobile phone, PDA, or other similar device. Figure 2 shows the case where the publisher terminal 20 is a smartphone.

The publisher terminal 20 is, for example, a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G and LTE. Additionally, the publisher terminal 20 may also have a screen such as an LCD display with touch panel functionality, allowing the publisher to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or styluses. In Figure 2, the publisher terminal 20 is used by publishers P11 and P12. Publisher terminal 20 ₁ and Publisher terminal 20 ₂ are the publisher terminals 20 for publishers P11 and P12, respectively.

Information processing device 100 is an information processing device aimed at further improving usability for users who use other people's makeup as a reference. Any device is acceptable as long as it can perform the processing described in the embodiment. Information processing device 100 is implemented, for example, by a server device, cloud system, etc. For example, information processing device 100 is implemented by a server device, cloud system, etc., that provides a prescribed electronic mall for selling cosmetics.

(2. An example of information processing)

Figure 2 is a diagram illustrating an example of information processing in the information processing system 1 according to the embodiment. Hereinafter, in the embodiment, the processing for a user U11 who wants to confirm whether the publisher P11's makeup suits them will be described as an example.

Information processing device 100 acquires information about the facial image selected by user U11 from facial images published by a publisher (step S101). This publisher is, for example, a celebrity or influencer whose number of followers on an SNS (Social Networking Service) exceeds a predetermined threshold. Furthermore, the facial images that user U11 can select may include multiple facial images published by a single publisher. In Figure 2, this is assumed to include facial image IM11 published by publisher P11, facial image IM12 published by publisher P12, etc. The following explanation assumes that user U11 selected facial image IM11 from content CT11. Furthermore, the facial images displayed in content CT11 can be selected based on any criterion. For example, facial images published by publishers followed by user U11 can be preferentially selected. Additionally, content CT11 is content within a specified service that manages the facial images of publishers.

Facial image IM11 is a facial image of publisher P11 after applying prescribed makeup. Furthermore, facial image IM21 is a facial image of publisher P11 before makeup, corresponding to the makeup applied to facial image IM11. Facial image IM21 can be displayed together with facial image IM11 on content CT11, or it can be displayed through operations on facial image IM11 (e.g., clicking or tapping). Alternatively, facial image IM21 may not be displayed to the user. Therefore, the publisher can avoid providing a bare-faced image to the user, thus improving usability for the publisher. Additionally, facial image IM12 is a facial image of publisher P12 after applying prescribed makeup. Furthermore, facial image IM22 is a facial image of publisher P12 before makeup, corresponding to the makeup applied to facial image IM12. Facial image IM22 can be displayed together with facial image IM12 on content CT11, or it can be displayed through operations on facial image IM12. Alternatively, facial image IM22 may not be displayed to the user.

Here, as an example of a method for generating makeup information, we cite, for instance, the method disclosed in Non-Patent Document 1, known as PSGAN (Pose and Expression Robust Spatial-Aware Generative Adversarial Network), which simply moves makeup information from one face image to another (Non-Patent Document 1). This method uses deep learning and is able to move makeup information to another face regardless of expression or pose. The information processing device 100 uses a method such as PSGAN to generate makeup information, which is used to generate a face image after applying the same makeup as that used by publisher P11 in the face image IM11 selected by user U11 (step S102). Then, the information processing device 100 uses a method such as PSGAN to apply the makeup information generated in step S102 to the face image IM1 of user U11 (step S103). At this time, the face image IM1 can also be any arbitrarily acquired face image. Facial image IM1 can be, for example, a facial image pre-registered by user U11 and stored in an external information processing device or within this device, or a facial image sent from user terminal 10. Furthermore, facial image IM1 can be a facial image selected as the application target by user U11, or a facial image automatically selected as the application target. Additionally, facial image IM2 is the applied facial image corresponding to facial image IM1.

The information processing device 100 sends information to the user terminal 10 for displaying a facial image IM2 of the user U11 after the makeup information was applied in step S103, and a description of the cosmetics used in the makeup (corresponding to the item) (step S104). Then, the user terminal 10 displays content CT1 based on the information sent from the information processing device 100.

Here, content CT1 includes descriptions of the cosmetics used by publisher P11 in the makeup application related to facial image IM11. For example, it includes information such as the product name and price of the cosmetics. Figure 2 shows the case where two cosmetics (PR1 and PR2) are included, namely lipstick and foundation, but there is no particular limitation on the number of cosmetics. For example, content CT1 may also include descriptions of the complete set of cosmetics used by publisher P11 to create the prescribed post-makeup state of facial image IM11 from the pre-makeup state of facial image IM21. Additionally, the descriptions may include operation acceptance information B1 for accessing a prescribed online store where users can purchase the complete set of cosmetics used in the makeup application. When operation acceptance information B1 is accessed, users can access either the product page introducing the complete set of cosmetics or the purchase page after adding the complete set to their cart. Furthermore, operation acceptance information B1 can be integrated with facial image IM2 and other content in the descriptions, allowing access to the product page and purchase page when operations are performed on facial image IM2 and other content in the descriptions.

Information processing device 100 determines whether user U11 has purchased cosmetics associated with the description content. For example, information processing device 100 determines whether user U11 purchased a complete set of cosmetics, including cosmetics PR1 and cosmetics PR2, or a portion of cosmetics from a designated online store accessed through operation acceptance information B1. If information processing device 100 determines that user U11 has made a purchase, it decides to award the corresponding reward to publisher P11, who is the publisher who published the facial image after makeup using the makeup information corresponding to image IM2. Then, information processing device 100 performs processing to award the corresponding reward to publisher P11. Alternatively, information processing device 100 may decide to send information for publisher P11 to receive the corresponding reward for user U11's purchase. In this case, information processing device 100 sends information for publisher P11 to receive the corresponding reward for user U11's purchase. Alternatively, the information processing device 100 may decide to transfer the payment to the publisher P11's account. In this case, the information processing device 100 performs the processing for transferring the payment to the publisher P11's account.

(Version 1: Display using a pre-makeup facial image)

In the above embodiment, a case is shown where pre-defined post-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, but this is not limited to this example. For example, facial images such as facial image IM21 and facial image IM22 before makeup can also be displayed in content CT11, and the post-makeup facial image can be displayed by manipulating the pre-makeup facial image. Furthermore, the information processing device 100 can also generate makeup information applied to the user's facial image by allowing the user U11 to view and select the post-makeup facial image corresponding to the pre-makeup facial image.

(Processed Variation 2: Display of dynamic makeup images)

The above embodiment illustrates a case where pre-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, and a pre-makeup facial image is displayed by manipulating the pre-makeup facial images, but this is not limited to this example. For instance, a makeup animation image showing the publisher applying makeup to create a finished look from a pre-makeup state can also be displayed by manipulating the pre-makeup facial images. Furthermore, the makeup animation image may include, for example, pre-makeup facial images such as facial image IM21 and facial image IM22, as well as pre-makeup facial images such as facial image IM11 and facial image IM12. Additionally, the makeup animation image may be displayed together with the pre-makeup facial image in content CT11.

(Version 3: Limitations of the facial image before makeup)

In the above embodiment, the case shown is that the facial images before makeup, such as facial image IM21 and facial image IM22, are images of the publisher's bare face, but this is not limited to this example. For example, the facial images before makeup, such as facial image IM21 and facial image IM22, can be any facial images before makeup, such as facial image IM11 and facial image IM12, or they can be images of the face after some degree of makeup, such as lipstick and foundation. In this case, the information processing device 100 can generate makeup information such as makeup information for the user's facial image that does not use the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22, or makeup information for the user's facial image that uses the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22. Furthermore, since the facial images before makeup, such as facial image IM21 and facial image IM22, are used for bone learning after generating makeup information, appropriate makeup information can be generated even if the facial image is not a bare face. Therefore, the publisher's pre-makeup facial image only needs to be registered once in the service that manages the publisher's facial images. Furthermore, the description of the cosmetics used in the pre-makeup facial images such as facial image IM21 and facial image IM22 may or may not be included in content CT1.

(Version 4: Automatic extraction of item information)

In the above embodiments, for example, the cosmetic used by the publisher P11 in the makeup application related to the facial image IM11 can be determined in advance from the facial image IM11 and facial image IM21 through image analysis, and the cosmetic is stored in association with the facial image IM11 and facial image IM21 in an external information processing device or this device. Alternatively, the cosmetic can be determined based on the publisher P11's purchase history of cosmetics purchased from a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Alternatively, the publisher P11 can select from a list of candidates based on the cosmetic purchase history to determine the cosmetic, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Furthermore, the cosmetic can be determined by the publisher P11's registration of cosmetics not sold in a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21.

(Version 5: Reflecting the order of makeup application)

In the above embodiment, the information processing device 100 can also generate makeup information reflecting the makeup order by determining the makeup order performed by the publisher P11 in relation to the facial image IM11. In this case, the information processing device 100 can determine the makeup order by receiving the makeup order input from the publisher P11, and in the case of a makeup animation image related to the facial image IM11, it can also determine the makeup order based on the makeup animation image. Alternatively, the information processing device 100 can determine the makeup order by learning from multiple makeup animation images. Furthermore, in the case of a makeup animation image, the information processing device 100 can generate makeup information according to each category of cosmetics, thus allowing for the selection of applicable facial images. Additionally, explanatory content reflecting the makeup order can also be included in the content CT1. Therefore, the information processing device 100 can provide appropriate suggestions, including the makeup order.

(Processing Variation 6: Prioritized Display of Facial Images 1)

In the above embodiments, the facial images displayed in content CT1 can be selected based on any criterion or according to any priority. For example, facial images published by publishers followed by user U11 can be prioritized, or the more followers a publisher has, the higher the priority of displaying the publisher's facial images. Alternatively, the selection or display can be prioritized based on user U11's facial information or other body information.

Here, examples of facial information include the shape, size, and arrangement of the components that make up the face; the color and contour of the face; and the color and size of the pupils. Additionally, facial color includes, for example, the color of a bare face. Examples include, for instance, the average color of the entire measurement area excluding the lips; the average color of the forehead area; the average color of the area under the left eye; the average color of the area under the right eye; the average color of the left cheek area; the average color of the right cheek area; the average color of the nose area; the average color of the upper lip; the average color of the lower lip; and the average color of the chin area. Furthermore, examples of other body information besides facial information include, for instance, hair (hairstyle, hair texture, hair color, accessories, etc.); skin (skin tone, skin texture, wrinkles, etc.); and body shape (the size of the components, overall balance).

In the above embodiments, priority can be increased by prioritizing the publisher's facial image in order to be selected or displayed more preferentially, based on the similarity of the publisher's body information to that of user U11. For example, the more similar or identical the publisher's facial and other body information is to user U11, the higher the priority of the publisher's facial image. Alternatively, the more similar or identical the publisher's age, skin age, and skin concerns are to user U11, the higher the priority of the publisher's facial image. Similarly, the more similar or identical the publisher's concerns about makeup are to user U11, the higher the priority of the publisher's facial image. Furthermore, the more similar or identical the publisher's hairstyle, hair color, and hair preferences are to user U11, the higher the priority of the publisher's facial image. Conversely, priority can be decreased by prioritizing the publisher's facial image in order to be less preferentially selected or displayed, based on the disparity between the publisher's body information and that of user U11. For example, the more different the balance of facial features is from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the facial contours are from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the bare-faced facial color is from user U11, the lower the priority of the publisher's facial image.

Alternatively, priority can be given to selecting or displaying facial images based on indicators of their popularity, such as user browsing history, purchase history, and review history. Specifically, priority can be given based on user browsing history of facial images selected by multiple users, user purchase history generated through customer guidance provided by content offered via facial image selection, user review history when applying makeup using facial images, and user review history of facial images or purchased cosmetics after purchase. For example, higher user browsing history, purchase history, and review history, indicating the popularity of a facial image, can result in higher priority for that image. Alternatively, priority can also be given based on the total price of the complete set of cosmetics used by the user in the makeup application. For example, a higher total price of the complete set of cosmetics used by the user in the makeup application can either result in higher or lower priority for the facial image.

(Processing Variation 7: Prioritized Display of Facial Images 2)

In the above embodiments, the selection or display of cosmetics can also be prioritized based on the user U11's purchase history. For example, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the publisher's facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are included in the publisher's purchase history, the higher the priority of the publisher's facial image. For example, the priority of a publisher's facial image that includes both cosmetics PR2 and PR3 in their purchase history may be higher than that of a publisher whose purchase history only includes cosmetic PR1. Furthermore, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are used in makeup, the higher the priority of the facial image after using the cosmetics.

Figure 3 is an illustration of information processing where the higher the priority of a facial image, the more identical or similar it is to the cosmetics available to user U11. This priority is based on user U11's cosmetic purchase history. Facial images IM11 and IM12 are facial images published by publisher P11, while IM12 is a facial image published by publisher P12. Furthermore, the cosmetics used in the makeup associated with facial image IM11 are cosmetics PR1, PR2, and PR3; those used in the makeup associated with facial image IM11 are cosmetics PR1, PR4, and PR5; and those used in the makeup associated with facial image IM12 are cosmetics PR2, PR6, PR7, and PR8. In the user U11's purchase history, which includes cosmetics PR1, PR2, PR3, and PR6, the facial image IM11, which uses the common cosmetics PR1, PR2, and PR3, has the highest priority; the facial image IM111, which uses the common cosmetics PR2 and PR6, has a medium priority; and the facial image IM12, which uses the common cosmetics PR1, has the lowest priority.

(Processing Variation 8: Acquisition of Color Information)

In the above embodiment, skin color, such as facial color, can be measured by measuring equipment such as ZOZOGLASS (registered trademark), which can be used by the user U11 for measurement. By accurately measuring skin color after light correction, the accuracy of the makeup information generated by the information processing device 100 can be improved.

[3. Structure of the user terminal]

Next, the structure of the user terminal 10 according to the embodiment will be described using FIG4. FIG4 is a diagram showing an example of the structure of the user terminal 10 according to the embodiment. As shown in FIG4, the user terminal 10 includes a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

(Communications Department 11)

The communication unit 11 is implemented, for example, by a NIC (Network Interface Card). Furthermore, the communication unit 11 establishes a wired or wireless connection with a designated network N, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 12)

The input unit 12 accepts various operations from the user. In Figure 2, it accepts various operations from the user U11. For example, the input unit 12 can also accept various operations from the user via the display surface through the touch panel function. In addition, the input unit 12 can also accept various operations from buttons provided on the user terminal 10, keyboards connected to the user terminal 10, and mice.

(Output Section 13)

The output unit 13 is, for example, a display screen of a flat panel terminal implemented with a liquid crystal display, an organic EL (electro-Luminescence) display, etc., and is a display device used to display various information. For example, the output unit 13 displays information sent from the information processing device 100.

(Control Unit 14)

The control unit 14 is, for example, a controller, implemented by a CPU (Central Processing Unit), MPU (Micro Processing Unit), or similar device that uses RAM (Random Access Memory) as its operating area to execute various programs stored in the internal storage device of the user terminal 10. These programs include, for example, programs for applications installed on the user terminal 10, such as programs for displaying content containing facial images sent from the information processing device 100. Furthermore, the control unit 14 is implemented using integrated circuits such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

As shown in Figure 4, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and performs the information processing described below.

(Receiving Unit 141)

The receiving unit 141 receives various types of information from other information processing devices such as the information processing device 100. For example, the receiving unit 141 receives information for displaying a user's facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user, as well as instructions on the cosmetics used in the makeup.

(Sending Department 142)

The sending unit 142 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends information about a facial image selected by the user to confirm whether the publisher's makeup suits them.

[4. Structure of the publisher's terminal]

Next, the structure of the publisher terminal 20 according to the embodiment will be described using FIG5. FIG5 is a diagram showing an example of the structure of the publisher terminal 20 according to the embodiment. As shown in FIG5, the publisher terminal 20 includes a communication unit 21, an input unit 22, an output unit 23, and a control unit 24.

(Communications Department 21)

The communication unit 21 is implemented, for example, by a NIC. Furthermore, the communication unit 21 is connected to a designated network N via wired or wireless means, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 22)

The input unit 22 accepts various operations from the publisher. In Figure 2, it accepts various operations from publishers P11 and P12. For example, the input unit 22 can also accept various operations from the publisher via the display surface through the touch panel function. In addition, the input unit 22 can also accept various operations from buttons provided on the publisher terminal 20, keyboards connected to the publisher terminal 20, and mice.

(Output Section 23)

The output unit 23 is, for example, a display screen of a flat panel terminal implemented by a liquid crystal display, an organic EL display, or the like, and is a display device used to display various information. For example, the output unit 23 displays information sent from the information processing device 100.

(Control Department 24)

The control unit 24 is, for example, a controller, implemented by executing various programs stored in the internal storage device of the publisher terminal 20 using RAM as the operating area, such as a CPU or MPU. These various programs include, for example, programs for applications installed on the publisher terminal 20. For example, these programs include programs for applications that send facial images of the publisher based on the publisher's actions. Furthermore, the control unit 14 is implemented using integrated circuits such as ASICs or FPGAs.

As shown in Figure 5, the control unit 24 has a receiving unit 241 and a transmitting unit 242, and performs the information processing described below.

(Receiving Unit 241)

The receiving unit 241 receives various types of information from the information processing device 100 and other information processing devices. For example, if a user who has viewed a facial image provided by a publisher and applied makeup to their own facial image purchases cosmetics corresponding to the applied makeup from a designated online store, the receiving unit 241 receives information related to the remuneration that the publisher can collect based on the user's purchase. Furthermore, the receiving unit 241 may also receive information for the publisher to collect remuneration corresponding to the user's purchase.

(Department 242)

The sending unit 242 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends a facial image of the publisher based on the publisher's operation.

(5. Configuration of information processing devices)

Next, the structure of the information processing apparatus 100 according to the embodiment will be described using FIG6. FIG6 is a diagram showing an example of the structure of the information processing apparatus 100 according to the embodiment. As shown in FIG6, the information processing apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. In addition, the information processing apparatus 100 may also include an input unit (e.g., a keyboard, mouse, etc.) for receiving various operations from the administrator of the information processing apparatus 100, and a display unit (e.g., a liquid crystal display, etc.) for displaying various information.

(Communication Department 110)

The communication unit 110 is implemented, for example, by a NIC. Furthermore, the communication unit 110 is connected to the network N via a wired or wireless connection, and transmits and receives information with the user terminal 10 via the network N.

(Storage Department 120)

The storage unit 120 is implemented, for example, by semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks and optical disks. As shown in FIG6, the storage unit 120 includes a publisher information storage unit 121 and a cosmetics information storage unit 122.

The publisher information storage unit 121 stores the publisher's facial image. Here, an example of the publisher information storage unit 121 according to the embodiment is shown in FIG7. As shown in FIG7, the publisher information storage unit 121 has items such as "publisher ID" and "facial image".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a bare-faced facial image of the publisher registered upon initial use. The example shown in Figure 7 illustrates a case where conceptual information such as "Facial Image #11" and "Facial Image #12" is stored for "Facial Image," but in reality, image data is stored. Additionally, for "Facial Image," the URL containing the image data and the file path indicating the storage location may also be stored.

That is, Figure 7 shows an example of a bare-faced facial image of the publisher identified according to the publisher "P11" as "Face Image #11".

The cosmetics information storage unit 122 stores information about the cosmetics used by the publisher in applying makeup. As shown in Figure 8, the cosmetics information storage unit 122 has items such as "Publisher ID", "Facial Image", and "Complete Set of Cosmetics".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a facial image published by the publisher. The example shown in Figure 8 illustrates a conceptual information storage term such as "Facial Image #21" and "Facial Image #22" for "Facial Image," but in reality, it stores image data. Additionally, "Facial Image" could also store, for example, the URL containing the image data, the file path indicating the storage location, etc. "Full Makeup Set" represents the complete set of cosmetics used by the publisher in their makeup routine.

That is, in Figure 8, the following example is shown: the facial image posted by the publisher identified according to the publisher "P11" is "Facial Image #21", and the full set of cosmetics used by the publisher in applying makeup is "Brand 〇〇 Product A1 (lipstick), Brand ×× Product A2 (foundation), ...". In addition, the categories of cosmetics are indicated in parentheses.

(Control Department 130)

The control unit 130 is a controller, which is implemented, for example, by executing various programs stored in the storage device inside the information processing device 100 using RAM as the operating area by a CPU, MPU, or the like. Alternatively, the control unit 130 is a controller, which is implemented, for example, by an integrated circuit such as an ASIC or FPGA.

As shown in Figure 6, the control unit 130 includes an acquisition unit 131, a generation unit 132, an application unit 133, a provision unit 134, and a decision unit 135, and performs the information processing described below. Furthermore, the internal structure of the control unit 130 is not limited to the structure shown in Figure 6; it can be any structure that performs the information processing described later, or any other structure.

(Acquisition Department 131)

The acquisition unit 131 acquires various information from external information processing devices. The acquisition unit 131 also acquires various information from other information processing devices such as the user terminal 10.

The acquisition unit 131 acquires various information from the storage unit 120. Furthermore, the acquisition unit 131 stores the acquired information in the storage unit 120.

The acquisition unit 131 acquires information about facial images selected by users from facial images published by publishers in the prescribed service.

The acquisition unit 131 acquires information about the cosmetics used by the publisher in relation to the facial image selected by the user.

(Generation Department 132)

Based on the information acquired by the acquisition unit 131, the generation unit 132 generates makeup information using methods such as PSGAN. This makeup information is used to generate a facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user. Alternatively, if the makeup order in relation to the facial image selected by the user and the order in which the publisher applied the makeup is determined, the generation unit 132 generates makeup information reflecting the makeup order.

The generation unit 132 generates information for displaying a facial image containing the makeup information applied by the application unit 133 (described later) and a description of the cosmetics determined based on the information obtained by the acquisition unit 131.

(Application Department 133)

The application unit 133 applies makeup information generated by the generation unit 132 to the facial image of the user selected as the application object, thereby generating a facial image that is displayed on the user's terminal 10.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed according to priority. The priority is determined based on the user's facial information, body information, cosmetic purchase history, etc. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed with priority. The priority is determined based on indicators representing the popularity of facial images, such as access history, purchase history, and evaluation history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected from those selected or displayed with priority. The priority is determined based on the total price of the cosmetics used by the publisher in the makeup. The higher the priority of the facial image, the more preferentially it is selected or displayed.

(Provide Department 134)

The providing unit 134 sends information generated by the generating unit 132 for displaying content to the user terminal 10. When the user terminal 10 receives the information sent by the providing unit 134, it displays the content corresponding to the information generated by the generating unit 132.

(Decision Section 135)

The decision unit 135 determines whether the user has purchased cosmetics associated with the content provided by the provision unit 134. If it determines that the user has purchased cosmetics, it decides to award the corresponding reward to the publisher who published the facial image selected by the user. Then, the provision unit 134 performs the processing for awarding the corresponding reward to the publisher. Alternatively, the decision unit 135 may also decide to send information for the publisher to receive the corresponding reward. In this case, the provision unit 134 sends the information for the publisher to receive the corresponding reward. Alternatively, the decision unit 135 may also decide to transfer the reward to the publisher's account. In this case, the provision unit 134 performs the processing for transferring the reward to the publisher's account.

(6. Information processing flow)

Next, FIG9 will be used to explain the information processing process performed by the information processing system 1 according to the embodiment. FIG9 is a flowchart showing the information processing process performed by the information processing system 1 according to the embodiment.

As shown in Figure 9, the information processing device 100 acquires information about the facial image selected by the user from facial images published by the publisher in the specified service (step S201).

The information processing device 100 generates makeup information based on the acquired information, which is used to generate makeup that is the same as the publisher's makeup used in the facial image selected by the user (step S202).

The information processing device 100 generates a facial image to be provided to the user by applying the generated makeup information to the facial image of the user selected as the application object (step S203).

The information processing device 100 sends information for displaying content including the generated facial image and descriptions of the cosmetics used by the publisher in the makeup (step S204).

[7. Variations]

The information processing system 1 described above can be implemented in various other ways besides the embodiments described above. Therefore, other embodiments of the information processing system 1 will be described below.

In the above embodiment, a case is shown where the user's facial image after applying makeup information is displayed on the user's terminal 10, but this is not limited to this example. For example, it could also be displayed on the terminal devices of other people communicating online with the user. For example, when a user selects a publisher's facial image to participate in an online meeting, makeup related to the selected facial image could be virtually applied to the user's face, and the facial image after applying the makeup related to the facial image selected by the user could be displayed on the terminal devices of other people. Furthermore, when a user uses the publisher's makeup to participate in a meeting, the user could be charged based on usage time, frequency, and other usage conditions. Moreover, a process could be implemented to give a portion of the fee charged to the user as compensation to the publisher. Thus, since the user's made-up face is displayed to others online, the user can participate in meetings with a natural face.

In this case, the application unit 133 applies makeup information generated by the generation unit 132 to the user's facial image captured by the camera in real time, thereby generating a facial image to be displayed on other people's terminal devices in real time. The generation unit 132 uses the facial image generated by the application unit 133 to generate information in real time for displaying the facial image after applying makeup information to the user's facial image on other people's terminal devices. The providing unit 134 transmits the facial image after applying makeup information via the application unit 133 to other people communicating online in real time with the user by sending the information generated by the generation unit 132. The decision unit 135 determines the reward given to the publisher of the facial image selected by the user based on the online usage of the facial image generated by the application unit 133 (e.g., usage time, usage frequency). In other words, the decision unit 135 determines the reward given to the publisher of the facial image that serves as the source of the makeup information based on the usage of the facial image after applying makeup information to the user's facial image. Furthermore, applications can also be performed on the user's cloud, in which case the providing unit 134 may, for example, only provide the right to use the makeup information. Moreover, the user can also exercise the provided right to use the information to display the facial image of the user after applying the makeup information to their facial image on other people's terminal devices.

[8. Effects]

As described above, the information processing apparatus 100 according to the embodiment includes a generation unit 132 and a determination unit 135. The generation unit 132 generates makeup information based on a facial image of a publisher who has applied prescribed makeup. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image. The determination unit 135 determines the reward to be given to the publisher who published the facial image that serves as the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

Therefore, the information processing device 100 according to the embodiment can appropriately meet the user's need to participate in online meetings, etc., by using a facial image of themselves with other people's makeup applied.

In addition, the decision unit 135 determines the compensation based on the usage time or frequency of the user's facial image after applying makeup information to it.

Therefore, the information processing apparatus 100 according to the embodiment can appropriately encourage the publisher to publish facial images.

In addition, the application unit 133 applies makeup information generated based on a facial image selected from facial images that are preferentially selected or displayed based on prescribed criteria.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on a predetermined benchmark, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority. The priority is determined based on at least one of the user's facial information, body information, and item purchase history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing device 100 according to the embodiment can provide appropriate information to the user based on facial information, body information, purchase history of items, etc., and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, wherein the priority is determined based on at least one of the facial image's access history, purchase history, and review history, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on access history, purchase history, evaluation history, etc. of facial images, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, the priority being determined according to the total price of the item, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on the total price of the item, and thus can more appropriately meet the user's needs.

[9. Hardware Structure]

Furthermore, the user terminal 10, publisher terminal 20, and information processing device 100 involved in the above-described embodiments can be implemented, for example, by a computer 1000 with the structure shown in FIG10. FIG10 is a hardware structure diagram showing an example of a computer that implements the functions of the user terminal 10, publisher terminal 20, and information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

The CPU 1100 operates based on the programs stored in the ROM 1300 or HDD 1400 to control various components. The ROM 1300 stores the startup program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 stores the program executed by CPU 1100 and the data used by that program. Communication interface 1500 obtains data from other devices and sends it to CPU 1100 via a specified communication network, and sends data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as monitors and printers, as well as input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. In addition, the CPU 1100 outputs the generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads the program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 into the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as DVD (Digital Versatile Disc), PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical Disk), magnetic tape, magnetic recording media, or semiconductor memory.

For example, when the computer 1000 functions as the terminal device 10, publisher terminal 20, and information processing device 100 in the implementation, the CPU 1100 of the computer 1000 implements the functions of the control units 14, 24, and 130 by executing programs loaded into the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800; as another example, these programs can also be obtained from other devices via a defined communication network.

[10. Others]

Furthermore, all or part of the processes described in the above embodiments that are intended to be performed automatically can also be performed manually, or all or part of the processes described as intended to be performed manually can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, various data, and parameters shown in the foregoing and accompanying drawings can be arbitrarily changed unless otherwise stated. For example, the various information shown in the figures is not limited to the information illustrated.

Furthermore, the constituent elements of each device illustrated are functional concepts and do not need to be physically configured as shown in the illustrations. That is, the specific way in which the devices are distributed/integrated is not limited to the way illustrated, and they can be configured to be distributed/integrated functionally or physically in any unit according to various loads, usage conditions, etc.

Furthermore, the above-described implementation methods can be appropriately combined within the scope of not contradicting the processing content.

Several embodiments of this application have been described in detail above with reference to the accompanying drawings. However, these are illustrative, and the invention can be implemented in other ways by means of various modifications and alterations based on the knowledge of those skilled in the art, based on the manner described in the Summary of the Invention section.

Furthermore, the terms "section, module, unit" mentioned above can be replaced with "unit," "circuit," etc. For example, "acquisition section" can be replaced with "acquisition unit" or "acquisition circuit."

Explanation of reference numerals in the attached figures

1: Information processing system; 10: User terminal; 11: Communication unit; 12: Input unit; 13: Output unit; 14: Control unit; 20: Publisher terminal; 21: Communication unit; 22: Input unit; 23: Output unit; 24: Control unit; 100: Information processing device; 110: Communication unit; 120: Storage unit; 121: Publisher information storage unit; 122: Cosmetic information storage unit; 130: Control unit; 131: Acquisition unit; 132: Generation unit; 133: Application unit; 134: Provision unit; 135: Decision unit; 141: Receiving unit; 142: Transmitting unit; 241: Receiving unit; 242: Transmitting unit; N: Network.

Information processing device, information processing method and information processing program

Technical Field

This invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Technology

When researching cosmetics purchases, the following situation often occurs: even when watching makeup applied to the faces of models or other people, one cannot imagine the image of oneself in such a situation, and therefore does not know whether it is suitable for oneself.

Conventionally, as a technique for fulfilling needs such as confirming whether makeup applied to another person's face is suitable for oneself, techniques that use other people's makeup as a reference to assist in makeup application are known. For example, techniques that present other people's makeup information corresponding to areas of the user's facial features are known.

Existing technical documents

Patent documents

Patent Document 1: Japanese Patent No. 5991536

Non-Patent Literature 1: W. Jiang, S. Liu, C. Gao, J. Cao, R. He, J. Feng, S. Yan, “PSGAN: Pose and Expression Robust Spatial-Aware CAN for Customizable Makeup Transfer,” [online], [searched January 24, 2022], Internet <https://openaccess.thecvf.com/content_CVPR_2020/html/Jiang_PSGAN_Pose_and_Expression_Robust_Spatial-Aware_GAN_for_Customizable_Makeup_CVPR_2020_paper.html>

Summary of the Invention

The problem the invention aims to solve

However, current technology lacks features such as the ability to guide customers to sales websites after they've confirmed whether other people's makeup looks suit them, and the inability to use performance-based rewards to incentivize makeup creators to actively guide customers. Therefore, there is room for improvement in usability for users who use others' makeup looks as a reference.

This application was made in view of the above circumstances, and its purpose is to further improve usability for users who use other people's makeup as a reference.

Solution for solving the problem

The information processing apparatus of this application is characterized by comprising: a generation unit that generates makeup information based on a facial image of a publisher who has applied prescribed makeup, the makeup information being used to generate a facial image after applying the same makeup as the prescribed makeup to the facial image; and a determination unit that determines the reward to be given to the publisher who published the facial image that is the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

The effects of the invention

According to one aspect of the implementation, this achieves the effect of further improving usability for users who can use other people's makeup as a reference.

Attached Figure Description

Figure 1 is a diagram illustrating a structural example of an information processing system according to an embodiment.

Figure 2 is a diagram illustrating an example of information processing involved in the implementation method.

Figure 3 is an explanatory diagram illustrating the information processing in the case of increasing the priority of facial images according to the embodiment.

Figure 4 is a diagram showing an example of the structure of a user terminal according to the embodiment.

Figure 5 is a diagram illustrating an example of the structure of the publisher terminal involved in the implementation method.

Figure 6 is a diagram showing a structural example of the information processing apparatus according to the embodiment.

Figure 7 is a diagram illustrating an example of the publisher information storage unit involved in the embodiment.

Figure 8 is a diagram illustrating an example of the cosmetic information storage unit involved in the embodiment.

Figure 9 is a flowchart illustrating an example of information processing involved in the implementation method.

Figure 10 is a hardware structure diagram illustrating an example of a computer that implements the functions of an information processing device.

Detailed Implementation

Hereinafter, with reference to the accompanying drawings, we will describe in detail the methods (hereinafter referred to as "implementations") for carrying out the information processing apparatus, information processing method, and information processing program involved in this application. However, the information processing apparatus, information processing method, and information processing program involved in this application are not limited to these embodiments. Furthermore, in the following embodiments, the same reference numerals will be used to label the same parts, and repeated descriptions will be omitted.

(Implementation Method)

(1. The structure of an information processing system)

The information processing system 1 shown in FIG1 will be described. As shown in FIG1, the information processing system 1 includes a user terminal 10, a publisher terminal 20, and an information processing device 100. The user terminal 10, the publisher terminal 20, and the information processing device 100 are connected via a predetermined communication network (network N) in a wired or wireless communication manner. FIG1 is a diagram showing an example of the structure of the information processing system 1 according to the embodiment.

User terminal 10 is an information processing device used by a user who wants to confirm whether makeup applied to another person's face is suitable for them. Additionally, the user using user terminal 10 may be researching and purchasing cosmetics. User terminal 10 can be any device as long as it can perform the processing described in the embodiment. Furthermore, user terminal 10 can also be a smartphone, tablet, laptop PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), etc. Figure 2 shows the case where user terminal 10 is a smartphone.

User terminal 10 is a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G (Generation) and LTE (Long Term Evolution). Additionally, user terminal 10 may have a screen such as an LCD display with touch panel functionality, allowing the user to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or a stylus. In Figure 2, user terminal 10 is used by user U11.

The publisher terminal 20 is an information processing device used by a publisher who provides a facial image after makeup. Alternatively, a publisher using the publisher terminal 20 may publish, for example, a facial image after makeup is created by applying makeup from a bare face before makeup application. The publisher terminal 20 can be any device as long as it can perform the processing described in the embodiment. Furthermore, the publisher terminal 20 can also be a smartphone, tablet, laptop PC, desktop PC, mobile phone, PDA, or other similar device. Figure 2 shows the case where the publisher terminal 20 is a smartphone.

The publisher terminal 20 is, for example, a smart device such as a smartphone or tablet, a portable terminal device capable of communicating with any server device via wireless communication networks such as 3G-5G and LTE. Additionally, the publisher terminal 20 may also have a screen such as an LCD display with touch panel functionality, allowing the publisher to perform various operations on displayed data, such as tapping, swiping, and scrolling, using fingers or styluses. In Figure 2, the publisher terminal 20 is used by publishers P11 and P12. Publisher terminal 20 ₁ and Publisher terminal 20 ₂ are the publisher terminals 20 for publishers P11 and P12, respectively.

Information processing device 100 is an information processing device aimed at further improving usability for users who use other people's makeup as a reference. Any device is acceptable as long as it can perform the processing described in the embodiment. Information processing device 100 is implemented, for example, by a server device, cloud system, etc. For example, information processing device 100 is implemented by a server device, cloud system, etc., that provides a prescribed electronic mall for selling cosmetics.

(2. An example of information processing)

Figure 2 is a diagram illustrating an example of information processing in the information processing system 1 according to the embodiment. Hereinafter, in the embodiment, the processing for a user U11 who wants to confirm whether the publisher P11's makeup suits them will be described as an example.

Information processing device 100 acquires information about the facial image selected by user U11 from facial images published by a publisher (step S101). This publisher is, for example, a celebrity or influencer whose number of followers on an SNS (Social Networking Service) exceeds a predetermined threshold. Furthermore, the facial images that user U11 can select may include multiple facial images published by a single publisher. In Figure 2, this is assumed to include facial image IM11 published by publisher P11, facial image IM12 published by publisher P12, etc. The following explanation assumes that user U11 selected facial image IM11 from content CT11. Furthermore, the facial images displayed in content CT11 can be selected based on any criterion. For example, facial images published by publishers followed by user U11 can be preferentially selected. Additionally, content CT11 is content within a specified service that manages the facial images of publishers.

Facial image IM11 is a facial image of publisher P11 after applying prescribed makeup. Furthermore, facial image IM21 is a facial image of publisher P11 before makeup, corresponding to the makeup applied to facial image IM11. Facial image IM21 can be displayed together with facial image IM11 on content CT11, or it can be displayed through operations on facial image IM11 (e.g., clicking or tapping). Alternatively, facial image IM21 may not be displayed to the user. Therefore, the publisher can avoid providing a bare-faced image to the user, thus improving usability for the publisher. Additionally, facial image IM12 is a facial image of publisher P12 after applying prescribed makeup. Furthermore, facial image IM22 is a facial image of publisher P12 before makeup, corresponding to the makeup applied to facial image IM12. Facial image IM22 can be displayed together with facial image IM12 on content CT11, or it can be displayed through operations on facial image IM12. Alternatively, facial image IM22 may not be displayed to the user.

Here, as an example of a method for generating makeup information, we cite, for instance, the method disclosed in Non-Patent Document 1, known as PSGAN (Pose and Expression Robust Spatial-Aware Generative Adversarial Network), which simply moves makeup information from one face image to another (Non-Patent Document 1). This method uses deep learning and is able to move makeup information to another face regardless of expression or pose. The information processing device 100 uses a method such as PSGAN to generate makeup information, which is used to generate a face image after applying the same makeup as that used by publisher P11 in the face image IM11 selected by user U11 (step S102). Then, the information processing device 100 uses a method such as PSGAN to apply the makeup information generated in step S102 to the face image IM1 of user U11 (step S103). At this time, the face image IM1 can also be any arbitrarily acquired face image. Facial image IM1 can be, for example, a facial image pre-registered by user U11 and stored in an external information processing device or within this device, or a facial image sent from user terminal 10. Furthermore, facial image IM1 can be a facial image selected as the application target by user U11, or a facial image automatically selected as the application target. Additionally, facial image IM2 is the applied facial image corresponding to facial image IM1.

The information processing device 100 sends information to the user terminal 10 for displaying a facial image IM2 of the user U11 after the makeup information was applied in step S103, and a description of the cosmetics used in the makeup (corresponding to the item) (step S104). Then, the user terminal 10 displays content CT1 based on the information sent from the information processing device 100.

Here, content CT1 includes descriptions of the cosmetics used by publisher P11 in the makeup application related to facial image IM11. For example, it includes information such as the product name and price of the cosmetics. Figure 2 shows the case where two cosmetics (PR1 and PR2) are included, namely lipstick and foundation, but there is no particular limitation on the number of cosmetics. For example, content CT1 may also include descriptions of the complete set of cosmetics used by publisher P11 to create the prescribed post-makeup state of facial image IM11 from the pre-makeup state of facial image IM21. Additionally, the descriptions may include operation acceptance information B1 for accessing a prescribed online store where users can purchase the complete set of cosmetics used in the makeup application. When operation acceptance information B1 is accessed, users can access either the product page introducing the complete set of cosmetics or the purchase page after adding the complete set to their cart. Furthermore, operation acceptance information B1 can be integrated with facial image IM2 and other content in the descriptions, allowing access to the product page and purchase page when operations are performed on facial image IM2 and other content in the descriptions.

Information processing device 100 determines whether user U11 has purchased cosmetics associated with the description content. For example, information processing device 100 determines whether user U11 purchased a complete set of cosmetics, including cosmetics PR1 and cosmetics PR2, or a portion of cosmetics from a designated online store accessed through operation acceptance information B1. If information processing device 100 determines that user U11 has made a purchase, it decides to award the corresponding reward to publisher P11, who is the publisher who published the facial image after makeup using the makeup information corresponding to image IM2. Then, information processing device 100 performs processing to award the corresponding reward to publisher P11. Alternatively, information processing device 100 may decide to send information for publisher P11 to receive the corresponding reward for user U11's purchase. In this case, information processing device 100 sends information for publisher P11 to receive the corresponding reward for user U11's purchase. Alternatively, the information processing device 100 may decide to transfer the payment to the publisher P11's account. In this case, the information processing device 100 performs the processing for transferring the payment to the publisher P11's account.

(Version 1: Display using a pre-makeup facial image)

In the above embodiment, a case is shown where pre-defined post-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, but this is not limited to this example. For example, facial images such as facial image IM21 and facial image IM22 before makeup can also be displayed in content CT11, and the post-makeup facial image can be displayed by manipulating the pre-makeup facial image. Furthermore, the information processing device 100 can also generate makeup information applied to the user's facial image by allowing the user U11 to view and select the post-makeup facial image corresponding to the pre-makeup facial image.

(Processed Variation 2: Display of dynamic makeup images)

The above embodiment illustrates a case where pre-makeup facial images such as facial image IM11 and facial image IM12 are displayed in content CT11, and a pre-makeup facial image is displayed by manipulating the pre-makeup facial images, but this is not limited to this example. For instance, a makeup animation image showing the publisher applying makeup to create a finished look from a pre-makeup state can also be displayed by manipulating the pre-makeup facial images. Furthermore, the makeup animation image may include, for example, pre-makeup facial images such as facial image IM21 and facial image IM22, as well as pre-makeup facial images such as facial image IM11 and facial image IM12. Additionally, the makeup animation image may be displayed together with the pre-makeup facial image in content CT11.

(Version 3: Limitations of the facial image before makeup)

In the above embodiment, the case shown is that the facial images before makeup, such as facial image IM21 and facial image IM22, are images of the publisher's bare face, but this is not limited to this example. For example, the facial images before makeup, such as facial image IM21 and facial image IM22, can be any facial images before makeup, such as facial image IM11 and facial image IM12, or they can be images of the face after some degree of makeup, such as lipstick and foundation. In this case, the information processing device 100 can generate makeup information such as makeup information for the user's facial image that does not use the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22, or makeup information for the user's facial image that uses the cosmetics used in the facial images before makeup, such as facial image IM21 and facial image IM22. Furthermore, since the facial images before makeup, such as facial image IM21 and facial image IM22, are used for bone learning after generating makeup information, appropriate makeup information can be generated even if the facial image is not a bare face. Therefore, the publisher's pre-makeup facial image only needs to be registered once in the service that manages the publisher's facial images. Furthermore, the description of the cosmetics used in the pre-makeup facial images such as facial image IM21 and facial image IM22 may or may not be included in content CT1.

(Version 4: Automatic extraction of item information)

In the above embodiments, for example, the cosmetic used by the publisher P11 in the makeup application related to the facial image IM11 can be determined in advance from the facial image IM11 and facial image IM21 through image analysis, and the cosmetic is stored in association with the facial image IM11 and facial image IM21 in an external information processing device or this device. Alternatively, the cosmetic can be determined based on the publisher P11's purchase history of cosmetics purchased from a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Alternatively, the publisher P11 can select from a list of candidates based on the cosmetic purchase history to determine the cosmetic, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21. Furthermore, the cosmetic can be determined by the publisher P11's registration of cosmetics not sold in a designated online store, and the cosmetic can be stored in association with the facial image IM11 and facial image IM21.

(Version 5: Reflecting the order of makeup application)

In the above embodiment, the information processing device 100 can also generate makeup information reflecting the makeup order by determining the makeup order performed by the publisher P11 in relation to the facial image IM11. In this case, the information processing device 100 can determine the makeup order by receiving the makeup order input from the publisher P11, and in the case of a makeup animation image related to the facial image IM11, it can also determine the makeup order based on the makeup animation image. Alternatively, the information processing device 100 can determine the makeup order by learning from multiple makeup animation images. Furthermore, in the case of a makeup animation image, the information processing device 100 can generate makeup information according to each category of cosmetics, thus allowing for the selection of applicable facial images. Additionally, explanatory content reflecting the makeup order can also be included in the content CT1. Therefore, the information processing device 100 can provide appropriate suggestions, including the makeup order.

(Processing Variation 6: Prioritized Display of Facial Images 1)

In the above embodiments, the facial images displayed in content CT1 can be selected based on any criterion or according to any priority. For example, facial images published by publishers followed by user U11 can be prioritized, or the more followers a publisher has, the higher the priority of displaying the publisher's facial images. Alternatively, the selection or display can be prioritized based on user U11's facial information or other body information.

Here, examples of facial information include the shape, size, and arrangement of the components that make up the face; the color and contour of the face; and the color and size of the pupils. Additionally, facial color includes, for example, the color of a bare face. Examples include, for instance, the average color of the entire measurement area excluding the lips; the average color of the forehead area; the average color of the area under the left eye; the average color of the area under the right eye; the average color of the left cheek area; the average color of the right cheek area; the average color of the nose area; the average color of the upper lip; the average color of the lower lip; and the average color of the chin area. Furthermore, examples of other body information besides facial information include, for instance, hair (hairstyle, hair texture, hair color, accessories, etc.); skin (skin tone, skin texture, wrinkles, etc.); and body shape (the size of the components, overall balance).

In the above embodiments, priority can be increased by prioritizing the publisher's facial image in order to be selected or displayed more preferentially, based on the similarity of the publisher's body information to that of user U11. For example, the more similar or identical the publisher's facial and other body information is to user U11, the higher the priority of the publisher's facial image. Alternatively, the more similar or identical the publisher's age, skin age, and skin concerns are to user U11, the higher the priority of the publisher's facial image. Similarly, the more similar or identical the publisher's concerns about makeup are to user U11, the higher the priority of the publisher's facial image. Furthermore, the more similar or identical the publisher's hairstyle, hair color, and hair preferences are to user U11, the higher the priority of the publisher's facial image. Conversely, priority can be decreased by prioritizing the publisher's facial image in order to be less preferentially selected or displayed, based on the disparity between the publisher's body information and that of user U11. For example, the more different the balance of facial features is from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the facial contours are from user U11, the lower the priority of the publisher's facial image. Alternatively, for example, the more different the bare-faced facial color is from user U11, the lower the priority of the publisher's facial image.

Alternatively, priority can be given to selecting or displaying facial images based on indicators of their popularity, such as user browsing history, purchase history, and review history. Specifically, priority can be given based on user browsing history of facial images selected by multiple users, user purchase history generated through customer guidance provided by content offered via facial image selection, user review history when applying makeup using facial images, and user review history of facial images or purchased cosmetics after purchase. For example, higher user browsing history, purchase history, and review history, indicating the popularity of a facial image, can result in higher priority for that image. Alternatively, priority can also be given based on the total price of the complete set of cosmetics used by the user in the makeup application. For example, a higher total price of the complete set of cosmetics used by the user in the makeup application can either result in higher or lower priority for the facial image.

(Processing Variation 7: Prioritized Display of Facial Images 2)

In the above embodiments, the selection or display of cosmetics can also be prioritized based on the user U11's purchase history. For example, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the publisher's facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are included in the publisher's purchase history, the higher the priority of the publisher's facial image. For example, the priority of a publisher's facial image that includes both cosmetics PR2 and PR3 in their purchase history may be higher than that of a publisher whose purchase history only includes cosmetic PR1. Furthermore, the more similar or identical the cosmetics are to those currently owned by the user U11, the higher the priority of the facial image. For example, if the user U11's purchase history includes cosmetics PR1, PR2, and PR3, the more cosmetics PR1, PR2, and PR3 are used in makeup, the higher the priority of the facial image after using the cosmetics.

Figure 3 is an illustration of information processing where the higher the priority of a facial image, the more identical or similar it is to the cosmetics available to user U11. This priority is based on user U11's cosmetic purchase history. Facial images IM11 and IM12 are facial images published by publisher P11, while IM12 is a facial image published by publisher P12. Furthermore, the cosmetics used in the makeup associated with facial image IM11 are cosmetics PR1, PR2, and PR3; those used in the makeup associated with facial image IM11 are cosmetics PR1, PR4, and PR5; and those used in the makeup associated with facial image IM12 are cosmetics PR2, PR6, PR7, and PR8. In the user U11's purchase history, which includes cosmetics PR1, PR2, PR3, and PR6, the facial image IM11, which uses the common cosmetics PR1, PR2, and PR3, has the highest priority; the facial image IM111, which uses the common cosmetics PR2 and PR6, has a medium priority; and the facial image IM12, which uses the common cosmetics PR1, has the lowest priority.

(Processing Variation 8: Acquisition of Color Information)

In the above embodiment, skin color, such as facial color, can be measured by measuring equipment such as ZOZOGLASS (registered trademark), which can be used by the user U11 for measurement. By accurately measuring skin color after light correction, the accuracy of the makeup information generated by the information processing device 100 can be improved.

[3. Structure of the user terminal]

Next, the structure of the user terminal 10 according to the embodiment will be described using FIG4. FIG4 is a diagram showing an example of the structure of the user terminal 10 according to the embodiment. As shown in FIG4, the user terminal 10 includes a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

(Communications Department 11)

The communication unit 11 is implemented, for example, by a NIC (Network Interface Card). Furthermore, the communication unit 11 establishes a wired or wireless connection with a designated network N, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 12)

The input unit 12 accepts various operations from the user. In Figure 2, it accepts various operations from the user U11. For example, the input unit 12 can also accept various operations from the user via the display surface through the touch panel function. In addition, the input unit 12 can also accept various operations from buttons provided on the user terminal 10, keyboards connected to the user terminal 10, and mice.

(Output Section 13)

The output unit 13 is, for example, a display screen of a flat panel terminal implemented with a liquid crystal display, an organic EL (electro-Luminescence) display, etc., and is a display device used to display various information. For example, the output unit 13 displays information sent from the information processing device 100.

(Control Unit 14)

The control unit 14 is, for example, a controller, implemented by a CPU (Central Processing Unit), MPU (Micro Processing Unit), or similar device that uses RAM (Random Access Memory) as its operating area to execute various programs stored in the internal storage device of the user terminal 10. These programs include, for example, programs for applications installed on the user terminal 10, such as programs for displaying content containing facial images sent from the information processing device 100. Furthermore, the control unit 14 is implemented using integrated circuits such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

As shown in Figure 4, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and performs the information processing described below.

(Receiving Unit 141)

The receiving unit 141 receives various types of information from other information processing devices such as the information processing device 100. For example, the receiving unit 141 receives information for displaying a user's facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user, as well as instructions on the cosmetics used in the makeup.

(Sending Department 142)

The sending unit 142 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends information about a facial image selected by the user to confirm whether the publisher's makeup suits them.

[4. Structure of the publisher's terminal]

Next, the structure of the publisher terminal 20 according to the embodiment will be described using FIG5. FIG5 is a diagram showing an example of the structure of the publisher terminal 20 according to the embodiment. As shown in FIG5, the publisher terminal 20 includes a communication unit 21, an input unit 22, an output unit 23, and a control unit 24.

(Communications Department 21)

The communication unit 21 is implemented, for example, by a NIC. Furthermore, the communication unit 21 is connected to a designated network N via wired or wireless means, and transmits and receives information with the information processing device 100 and the like via the designated network N.

(Input Section 22)

The input unit 22 accepts various operations from the publisher. In Figure 2, it accepts various operations from publishers P11 and P12. For example, the input unit 22 can also accept various operations from the publisher via the display surface through the touch panel function. In addition, the input unit 22 can also accept various operations from buttons provided on the publisher terminal 20, keyboards connected to the publisher terminal 20, and mice.

(Output Section 23)

The output unit 23 is, for example, a display screen of a flat panel terminal implemented by a liquid crystal display, an organic EL display, or the like, and is a display device used to display various information. For example, the output unit 23 displays information sent from the information processing device 100.

(Control Department 24)

The control unit 24 is, for example, a controller, implemented by executing various programs stored in the internal storage device of the publisher terminal 20 using RAM as the operating area, such as a CPU or MPU. These various programs include, for example, programs for applications installed on the publisher terminal 20. For example, these programs include programs for applications that send facial images of the publisher based on the publisher's actions. Furthermore, the control unit 14 is implemented using integrated circuits such as ASICs or FPGAs.

As shown in Figure 5, the control unit 24 has a receiving unit 241 and a transmitting unit 242, and performs the information processing described below.

(Receiving Unit 241)

The receiving unit 241 receives various types of information from the information processing device 100 and other information processing devices. For example, if a user who has viewed a facial image provided by a publisher and applied makeup to their own facial image purchases cosmetics corresponding to the applied makeup from a designated online store, the receiving unit 241 receives information related to the remuneration that the publisher can collect based on the user's purchase. Furthermore, the receiving unit 241 may also receive information for the publisher to collect remuneration corresponding to the user's purchase.

(Department 242)

The sending unit 242 sends various types of information to other information processing devices such as the information processing device 100. For example, the sending unit 142 sends a facial image of the publisher based on the publisher's operation.

(5. Configuration of information processing devices)

Next, the structure of the information processing apparatus 100 according to the embodiment will be described using FIG6. FIG6 is a diagram showing an example of the structure of the information processing apparatus 100 according to the embodiment. As shown in FIG6, the information processing apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. In addition, the information processing apparatus 100 may also include an input unit (e.g., a keyboard, mouse, etc.) for receiving various operations from the administrator of the information processing apparatus 100, and a display unit (e.g., a liquid crystal display, etc.) for displaying various information.

(Communication Department 110)

The communication unit 110 is implemented, for example, by a NIC. Furthermore, the communication unit 110 is connected to the network N via a wired or wireless connection, and transmits and receives information with the user terminal 10 via the network N.

(Storage Department 120)

The storage unit 120 is implemented, for example, by semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks and optical disks. As shown in FIG6, the storage unit 120 includes a publisher information storage unit 121 and a cosmetics information storage unit 122.

The publisher information storage unit 121 stores the publisher's facial image. Here, an example of the publisher information storage unit 121 according to the embodiment is shown in FIG7. As shown in FIG7, the publisher information storage unit 121 has items such as "publisher ID" and "facial image".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a bare-faced facial image of the publisher registered upon initial use. The example shown in Figure 7 illustrates a case where conceptual information such as "Facial Image #11" and "Facial Image #12" is stored for "Facial Image," but in reality, image data is stored. Additionally, for "Facial Image," the URL containing the image data and the file path indicating the storage location may also be stored.

That is, Figure 7 shows an example of a bare-faced facial image of the publisher identified according to the publisher "P11" as "Face Image #11".

The cosmetics information storage unit 122 stores information about the cosmetics used by the publisher in applying makeup. As shown in Figure 8, the cosmetics information storage unit 122 has items such as "Publisher ID", "Facial Image", and "Complete Set of Cosmetics".

"Publisher ID" represents identification information used to identify the publisher. "Facial Image" represents a facial image published by the publisher. The example shown in Figure 8 illustrates a conceptual information storage term such as "Facial Image #21" and "Facial Image #22" for "Facial Image," but in reality, it stores image data. Additionally, "Facial Image" could also store, for example, the URL containing the image data, the file path indicating the storage location, etc. "Full Makeup Set" represents the complete set of cosmetics used by the publisher in their makeup routine.

That is, in Figure 8, the following example is shown: the facial image posted by the publisher identified according to the publisher "P11" is "Facial Image #21", and the full set of cosmetics used by the publisher in applying makeup is "Brand 〇〇 Product A1 (lipstick), Brand ×× Product A2 (foundation), ...". In addition, the categories of cosmetics are indicated in parentheses.

(Control Department 130)

The control unit 130 is a controller, which is implemented, for example, by executing various programs stored in the storage device inside the information processing device 100 using RAM as the operating area by a CPU, MPU, or the like. Alternatively, the control unit 130 is a controller, which is implemented, for example, by an integrated circuit such as an ASIC or FPGA.

As shown in Figure 6, the control unit 130 includes an acquisition unit 131, a generation unit 132, an application unit 133, a provision unit 134, and a decision unit 135, and performs the information processing described below. Furthermore, the internal structure of the control unit 130 is not limited to the structure shown in Figure 6; it can be any structure that performs the information processing described later, or any other structure.

(Acquisition Department 131)

The acquisition unit 131 acquires various information from external information processing devices. The acquisition unit 131 also acquires various information from other information processing devices such as the user terminal 10.

The acquisition unit 131 acquires various information from the storage unit 120. Furthermore, the acquisition unit 131 stores the acquired information in the storage unit 120.

The acquisition unit 131 acquires information about facial images selected by users from facial images published by publishers in the prescribed service.

The acquisition unit 131 acquires information about the cosmetics used by the publisher in relation to the facial image selected by the user.

(Generation Department 132)

Based on the information acquired by the acquisition unit 131, the generation unit 132 generates makeup information using methods such as PSGAN. This makeup information is used to generate a facial image after applying the same makeup as the publisher's makeup used in the facial image selected by the user. Alternatively, if the makeup order in relation to the facial image selected by the user and the order in which the publisher applied the makeup is determined, the generation unit 132 generates makeup information reflecting the makeup order.

The generation unit 132 generates information for displaying a facial image containing the makeup information applied by the application unit 133 (described later) and a description of the cosmetics determined based on the information obtained by the acquisition unit 131.

(Application Department 133)

The application unit 133 applies makeup information generated by the generation unit 132 to the facial image of the user selected as the application object, thereby generating a facial image that is displayed on the user's terminal 10.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed according to priority. The priority is determined based on the user's facial information, body information, cosmetic purchase history, etc. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected or displayed with priority. The priority is determined based on indicators representing the popularity of facial images, such as access history, purchase history, and evaluation history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Application unit 133 generates a facial image displayed on user terminal 10 based on makeup information generated from facial images selected from those selected or displayed with priority. The priority is determined based on the total price of the cosmetics used by the publisher in the makeup. The higher the priority of the facial image, the more preferentially it is selected or displayed.

(Provide Department 134)

The providing unit 134 sends information generated by the generating unit 132 for displaying content to the user terminal 10. When the user terminal 10 receives the information sent by the providing unit 134, it displays the content corresponding to the information generated by the generating unit 132.

(Decision Section 135)

The decision unit 135 determines whether the user has purchased cosmetics associated with the content provided by the provision unit 134. If it determines that the user has purchased cosmetics, it decides to award the corresponding reward to the publisher who published the facial image selected by the user. Then, the provision unit 134 performs the processing for awarding the corresponding reward to the publisher. Alternatively, the decision unit 135 may also decide to send information for the publisher to receive the corresponding reward. In this case, the provision unit 134 sends the information for the publisher to receive the corresponding reward. Alternatively, the decision unit 135 may also decide to transfer the reward to the publisher's account. In this case, the provision unit 134 performs the processing for transferring the reward to the publisher's account.

(6. Information processing flow)

Next, FIG9 will be used to explain the information processing process performed by the information processing system 1 according to the embodiment. FIG9 is a flowchart showing the information processing process performed by the information processing system 1 according to the embodiment.

As shown in Figure 9, the information processing device 100 acquires information about the facial image selected by the user from facial images published by the publisher in the specified service (step S201).

The information processing device 100 generates makeup information based on the acquired information, which is used to generate makeup that is the same as the publisher's makeup used in the facial image selected by the user (step S202).

The information processing device 100 generates a facial image to be provided to the user by applying the generated makeup information to the facial image of the user selected as the application object (step S203).

The information processing device 100 sends information for displaying content including the generated facial image and descriptions of the cosmetics used by the publisher in the makeup (step S204).

[7. Variations]

The information processing system 1 described above can be implemented in various other ways besides the embodiments described above. Therefore, other embodiments of the information processing system 1 will be described below.

In the above embodiment, a case is shown where the user's facial image after applying makeup information is displayed on the user's terminal 10, but this is not limited to this example. For example, it could also be displayed on the terminal devices of other people communicating online with the user. For example, when a user selects a publisher's facial image to participate in an online meeting, makeup related to the selected facial image could be virtually applied to the user's face, and the facial image after applying the makeup related to the facial image selected by the user could be displayed on the terminal devices of other people. Furthermore, when a user uses the publisher's makeup to participate in a meeting, the user could be charged based on usage time, frequency, and other usage conditions. Moreover, a process could be implemented to give a portion of the fee charged to the user as compensation to the publisher. Thus, since the user's made-up face is displayed to others online, the user can participate in meetings with a natural face.

In this case, the application unit 133 applies makeup information generated by the generation unit 132 to the user's facial image captured by the camera in real time, thereby generating a facial image to be displayed on other people's terminal devices in real time. The generation unit 132 uses the facial image generated by the application unit 133 to generate information in real time for displaying the facial image after applying makeup information to the user's facial image on other people's terminal devices. The providing unit 134 transmits the facial image after applying makeup information via the application unit 133 to other people communicating online in real time with the user by sending the information generated by the generation unit 132. The decision unit 135 determines the reward given to the publisher of the facial image selected by the user based on the online usage of the facial image generated by the application unit 133 (e.g., usage time, usage frequency). In other words, the decision unit 135 determines the reward given to the publisher of the facial image that serves as the source of the makeup information based on the usage of the facial image after applying makeup information to the user's facial image. Furthermore, applications can also be performed on the user's cloud, in which case the providing unit 134 may, for example, only provide the right to use the makeup information. Moreover, the user can also exercise the provided right to use the information to display the facial image of the user after applying the makeup information to their facial image on other people's terminal devices.

[8. Effects]

As described above, the information processing apparatus 100 according to the embodiment includes a generation unit 132 and a determination unit 135. The generation unit 132 generates makeup information based on a facial image of a publisher who has applied prescribed makeup. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image. The determination unit 135 determines the reward to be given to the publisher who published the facial image that serves as the source of the makeup information, based on the utilization status of the facial image after applying the makeup information to the user's facial image.

Therefore, the information processing device 100 according to the embodiment can appropriately meet the user's need to participate in online meetings, etc., by using a facial image of themselves with other people's makeup applied.

In addition, the decision unit 135 determines the compensation based on the usage time or frequency of the user's facial image after applying makeup information to it.

Therefore, the information processing apparatus 100 according to the embodiment can appropriately encourage the publisher to publish facial images.

In addition, the application unit 133 applies makeup information generated based on a facial image selected from facial images that are preferentially selected or displayed based on prescribed criteria.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on a predetermined benchmark, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority. The priority is determined based on at least one of the user's facial information, body information, and item purchase history. The higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing device 100 according to the embodiment can provide appropriate information to the user based on facial information, body information, purchase history of items, etc., and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, wherein the priority is determined based on at least one of the facial image's access history, purchase history, and review history, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on access history, purchase history, evaluation history, etc. of facial images, and thus can more appropriately meet the user's needs.

In addition, the application unit 133 applies makeup information generated from facial images selected or displayed based on priority, the priority being determined according to the total price of the item, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

Therefore, the information processing apparatus 100 according to the embodiment can provide appropriate information to the user based on the total price of the item, and thus can more appropriately meet the user's needs.

[9. Hardware Structure]

Furthermore, the user terminal 10, publisher terminal 20, and information processing device 100 involved in the above-described embodiments can be implemented, for example, by a computer 1000 with the structure shown in FIG10. FIG10 is a hardware structure diagram showing an example of a computer that implements the functions of the user terminal 10, publisher terminal 20, and information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

The CPU 1100 operates based on the programs stored in the ROM 1300 or HDD 1400 to control various components. The ROM 1300 stores the startup program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 stores the program executed by CPU 1100 and the data used by that program. Communication interface 1500 obtains data from other devices and sends it to CPU 1100 via a specified communication network, and sends data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as monitors and printers, as well as input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. In addition, the CPU 1100 outputs the generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads the program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 into the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as DVD (Digital Versatile Disc), PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as MO (Magneto-Optical Disk), magnetic tape, magnetic recording media, or semiconductor memory.

For example, when the computer 1000 functions as the terminal device 10, publisher terminal 20, and information processing device 100 in the implementation, the CPU 1100 of the computer 1000 implements the functions of the control units 14, 24, and 130 by executing programs loaded into the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800; as another example, these programs can also be obtained from other devices via a defined communication network.

[10. Others]

Furthermore, all or part of the processes described in the above embodiments that are intended to be performed automatically can also be performed manually, or all or part of the processes described as intended to be performed manually can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, various data, and parameters shown in the foregoing and accompanying drawings can be arbitrarily changed unless otherwise stated. For example, the various information shown in the figures is not limited to the information illustrated.

Furthermore, the constituent elements of each device illustrated are functional concepts and do not need to be physically configured as shown in the illustrations. That is, the specific way in which the devices are distributed/integrated is not limited to the way illustrated, and they can be configured to be distributed/integrated functionally or physically in any unit according to various loads, usage conditions, etc.

Furthermore, the above-described implementation methods can be appropriately combined within the scope of not contradicting the processing content.

Several embodiments of this application have been described in detail above with reference to the accompanying drawings. However, these are illustrative, and the invention can be implemented in other ways by means of various modifications and alterations based on the knowledge of those skilled in the art, based on the manner described in the Summary of the Invention section.

Furthermore, the terms "section, module, unit" mentioned above can be replaced with "unit," "circuit," etc. For example, "acquisition section" can be replaced with "acquisition unit" or "acquisition circuit."

Explanation of reference numerals in the attached figures

1: Information processing system; 10: User terminal; 11: Communication unit; 12: Input unit; 13: Output unit; 14: Control unit; 20: Publisher terminal; 21: Communication unit; 22: Input unit; 23: Output unit; 24: Control unit; 100: Information processing device; 110: Communication unit; 120: Storage unit; 121: Publisher information storage unit; 122: Cosmetic information storage unit; 130: Control unit; 131: Acquisition unit; 132: Generation unit; 133: Application unit; 134: Provision unit; 135: Decision unit; 141: Receiving unit; 142: Transmitting unit; 241: Receiving unit; 242: Transmitting unit; N: Network.

Claims (9)

1. An information processing device, characterized in that it comprises:

The generation unit generates makeup information based on a facial image of a publisher who has applied prescribed makeup. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image.

The decision-making department determines the reward for publishing the facial image that is the source of the makeup information, based on how the facial image is used after applying makeup information to the user's facial image.

2. The information processing device according to claim 1, characterized in that,

The decision-making unit determines the reward based on the usage time or frequency of the user's facial image after applying makeup information to it.

3. The information processing apparatus according to claim 1 or 2, characterized in that,

It also has an application unit that applies makeup information generated based on a facial image selected by the user from facial images published by a publisher that meets specified conditions to the user's facial image.

4. The information processing apparatus according to claim 3, characterized in that,

The application unit uses the makeup information generated based on a facial image selected from facial images that are preferentially selected or displayed based on prescribed criteria.

5. The information processing apparatus according to claim 4, characterized in that,

The application unit generates the makeup information based on facial images selected from those selected or displayed with priority, the priority being determined based on at least one of the user's facial information, body information, and item purchase history, with higher priority facial images being selected or displayed more frequently.

6. The information processing apparatus according to claim 4, characterized in that,

The application unit generates the makeup information based on facial images selected from those selected or displayed with priority, the priority being determined based on at least one of the facial image's access history, purchase history, and review history, with higher priority facial images being selected or displayed more frequently.

7. The information processing apparatus according to claim 4, characterized in that,

The application unit generates the makeup information based on facial images selected from those selected or displayed with priority, the priority being determined based on the total price of the item, and the higher the priority of the facial image, the more preferentially it is selected or displayed.

8. An information processing method, executed by a computer, characterized in that it includes:

The generation process involves generating makeup information based on a publisher's facial image with prescribed makeup applied. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image.

The process involves determining the compensation awarded to the publisher of the facial image that was the source of the makeup information, based on how the facial image is used after applying makeup information to the user's facial image.

9. An information processing program, characterized in that it causes a computer to perform the following process:

The generation process generates makeup information based on a publisher's facial image with prescribed makeup applied. This makeup information is used to generate a facial image after applying the same prescribed makeup to the facial image.

The decision-making process determines the compensation awarded to the publisher of the facial image that is the source of the makeup information, based on how the facial image is used after applying makeup information to the user's facial image.