FR3125612A1 - DIGITAL MAKEUP PALETTE - Google Patents

Abstract

DIGITAL MAKEUP PALETTE An augmented reality system for makeup includes a makeup lens unit having computational circuitry operatively coupled to a graphical user interface configured to generate one or more instances of selectable makeup lenses by the user and for receiving makeup lens information selected by the user, a makeup palette unit operatively coupled to the makeup lens unit, the makeup palette unit including computing circuitry configured to generate at least one digital makeup palette for a digital makeup product, and a makeup lens display unit including computing circuitry configured to analyze a user's face to determine one or more of the face shape, facial features, skin tone, color of hair wishes, eye color, lip shape, eyelid shape, hairstyle and lighting, and automatically create one or more instances of a personalized virtual try-on for a user in accordance with the makeup goal information selected by the user and the at least one digital makeup palette generated based on the analysis of the user's face. Figure for abstract: 1

Description

DIGITAL MAKEUP PALETTE

The present disclosure relates to a digital makeup palette and a method for a personalized augmented reality experience using the digital makeup palette.

CONTEXT

Smartphones having front cameras offer the possibility of taking photos and videos of the person having the camera at hand so that the person can view the image to be captured. Various mobile applications, also known as "apps", have been developed to use the front cameras. A common app is one that allows you to take a self-portrait photo, called a selfie, and insert the self-portrait into a social media context or forward it to share with others via email or text.

Some cosmetics companies have started developing apps that provide help in selecting cosmetics. Apps can provide tools to search for particular types of makeup, or to search for a product that might be the user's favorite, or simply to purchase a product that has already been used. Some apps offer tutorials on how to apply certain types of makeup. Some apps help choose lipstick or eye shadow colors by displaying color palettes. Some apps offer color matching features to help find a color that matches an item of clothing, an accessory, or the color from a photo.

Some cosmetics companies have started using the cameras of smart phones, tablets and laptops by offering product try-on apps. Some of these applications are implemented as web applications, or an app. These try-on apps work by taking a self-portrait photo with the smartphone camera, uploading the photo to the web app, and then applying virtual makeup products on the uploaded image. These try-on apps can offer a variety of options, such as skin smoothing, cheekbone lift, eye color adjustment. These try-on apps can give the user the ability to add any type and color of makeup product, as well as change the intensity of the color.

However, the try-on apps offered so far tend to style through photo editing tools. Some of the earlier try-on apps start from an uploaded photograph and offer one-step functions to apply types and colors of makeup, and then allow editing of the makeup photo. Such tools do not capture a personal makeup experience. Similarly, earlier try-on tools do not allow for the creation of custom styles. For example, a user might want a Friday night date style. Earlier try-on apps may offer a Friday night date style, but that style may not be what the user had in mind. The tools provided can be used to make additional edits in an attempt to achieve a style that matches what the user believes is a Friday night date style. However, such an approach is limited by the peculiarities of the editing tools. A user may want a Friday night date style that is based on their mood, or a mood the user may want to portray, which may require extensive editing.

These earlier try-on web or app apps lack full customization, in part because earlier virtual try-on apps are limited to certain brands of physical makeup products. A user may not be able to create the style they have in mind, or want to experiment with. It is necessary to provide a personalized fitting experience for a particular user, which allows them to have fun trying on makeup and not be disappointed by an unrealistic appearance.

Also, a user may have problem areas, such as redness, scar, age spots, hyperpigmentation, etc. that he wishes to treat with makeup. A user may also wish to highlight certain features of the face, such as the cheekbones, the eyes, the lips. There is a need to provide a personalized fitting experience that can help address particular problem areas or the best features of the particular user's face.

Alternatively, distributing special formula customization machines is time-consuming and expensive because a sampling system would have to be added to the machine to deliver a small dose of product for the consumer to try and then change the product recipe based on the bias measured on a consumer's skin.

In addition, e-commerce personalization fitting services are not adaptable to end consumers with smart phones, as the variation in problem areas of each consumer's skin and facial features is too great.

The above description of “background” is intended to provide a general background to the disclosure. The work of the inventors, to the extent described in this background section, and aspects of the description which may not qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art. prior to the present invention.

SUMMARY

An augmented reality system for makeup includes a makeup lens unit having computational circuitry operatively coupled to a graphical user interface configured to generate one or more instances of user-selectable makeup lenses and to receive information makeup lens units selected by the user, a makeup palette unit operatively coupled to the makeup lens unit, the makeup palette unit including computing circuitry configured to generate at least one makeup palette for a digital makeup product, and a makeup lens display unit comprising computing circuitry configured to analyze a user's face to determine one or more of facial shape, facial landmarks, skin tone, hair color, eye color, lip shape, shape e of eyelids, hairstyle and lighting, and automatically creating one or more instances of a personalized virtual try-on for a user in accordance with the user-selected makeup goal information and the at least one digital makeup generated based on the analysis of the user's face.

The foregoing general description of the illustrated embodiments and their following detailed description are merely exemplary aspects of the teachings of this disclosure, and are not limiting.

A fuller appreciation of this disclosure and its many attendant benefits will be readily obtained when it is better understood by reference to the following detailed description, taken in conjunction with the accompanying drawings, in which:

there is a diagram of a system in accordance with an example aspect of the disclosure;

there is a block diagram of a computer system for a mobile device;

[Fig. 3A, 3B] Figures 3A, 3B are a flowchart of a method of creating a custom style, where the is a process in which a user creates their own custom style, and the is a method in which the custom style is created by a mobile application in accordance with an example aspect of the disclosure;

there is an example user interface for choosing between user-created and style-created in accordance with an example aspect of the disclosure;

there is a flowchart of a method of obtaining a digital makeup palette in accordance with an example aspect of the disclosure;

there illustrates an exemplary digital makeup palette in accordance with an exemplary aspect of the disclosure;

there illustrates an example of digital makeup in accordance with an example aspect of the disclosure;

there is a flowchart of the more detailed face analysis step in accordance with an example aspect of the disclosure;

there is a block diagram of a CNN for classifying a face shape;

there is a diagram of a deep learning neural network for facial cue detection;

there is a sample user interface for selecting a virtual product to apply;

there is an example of a user interface for choosing between applying makeup by the user and recommending how to apply makeup;

there is an example mobile application in accordance with an example aspect of the disclosure;

there is a schema for a recommender system;

there illustrates an example of a style-makeup matrix for the recommendation system of the ;

there illustrates a blending process that can be used to create a face image based on a desired feature and an original feature;

there is a flowchart for a step of applying virtual makeup in accordance with an exemplary aspect of the disclosure;

there is a flowchart of a step for recording areas and scans during makeup application;

there is a flowchart of a step for analyzing a user's makeup application steps to estimate problem areas or best features;

there is an example user interface for storing a makeup style in accordance with an example aspect of the disclosure;

there is a flowchart of a method of custom applying a digital palette in accordance with an exemplary aspect of the disclosure;

there is a sample UI showing the status of a custom makeup application;

there is a flowchart for a method of selecting makeup filters in accordance with an exemplary aspect of the disclosure;

there is a sample UI for saving makeup styles.

there is a block diagram of a reinforcement learning architecture; And

there is a flow diagram of a machine learning model in accordance with an example aspect of the disclosure.

In the following detailed description, reference is made to the accompanying drawings, which form part of this document. In the drawings, like symbols typically identify like components, unless the context indicates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be used, and other modifications may be made, without departing from the spirit or scope of the subject matter presented herein.

Claims (10)

Augmented reality system for make-up, including:
a makeup lens unit having computational circuitry operatively coupled to a graphical user interface configured to generate one or more instances of user-selectable makeup lenses and to receive makeup lens information selected by the user ;
a makeup palette unit operatively coupled to the makeup lens unit, the makeup palette unit including computing circuitry configured to generate at least one digital makeup palette for a digital makeup product; And
a makeup lens display unit having computational circuitry configured to analyze a user's face to determine one or more of facial shape, facial landmarks, skin tone, hair color, eye color, lip shape, eyelid shape, hairstyle and lighting, and automatically create one or more instances of a personalized virtual try-on for a user in accordance with the user-selected makeup goal information user and the at least one digital makeup palette generated based on the analysis of the user's face. The makeup augmented reality system of claim 1, wherein the makeup lens display unit computing circuitry is further configured to:
receiving one or more digital images of the user including at least a portion of the user's face, and
analyzing the user's face image to identify parts of the face. The makeup augmented reality system of claim 2, wherein the makeup lens display unit computing circuitry is further configured to:
correcting lighting in one or more of the digital images to brighten portions of the face. The makeup augmented reality system of claim 1, wherein the calculation circuitry of the makeup lens display unit is further configured to select at least one makeup filter based on the at least one resulting digital makeup palette, and creating the personalized virtual try-on for the user using the at least one makeup filter. The makeup augmented reality system of claim 1, wherein the computational circuitry of the makeup lens display unit is further configured to display the one or more instances of the personalized virtual try-on with a function to save the one or more instances in a memory. The makeup augmented reality system of claim 1, wherein the computational circuitry of the makeup lens display unit is further configured to store in memory the one or more instances of the personalized virtual try-on that are created based on a plurality of digital makeup palettes. The augmented reality makeup system of claim 1, wherein the computational circuitry of the makeup lens display unit is further configured to transfer the personalized virtual try-on to a platform providing live video. The augmented reality makeup system of claim 1, further comprising a recommender system that includes an indexed database containing a plurality of makeup filters,
in which the recommender system is configured to:
retrieving one or more makeup filters from the indexed database in response to a request to demonstrate how to apply digital makeup;
generating a sequence of video frames by mixing the face image with the one or more retrieved makeup filters; And
display the sequence of video frames. The augmented reality makeup system of claim 8, wherein the indexed database contains a plurality of makeup filter categories, and
wherein the recommender system is further configured to:
retrieving one or more makeup filters from the indexed database in response to a request to apply digital makeup for one of the plurality of makeup filter categories. The augmented reality makeup system of claim 9, wherein the makeup filter category is subculture makeup styles,
wherein the calculation circuitry of the makeup lens unit is configured to generate the subculture makeup styles as one or more instances of user-selectable makeup lenses,
wherein the computational circuitry of the makeup lens display unit is configured to automatically create one or more instances of a personalized virtual fitting for a user in accordance with the subculture makeup styles selected by the user.