WO2024058468A1

WO2024058468A1 - Display device and control method thereof

Info

Publication number: WO2024058468A1
Application number: PCT/KR2023/012885
Authority: WO
Inventors: 한지연; 김선혜; 이가희; 이교희
Original assignee: 삼성전자주식회사
Priority date: 2022-09-16
Filing date: 2023-08-30
Publication date: 2024-03-21

Abstract

Disclosed is a control method of a display device. This control method comprises the steps of: identifying a makeup style selected by a user; displaying a facial image of the user when the user's face is photographed through a camera; obtaining makeup information comprising color information, texture information, and highlight degree information for each area of the user's face by analyzing the facial image of the user; calculating, when an area of interest is selected from the facial image of the user, a makeup matching rate between the area of interest and the remaining areas on the basis of a matching database corresponding to the obtained makeup information and the identified makeup style; and displaying a makeup guide for at least one of the area of interest or the remaining areas on the basis of the calculated makeup matching rate.

Description

Display device and control method thereof

The present disclosure relates to a display device and a control method thereof, and more specifically, to a display device that provides a makeup guide and a control method thereof.

Recently, people are sharing their makeup and fashion with others through social platforms. In particular, in the case of makeup, many influencers are producing and distributing content in various styles, and people often follow the makeup method or style after seeing this.

However, it is very time-consuming to copy the influencer's content, and even if you follow it, you often end up with makeup failure because it does not suit you well. Therefore, there is a need for technology that provides guidance to achieve the desired makeup style that suits oneself.

According to an embodiment of the present disclosure, a method of controlling a display device including a camera includes identifying a makeup style selected by a user, displaying an image of the user's face when the user's face is captured through the camera. A step of analyzing the user's face image to obtain makeup information including color information, texture information, and emphasis level information for each area of the user's face. When a region of interest is selected from the user's face image, Calculating a makeup matching rate between the region of interest and another region of the user's face based on acquired makeup information and a matching database corresponding to the identified makeup style, and calculating a makeup matching rate between the region of interest and another region of the user's face based on the calculated makeup matching rate Displaying a makeup guide for at least one of a region of interest or the other region, wherein the matching database includes color information for each region of the face corresponding to the identified makeup style, the identification It includes a texture matching database including texture information of each area of the face corresponding to the identified makeup style, and an emphasis degree matching database including emphasis degree information of each region of the face corresponding to the identified makeup style.

In addition, the step of calculating the makeup matching rate includes color information and texture information of the other region included in the color matching database based on color information, texture information, and emphasis degree information of the region of interest included in the makeup information. and identifying emphasis degree information, respectively, and comparing the color information, texture information, and emphasis degree information of the identified other region with the color information, texture information, and emphasis degree information of the other region included in the makeup information. Thus, it may include calculating the color matching rate, texture matching rate, and emphasis matching rate of the other regions based on the region of interest, respectively.

In addition, displaying the makeup guide may include displaying a makeup guide for the color of the other area based on the color matching rate of the other area based on the area of interest, and displaying the makeup guide for the color of the other area based on the area of interest. Displaying a makeup guide for the texture of the other area based on the texture matching rate of the other area, and displaying a makeup guide for the texture of the other area based on the emphasis information matching rate of the other area based on the area of interest. It may include displaying a makeup guide.

In addition, the step of calculating the makeup matching rate includes color information and texture information of the region of interest included in the color matching database based on color information, texture information, and emphasis degree information of the other region included in the makeup information. and identifying emphasis degree information, respectively, and comparing color information, texture information, and emphasis degree information of the identified region of interest with color information, texture information, and emphasis degree information of the region of interest included in the makeup information. Thus, it may include calculating the color matching rate, texture matching rate, and emphasis matching rate of the region of interest based on the other region.

In addition, displaying the makeup guide may include displaying a makeup guide for a color of the region of interest based on a color matching rate of the region of interest based on the other region, Displaying a makeup guide for a texture of the region of interest based on a texture matching rate of the region of interest, and determining a degree of emphasis of the region of interest based on a matching rate of emphasis information of the region of interest based on the other regions. It may include displaying a makeup guide.

Additionally, the identifying step includes displaying a plurality of images corresponding to a plurality of makeup styles, and when one image among the plurality of images is selected, identifying a makeup style corresponding to the selected image. can do.

Additionally, the makeup guide may include at least one of an information message, a video, or a makeup product recommendation related to a makeup method to increase the calculated makeup matching rate.

In addition, it includes receiving user context information including at least one of information about the purpose of makeup, a place to visit, or information about products used for makeup in the area of interest, and the step of displaying the makeup guide includes the step of receiving the user's context information. The makeup guide can be displayed by reflecting the information.

Meanwhile, a display device according to an embodiment of the present disclosure identifies a display, a camera, and a makeup style selected by a user, and when the user's face is captured through the camera, the display device displays the user's face image. Controls and analyzes the user's face image to obtain makeup information including color information, texture information, and emphasis level information for each area of the user's face, and when a region of interest is selected from the user's face image, Based on the obtained makeup information and a matching database corresponding to the identified makeup style, a makeup matching rate between the area of interest and another area of the user's face is calculated, and the interest rate is calculated based on the calculated makeup matching rate. At least one processor to control the display to display a makeup guide for at least one of the region or the other region, wherein the matching database includes color information for each region of the face corresponding to the identified makeup style. a color matching database, a texture matching database including texture information of each region of the face corresponding to the identified makeup style, and an emphasis degree matching including emphasis degree information of each region of the face corresponding to the identified makeup style. Includes database.

In addition, the at least one processor is configured to store color information, texture information, and emphasis degree of the other region included in the color matching database based on color information, texture information, and emphasis degree information of the region of interest included in the makeup information. Each piece of information is identified, and the color information, texture information, and emphasis degree information of the identified other region are compared with the color information, texture information, and emphasis degree information of the other region included in the makeup information, respectively, to determine the region of interest. Based on , the color matching rate, texture matching rate, and emphasis degree matching rate of the different areas can be calculated, respectively.

In addition, the at least one processor displays a makeup guide for the color of the other area based on the color matching rate of the other area based on the area of interest, and displays a makeup guide for the color of the other area based on the area of interest. To display a makeup guide for the texture of the other area based on the texture matching rate, and to display a makeup guide for the degree of emphasis of the other area based on the emphasis information matching rate of the other area based on the region of interest. The display can be controlled.

In addition, the at least one processor is configured to store color information, texture information, and emphasis degree of the region of interest included in the color matching database based on color information, texture information, and emphasis degree information of the other region included in the makeup information. Each information is identified, and the color information, texture information, and emphasis degree information of the identified region of interest are compared with the color information, texture information, and emphasis degree information of the region of interest included in the makeup information, respectively, and the other region is Based on , the color matching rate, texture matching rate, and emphasis degree matching rate of the region of interest can be calculated, respectively.

In addition, the at least one processor displays a makeup guide for the color of the region of interest based on a color matching rate of the region of interest based on the other region, and displays a makeup guide for the color of the region of interest based on the other region. Display a makeup guide for the texture of the region of interest based on the texture matching rate, and display a makeup guide for the degree of emphasis of the region of interest based on the matching rate of emphasis information of the region of interest based on the other regions. The display can be controlled.

In addition, the at least one processor controls the display to display a plurality of images corresponding to a plurality of makeup styles, and when one image of the plurality of images is selected, identifies a makeup style corresponding to the selected image. can do.

In addition, when the user's context information including at least one of the purpose of makeup, the place to visit, or information about the product used for makeup in the area of interest is input, the at least one processor reflects the user's context information to apply the makeup The display can be controlled to display a guide.

Meanwhile, according to an embodiment of the present disclosure, in a non-transitory computer-readable recording medium that stores computer instructions that cause the display device to perform an operation when executed by a processor of the display device, the operation is performed by a user. Identifying the makeup style selected by the camera, displaying the user's face image when the user's face is captured by the camera, analyzing the user's face image to include color information, texture information, and emphasis level information. Obtaining makeup information for each area of the user's face, when a region of interest is selected from the face image of the user, based on the obtained makeup information and a matching database corresponding to the identified makeup style, calculating a makeup matching rate between the region of interest and another region of the user's face, and displaying a makeup guide for at least one of the region of interest or the other region based on the calculated makeup matching rate, The matching database includes a color matching database containing color information of each area of the face corresponding to the identified makeup style, a texture matching database containing texture information of each area of the face corresponding to the identified makeup style, and It includes an emphasis degree matching database that includes information on the degree of emphasis of each area of the face corresponding to the identified makeup style.

1 is a diagram for explaining the operation of a display device according to an embodiment of the present disclosure;

2 is a block diagram of a display device according to an embodiment of the present disclosure;

Figure 3 is an example of a makeup style recommendation video,

FIG. 4A is a diagram illustrating a screen displayed on a display device according to an embodiment of the present disclosure;

FIG. 4B is a diagram illustrating a screen displayed on a display device according to an embodiment of the present disclosure;

FIG. 4C is a diagram illustrating a screen displayed on a display device according to an embodiment of the present disclosure;

FIG. 5A is a diagram illustrating a screen displayed on a display device according to an embodiment of the present disclosure;

5B is a diagram illustrating a screen displayed on a display device according to an embodiment of the present disclosure;

6 is a block diagram of a display device according to an embodiment of the present disclosure, and

Figure 7 is a flowchart showing a method of controlling a display device according to an embodiment of the present disclosure.

Various embodiments of the present disclosure described below with reference to the drawings are not intended to limit the scope to specific embodiments, but are understood to include various modifications, equivalents, and/or alternatives. It has to be. In connection with the description of the drawings, similar reference numbers may be used for similar components.

In describing the present disclosure, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. Additionally, duplicate descriptions of the same configuration will be omitted as much as possible.

The suffix "part" for the components used in the following description is given or used interchangeably only considering the ease of preparing the specification, and does not have a distinct meaning or role in itself.

The terms used in this disclosure are used to describe embodiments and are not intended to limit and/or limit the present disclosure. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, terms such as 'include' or 'have' are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is referred to as being “directly connected” or “directly connected” to another component (e.g., a second component), it means that there is a connection between said component and said other component. It may be understood that other components (e.g., a third component) do not exist.

Unless otherwise defined, terms used in the embodiments of the present disclosure may be interpreted as meanings commonly known to those skilled in the art.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

1 is a diagram for explaining the operation of a display device according to an embodiment of the present disclosure.

According to FIG. 1, the display device 100 can recommend various makeup styles (11). For example, a makeup guide app may be installed on the display device 100 to perform functions according to various embodiments of the present disclosure, which will be described later, and the user may execute the makeup guide app.

When the makeup guide app is executed, the display device 100 allows the user to select, for example, various makeup style images (still images, videos) of influencers or images of the user stored in the display device 100. By displaying it so that various makeup styles can be recommended to the user.

When the user selects one makeup style (12), the display device 100 can run a camera app and allow the user to take a picture of his or her current makeup state (13). At this time, a guide UI (User Interface) may be displayed so that the user's face is captured.

When the user's current state is captured, the display device 100 may display the captured image and analyze the displayed image to obtain makeup information for each facial area of the user. At this time, the makeup information for each face area includes color information, texture information, and degree of emphasis for each of the user's face areas (e.g., forehead, eyes, eyebrows, nose, lips, cheeks, skin tone, etc.). It may include information, so it is not limited to this.

Meanwhile, the user can select an area of interest among facial areas in the displayed image. At this time, the display device 100 may display a guide UI that guides the user to select an area of interest.

When the user selects an area of interest based on the captured photo (14), the display device 100 is based on makeup information for each facial area obtained by analyzing the user's image and a matching database corresponding to the makeup style selected in the recommended image. Thus, the makeup matching rate between the area of interest and other areas of the face can be calculated.

At this time, the matching database may include a color matching database, a texture matching database, and an emphasis information matching database. The color matching database may include color information for each area of the face to implement the makeup style selected by the user. The texture matching database may include texture information for each area of the face to implement the makeup style selected by the user. The highlight information matching database may include texture information for each area of the face to implement the makeup style selected by the user.

For example, if a user applies eye makeup using a newly purchased eye makeup product and then wants to apply a “natural style” makeup to match the current eye makeup condition, the user can select the “natural style” corresponding to the recommended video. You can select an image and select “eyes” as the region of interest in the displayed image.

In this case, the display device 100 may analyze the displayed image to obtain current color information, texture information, and emphasis level information about the user's forehead, eyes, eyebrows, nose, lips, cheeks, skin tone, etc. Additionally, the display device 100 may obtain a matching database corresponding to the natural style.

Accordingly, the display device 100 can identify information about the colors of the remaining areas that match the color of the current user's eye area from the color matching database corresponding to the natural style. Additionally, the display device 100 may calculate the color matching rate for each of the remaining areas by comparing the current color information of the remaining areas with the color information of the remaining areas identified in the color matching database for each area. At this time, the calculated color matching rate indicates how much the current color of the remaining areas matches the current color of the eye area in order to implement a natural makeup style, so it is a standard for providing a makeup guide for the colors of the remaining areas. It can be.

Meanwhile, the display device 100 may identify information about the textures of the remaining areas that match the texture of the current user's eye area from a texture matching database corresponding to the natural style. Additionally, the display device 100 may calculate a texture matching rate for each of the remaining regions by comparing the current texture information of the remaining regions with the texture information of the remaining regions identified in the texture matching database for each region. At this time, the calculated texture matching rate indicates how much the current texture of the remaining areas matches the current texture of the eye area in order to implement a natural makeup style, so it is a standard for providing a makeup guide for the textures of the remaining areas. It can be.

Additionally, the display device 100 may identify the degree of emphasis of the color of the remaining areas that matches the degree of emphasis of the current user's eye region from the degree of emphasis matching database corresponding to the natural style. Additionally, the display device 100 may compare the current emphasis level information of the remaining areas with the emphasis level information of the remaining areas identified in the color matching database for each area, and calculate an emphasis level matching rate for each of the remaining areas. At this time, the calculated emphasis degree matching rate indicates how much the current emphasis degree of the remaining areas matches the current emphasis degree of the eye area in order to implement a natural makeup style, and thus provides a makeup guide for the degree of emphasis of the remaining areas. It can be a standard for doing so.

In this way, when the makeup matching rate between the region of interest and other areas of the face is calculated, the display device 100 may display a makeup guide related to a makeup method to increase the makeup matching rate.

In the above example, the user's current eye area color is "a", the lip area color is "b", but the lip area color that matches the eye area color "a" is "c", and the color matching database When identified, the display device 100 can calculate the color matching rate of the eye area and the lip area by comparing the color information of the eye area, which is the area of interest, and the lip area, which is another area, that is, "b" and "c." Accordingly, a makeup guide may be displayed that guides changing the color “b” of the lip area to “c” to implement a natural makeup style. Makeup guides for texture and level of emphasis could also be provided in a similar way.

At this time, depending on the embodiment, the makeup guide may be displayed in the form of a guide message in text form, a related video showing a makeup method, or a recommendation for a related makeup product.

According to an embodiment of the present disclosure as described above, the user selects a desired makeup style among recommended makeup styles and selects a region of interest among areas of the user's face, so that the desired makeup style is completed based on the user's current state. You can receive a makeup guide to help you achieve your goal. Accordingly, users can complete the makeup style they want more conveniently and accurately.

Figure 2 is a block diagram of a display device according to an embodiment of the present disclosure. According to FIG. 2, the display device 100 includes a camera 110, a processor 120, and a display 130.

The camera 110 may acquire an image by photographing the surrounding environment. An image may include a static image containing one frame and a dynamic image containing multiple frames. For example, the camera 110 may include, but is not limited to, an RGB camera, a depth camera, a wide-angle camera, a telephoto camera, etc.

The display 130 can display various visual information such as still images, moving images, user interfaces (UI), icons, etc. under the control of the processor 120. To this end, the display 170 may be implemented in various forms such as liquid crystal display panel (LCD), light emitting diode (LED), organic light emitting diode (OLED), micro LED, etc. Additionally, the display 130 may be combined with a touch panel and implemented in the form of a touch screen.

The processor 120 controls the overall operation of the display device 100. At this time, the processor 120 may include at least one processor 120. For example, the processor 120 may include one or more central processing units (CPU), application processors (AP), or communication processors (CP). .

In particular, the processor 120 may identify the makeup style selected by the user. For example, the processor 120 may display a plurality of images corresponding to a plurality of makeup styles. At this time, the plurality of images may be an influencer's makeup style video, a thumbnail of a video showing a makeup method, or a user's makeup image stored in the display device 100.

Accordingly, when one image is selected by the user, the processor 120 can identify the makeup style corresponding to the selected image. Specifically, when a plurality of images are displayed for each makeup style, when one image is selected by the user, the processor 120 can immediately identify which makeup style the selected image corresponds to. Alternatively, the processor 120 may analyze the image selected by the user and identify which makeup style the image corresponds to. At this time, various image recognition technologies or artificial intelligence models that are learned to output a makeup style for the input image when an image is input may be used. Since more specific details are unrelated to the gist of the present disclosure, description is omitted.

Additionally, according to one embodiment, the processor 120 displays a makeup style list including various makeup styles in text form, and when the user selects one makeup style from the list, the processor 120 can identify the selected makeup style. At this time, depending on the embodiment, it goes without saying that an image corresponding to the makeup style and text representing the makeup style may be displayed together.

Figure 3 is an example of a makeup style recommendation image displayed on the display 130. Referring to Figure 3, you can see that various makeup styles such as classic makeup, natural makeup, spring makeup, chic makeup, etc. are displayed along with related images. The user can select one of the displayed images and select a desired makeup style from among a plurality of makeup styles.

In this way, when a makeup style is selected, the processor 120 can control the camera 110 to capture the user's current state. For example, the processor 120 may display a guide UI to guide the user in taking a picture of the user's face, and execute a camera app to allow the camera 110 to take a picture of the user's current state.

When the user's face is captured through the camera 110, the processor 120 can control the display 130 to display the user's face image.

In the above, the case where the user's face is photographed after selecting a makeup style is given as an example, but the embodiment is not limited to this. That is, depending on the embodiment, the makeup style may be selected after photographing the user's face.

Accordingly, the processor 120 may analyze the displayed image of the user's face and obtain makeup information including color information, texture information, and emphasis level information. At this time, makeup information may be obtained for each region of the user's face.

Specifically, the processor 120 may use various image analysis technologies to identify each area of the face and obtain makeup information for each identified area. Alternatively, the processor 120 may obtain makeup information for each area of the face using an artificial intelligence model that has been learned to output makeup information for each area of the face when a face image is input. More detailed information regarding this will be omitted as detailed information is irrelevant to the gist of the present disclosure.

For example, color information may include information about the color of each area of the face, such as forehead, eyes, eyebrows, nose, lips, cheeks, skin tone, etc., but is not limited to this. Texture information may include information about the degree of matte, degree of shimmer, degree of glitter, and degree of gloss of each area of the face, such as forehead, eyes, eyebrows, nose, lips, cheeks, skin tone, etc., but is limited to this. It doesn't work. The emphasis level information may include, but is not limited to, information about the area, thickness, density, saturation, etc. of each area of the face such as the forehead, eyes, eyebrows, nose, lips, cheeks, skin tone, etc. For example, the emphasis level information includes the area, width, and concentration of eye makeup, the thickness of the eye line, the tail direction of the eye line, the lip line, information about the direction of the corner of the mouth, information about the gradation of the eyes, nose, and cheek areas, etc. can do.

Meanwhile, when an area of interest is selected from the user's face image, the processor 120 calculates a makeup matching rate between the area of interest and other areas of the user's face based on the matching database corresponding to the makeup information and the identified makeup style. You can.

For example, the processor 120 may control the display 130 to display a guide UI that guides selection of a region of interest from the displayed face image of the user. Accordingly, the user can select a region of interest from the displayed face image. However, it is not limited to this.

Once an area of interest is selected, processor 120 may calculate a makeup matching rate between the area of interest and other areas of the face. Specifically, the processor 120 may calculate the makeup matching rate between the region of interest and other areas of the face based on makeup information obtained by analyzing the user's face image and a matching database corresponding to the makeup style selected by the user. .

At this time, the matching database corresponding to the makeup style selected by the user may be obtained from the storage unit 150 of the display device 100 or an external server. According to one embodiment, the display device 100 may store a matching database for each makeup style. In this case, a matching database corresponding to the makeup style selected by the user can be obtained from the matching database for each makeup style stored in the processor 120. Or, according to one embodiment, the processor 120 transmits information about the makeup style selected by the user to an external server that stores a matching database for each makeup style, and transmits a matching database corresponding to the makeup style selected by the user from the external server. It can also be obtained by receiving. The matching database obtained in this way may include a color matching database, a texture matching database, and an emphasis information matching database corresponding to the makeup style selected by the user, as described above with reference to FIG. 1 .

Meanwhile, according to an embodiment of the present disclosure, the area of interest selected by the user may be a point area. The point area may be an area in which the user satisfactorily applied makeup among areas of the face. The user may want to complete the selected makeup style by applying makeup to the remaining areas based on the point area. For example, the user completes the eye makeup using a newly purchased eye makeup product or an eye makeup product that they really want to use, and then makes up the remaining areas to match the desired makeup style (for example, a natural makeup style). You may want to complete .

Accordingly, when the area of interest selected by the user is a point area, the processor 120 may calculate the matching rate of other areas of the face based on the point area and display a makeup guide based on this.

For example, if the user selects “natural style” and selects the point area as “eyes,” the processor 120 provides information about the colors of the remaining areas that match the color of the current eye area of the user, and selects the point area as “eyes.” It can be identified in the corresponding color matching database. Additionally, the processor 120 may compare the current color information of the remaining regions with the color information of the remaining regions identified in the color matching database for each region, and calculate the color matching rate for each of the remaining regions. At this time, since the color information compared to the current color information is identified in the color matching database based on the color of the eye region, the calculated color matching rate can be the color matching rate of each of the remaining regions based on the eye region. there is. Likewise, the processor 120 may calculate the texture matching rate and the emphasis degree matching rate of the remaining regions based on the eye region, respectively.

In this way, when the makeup matching rate is calculated, the processor 120 may control the display 130 to display a makeup guide for the point area based on the calculated makeup matching rate.

Specifically, the processor 120 displays a makeup guide for the color of another area based on the color matching rate of the other area based on the point area, and displays a makeup guide for the color of the other area based on the point area. The display 130 can be controlled to display a makeup guide for the texture of another area and to display a makeup guide for the degree of emphasis of another area based on the matching rate of emphasis information of the other area based on the point area.

FIGS. 4A, 4B, and 4B illustrate an example in which a makeup guide is displayed when a point area is selected according to an embodiment of the present disclosure.

First, as shown in FIG. 4A, a captured image of the user's face may be displayed on the display 130. In this case, the processor 120 may analyze the displayed image of the user's face and obtain makeup information including color information, texture information, and emphasis level information for each region of the user's face.

Afterwards, the user can select a point area. Figure 4b shows an example in which the eye area is selected as the point area. In this way, when the eye area is selected as the point area, the processor 120 calculates the color matching rate of other areas based on the selected eye area, and based on this, creates a makeup guide for other areas as shown in FIG. 4C. can be displayed.

Meanwhile, in Figure 4b, an example is given where the guidance message is displayed as a makeup guide, but it is not limited thereto. For example, the makeup guide may include at least one of a video or makeup product recommendation related to the makeup method as well as a guidance message related to the makeup method to increase the calculated makeup matching rate.

At this time, the processor 120 uses an artificial intelligence model learned to output an appropriate makeup guide when the matching rate is input, or stores a makeup guide table according to the matching rate and uses it to display a makeup guide according to the matching rate. You can.

Meanwhile, according to an embodiment of the present disclosure, the area of interest selected by the user may be a complementary area. For example, after completing makeup, the user may have an area that they would like to further improve and can select this as the area of interest.

Unlike the point area, the complementary area is an area that the user wants to supplement, so when the area of interest selected by the user is the complementary area, the processor 120 calculates the matching rate of the complementary area based on the remaining areas of the face, and based on this Thus, the makeup guide can be displayed.

For example, if the user selects “natural style” and selects “cheeks” as the complementary area, the processor 120 provides information about the color of the cheek area that matches the current color of the remaining areas excluding the cheek area, It can be identified in the color matching database corresponding to the natural style. Additionally, the processor 120 may calculate the color matching rate of the cheek area by comparing the current color information of the cheek area with the color information of the cheek area identified in the color matching database. At this time, the color information compared to the current color information of the current ball is identified in the color matching database based on the colors of the remaining areas, so the calculated color matching rate will be the color matching rate of the ball area based on the remaining areas. You can.

Likewise, the processor 120 may calculate the texture matching rate and the emphasis level matching rate of the ball area based on the remaining areas excluding the ball. For example, the processor 120 may identify information about the texture of the cheek area that matches the current texture of the remaining areas excluding the cheek area from a texture matching database corresponding to the natural style. Additionally, the processor 120 may compare the current texture information of the ball area with the texture information of the ball area identified in the texture matching database and calculate the texture matching rate of the ball area based on the remaining areas. Additionally, the processor 120 may identify information about the emphasis level of the cheek area that matches the current emphasis level of the remaining areas excluding the cheek area from an emphasis level matching database corresponding to the natural style. Additionally, the processor 120 may compare the current emphasis level information of the cheek area with the emphasis level information of the ball area identified in the texture matching database, and calculate the matching rate of the emphasis level of the cheek area based on the remaining areas.

In this way, when the makeup matching rate is calculated, the processor 120 may control the display 130 to display a makeup guide for the complementary area based on the calculated makeup matching rate.

Specifically, the processor 120 displays a makeup guide for the color of the complementary area based on the color matching rate of the complementary area based on the remaining areas, and displays a makeup guide for the color of the complementary area based on the texture matching rate of the complementary area based on the remaining areas. The display 130 may be controlled to display a makeup guide for the texture of the complementary area and to display a makeup guide for the degree of emphasis of the complementary area based on the matching rate of emphasis information of the complementary area based on the remaining areas.

FIGS. 5A and 5B illustrate an example in which a makeup guide is displayed when a complementary area is selected according to an embodiment of the present disclosure.

Afterwards, the user can select a supplementary area, as shown in FIG. 5A. Figure 4a shows an example in which the cheek area is selected as the supplementary area. In this way, when the cheek area is selected as the complementary area, the processor 120 calculates the color matching rate of the cheek area based on the remaining areas of the face, and based on this, applies makeup to the cheek area as shown in FIG. 5B. The guide can be displayed.

At this time, the processor 120 may display a makeup guide including a comment about the current makeup state of the cheek area and a video about a makeup method to increase the matching rate, as indicated by reference number 51 in FIG. 5B. Alternatively, the processor 120 may display a makeup guide including recommendations for makeup products to increase the matching rate, as shown by reference number 52 in FIG. 5B. However, this is only an example, and the makeup guide is not limited to this.

The processor 120 may use an artificial intelligence model learned to output an appropriate makeup guide when the matching rate is input, or may store a makeup guide table according to the matching rate and use this to display a makeup guide according to the matching rate. .

Meanwhile, according to an embodiment of the present disclosure, the processor 120 may display a guide UI that guides the user to select one of the point area or the supplementary area. At this time, a guidance message explaining the point area and the supplementary area may be displayed together.

When a user selects a point area and selects an area of the face, the selected area becomes the point area, and accordingly, a makeup guide for another area based on the point area may be displayed. If the user selects the complementary area and selects one area of the face, the selected area becomes the complementary area, and accordingly, a makeup guide for the complementary area based on the remaining areas of the face may be displayed.

Meanwhile, according to an embodiment of the present disclosure, a user can input context information. For example, while selecting an area of interest, the user may input the user's context information including at least one of the purpose of makeup, a place to visit, or information about products used for makeup in the area of interest. At this time, context information may be input to the display device 100 through the user's voice or message input.

Accordingly, the processor 120 may control the display 130 to display the makeup guide by reflecting the user's context information. At this time, the processor 120 uses an artificial intelligence model learned to output an appropriate makeup guide when the matching rate and context information is input, or stores a makeup guide table according to the matching rate and context and uses this to provide the user's context information. A makeup guide reflecting the may be displayed, but is not limited to this.

For example, when an “ID photo” is input for makeup purposes, the processor 120 may display an additional makeup guide suitable for the context, such as recommending a full lip rather than a gradient lip for lip makeup.

Additionally, for example, when information about the product used for makeup in the eye area selected by the user as the point area is input, the processor 120 may display a makeup guide by reflecting the characteristics of the product used.

Meanwhile, according to an embodiment of the present disclosure, when the user does not wear any makeup, a guide for makeup for the point area selected by the user may be provided.

For example, the processor 120 may receive information about the user's current makeup state. For example, the processor 120 displays a UI that allows selection of whether the user is before or during makeup, and can identify whether the user is before or during makeup. Alternatively, the user's current makeup state may be determined by analyzing the user's face image.

Accordingly, when the user is identified as being in a pre-makeup state, the processor 120 provides a makeup guide image regarding the makeup method of the point area corresponding to the makeup style selected by the user so that the user can perform point makeup from the beginning. can do.

On the other hand, if it is identified as being in a makeup state, but the makeup of the point area is identified as insufficient, that is, if it is determined that there is no information corresponding to the makeup information of the current user's point area in the matching database, the processor 120 applies the makeup selected by the user. A makeup guide video can be provided on how to apply makeup in the point area corresponding to the style. Alternatively, if the history of performing makeup using the makeup style selected by the user is stored in the display device 100 as an image, the image may be provided to provide a makeup guide for the point area.

Meanwhile, according to an embodiment of the present disclosure, the processor 120 may control the display 130 to display the matching rate of each of the remaining areas based on the point area. For example, the processor 120 may display the matching rate of each of the remaining areas based on the point area adjacent to each of the remaining areas in the face image, but the method is not limited to this.

At this time, the displayed matching rate may be displayed as a single number by considering all the color matching rate, texture matching rate, and emphasis level matching rate based on the point area. For example, the processor 120 calculates the average matching rate for the color matching rate, texture matching rate, and emphasis degree matching rate of other areas based on the point area, and displays the calculated average value as the matching rate of the other area. can do. Through this, users can visually check the makeup matching rate of the remaining areas.

Figure 6 is a block diagram showing the configuration of a display device according to an embodiment of the present disclosure. FIG. 6 comprehensively illustrates various components that may be included in the display device 100. Therefore, depending on the embodiment, some of the components shown in FIG. 6 may be omitted or changed, and other components may be added. In explaining FIG. 6, the description of content that overlaps with the above will be omitted or simplified.

According to FIG. 6, the display device 100 includes a camera 110, a processor 120, a display 130, a communication unit 140, a storage unit 150, a microphone 160, a speaker 170, and a user input unit. It may include (180).

The communication unit 140 can communicate with various external devices according to various types of communication methods.

For example, the communication unit 130 may communicate with an external server that stores a matching database for each makeup style. Accordingly, the processor 120 may request and receive a matching database corresponding to the makeup style selected by the user from an external server through the communication unit 140. Additionally, the communication unit 140 can perform Internet communication. Accordingly, the processor 120 can receive various contents related to makeup methods or various information about makeup products through the communication unit 140.

For this purpose, the communication unit 140 may include wired and wireless LAN communication modules. Additionally, the communication unit 130 may include a short-range wireless communication module (not shown) or a mobile communication module, depending on the embodiment. Here, the short-range wireless communication module (not shown) is a communication device that wirelessly communicates data with an electronic device located nearby, such as a Bluetooth module, ZigBee module, or NFC (Near Field Communication) module. It says module. In addition, the mobile communication module refers to a communication module that connects to a mobile communication network and performs communication according to various mobile communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), and LTE (Long Term Evolution).

Depending on the embodiment, the communication unit 140 may include a broadcast reception module, and at least one of wired communication modules (not shown) such as USB (Universal Serial Bus), IEEE (Institute of Electrical and Electronics Engineers) 1394, and RS-232. It may also contain one.

The storage unit 150 may store an operating system (OS) for controlling the overall operation of the electronic device 100. Additionally, the storage unit 150 may store various algorithms, programs, data, etc. for performing various functions of the display device 100 described above. In particular, the storage unit 150 can store the various artificial intelligence models, makeup guide tables, and matching databases described above. Additionally, the storage unit 150 may store a camera app for driving the camera 110, a makeup guide app for performing the above-described makeup guide operation, etc. Additionally, the storage unit 150 may store a voice recognition module for performing a voice recognition function, but is not limited thereto.

To this end, the storage unit 150 may include volatile memory such as DRAM and SRAM, and non-volatile memory such as EEPOM and flash memory that can retain information even when the power is turned off. Additionally, the storage unit 150 may include a hard disk drive (HDD), solid state drive (SSD), or a memory card (e.g., SD card, memory stick) that is removable or attachable to the electronic device 100. .

The microphone 160 can receive sound from the surrounding environment or the user's voice. The processor 120 can identify an input sound or voice signal and perform an operation corresponding to the input sound or voice signal. For example, the microphone 160 may include a general microphone, a surround microphone, a directional microphone, etc.

The speaker 170 can output a sound signal. For example, the speaker 170 may output various messages, inquiries, notifications, guides, etc. related to the operation of the display device 100 described above as voice or notification sounds.

The user input unit 180 can receive various user commands such as control commands, selection commands, input commands, etc. For example, the user input unit 180 may receive the user's context information described above. Additionally, the user can input selection commands through the various UIs described above.

To this end, the user input unit 180 may include a keyboard, buttons, key pad, touch pad, and touch screen. Alternatively, the user input unit 180 may be implemented as an input/output port. In this case, the user input unit 180 may be called an input/output interface. When the user input unit 180 is implemented as an input/output port, the input/output ports include High-Definition Multimedia Interface (HDMI), DisplayPort (DP), RGB, Digital Visual Interface (DVI), Universal Serial Bus (USB), Thunderbolt, and audio. May include jacks, video jacks, etc. The user input unit 180 may be called an input device, an input unit, an input module, etc., and when implemented as an input/output port, it may be called an input/output device, an input/output unit, an input/output module, etc.

The processor 120 controls the overall operation of the display device 100. In particular, the processor 120 may perform various operations and data processing to perform the operation of the display device 100 according to various embodiments of the present disclosure described above, and may control the operations of other components. .

To this end, the processor 120 can control hardware or software components connected to the processor 120 by running an operating system or application program, and can perform various data processing and calculations. Additionally, the processor 120 may load and process commands or data received from at least one of the other components into volatile memory and store various data in non-volatile memory.

Figure 7 is a flowchart illustrating a method of controlling a display device according to an embodiment of the present disclosure.

According to FIG. 7, the display device 100 can identify the makeup style selected by the user (S710). For example, the display device 100 may display a plurality of images corresponding to a plurality of makeup styles, and when one image among the plurality of images is selected, the makeup style corresponding to the selected image may be identified.

Thereafter, when the user's face is photographed through the camera 110, the display device 100 may display the user's face image on the display 130 (S720).

Meanwhile, the display device 100 may analyze the user's face image and obtain makeup information including color information, texture information, and emphasis level information for each region of the user's face (S730).

In addition, when an area of interest is selected from the user's face image, the display device 100 calculates a makeup matching rate between the area of interest and other areas of the user's face based on the matching database corresponding to the makeup information and the identified makeup style. You can do it (S740). At this time, the matching database is a color matching database containing color information of each area of the face corresponding to the identified makeup style, a texture matching database containing texture information of each area of the face corresponding to the identified makeup style, and identification An emphasis degree matching database containing information on the degree of emphasis of each area of the face corresponding to the applied makeup style may be included.

In this way, when the makeup matching rate is calculated, the display device 100 may display a makeup guide for at least one of the area of interest or another area based on the makeup matching rate.

Meanwhile, according to an embodiment of the present disclosure, the display device 100 identifies color information of another region included in the color matching database based on color information of the region of interest included in the makeup information, and identifies color information of another region included in the color matching database. By comparing the color information of the identified other area with the color information of the other area included in the makeup information, the color matching rate of the other area based on the area of interest can be calculated. Additionally, the display device 100 identifies texture information of another area included in the texture matching database based on texture information of the area of interest included in the makeup information, and matches the makeup with the texture information of the other area identified in the texture matching database. By comparing texture information of other areas included in the information, the texture matching rate of other areas based on the area of interest can be calculated. In addition, the display device 100 identifies the emphasis level information of other areas included in the emphasis degree matching database based on the emphasis degree information of the region of interest included in the makeup information, and identifies the emphasis degree information of other areas identified in the emphasis degree matching database. By comparing the emphasis level information with the emphasis level information of other areas included in the makeup information, the matching rate of the emphasis level of other areas based on the area of interest can be calculated.

Meanwhile, the display device 100 may display a makeup guide for the color of another area based on the color matching rate of the other area based on the area of interest. Additionally, the display device 100 may display a makeup guide for the texture of another area based on the texture matching rate of the other area based on the area of interest. Additionally, the display device 100 may display a makeup guide for the degree of emphasis of another area based on the matching rate of the emphasis information of the other area based on the area of interest.

Meanwhile, according to an embodiment of the present disclosure, the display device 100 identifies color information of a region of interest included in the color matching database based on color information of another region included in the makeup information, and identifies the region of interest in the color matching database. By comparing the color information of the region of interest included in the makeup information with the color information of the region of interest included in the makeup information, the color matching rate of the region of interest based on another region can be calculated. In addition, the display device 100 identifies texture information of a region of interest included in the texture matching database based on texture information of another region included in the makeup information, and matches the texture information of the region of interest identified in the texture matching database, By comparing the texture information of the region of interest included in the makeup information, the texture matching rate of the region of interest based on other regions can be calculated. In addition, the display device 100 identifies the emphasis level information of the region of interest included in the emphasis degree matching database based on the emphasis degree information of other regions included in the makeup information, and emphasizes the region of interest identified in the emphasis degree matching database. By comparing the degree information and the emphasis degree information of the region of interest included in the makeup information, the matching rate of the degree of emphasis of the region of interest based on other regions can be calculated.

Meanwhile, the display device 100 may display a makeup guide for the color of the area of interest based on the color matching rate of the area of interest based on other areas. Additionally, the display device 100 may display a makeup guide for the texture of the region of interest based on the texture matching rate of the region of interest based on other regions. Additionally, the display device 100 may display a makeup guide for the degree of emphasis of the area of interest based on the matching rate of emphasis information of the area of interest based on other areas.

Meanwhile, according to an embodiment of the present disclosure, the makeup guide may include at least one of a guidance message, a video, or a makeup product recommendation related to a makeup method to increase the calculated makeup matching rate.

In addition, according to an embodiment of the present disclosure, the display device 100 receives user context information including at least one of information about the purpose of makeup, a place to visit, or a product used for makeup in an area of interest, and the input user The makeup guide can be displayed by reflecting the context information.

According to the various embodiments of the present disclosure as described above, a user can select a desired makeup style and select a region of interest among areas of the user's face, thereby providing an appropriate makeup guide to complete the desired makeup style. Accordingly, users can more conveniently and accurately complete the desired makeup style based on their current state.

Meanwhile, various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media (e.g., a computer). Here, the device is a device capable of calling commands stored from a storage medium and operating according to the called commands, and may include the display device 100 according to the disclosed embodiments.

When the instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or using other components under the control of the processor. Instructions may contain code generated or executed by a compiler or interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium does not contain signals and is tangible, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

According to one embodiment, methods according to various embodiments disclosed in the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed on a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or online through an application store (e.g. Play Store™). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or created temporarily in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

Each component (e.g., module or program) according to various embodiments may be composed of a single or plural entity, and some of the above-described sub-components may be omitted, or other sub-components may be variously used. Further examples may be included. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added. You can.

The above description is merely an illustrative explanation of the technical idea of the present disclosure, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present disclosure. In addition, the embodiments according to the present disclosure are not intended to limit the technical idea of the present disclosure, but are provided for explanation, and the scope of the technical idea of the present disclosure is not limited by these embodiments. Therefore, the scope of protection of this disclosure should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this disclosure.

Claims

In a method of controlling a display device including a camera,

identifying the makeup style selected by the user;

When the user's face is photographed through the camera, displaying the user's face image;

Analyzing the user's face image to obtain makeup information including color information, texture information, and emphasis level information for each area of the user's face;

When an area of interest is selected in the face image of the user, calculating a makeup matching rate between the area of interest and other areas of the user's face based on the obtained makeup information and a matching database corresponding to the identified makeup style. step; and

Comprising: displaying a makeup guide for at least one of the area of interest or the other area based on the calculated makeup matching rate,

The matching database is,

A color matching database containing color information of each area of the face corresponding to the identified makeup style, a texture matching database containing texture information of each area of the face corresponding to the identified makeup style, and the identified makeup style. A control method including an emphasis degree matching database containing information on the degree of emphasis of each region of the face corresponding to.
According to claim 1,

The step of calculating the makeup matching rate is,

identifying color information, texture information, and emphasis degree information of the other region included in the color matching database based on color information, texture information, and emphasis degree information of the region of interest included in the makeup information; and

The color information, texture information, and emphasis degree information of the identified other region are compared with the color information, texture information, and emphasis degree information of the other region included in the makeup information, and the other region based on the region of interest is compared. A control method including calculating the color matching rate, texture matching rate, and emphasis degree matching rate of each area.
According to claim 2,

The step of displaying the makeup guide is,

displaying a makeup guide for the color of the other area based on the color matching rate of the other area based on the area of interest;

displaying a makeup guide for the texture of the other area based on the texture matching rate of the other area based on the area of interest; and

A control method comprising: displaying a makeup guide for the degree of emphasis of the other area based on a matching rate of the emphasis information of the other area based on the area of interest.
According to claim 1,

The step of calculating the makeup matching rate is,

identifying color information, texture information, and emphasis degree information of the region of interest included in the color matching database based on color information, texture information, and emphasis degree information of the other region included in the makeup information; and

By comparing the color information, texture information, and emphasis degree information of the identified region of interest with the color information, texture information, and emphasis degree information of the region of interest included in the makeup information, the interest based on the other region is determined. A control method including calculating the color matching rate, texture matching rate, and emphasis degree matching rate of each area.
According to claim 4,

The step of displaying the makeup guide is,

displaying a makeup guide for a color of the region of interest based on a color matching rate of the region of interest based on the other region;

displaying a makeup guide for a texture of the region of interest based on a texture matching rate of the region of interest based on the other region; and

A control method comprising: displaying a makeup guide for the degree of emphasis of the region of interest based on a matching rate of emphasis information of the region of interest with respect to the other regions.
According to claim 1,

The identification step is,

Displaying a plurality of images corresponding to a plurality of makeup styles; and

When one of the plurality of images is selected, identifying a makeup style corresponding to the selected image.
According to claim 1,

The makeup guide is,

A control method including at least one of an information message, a video, or a makeup product recommendation related to a makeup method to increase the calculated makeup matching rate.
According to claim 1,

Receiving context information of the user including at least one of the purpose of makeup, the place of visit, or information about products used for makeup in the area of interest,

The step of displaying the makeup guide is,

A control method for displaying the makeup guide by reflecting the user's context information.
In the display device,

display;

camera; and

Identify the makeup style selected by the user,

When the user's face is captured through the camera, control the display to display the user's face image,

By analyzing the user's face image, makeup information including color information, texture information, and emphasis level information is obtained for each area of the user's face,

When an area of interest is selected in the face image of the user, a makeup matching rate between the area of interest and other areas of the user's face is calculated based on the obtained makeup information and a matching database corresponding to the identified makeup style; ,

At least one processor that controls the display to display a makeup guide for at least one of the area of interest or the other area based on the calculated makeup matching rate,

The matching database is,

A color matching database containing color information of each area of the face corresponding to the identified makeup style, a texture matching database containing texture information of each area of the face corresponding to the identified makeup style, and the identified makeup style. A display device including an emphasis degree matching database that includes information on the degree of emphasis of each region of the face corresponding to.
According to clause 9,

The at least one processor,

Identifying color information, texture information, and emphasis degree information of the other regions included in the color matching database based on color information, texture information, and emphasis degree information of the region of interest included in the makeup information, respectively,

The color information, texture information, and emphasis degree information of the identified other region are compared with the color information, texture information, and emphasis degree information of the other region included in the makeup information, and the other region based on the region of interest is compared. A display device that calculates the color matching rate, texture matching rate, and emphasis matching rate of each area.
According to claim 10,

The at least one processor,

Displaying a makeup guide for the color of the other area based on the color matching rate of the other area based on the area of interest, and displaying a makeup guide for the color of the other area based on the texture matching rate of the other area based on the area of interest. Displaying a makeup guide for the texture of and controlling the display to display a makeup guide for the degree of emphasis of the other region based on a matching rate of emphasis information of the other region based on the region of interest.
According to clause 9,

The processor,

Identifying color information, texture information, and emphasis degree information of the region of interest included in the color matching database based on color information, texture information, and emphasis degree information of the other region included in the makeup information, respectively,

By comparing the color information, texture information, and emphasis degree information of the identified region of interest with the color information, texture information, and emphasis degree information of the region of interest included in the makeup information, the interest based on the other region is determined. A display device that calculates the color matching rate, texture matching rate, and emphasis matching rate of each area.
According to claim 12,

The display device is,

Displaying a makeup guide for the color of the region of interest based on the color matching rate of the region of interest based on the other region, and displaying a makeup guide for the color of the region of interest based on the texture matching rate of the region of interest based on the other region. Displaying a makeup guide for a texture, and controlling the display to display a makeup guide for a degree of emphasis of the region of interest based on a matching rate of emphasis information of the region of interest based on the other regions.
According to clause 9,

The at least one processor,

A display device that controls the display to display a plurality of images corresponding to a plurality of makeup styles, and when one image among the plurality of images is selected, identifies a makeup style corresponding to the selected image.
In the computer-readable recording medium including a program executing a method of controlling a display device including a camera,

The control method of the display device is,

identifying the makeup style selected by the user;

When the user's face is photographed through the camera, displaying the user's face image;

Analyzing the user's face image to obtain makeup information including color information, texture information, and emphasis level information for each area of the user's face;

When an area of interest is selected in the face image of the user, calculating a makeup matching rate between the area of interest and other areas of the user's face based on the obtained makeup information and a matching database corresponding to the identified makeup style. step; and

Comprising: displaying a makeup guide for at least one of the area of interest or the other area based on the calculated makeup matching rate,

The matching database is,

A color matching database containing color information of each area of the face corresponding to the identified makeup style, a texture matching database containing texture information of each area of the face corresponding to the identified makeup style, and the identified makeup style. A computer-readable recording medium including an emphasis degree matching database containing information on the degree of emphasis of each region of the face corresponding to