KR101793468B1 - Self-Beauty App platform System using AR and method - Google Patents

Abstract

The present invention relates to a self-beauty application platform system using AR and a method thereof, and more particularly, to a self-beauty application platform system capable of determining a diagnosis position in a smart device through a marker formed in a skin diagnosing unit if diagnosis result information is provided to the smart device by diagnosing (moisture, oil, pores, melanin (pigmentation), sebum, wrinkles, sensitivity, etc.) by the skin diagnosing unit of a pen type, and matching the position of the corresponding skin diagnosis with the diagnosis result information diagnosed in the skin diagnosing unit by enabling the smart device with a self-beauty application to determine which part is the position of the skin diagnosed by the skin diagnosing unit in a photographed skin image, to output a matched result on a screen or transmit the matched result to a skin management cloud server.

Description

{Self-Beauty App platform System using AR and method}

The present invention relates to a self-beauty app platform system and method using an AR, and more particularly, to a self-beauty app platform system and method using an AR In case of providing the diagnostic result information to the smart device, the diagnosis position can be judged by the smart device through the marker formed in the skin diagnosis device, and the smart device equipped with the self-beauty app can be diagnosed by the skin diagnosis device A self-beauty app platform system and method using an AR that determines the location of a skin and outputs the result to the screen or transmits it to the skin management server by matching the diagnosis diagnosis position and the diagnosis result information diagnosed by the skin diagnosis device .

As the interest in skin beauty has increased, many users have checked the condition of the skin through various routes such as skin care room, massage shop, and dermatologist's office, and managed the skin by using appropriate cosmetics or medicines.

People's skin condition varies greatly according to the seasons, and individual differences are so severe that accurate diagnosis is the most important for proper prescription.

In order to diagnose the skin condition, specialist medical equipments are provided in the dermatologist's clinic and the clinic, and the diagnosis act such as photographing the enlarged pores is possible.

However, due to the increased interest in skin care, the level of knowledge about the skin of the user has also increased, and users have been wishing to know more about their skin condition or skin condition changes.

Nevertheless, it is very troublesome to visit the clinic regularly in order to monitor the skin condition and to understand the change in the use of cosmetics.

Therefore, it is necessary to provide a method for enabling users to easily check and diagnose their skin condition while carrying it.

In addition, a camera, a sensor, and the like, which provide comprehensive information about the conventional skin, are merely manually contacted by the operator to photograph the skin and obtain information.

In addition, when measuring the approximate state of the face skin, a 2D image is acquired and analyzed using a high-resolution camera. In order to obtain comprehensive information on the facial skin, a contact- There was a hassle to re-shoot.

In such a case, there has been a problem in that a face is separated from the device after the facial skin photographing in front of the high-resolution camera, and then the contact type local photographing camera is manually brought into close contact with the photographer.

As a result, it is possible to acquire information of skin diagnosis result at any time by using a portable pen type skin diagnosis device, so that users can easily check their skin condition by using a portable smart device and perform skin care at home The system can be used.

(Patent Document) Korean Patent Laid-Open Publication No. 10-2005-0083197

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is a primary object of the present invention to provide a skin type diagnostic apparatus for diagnosing skin (moisture, oil, pores, melanin , A wrinkle, a sensitivity, etc.) to provide the diagnosis result information to the smart device 300, the diagnosis position can be determined by the smart device through the marker formed on the skin diagnosis device, and the smart device 300 with the self- It is determined whether the position of the skin diagnosed by the skin diagnosis unit in the captured skin image is the location of the skin, and the position of the skin diagnosis is matched with the diagnosis result information diagnosed by the skin diagnosis unit and output to the screen or the skin management cloud server 400).

A second object of the present invention is to provide a skin type diagnostic device (100) for acquiring skin diagnosis information at a skin management cloud server (400) and enabling continuous management, and a smart device So that the user can freely access the skin management cloud server 400 anywhere and perform skin management according to the skin diagnosis location and items.

The third object of the present invention is to process the position and the item of the diagnosis result in the real-time skin image information photographed using the smart device 300 equipped with the self-beauty app as an augmented reality and display it on the screen.

The fourth object of the present invention is to recommend a cosmetic product suitable for each skin trouble position or item.

According to an aspect of the present invention, there is provided a self-beauty app platform system using an AR,

A pen type skin diagnosis device (100) for diagnosing the skin and providing diagnosis result information to the smart device (300);

A marker 200 formed on the pen type skin diagnosis device to allow the smart device to determine a diagnostic position;

In order to determine the position of the skin diagnosis unit in the whole skin, the position of the skin diagnosis unit is extracted, and the position of the skin diagnosis unit is extracted from the skin diagnosis unit And a smart device 300 equipped with a self-beauty app for matching and outputting diagnostic result information to a screen or transmitting the information to a skin management cloud server 400.

A self-beauty app platform system and method using an AR according to the present invention includes:

When the diagnostic result information is provided to the smart device by a pen type skin diagnosis device (moisture, oil, pores, melanin (pigmentation), sebum, wrinkle, sensitivity, etc.) It is also possible to determine the location of the skin by the skin diagnosis device in the skin image of the smart device equipped with the self-beauty app, and determine the location of the skin diagnosis and the diagnosis The diagnosis result information is matched and output to the screen or transmitted to the skin management cloud server 400, thereby providing an effect of self skin management using a portable skin diagnosis device.

In addition, skin diagnosis information can be obtained from a skin management cloud server through a pen type skin diagnosis device, and it is possible to continuously manage the skin diagnosis server. By using a smart device equipped with a self-beauty app, By allowing the skin management to be performed according to the diagnosis position and item, convenience and efficiency of use are provided.

In addition, by using the smart device equipped with the self-beauty app, the position and the item of the diagnosis result are processed as an augmented reality on the real time skin image information and the screen output is performed. The effect will be demonstrated.

In addition, by recommending a cosmetic product suitable for each skin trouble position or item, it is possible to maximize the skin care effect since the cosmetic product most suitable for each type of skin trouble currently used is used.

FIG. 1 is an overall configuration view of a self-beauty app platform system using an AR according to an embodiment of the present invention; FIG.
FIG. 2 is a block diagram of a smart device of a self-beauty app platform system using an AR according to an embodiment of the present invention; FIG.
3 is a block diagram of a skin management cloud server of a self-beauty app platform system using an AR according to an embodiment of the present invention;
4 is a flowchart illustrating a method of providing a self-beauty app platform using an AR according to an exemplary embodiment of the present invention.
5 to 6 are views showing skin trouble positions displayed on a smart device of a self-beauty app platform system using an AR according to an embodiment of the present invention.
7 is a view illustrating another example of a skin trouble item on a screen of a smart device of the self-beauty app platform system using the AR according to an embodiment of the present invention. FIG. 8 is an exemplary view showing recommended beauty product information for each skin trouble item .

The following merely illustrates the principles of the invention. Therefore, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or illustrated herein, embody the principles of the invention and are included in the concept and scope of the invention.

Furthermore, all of the conditional terms and embodiments listed herein are, in principle, only intended for the purpose of enabling understanding of the concepts of the present invention, and are not to be construed as limited to such specifically recited embodiments and conditions do.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a self-beauty app platform system using AR and a method of providing the same according to the present invention will be described in detail.

FIG. 1 is an overall configuration diagram of a self-beauty app platform system using an AR according to an embodiment of the present invention.

1, the self-beauty app platform system using the AR of the present invention includes a pen type skin diagnosis device 100, a marker 200, and a smart device 300.

The skin diagnosis device 100 and the smart device 300, and the smart device and the skin care cloud server are exchanged with each other by using wired / wireless communication.

Although only three basic blocks are shown in FIG. 1 for the sake of convenience, the respective components may be composed of a plurality of components.

The skin diagnosis device 100 uses a pen type skin diagnosis device and performs diagnosis of the skin (moisture, oil, pores, melanin (pigment deposition), sebum, wrinkle, sensitivity, .

In addition, the skin diagnosis device can be a pen-type contactless or a contactless-type device, and can be easily carried by consumers. Therefore, it is possible to measure the skin anytime and anywhere, and the diagnostic information can be confirmed in real- .

At this time, the marker 200 is formed on the pen type skin diagnosis device so that the diagnosis position can be determined by the smart device.

The marker may be a special type of marker, but it may be colored to determine the skin location measured using the image processing method of the smart device.

That is, the image processing is performed based on the marker.

In addition, the smart device 300 is equipped with a self-beauty app. By the operation of the self-beauty app, the skin image is photographed, and the position of the skin diagnosed by the skin diagnosis device is acquired as an image, The position of the skin diagnosis device is extracted to determine the position of the diagnosis device.

The extracted location may be matched with the diagnosis result information diagnosed by the skin diagnosis device and output to the screen or may be transmitted to the skin management cloud server 400 configured at a remote location and stored in a separate database so that continuous management can be performed.

According to an additional aspect, the smart device 300 equipped with the self-beauty app processes the position and the item of the diagnosis result diagnosed by the skin diagnostic device on the photographed real-time skin image information as an augmented reality, Or transmitted to the skin management cloud server 400 configured at a remote site.

2 is a block diagram of a smart device of a self-beauty app platform system using an AR according to an embodiment of the present invention.

As shown in FIG. 2, the smart device 300 includes:

A smart device photographing unit 310 for photographing a face image of the user,

A skin recognition information generation unit 320 for generating skin recognition information by acquiring the photographed skin image,

A skin recognition and alignment processing unit 330 for mapping and mapping the skin recognition information included in the skin recognition information and the skin recognition map information to the generated skin recognition information and processing the position of the skin diagnosis for each item of the skin diagnosis .

In the following example of the present invention, the face skin is exemplarily described, but it is not limited to the face, and it is natural that diagnosis of the skin of the other area is also possible.

More specifically, the facial image of the diagnosis person himself is photographed through the photographing unit 310 for the smart device, and the photographed face image is stored in a separate memory.

At this time, the skin recognition information generation unit 320 acquires the photographed face (skin) image to generate facial (skin) recognition information.

The facial recognition information includes coordinate information of the face including eyes, nose, and mouth.

Therefore, the face image of the diagnoser as shown in Figs. 5 to 6 is generated and displayed on the screen.

Thereafter, the skin recognition and alignment processor 330 maps the generated facial recognition information and facial recognition map information included in the facial skin diagnosis information to perform the alignment process.

At this time, the position of the skin trouble is processed for each item of the skin trouble.

As shown in FIG. 5, the diagnostic information is displayed at the position where the oil 85AU, water 67%, water 71%, and the like are respectively measured.

In addition, the portion C in Fig. 6 means the eye wrinkle portion.

Thus, the diagnoser can see the position on his or her face image and easily identify which skin trouble exists at which position.

As described above, it is possible to check which part of the skin trouble, that is, the skin diagnosis information, has been diagnosed using the marker.

That is, when the position information of the skin diagnosis is acquired through the marker and the information is provided to the screen, the diagnostic information is matched to the corresponding position and provided to the screen.

According to the embodiment, the smart device 300 may acquire the real-time skin image information photographed through the smart device photographing unit 310 and perform augmented reality processing for processing the position and items of the skin diagnosis as an augmented reality (340).

That is, by expressing the positions and items of the skin troubles in the real-time skin image information by the augmented reality processing unit as the augmented reality, it is possible to check the accurate skin trouble positions and items of the own face region in real time.

It is also possible to provide information in the form of a report.

According to the third embodiment, the smart device 300 may further include a classification display processor 350 for classifying the display screen according to the location and items of the skin diagnosis and items.

The skin trouble items such as pigmentation, pores, wrinkles and wrinkles on the screen of the smart device are displayed on the screen of the smart device. When any item is selected, the skin trouble position of the item on the face image is displayed.

For example, as shown in Fig. 6, when the eye wrinkles are selected, the positions corresponding to the wrinkles of the eyes are displayed.

In the example, the area is indicated in yellow.

In other words, as described above, when the sorting display processing unit 350 for each skin trouble item is configured, the display screen can be provided as a whole by each trouble item (sorting), and the screen can be classified and provided.

Meanwhile, in accordance with the fourth embodiment, the smart device 300 includes:

A recommended beauty product information display unit 360 for displaying cosmetic product component information or cosmetic product information matched to each item of the skin diagnosis on the screen,

And a progressive bar display unit 370 for displaying on the screen whether the cosmetic product displayed on the screen is used in which stage of the skin care stage.

That is, as shown in FIG. 8, the beauty product component information matched to each skin trouble item is displayed on the screen.

For example, as shown in FIG. 8, if a pore trouble item among the icon bars (tabs) that can select the skin trouble item set on the lower side on the screen is selected, the component information of the cosmetic product optimized for the pore trouble is provided.

The component information of the cosmetic product may be provided in a text form or an image form.

As shown in FIG. 8, the beauty product information optimized for the pore trouble, for example, the model name of the beauty product, the manufacturer of the beauty product, the link address information for viewing the beauty product, .

In the case of FIG. 8, the types of beauty products are indicated on the right side of the facial image, and a progressive bar is displayed on the lower side of the facial image.

The progressive bar is a representation of the product to be used at what time.

Meanwhile, according to yet another embodiment, the smart device 300 may include:

And a progressive bar display unit 370 for displaying on the screen which cosmetic product displayed on the screen is a product to be used during the skin care stage.

As shown in FIG. 8, through the progressive bar display unit, a screen is displayed on the screen to determine which step of the skin care step the cosmetic product is displayed on the screen. For example, from the cleansing- It indicates the time before or before bedtime.

3 is a block diagram of a skin management cloud server of a self-beauty app platform system using an AR according to an embodiment of the present invention.

As shown in Fig. 3, the skin management cloud server 400,

A skin diagnosis information matching unit 410 for acquiring skin diagnosis information provided from the smart device and matching the skin diagnosis information with the ID information of the diagnosis person,

A skin diagnosis information database 420 for storing and managing the matched information by ID,

A skin recognition map converting unit 430 for acquiring skin diagnosis information stored in the skin diagnosis information database and converting the skin diagnosis information into a skin recognition map,

A recommendation information divider 440 for each skin trouble item which stores information of at least one of cosmetic product component information corresponding to each skin diagnosis item, cosmetic product component information per bar code, and cosmetic product information.

More specifically, the skin diagnosis information matching unit 410 acquires the skin diagnosis information provided from the smart device and matches the ID information of the diagnostic person, and stores the matched information in the skin diagnosis information database 420 do.

Accordingly, the matching information is stored and managed for each ID in the skin diagnosis information database 420.

The skin recognition map converting unit 430 acquires skin diagnosis information stored in the skin diagnosis information database and converts the obtained skin diagnosis information into a skin recognition map.

Meanwhile, a self-beauty application for accessing the skin management cloud server through the smart device 300 to acquire the skin diagnosis information and displaying the location of the skin trouble on the skin trouble items is downloaded and loaded.

4 is a flowchart illustrating a method of providing a self-beauty app platform using an AR according to an exemplary embodiment of the present invention.

4, in the self-beauty app platform providing method using the AR,

A self-beauty app execution step (S100) of executing a self-beauty app installed in the smart device (300);

A skin diagnostic device pairing step S200 for turning on the skin diagnostic device 100 and pairing the skin diagnostic device 100 with the smart device 300;

A diagnosis start step (S300) for performing diagnosis after contacting or non-contacting the skin diagnosis device (100) at a position of the skin to be diagnosed;

A start signal sending step (S400) of sending the start signal to the smart device 300 by the skin diagnosis device 100;

A skin diagnosis location image acquisition step (S500) of acquiring an image through a camera to confirm where the smart device 300 has performed the skin diagnosis at the skin;

A diagnosis end signal sending step (S600) of causing the skin diagnosis device (100) to send a diagnosis end signal to the smart device (300);

A skin diagnosis result information obtaining step (S700) of obtaining, from the skin diagnosis device (100), diagnostic result information corresponding to a position where the smart device (300) diagnoses with reference to the marker (200) formed in the skin diagnosis device;

A skin diagnosis result matching step S800 for matching the diagnosis result with the position where the smart device 300 diagnoses the skin coordinates;

And a skin diagnosis result storing step (S900) in which the smart device 300 stores the position of the skin diagnosis in the memory and the diagnosis result information.

First, the self-beauty app platform of the present invention is downloaded and installed in a smart device.

At this time, the self-beauty app installed in the smart device 300 is executed (S100), and when the pen type skin diagnosis device 100 is turned on, the device is paired with the smart device 300 through wired / wireless communication (S200) , The diagnosis is performed (S300) if the diagnosis start is performed after contacting or non-contacting the skin diagnosis device 100 at the position of the skin to be diagnosed after pairing.

For this purpose, a start button can be configured in the skin diagnosis device.

At this time, the skin diagnosis device 100 sends a start signal to the smart device 300 (S400).

Then, an image including the marker is acquired through the camera (S500) in order to check where the smart device 300 has performed the skin diagnosis at the skin.

When the diagnosis is completed, the skin diagnosis device 100 sends a diagnosis completion signal to the smart device 300 (S600).

The diagnosis termination may be performed when the diagnosis is completed, or the diagnosis termination may be performed through a separate termination button.

Then, the smart device 300 refers to the marker 200 formed on the skin diagnosis device and acquires diagnostic result information corresponding to the diagnosed position from the skin diagnosis device 100 (S700).

Then, the smart device 300 matches the diagnostic position with the diagnosis result information (S800), and the smart device 300 stores the skin diagnosis position and diagnostic result information in the memory (S900) .

After the diagnosis of the specific position is completed, the diagnosis start position S300, the start signal transmitting step S400, the skin diagnosis position image obtaining step S500, and the diagnosis end signal transmitting step S600 The skin diagnosis result information acquisition step S700, the skin diagnosis result matching step S800, and the skin diagnosis result storage step S900.

After finishing all the diagnoses finally, the results of diagnosis so far can be collected and stored in the memory, or the diagnosis information can be transmitted to the skin management cloud server 400 configured at a remote site.

In the meantime, according to the present embodiment, the real-time skin image information photographed by the smart device 300 may be processed as an augmented reality by processing the position and the item of the diagnosis result stored in the memory and outputting the result to the output or skin management cloud server 400 And a real processing step (S1000).

In other words, when the smart device 300 processes the position and the item of the diagnosis result stored in the memory as augmented reality and outputs the result to the screen, the user views the skin of the user in real time and confirms what diagnosis result It becomes possible.

In addition, by sending the data to the skin management cloud server 400, the user can freely access the skin management cloud server 400 anytime and anywhere using the smart device equipped with the self-beauty app and perform skin management according to the skin diagnosis position and items Thereby providing convenience in use and efficiency.

According to another embodiment of the present invention, the smart device 300 may include a skin diagnosis item classification display step S1100 for classifying the display screen by item referring to the location and item of the skin diagnosis stored in the memory;

And a recommended beauty product information display step (S1200) for displaying the beauty product component information or the beauty product information matched by the skin diagnosis item on the screen to recommend the beauty product to the smart device 300 .

That is, the smart device 300 refers to the location and items of the skin diagnosis stored in the memory, and displays and displays the display screen for each item (S 1100). [

Then, in order to recommend the beauty product, the smart device 300 displays the beauty product component information or the beauty product information matched with the skin diagnosis item on the screen (S1200).

According to the present invention, diagnostic information is provided to the smart device 300 by diagnosing the skin (moisture, oil, pores, melanin (pigmentation), sebum, wrinkles, sensitivity, etc.) The smart device 300 allows the user to determine the diagnosis position from the smart device through the markers formed on the skin diagnosis device. In addition, the skin device diagnosed by the skin diagnosis device in the skin image captured by the self- And outputs the result to the screen or transmits it to the skin management cloud server 400 by matching the position of the skin diagnosis and the diagnosis result information diagnosed by the skin diagnosis device to the screen, thereby enabling self-skin management using a portable skin diagnosis device Effect.

In addition, skin diagnosis information can be obtained from the skin management cloud server 400 through the pen type skin diagnosis device 100 to enable continuous management, and a smart device equipped with a self-beauty application can freely perform skin management The user can access the cloud server 400 to perform skin management according to the skin diagnosis location and items, thereby providing convenience and efficiency in use.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

100: Skin Diagnostic System
200: Marker
300: Smart device
400: skin care cloud server

Claims (7)

In a self-beauty app platform system using AR,
A pen type skin diagnosis device (100) for diagnosing the skin and providing diagnosis result information to the smart device (300);
A marker 200 formed on the pen type skin diagnosis device to allow the smart device to determine a diagnostic position;
In order to determine the position of the skin diagnosis unit in the whole skin, the position of the skin diagnosis unit is extracted, and the position of the skin diagnosis unit is extracted from the skin diagnosis unit And a smart device 300 equipped with a self-beauty app for matching and outputting diagnostic result information to a screen or sending it to a skin management cloud server 400,
The smart device (300)
A smart device photographing unit 310 for photographing a face image of the user,
A skin recognition information generation unit 320 for generating skin recognition information by acquiring the photographed skin image,
A skin recognition and alignment processor 330 for mapping and mapping the generated skin recognition information and skin recognition map information included in the skin diagnosis information and processing the position of the skin diagnosis for each item of the skin diagnosis,
An augmented reality processing unit 340 for acquiring real-time face image information photographed through the smart device photographing unit 310 and processing the position and items of the skin diagnosis as an augmented reality,
(350) for classifying the display screen according to the position and the item of the skin diagnosis,
Wherein the skin recognition information includes coordinate information of a face including eyes, nose, and mouth.
In a self-beauty app platform system using AR,
A pen type skin diagnosis device (100) for diagnosing the skin and providing diagnosis result information to the smart device (300);
A marker 200 formed on the pen type skin diagnosis device to allow the smart device to determine a diagnostic position;
In order to determine the position of the skin diagnosis unit in the whole skin, the position of the skin diagnosis unit is extracted, and the position of the skin diagnosis unit is extracted from the skin diagnosis unit And a smart appliance 300 equipped with a self-beauty app for matching the diagnosis result information with the real-time skin image information and outputting the diagnosis result as an augmented reality to the output or skin management cloud server 400 )Wow;
And a skin management cloud server (400) for matching and storing the diagnosis result information provided from the smart device and the ID information of the diagnosis person,
The smart device (300)
A smart device photographing unit 310 for photographing a face image of the user,
A skin recognition information generation unit 320 for generating skin recognition information by acquiring the photographed skin image,
A skin recognition and alignment processor 330 for mapping and mapping the generated skin recognition information and skin recognition map information included in the skin diagnosis information and processing the position of the skin diagnosis for each item of the skin diagnosis,
An augmented reality processing unit 340 for acquiring real-time face image information photographed through the smart device photographing unit 310 and processing the position and items of the skin diagnosis as an augmented reality,
(350) for classifying the display screen according to the position and the item of the skin diagnosis,
Wherein the skin recognition information includes coordinate information of a face including eyes, nose, and mouth.
delete 3. The method according to claim 1 or 2,
The smart device (300)
A recommended beauty product information display unit 360 for displaying cosmetic product component information or beauty product information matched with each item of skin diagnosis on the screen,
And a progressive bar display unit (370) for providing at least one of the skin care step displayed on the screen to the user, the progressive bar display unit (370).


delete delete delete

Images