KR102114273B1 - Method for personal image diagnostic providing and computing device for executing the method - Google Patents

Abstract

Disclosed are a method for providing personal image diagnosis to provide a personal image corresponding to a user′s image and a computing device for executing the same. According to one embodiment of the present invention, the computing device, as a computing device which comprises one or more processors and a memory storing one or more programs executed by the one or more processors, comprises: a communication module acquiring a user′s facial image; a facial image analysis module performing pretreatment based on the facial image to generate a data set and classifying a type of a user′s personal image based on the data set through a first machine learning module; and an image matching module calculating suitability of a test type in accordance with the type of the user′s personal image; and a content providing module providing the user with the personal image type and the suitability of the test type.

Description

METHOD FOR PERSONAL IMAGE DIAGNOSTIC PROVIDING AND COMPUTING DEVICE FOR EXECUTING THE METHOD}

Embodiments of the present invention relate to techniques for providing personal image diagnostics.

Personal color can be said to be the 'color tone' that represents the most ideal combination for all styling according to an individual's body color, mood, and appearance. Personal color has the basic concept of 'harmony' and 'combination' of colors. Therefore, 'seasonal' classification is used to represent the most harmonious color combination in nature. The seasonal classification is classified into spring, summer, autumn, and winter, and is the most popular personal color classification. However, simply classifying personal colors into four seasons cannot reflect the characteristics of various individuals.

In other words, in the process of analyzing a person's image, it is diagnosed only with color in four seasons, and recommending make-up color, fashion color, and hair color within the applicable range is less objectivity in the narrow spectrum, and does not fit well with the real person's image There is this.

Republic of Korea Patent Registration No. 10-1563124 (2015.10.20.)

Embodiments of the present invention is to diagnose a user's image by analyzing a user's face image, and to provide a personal image suitable for the user's image.

In addition, embodiments of the present invention is to provide a suitability for a test type by combining a real fabric with a user's face image for each test type.

According to an exemplary embodiment of the present invention, a computing device having one or more processors, and a memory storing one or more programs executed by the one or more processors, comprising: a communication module for obtaining a face image of a user; A face image analysis module generating a data set by performing pre-processing based on the face image, and classifying the personal image type of the user based on the data set through a first machine learning module; An image matching module for calculating a suitability of a test type according to the user's personal image type; And a content providing module that provides the user with the suitability of the personal image type and the test type.

When the data set is input, the first machine learning module extracts a feature value of a preset feature point region using a convolutional neural network (CNN) model from the input data set, and a feature value of the extracted feature point region Similarity scores are calculated by comparing similarity with a previously produced personal image type based on, and one or more personal image types having a similarity score equal to or greater than the predetermined score among the personal image types may be detected.

The image matching module may calculate the suitability of each of the test types for the personal image type based on the personal image type and the test type through a second machine learning module.

The test type includes at least one of a color type, a pattern type, a pattern type, and a material type, and the content providing module divides the test type into color, pattern, pattern, or material, and provides the list. Upon receiving a user input for one of the provided test type lists, an image corresponding to the test type list may be combined with the face image of the user and provided to the user.

The test type includes at least one of a color type, a pattern type, a pattern type, and a material type, and the content providing module divides the test type into color, pattern, pattern, or material, and provides the list. Upon receiving a user's input for one of the list of test types provided, the personal image type and the suitability of the input test type can be provided to the user.

The image matching module calculates each suitability for brightness and saturation for each color in the color type, calculates each suitability for the size of a pattern in the pattern type, and each for a type of pattern in the pattern type Calculate the goodness of fit, and calculate the goodness of fit for the gloss, matte and density of the material in the material type.

According to another exemplary embodiment of the present invention, as a method performed in a computing device having one or more processors, and a memory storing one or more programs executed by the one or more processors, obtaining a face image of a user To do; Generating a data set by performing pre-processing based on the face image; Classifying the personal image type of the user based on the data set through a first machine learning module; Calculating a goodness of fit of the test type according to the user's personal image type; And providing the fitness of the personal image type and the test type to the user.

When the data set is input, the first machine learning module extracts a feature value of a preset feature point region using a convolutional neural network (CNN) model from the input data set, and a feature value of the extracted feature point region Calculating similarity scores by comparing similarities with previously produced personal image types based on, detecting one or more personal image types having similarity scores equal to or greater than the preset score among the personal image types, and calculating the fitness , May calculate the suitability of each of the test types for the personal image type based on the personal image type and the test type through a second machine learning module.

The test type includes at least one of a color type, a pattern type, a pattern type, and a material type, and the providing step to the user provides the test type as a list by dividing the test type into color, pattern, pattern, or material. To do; Receiving a user input for one of the provided test type lists; And combining the image corresponding to the test type list with the face image of the user and providing the image to the user.

The test type includes at least one of a color type, a pattern type, a pattern type, and a material type, and the providing step to the user provides the test type as a list by dividing the test type into color, pattern, pattern, or material. To do; Receiving a user input for one of the provided test type lists; And providing suitability of the personal image type and the input test type to the user.

According to embodiments of the present invention, a user's image may be diagnosed by analyzing a user's face image, and a personal image suitable for the user's image may be provided.

In addition, according to embodiments of the present invention, the user can accurately know the image of the user by checking the suitability for the test type by combining the actual fabric with the user's face image for each test type.

1 is a view showing the configuration of a personal image diagnosis providing system according to an embodiment of the present invention
2 is a block diagram showing the configuration of a smart device in a personal image diagnosis providing system according to an embodiment of the present invention
3 is a block diagram showing the configuration of a management server in a personal image diagnosis providing system according to an embodiment of the present invention
4 is a block diagram illustrating an image analysis module and an image matching module in the management server of the personal image diagnosis providing system according to an embodiment of the present invention
5 and 6 are views showing a screen of a smart device provided with a personal image diagnosis providing service according to an embodiment of the present invention
7 is a block diagram illustrating and illustrating a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to aid in a comprehensive understanding of the methods, devices and / or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, when it is determined that a detailed description of known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to a user's or operator's intention or practice. Therefore, the definition should be made based on the contents throughout this specification. The terminology used in the detailed description is only for describing embodiments of the present invention and should not be limiting. Unless expressly used otherwise, a singular form includes a plural form. In this description, expressions such as “including” or “equipment” are intended to indicate certain characteristics, numbers, steps, actions, elements, parts or combinations thereof, and one or more other than described. It should not be interpreted to exclude the presence or possibility of other characteristics, numbers, steps, actions, elements, or parts or combinations thereof.

In the following description, the terms “transmission”, “communication”, “transmission”, “reception” and the like of a signal or information are not only those in which a signal or information is transmitted directly from one component to another. This includes passing through other components. In particular, "transmitting" or "transmitting" a signal or information to a component indicates the final destination of the signal or information, not a direct destination. The same is true for the "reception" of a signal or information. Also, in this specification, when two or more data or information is “related” means that acquiring one data (or information) may acquire at least a portion of the other data (or information) based thereon.

Meanwhile, an embodiment of the present invention may include a program for performing the methods described herein on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include program instructions, local data files, local data structures, or the like alone or in combination. The media may be specially designed and constructed for the present invention, or may be commonly used in the field of computer software. Examples of computer readable recording media include specially configured to store and execute magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and program instructions such as ROM, RAM, and flash memory. Hardware devices are included. Examples of the program may include high-level language codes that can be executed by a computer using an interpreter as well as machine language codes made by a compiler.

1 is a block diagram showing the configuration of a personal image diagnosis providing system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the configuration of a smart device of a personal image diagnosis providing system according to an embodiment of the present invention, 3 is a block diagram showing the configuration of a management server of a personal image diagnosis providing system according to an embodiment of the present invention, and FIG. 4 is an image analysis module in a management server of a personal image diagnosis providing system according to an embodiment of the present invention And a block diagram for describing the image matching module.

1 to 4, an image diagnosis providing system according to an embodiment of the present invention may include a smart device 100 and a management server 200. The smart device 100 and the management server 200 are communicatively connected through a communication network.

In the following description, "image" refers to an image photographed using a photographing device such as a camera, that is, an image such as a photograph, and "image" refers to an intuitive impression that emerges from the inside of an individual about an object or object, or It means representation or image. That is, in the disclosed embodiment, diagnosing a personal image means determining an impression or feeling of an individual felt according to characteristics of the individual.

In some embodiments, the communication network includes the Internet, one or more local area networks, wire area networks, cellular networks, mobile networks, other types of networks, or combinations of these networks. can do.

The smart device 100 is a smart phone, a tablet PC, and the like that can use wireless communication, and the smart device 100 may include a photographing unit 110, a communication unit 120, and a display unit 130.

The photographing unit 110 may photograph a user's face. The photographing unit 110 may be integrally formed with a terminal such as the smart device 100. Specifically, the photographing unit 110 may be integrally formed with a terminal such as the smart device 100 to photograph a user's face. The photographing unit 110 may provide the photographed image to the management server 200 through the communication unit 120. For example, as shown in FIG. 5, it is shown that the image capturing is performed using the photographing unit 110 of the smart device 100. When taking an image using the photographing unit 110, recommendations may be provided to the user. When taking an image, the front image of the face is required for accurate diagnosis, and color change according to the surrounding environment such as illuminance and color change according to the performance of the camera module may occur, so shoot in front, shoot in natural light, We recommend that you shoot in an environment where the skin color is best exposed, and in the most natural way.

The communication unit 120 may transmit the captured image to the management server 200 through a network. In addition, the communication unit 120 may receive the assignment diagnosis content implemented so that the user can test the color type, pattern type, pattern type, and material type from the management server 200. The communication unit 120 may receive the fitness of the personal image type and the test type from the management server 200.

The display 130 may display the screening diagnosis contents implemented on the screen so that the user can test the color type, pattern type, pattern type, and material type. The display unit 130 may output the assignment diagnosis content through the screen of the smart device 100. The assignment diagnosis content can be tested by the user for the color type, pattern type, pattern type, and material type through the screen of the smart device 100. Here, the assignment diagnosis content can be tested for brightness and saturation for each color of the fabric in the color type. In addition, the assignment diagnosis content can be tested for the size of the fabric pattern in the pattern type. In addition, the assignment diagnosis content can be tested for the type of fabric pattern (for example, polka dot, geometric pattern, checkered pattern, sprite pattern, plain fabric, etc.) from the pattern type. In addition, the assignment diagnosis content can be tested for the gloss, matte and density of the fabric material in the material type.

In addition, the display 130 may display a user's personal image type and test type suitability. The display unit 130 may process the user's personal image type and test type fitness to be output on the smart device 100 and output the screen through the screen of the smart device 100.

At this time, an application for providing an image diagnosis providing service may be installed in the smart device 100 according to an embodiment of the present invention. The application may be stored in a computer-readable storage medium of the smart device 100. The application includes a predetermined set of instructions executable by the processor of the smart device 100. The command may cause the processor of the smart device 100 to perform an operation according to an exemplary embodiment. The computer-readable storage medium of the smart device 100 includes components of an operating system for executing a set of instructions such as the application on the smart device 100. For example, such an operating system may be Apple's iOS or Google's Android.

Here, the application for providing the image diagnosis providing service may provide an input interface for inputting user information, and may receive user data through the input interface. For example, an input interface for inputting a user's name, gender, age, and smart device 100 number may be provided to the user.

Meanwhile, an application for providing an image diagnosis providing service may be set to perform various analysis processing and information providing operations in addition to the functions described above. For example, an application for providing an image diagnosis providing service uploads or backs up the suitability of a user's personal image type and test type to a preset web server (for example, an SNS server), or from a preset web server. The suitability of the personal image type and the test type of the pre-registered user matching the information may be acquired and displayed through the display unit 130.

The management server 200 may include a communication module 210, a storage module 220, a face image analysis module 230, an image matching module 240, and a content providing module 250.

The communication module 210 may receive an image captured from the smart device 100 through a network. The communication module 210 may receive user information from the smart device 100 through a network. In addition, the communication module 210 may transmit the assignment diagnosis content to the smart device 100. The communication module 210 may transmit the fitness of the personal image type and the test type to the smart device 100.

The storage module 220 may store user information and photographed images received through the communication module 210. Also, the storage module 220 may store an image of a pre-recorded user. Also, the storage module 220 may store an image for the test type. Here, the test type may include a color type, a pattern type, a pattern type, and a material type. In addition, images for color type, pattern type, pattern type, and material type may be images of actual fabrics for each test type.

The face image analysis module 230 extracts a face image from the image captured from the photographing unit 110 and uses a machine learning-based technology from the extracted face image to determine a user's personal image type. I can judge. Here, diagnosing a personal image means determining an individual's impression or feeling according to characteristics of the individual. The face image analysis module 230 may include a pre-processing module 231 and a first machine learning module 233.

The pre-processing module 231 may extract a face image from the image photographed from the photographing unit 110 and perform a sharpening process to generate a data set. That is, the image captured by the photographing unit 110 may blur the face image due to the photographing environment. Accordingly, sharpening may be performed by sharpening the feature points of the image by performing a sharpening process on each image.

The pre-processing module 231 may transmit the generated data set to a machine learning module. The pre-processing module 231 may use a sharpening processing technique, but is not limited thereto, and may further perform blur, Gaussian, geometric processing, and the like.

The first machine learning module 233 may receive the generated data set and classify a user's personal image type in the data set. When the data set is input, the first machine learning module 233 may classify the personal image type of the user from the data set based on the already learned personal image classification model.

In an exemplary embodiment, the first machine learning module 233 may extract feature values of a predetermined feature point region from a data set using a convolutional neural network (CNN) model as a machine learning or deep learning technique. Specifically, the first machine learning module 233 may extract a feature value of a preset feature point region of a sharpening image (a face image that has undergone sharpening processing).

Also, the first machine learning module 233 may calculate a similarity score for classifying the personal image type with respect to the predetermined feature point area. Specifically, the first machine learning module 233 compares the similarity with the previously created personal image type (for example, an image type pre-classified according to the feature value of the face image) based on the feature value of the extracted feature point region. Similarity scores can be calculated by comparison. For example, the first machine learning module 233 may use an Inception Model, but is not limited thereto. Here, the personal image type may be 22 types classified as previously known, cute, sporty, clear sporty, casual, soft casual, ethnic, dynamic, wild, romantic, feminine, elegance, soft elegance, softmodern, hardmodern, Dandy, formal, noble, natural, traditional, classic, gorgeous, elegance gorgeous. Meanwhile, the personal image type is classified into 22 types that have been previously classified, but the present invention is not limited thereto, and the previously known image types can be added, and the 22 types previously classified can be grouped and classified.

Also, the first machine learning module 233 may classify the personal image type into one or more personal image types having similarity scores equal to or greater than a preset score.

The image matching module 240 uses machine learning based technology from the personal image type to fit each test type (eg, color type, pattern type, pattern type, material type) for the personal image type Can be calculated. The image matching module 240 may include a second machine learning module 243.

The second machine learning module 243 may use machine learning technology or deep learning technology. In this case, when the personal image type and the user-selected test type are input, the second machine learning module 243 may calculate the suitability of each test type based on the already-trained fitness classification model.

In an exemplary embodiment, the second machine learning module 243, when the personal image type and the test type selected by the user are input, regression learned to calculate and output the fitness of the test type from the input personal image type and the test type. It may be a regression model. The regression model may be a multiple linear regression model or a neural network. The second machine learning module 243 may compare the significance of the test type with respect to the personal image type to calculate the fitness of the test type with respect to the significance. For example, the second machine learning module 243 may calculate the suitability for each color type, pattern type, pattern type, and material type for the personal image type selected in the face image analysis module 230. In the color type, each suitability for brightness and saturation can be calculated for each color of fabric. In addition, in the pattern type, each suitability for the size of the fabric pattern can be calculated. In addition, in the pattern type, each suitability for the type of the fabric pattern (for example, a polka dot pattern, a geometric pattern, a check pattern, a sprite pattern, a plain fabric, etc.) can be calculated. In addition, in the material type, each suitability for gloss, matte and density of the fabric material can be calculated.

The content providing module 250 provides a screen through which the user can check through the smart device 100, and provides an input diagnostic content implemented so that the user can test the color type, pattern type, pattern type, and material type. It can be provided as (100). The assignment diagnosis content can be tested by the user for the color type, pattern type, pattern type, and material type through the screen of the smart device 100. That is, the user can combine the face image of the user and each test type through the screen of the smart device 100. For example, as shown in FIG. 6, one of color type, pattern type, pattern type, and material type may be selected and the actual fabric image may be combined with the user's face and output through the screen of the smart device 100. Here, the assignment diagnosis content can be tested for brightness and saturation for each color of the fabric in the color type. In addition, the assignment diagnosis content can be tested for the size of the fabric pattern in the pattern type. In addition, the assignment diagnosis content can be tested for the type of fabric pattern (for example, polka dot pattern, geometric pattern, check pattern, sprite pattern, plain fabric, etc.) from the pattern type. In addition, the assignment diagnosis content can be tested for the gloss, matte and density of the fabric material in the material type.

In addition, when the user selects a test type, the content providing module 250 may display the suitability of the test type for the user's personal image type. Also, the content providing module 250 may display a personal image type selected from the image matching module 240. Specifically, when the user selects a test type, the content providing module 250 may display the suitability of the corresponding test type calculated from the image matching module 240. In addition, when the user selects a test type, the content providing module 250 may display the suitability of the test type together with the personal image type that has received a similarity score equal to or greater than a preset score.

Therefore, the personal image diagnosis providing system according to an embodiment of the present invention can analyze a user's face image to diagnose a user's image and provide a personal image suitable for the user's image.

In addition, the personal image diagnosis providing system according to an embodiment of the present invention enables the user to directly match the test type by confirming the suitability for the type by combining the actual fabric with the user's face image for each type. By doing so, the user's image can be accurately known.

7 is a block diagram illustrating and illustrating a computing environment including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. In one embodiment, computing device 12 may be smart device 100. Further, the computing device 12 may be the management server 200.

The computing device 12 includes at least one processor 14, a computer readable storage medium 16 and a communication bus 18. The processor 14 can cause the computing device 12 to operate in accordance with the exemplary embodiment mentioned above. For example, processor 14 may execute one or more programs stored on computer readable storage medium 16. The one or more programs may include one or more computer-executable instructions, which, when executed by the processor 14, configure the computing device 12 to perform operations according to an exemplary embodiment. Can be.

Computer readable storage medium 16 is configured to store computer executable instructions or program code, program data and / or other suitable types of information. The program 20 stored on the computer readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer-readable storage medium 16 is a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash Memory devices, other types of storage media that can be accessed by the computing device 12 and store desired information, or suitable combinations thereof.

The communication bus 18 interconnects various other components of the computing device 12, including a processor 14 and a computer readable storage medium 16.

Computing device 12 may also include one or more I / O interfaces 22 and one or more network communication interfaces 26 that provide an interface for one or more I / O devices 24. The input / output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input / output device 24 may be connected to other components of the computing device 12 through the input / output interface 22. Exemplary input / output devices 24 include pointing devices (such as a mouse or trackpad), keyboards, touch input devices (such as touch pads or touch screens), voice or sound input devices, various types of sensor devices, and / or imaging devices. Input devices and / or output devices such as display devices, printers, speakers, and / or network cards. The exemplary input / output device 24 is a component constituting the computing device 12 and may be included in the computing device 12 or connected to the computing device 12 as a separate device distinct from the computing device 12. It might be.

Although the exemplary embodiments of the present invention have been described in detail above, those skilled in the art to which the present invention pertains will understand that various modifications are possible within the limits of the embodiments described above without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims to be described later, but also by the claims and equivalents.

100: smart device
110: filming unit
120: communication unit
130: display unit
200: management server
210: communication module
220: storage module
230: face image analysis module
240: image matching module
250: content providing module

Claims (10)

One or more processors, and
A computing device having a memory for storing one or more programs executed by the one or more processors,
A communication module for acquiring a user's face image;
A face image analysis module that generates a data set by performing pre-processing based on the face image, and classifies the personal image type of the user based on the data set through a first machine learning module;
An image matching module for calculating a suitability of a test type according to the user's personal image type; And
And a content providing module that provides the user with the suitability of the personal image type and the test type,
The first machine learning module,
When the data set is input, a feature value of a preset feature point region is extracted from the input data set using a convolutional neural network (CNN) model, and a personal image previously produced based on the feature value of the extracted feature point region The similarity score is calculated by comparing similarity with a type, and one or more personal image types having a similarity score equal to or greater than the predetermined score among the personal image types are detected,
The image matching module,
A second machine learning module calculates the suitability of each of the test types for the personal image type based on the personal image type and the test type,
The test type is,
At least one of color type, pattern type, pattern type, and material type,
The content providing module,
When the test type is divided into color, pattern, pattern, or material, and provided as a list, and when a user input for one of the provided test type lists is received, an image corresponding to the test type list is combined with the face image of the user And provide the user with the suitability of the personal image type and the input test type.
delete delete delete delete The method according to claim 1,
The image matching module,
In the color type, each suitability for brightness and saturation for each color is calculated, each suitability for size of a pattern is calculated in the pattern type, and each suitability for a type of pattern is calculated in the pattern type, A computing device that calculates each suitability for a material's gloss, matte and density from the material type.
One or more processors, and
A method performed in a computing device having a memory that stores one or more programs executed by the one or more processors,
Obtaining a user's face image;
Generating a data set by performing pre-processing based on the face image;
Classifying the personal image type of the user based on the data set through a first machine learning module;
Calculating a goodness of fit of the test type according to the user's personal image type; And
Providing the fitness of the personal image type and the test type to the user,
The first machine learning module,
When the data set is input, a feature value of a preset feature point region is extracted from the input data set using a convolutional neural network (CNN) model, and a personal image previously produced based on the feature value of the extracted feature point region The similarity score is calculated by comparing similarity with a type, and one or more personal image types having a similarity score equal to or greater than the predetermined score among the personal image types are detected,
The step of calculating the goodness of fit,
Calculating the fitness of each of the test types for the personal image type based on the personal image type and the test type through a second machine learning module,
The test type is,
At least one of color type, pattern type, pattern type, and material type,
The step of providing to the user,
Dividing the test type into a color, a pattern, a pattern, or a material and providing it as a list;
Receiving a user input for one of the provided test type lists;
Providing an image corresponding to the test type list to the user in combination with the user's face image; And
And providing the user with the fitness of the personal image type and the input test type.
delete delete delete

Images