KR102316723B1 - Body-tailored coordinator system using artificial intelligence - Google Patents

Abstract

The present invention relates to a body-tailored coordination system using artificial intelligence, which comprises: a fashion information collecting module for collecting, from an external server, fashion information including a first coordination information including various pieces of clothing information, accessory information, hat information, sock information, and shoe information and a first beauty information including hair style information, nail art information and cosmetic information; a database module for collecting, from the external server, a first body information including face information, race information, skin tone information, age information, body type information, gender information, height information, hair length information, hair color information, hair style information and nail shape information on a plurality of persons, and performing learning for each information on the basis of a convolutional neural network to generate a learning model; an input module for inputting a user image obtained by photographing the whole body of a user and user personal information; a processing module for receiving the user image from the input module to pre-process the same; a prediction module for analyzing the user image processed by the processing module on the basis of information from the database module to predict a face shape, skin tone, body type, height, gender, race and age of the user so as to generate a second body information; a recommendation module for matching the fashion information corresponding to the second body information generated by the prediction module to generate a second beauty information and a second coordination information for the user; and an interface module for receiving the second beauty information and the second coordination information from the recommendation module, and providing the same for the user. Therefore, beauty information and coordination information suitable for user body information can be provided.

Description

Body-tailored coordinator system using artificial intelligence

The present invention relates to a body-tailored coordination system using artificial intelligence, and more particularly, to a body-tailored coordination system using artificial intelligence that analyzes a user's body and provides beauty information and coordination information suitable for the user's body information. .

Coordination refers to decorating clothes, accessories, and shoes in a harmonious manner according to the situation.

Recently, airport coordinating, interview coordinating, meeting coordinating coordinator, lower body obesity coordinating, etc. have taken a place in Internet search terms, so the coordination method according to the user's body type and situation is attracting attention from many people.

If you are interested in fashion, you can easily choose a product that can cover your body type and coordinate it, but most people have a hard time choosing a product that suits their body type and situation.

In order to solve this problem, people are getting information about coordinating by doing an Internet search using the situation and body shape as keywords. However, in this search method, since the user's situation and body type are simply keywords, it is difficult to search for coordination information that exactly fits the user's situation and body type, and since the integrated coordination information is searched, it is difficult to obtain various coordination information.

In addition, there is a problem in that the accuracy of the search result is significantly lowered because the user often misunderstands his or her body type.

Meanwhile, as the Internet and applications become popular, the number of people who purchase products such as clothes, cosmetics, and accessories through mobile devices has increased significantly.

However, when purchasing a product using a mobile device, it is not possible to test or try on the product, so you have no choice but to purchase it after checking the color, size, material, etc. of the product only with pictures.

According to this, when the product is used after purchase, there is a problem that the design and color do not suit the user or the clothing size does not fit.

SUMMARY OF THE INVENTION An object of the present invention is to provide a body-tailored coordination system using artificial intelligence that analyzes a user's body and provides beauty information and coordination information suitable for the user's body information.

The present invention includes first coordination information including various clothes information, accessory information, hat information, socks information and shoes information, and first beauty information including hair style information, nail art information and cosmetics information from an external server. a fashion information collection module for collecting fashion information; A first body including face shape information, race information, skin tone information, age information, body type information, gender information, height information, hair length information, hair color information, hair shape information and nail shape information of a plurality of people from an external server a database module that collects information and generates a learning model by learning each information based on a convolutional neural network; an input module into which a user image obtained by photographing the user's whole body and personal information of the user is input; a processing module receiving the user image from the input module and pre-processing the user image; Based on the information of the database module, by analyzing the user image processed by the processing module, the user's face shape, the user's skin tone, the user's body type, the user's height, the user's gender, the user's race, and the user's age a prediction module for predicting and generating second body information; a recommendation module for matching the fashion information corresponding to the second body information generated by the prediction module to generate second beauty information and second coordination information for the user; and an interface module that receives the second beauty information and the second coordination information from the recommendation module and provides them to the user.

The prediction module detects the user's face contour from the user image, and compares it with the first body information of the database module to predict the user's face shape by matching the face shape most similar to the user's face contour line module; a skin tone prediction module that detects the user's skin from the user image and predicts the user's skin tone by matching the skin tone most similar to the user's skin tone by comparing it with the first body information of the database module; a body shape prediction module for detecting a user's body in the user image, and predicting the user's body shape by matching the body shape most similar to the user's body shape by comparing it with the first body information of the database module; a height prediction module for detecting the user's face and the user's body from the user image, and predicting the height of the user by comparing it with the first body information of the database module; a gender prediction module for detecting the user's face contour, eye shape, nose shape, mouth shape, hair length, body type, and skin color from the user image, and predicting the user's gender by comparing it with the first body information of the database module; a race prediction module for detecting the user's face contour, eye shape, nose shape, mouth shape, hair length, body type, and skin color from the user image, and comparing it with the first body information of the database module to predict the user's race; and an age prediction module for predicting the age of the user based on the prediction results of the face shape prediction module, the skin tone prediction module, the body shape prediction module, the height prediction module, the gender prediction module, and the race prediction module. can

The recommendation module is characterized in that it recommends the second beauty information and the second coordination information matching the season, place and situation set by the user to the user, wherein the second beauty information is the second body information including skin cosmetics information, lip cosmetics information, cheek cosmetics information, hair style information, nail art information, and color lens information corresponding to , shoe information, socks information, hat information, and accessory information.

The recommendation module is configured to determine the user's location through the user terminal possessed by the user, collect weather information about the user's location from an external server, and the second beauty information and the second coordination information matching the weather information can recommend

The customized coordination system using the artificial intelligence further includes an electronic transaction server for trading the second beauty information and a product included in the second coordination information, wherein the user communicates with the second beauty information and the second beauty information through the user terminal. If any one or more of the second coordination information is selected, the electronic transaction server may be accessed.

According to the present invention, since beauty information and coordination information suitable for the user are provided to the user by analyzing the user's body, the hassle of the user having to search for a product suitable for him or try on it can be reduced.

In addition, according to the present invention, since beauty information and coordination information suitable for the user are provided to the user, there is an effect that the user can coordinate suitable for the user without directly coordinating the user.

1 is a schematic diagram of an entire network system for the operation of a body-customized coordination system using artificial intelligence according to an embodiment of the present invention;
2 is a block diagram of a body-customized coordination system using artificial intelligence according to an embodiment of the present invention;
3 is a block diagram of a prediction module of a body-customized coordination system using artificial intelligence of FIG. 2;
4 is a block diagram of a database module of the body-customized coordination system using artificial intelligence of FIG. 2;
5 is a configuration diagram of a body-customized coordination system using the artificial intelligence of FIG. 2 that further includes an electronic transaction server;
6 is a block diagram of a body-customized coordination system using artificial intelligence of FIG. 2;
FIG. 7 is a diagram illustrating a feature point for analyzing a face region of a user image by a prediction module of a body-customized coordination system using artificial intelligence of FIG. 2 .

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings.

In the description of the embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function interferes with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the component from other components, and the essence, order, or order of the component is not limited by the term.

Hereinafter, a body-tailored coordination system using artificial intelligence according to an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a schematic diagram of an entire network system for the operation of a body-customized coordination system using artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1 , a user terminal 10 , an external server 2000 , and a body-customized coordination system 100 using artificial intelligence may transmit/receive necessary information to each other through a single network 1000 . Network 1000 may include connections of elements of network 1000 such as hubs, bridges, routers, switches, and gateways. It may also include a multi-network 1000 environment, including a public network 1000 such as the Internet and a private network 1000 .

2 is a block diagram of a body-customized coordination system using artificial intelligence according to an embodiment of the present invention, FIG. 3 is a block diagram of a prediction module of the body-customized coordination system using artificial intelligence of FIG. 2, and FIG. 4 is FIG. 2 is a block diagram of a database module of the body-customized coordination system using artificial intelligence, FIG. 5 is a block diagram of the body-customized coordination system using artificial intelligence of FIG. 2 further including an electronic transaction server, and FIG. 6 is FIG. It is a block diagram of a body-tailored coordination system using artificial intelligence, and FIG. 7 is a diagram illustrating a feature point for analyzing a face region of a user image by the prediction module of the body-customized coordination system using artificial intelligence of FIG. 2 .

Referring to FIG. 2 , the body-customized coordination system 100 using artificial intelligence according to an embodiment of the present invention includes a fashion information collection module 110 , a database module 120 , an input module 130 , and a processing module 140 . ), a prediction module 150 , a recommendation module 160 and/or an interface module 170 .

The fashion information collection module 110 may collect fashion information including the first coordination information and the first beauty information from the external server 2000 .

The first coordination information collected by the fashion information collection module 110 may include clothing information, accessory information, hat information, socks information, shoe information, and bag information, and the first beauty information is hair style information and nail art information. and cosmetic information.

Specifically, the first coordination information is information about each fashion product, and the image of the fashion product, the appearance form of the fashion product, the color of the fashion product, the pattern pattern of the fashion product, the length of the fashion product, the material of the fashion product, and the addition of the fashion product Information on a configuration location and size, a sales place of a fashion product, and a price of a fashion product may be included. In addition, information on skin tones, body types, etc. suitable for each fashion product may be included.

For example, costume information includes the appearance of the clothes, the color of the clothes, the pattern of the clothes, the length of the clothes, the material of the clothes, the location and size of the additional components of the clothes, the seller of the clothes, the price of the clothes, and the appearance of the clothes. It may include information about the body type that suits the outfit, the skin tone that suits the color of the clothes, and the like.

The first beauty information is information on hairstyles, nail art, and cosmetics, and includes images of various hairstyles, hair length that suits each face shape, hair color that suits skin tone, hair salon location, hair salon price, and nail art. Information on design, nail art shop location, nail art treatment price, cosmetic type, cosmetic color, skin tone suitable for each cosmetic color, cosmetic size, cosmetic efficacy, cosmetic price, and cosmetics vendor may be included.

For example, the hairstyle information may include information about a hair shape and hair length suitable for a face type, a hair color suitable for a skin tone, a hair length suitable for a face type, and the like.

Meanwhile, the fashion information collection module 110 may collect information about the vehicle. In this case, the vehicle information on the vehicle may include price information of the vehicle, exterior information of the vehicle, color information of the vehicle, year information of the vehicle, model name information of the vehicle, information on the age group that prefers the vehicle, and the like.

The fashion information collection module 110 may collect or crawl fashion information including the first coordination information and the first beauty information from the website.

For example, the fashion information collection module 110 is a representative open market server such as Amazon, Taobao, Coupang, 11st, Gmarket, WeMakep, Google, etc. using a Scrapy library or Google Spreadsheet written based on the Python language. Fashion information can be collected and stored from search site servers such as Naver and Daum, and from Internet shopping mall sites operated by individuals.

However, the fashion information collection module 110 is not limited to the above-described Python language and may be implemented in any way as long as it is a method capable of collecting information from a web site.

Meanwhile, the fashion information collection module 110 may classify and store the collected first coordination information according to the type of each fashion product. In addition, the fashion information collection module 110 may store the collected first beauty information by dividing it by age, gender, race, skin tone, and face type.

The database module 120 may collect the first body information from the external server 2000 and generate a learning model.

In more detail, the database module 120 may include a body information collection unit 121 , a learning unit 122 , and a memory 123 .

The body information collection unit 121 receives from the external server 2000 people's face shape information, race information, skin tone information, age information, body type information, gender information, height information, hair length information, hair color information, and hair shape information. , first body information including nail shape information may be collected.

The body information collection unit 121 may collect the first body information from various external servers 2000 such as websites, blogs, YouTube, and SNS. The external server 2000 is not limited to the above type, and any server may be used as long as it can collect the above-mentioned first body information.

The first body information collected by the body information collection unit 121 includes face shape information, race information, skin tone information, age information, body type information, gender information, height information, and hair according to race, age, and gender of a plurality of people. It may include length information, hair color information, hair shape information, and nail shape information.

The form of the first body information collected by the body information collection unit 121 may be in various forms including images, videos, texts, and the like.

The body information collection unit 121 may classify the collected first body information by race, gender, and age and store it in the memory 123 .

For example, the body information collection unit 121 separates and stores face shape information, skin tone information, body type information, height information, hair length information, hair color information, hair shape information, and nail shape information according to race or by gender. Face shape information, skin tone information, body type information, height information, hair length information, hair color information, hair shape information, and nail shape information can be classified according to the corresponding information.

Here, the face shape information is information about the shape of the face outline, and includes square face shape information, triangular face shape information, inverted triangular face shape information, elliptical face shape information, long oval face shape information, round face shape information, and rhombus face shape information. may include.

The skin tone information may be expressed by being divided into RGB values as skin color information, and may be divided into spring warm tones, summer cool tones, autumn warm tones, and winter cool tones according to skin color.

Body shape information is information about the overall body shape of a person, and is classified according to the shape of the body. It can be classified by diamond body type information.

The height information is information on the straight length from the head to the toes of a person, and may be classified in cm units.

The hair length information is information about the length of human hair, and may be classified into short hair, short hair, medium hair, and long hair according to the length of hair.

The hair color information, as hair color information, may be expressed by being divided into RGB values like skin tone information. Such hair color information may include information on hair color suitable for each skin tone.

The nail shape information is information on the shape of each nail, and may include information on the left length, the right length, the front end length, and the rear end length of the nail, and may be classified into a short nail, a long nail, a wide nail, and the like.

The hair shape information is information on various hair shapes, and may include information on hair shapes suitable for each face shape (eg, short cut, S curl perm, see-through bang, layered cut, etc.).

The learning unit 122 may learn to extract and classify the features of the first image by analyzing the first image made in the form of a single image or a plurality of images among the pieces of information collected by the body information collection unit 121 . .

In this case, the learning unit 122 may generate a learning model by performing learning based on a convolutional neural network (CNN), which is a kind of deep learning technology, and may include a plurality of convolutional neural networks.

Deep learning is a technology that performs learning by classifying a large amount of data based on a multi-layered neural network, tying the same sets to each other, and grasping the upper-lower relationship.

This deep learning technology is a field of machine learning and is widely used for voice and image recognition and photo analysis because it can recognize, reason, and judge by itself.

The convolutional neural network used in the present invention is a kind of deep learning technology, and it generates a learning result as a model, and analyzes and predicts newly acquired information through the generated model.

A convolutional neural network is largely composed of a convolution layer, a pooling layer, an unpooling layer, and a deconvolution layer.

A convolutional neural network extracts object features from an image by alternately performing a convolutional layer and a pooling radar, and reduces the number of dimensions of the extracted features.

The unpooling layer solves the problem of spatial information lost in the pooling layer, and the deconvolution layer is used to generate new values using the learned filter for elements that have been reduced through the unpooling layer.

Meanwhile, the pooling layer serves to reduce the size of the feature map of the convolutional layer. In this case, a max pooling layer or an average pooling layer may be selectively used.

The max pooling layer is a method of cropping a feature map to an arbitrary size, and then extracting the largest value from the image area of the cropped size. This has the advantage of reducing the resources required for calculation because the size of the entire data is reduced.

The average pooling layer reduces the size of the feature map by selecting one average value obtained by averaging the four parameter values in the pooling filter.

Here, there may be one or more pooling layers, or two or more fully connected layers may be configured.

In the fully connected layer, all input nodes are connected to all output nodes, and feature data is actually learned through the fully connected layer.

The fully connected layer introduces feature values extracted from the convolutional layer and the pooling layer into an existing neural network.

At the output end of the fully connected layer, a Softmax function may be used to display each element included in the input image as a probability value.

In addition, the present invention can apply a Leaky Rectified Linear Unit (ReLU) activation function to a feature map extracted by applying a convolution filter.

This is to prevent a gradient vanishing phenomenon that occurs when learning becomes difficult because the layer layer increases as the neural network deepens.

The present invention refers to the difference between the result output from the output end of the convolutional neural network and the expected output value, and then changes the weight value corresponding to the net of the convolutional neural network through a backpropagation algorithm and repeatedly applies the image. It includes the step of allowing the result output from the output end of the convolutional neural network to converge to the expected output value, and finishing the learning when the difference between the actual output value and the expected value is minimized.

The learner 122 may include a convolution layer that converts the first image into a feature map by applying a convolution filter to the first image.

Furthermore, the learning unit 122 may include a pooling layer that reduces the data size of the first image. That is, the learning unit 122 may reduce the size of data obtained by extracting each feature of the first image while repeatedly configuring the convolution layer and the pooling layer to classify the data by type.

For example, the learning unit 122 may extract the features of the first image and classify them by race, gender, and age, or classify them by each fashion product and each beauty product.

For example, the learning unit 122 divides and stores the data extracted from the features of the first image into face type, skin tone, body type, height, hair length, hair color, hairstyle, and nail shape according to race, or stores the data according to gender. Depending on the face type, skin tone, body type, height, hair length, hair color, hairstyle, and nail shape, it can be stored.

Thereafter, the learner 122 may generate first vector information arranged by converting each classified data into a vector value. In this case, the learning unit 122 may analyze one or more first images.

Meanwhile, the learning unit 122 may include a plurality of convolutional neural networks and may generate a plurality of learning models.

The database module 120 may include one or more memories 123 , and the memory 123 may be a flash memory type, a hard disk type, or a multimedia card micro type. ), card type memory (such as SD or XD memory), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read) -Only memory), a programmable read-only (PROM), a magnetic memory, a magnetic disk, and may include at least one type of storage medium among optical disks.

The input module 130 may input a user image obtained by photographing the user's whole body and personal information of the user from the user terminal 10 . In this case, the user terminal 10 may include a smart phone, a tablet computer, a notebook computer, a desktop, and the like.

A user may photograph the whole body using an image acquisition device such as a camera or a camcorder, and the captured image may include a plurality of images or videos as well as a single image.

The user's personal information may include user name information, user location information, user gender information, user age information, user height information, user weight information, and user contact information.

Each item of the user's personal information may be output to the screen of the user terminal 10 . In addition, the user may input personal information about the item output on the screen using the user terminal 10 .

The items for the user's personal information are not limited to the above items, and various items may be set by the administrator.

The input user's personal information is converted into one code and can be used to distinguish a plurality of users.

The processing module 140 may receive the user image input to the input module 130 and perform a pre-processing operation for improving the user image.

For example, the improvement of the received image includes Denoise operation processing, Deblur operation processing, High Dynamic Range operation processing, Color Tone Mapping operation processing, Defog operation processing, Brightness operation processing, Contrast operation processing, Auto White Balance operation processing, Back Light. It may include at least one arithmetic process of a compression arithmetic process and a decompression arithmetic process.

At this time, the pre-processing condition of the processing module 140 may improve the object recognition rate of the prediction module 150 so that the prediction module 150 to be described later can more accurately recognize and analyze the user's whole body, that is, the object in the user image. It is desirable to make it under the conditions that exist.

For example, if the object recognition rate of the prediction module 150 is improved as the sharpness of the user image is higher, the processing module 140 may process the user image on the condition that the sharpness of the user image is increased.

On the other hand, the processing module 140 may pre-process the user image by removing noise and adjusting the contrast, brightness, etc. using the Open CV library.

The prediction module 150 may generate the second body information by analyzing the user image preprocessed by the processing module 140 to predict the user's face shape, skin tone, body type, height length, gender, race, and age.

The prediction module 150 can predict a user's face type, race, skin tone, age, body type, gender, and height by analyzing a user image based on an algorithm of a plurality of convolutional neural networks (CNN). .

In more detail, the prediction module 150 includes a face shape prediction module 151, a skin tone prediction module 152, a body shape prediction module 153, a height prediction module 154, a gender prediction module 155, and a race prediction module 156. ), an age prediction module 157 may be included.

The face shape prediction module 151 may receive the user image input to the input module 130 and detect the user's face contour from the user image to predict the user's face shape.

Detecting facial contours and feature points in user images is accomplished through face recognition algorithms such as Adaboost, SVM (Support Vector, Machine), and AAM (Active Appearance Model). Since it is a well-known technique in the field, a detailed description thereof will be omitted.

For example, the face shape prediction module 151 may predict the user's face shape by extracting the detected user's face contour and inputting it to the learning model generated by the learning unit 122 .

In more detail, the extracted user's facial contour line may be input to the learning model generated by the convolutional neural network algorithm, that is, the learning unit 122 and converted into data, and then, the facial contour information may be extracted and classified through a filter. .

The classified face contour information may be converted into a vector value to generate second vector information.

The face shape prediction module 151 may predict the user's face shape by comparing the second vector information with the first vector information and matching the first vector information most similar to the second vector information.

In this case, the face shape may be predicted as any one of a quadrangle, a triangle, an inverted triangle, an oval, a long oval, a round, and a rhombus. However, the classification of the face type is not limited to the above-described type of the face type and may be variously set by an administrator.

Meanwhile, the face shape prediction module 151 may predict the face shape by detecting the face region and feature points.

In more detail, the face shape prediction module 151 may detect a face region in the user image and detect a plurality of feature points in the detected face region. However, the present invention is not limited thereto, and a smaller number of feature points or a larger number of feature points may be detected as needed.

Referring to FIG. 7 , the first to thirteenth feature points may be feature points (hereinafter, 'face shape points') capable of representing the face shape of the subject, respectively, feature point A is the center of the nose, and feature point B is the eyes. The center between the eye and the C feature point may be a center of the forehead (hereinafter, 'face center point').

The face shape prediction module 151 connects the first to thirteenth feature points, which are the face shape points, with the feature points A, respectively, and then calculates each distance. In addition, the distance is calculated by connecting each of the face shape points with the B feature point and the C feature point. The face shape prediction module 151 may analyze the user's face shape by comparing each calculated distance value. In addition, the value calculated by calculating the curvature and length of the connected line after connecting the first to seventh feature points, and the value calculated by calculating the curvature and length of the connected line after connecting the seventh to thirteen feature points. Facial contour analysis is possible.

The face shape prediction module 151 may predict the user's face shape by comparing the calculation result with a preset value.

On the other hand, the face shape prediction module 151 inputs to the learning model and predicts the user's face shape and the user's terminal ( 10) can output text or images requesting a new user image.

The skin tone prediction module 152 may receive the user image input to the input module 130 , detect the user's skin from the user image, and predict the user's skin tone.

For example, the skin tone prediction module 152 may predict the user's skin tone by detecting the sensed user's skin region and inputting it to the learning model of the learning unit 122 .

In more detail, the skin tone prediction module 152 detects the user's skin area in the user image and extracts color information about the user's skin area.

Thereafter, the extracted user skin color information may be converted into a vector value to generate third vector information.

The skin tone prediction module 152 may predict the skin tone of the user by comparing the third vector information with the first vector information and matching the first vector information most similar to the third vector information.

The learning unit 122 of the database module 120 can predict the skin tone by dividing the user's skin color into RGB values. For example, the learning unit 122 determines that the extracted RGB values of the user's skin color are In the case of #F5F5DC, #FFF8DC, #FFFAF0, the user's skin tone can be distinguished as a summer cool tone, and if the extracted user skin color has RGB values of #FFDEAD, #FFEFD5, #FFDAB9, #FFE4B5, #FFC19E, the user's skin tone is autumn. It can be classified as a warm tone.

However, the classification of skin tones is not limited to the above-described RGB values, and may be flexibly changed by an administrator.

The body shape prediction module 153 may receive the user image input to the input module 130 and detect the user's body from the user image to predict the user's body shape.

Here, the body region means from the neck to the toes excluding the user's head, and means the user's overall outer shape.

For example, the body shape prediction module 153 may predict the body shape of the user by inputting the sensed user's body into the learning model of the learning unit 122 .

In more detail, the body shape prediction module 153 extracts the overall outer shape of the user's body from the user image.

The extracted outer shape of the user's body may be input to the learning model of the learning unit 122 and converted into data, and then, body shape information may be extracted and classified through a filter.

The classified body shape information may be converted into a vector value to generate fourth vector information.

The body shape prediction module 153 may predict the user's body shape by comparing the fourth vector information with the first vector information and matching the first vector information most similar to the fourth vector information.

The height prediction module 154 may receive the user image input to the input module 130 and predict the height of the user by detecting the user's face region and the user's body type from the user image.

In more detail, the height prediction module 154 detects the user's face area and the user's body area from the user image input to the input module 130 .

The height prediction module 154 detects the user's face region, that is, from the uppermost part of the head to the chin, and recognizes it as the head. And the height prediction module 154 detects the user's body region, that is, from the neck to the toes, and recognizes it as the body.

Each measured face region and body region may be input to the learning model of the learning unit 122 .

Each input face region and body region are converted into data, and height information of the user can be extracted and classified through a filter. The classified height information may be converted into a vector value to generate fifth vector information.

The height prediction module 154 may predict the height of the user by comparing the fifth vector information with the first vector information and matching the first vector information most similar to the fifth vector information.

Meanwhile, the height prediction module 154 may analyze the user image and predict the height of the user based on the ratio of the user's face region, body region, and shoulder region.

In detail, first, the height prediction module 154 may store the average face size of men and women by country, the average height of men and women by country, and the average shoulder width of men and women by country as data.

In addition, the height prediction module 154 may receive the user image and detect the user's face region, body region, and shoulder region from the user image.

Thereafter, the height prediction module 154 may measure the sensed linear distance from the top of the head to the chin, and measure the linear distance from the neck to the toes, and the linear distance from the left shoulder to the right shoulder.

The height prediction module 154 may predict the height of the user by comparing each measured value with the average face length, height, and shoulder width values of men and women by country.

The gender prediction module 155 receives the user image input to the input module 130 and detects the user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color from the user image to determine the user's gender. can be predicted.

The gender prediction module 155 may detect the user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color from the user image and input it to the learning model of the learning unit 122 .

The input user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color can be input to the learning model and converted into data, and then, gender information can be extracted and classified through a filter.

The classified gender information may be converted into a vector value to generate sixth vector information.

The gender prediction module 155 may predict the user's body type by comparing the sixth vector information with the first vector information and matching the first vector information most similar to the sixth vector information.

Meanwhile, the gender prediction module 155 may predict the gender by detecting the user's face area, body type area, and feature points.

For example, as shown in FIG. 7 , the gender prediction module 155 may detect a feature point A, a feature point B, a feature point C, and a 14th to 72nd feature point in the user image. However, the present invention is not limited thereto, and a smaller number of feature points or a larger number of feature points may be detected as needed.

A feature point, B feature point, and feature point C are the same face center points as in the face shape prediction module 151, and the 14th to 72nd feature points are feature points that can distinguish the user's facial configuration, that is, eyes, nose, and mouth (hereinafter, 'configuration'). characteristic point').

The gender prediction module 155 may analyze the user's face as a ratio to a distance between a feature point and a face center point and a distance between each feature point.

In more detail, the gender prediction module 155 may calculate the distance between the user's two eyes, the length and width of the nose, the thickness of the lips, and the like by calculating the distance between the constituent feature points and the face center point. Thereafter, the gender prediction module 155 may predict the gender of the user by comparing the calculated result with an average distance between two eyes, an average nose length and width, and an average lip thickness according to a preset gender.

For example, when the preset male average nose length is 4.9 to 5.5 mm, the male average nose width is 4.0 to 4.5 mm, the user's nose length is 52 mm, and the nose width is 4.3 mm, the gender prediction module 155 is It can be predicted to be male.

The race prediction module 156 may predict the user's race by detecting the user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color from the user image.

The race prediction module 156 may detect the user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color from the user image and input it to the learning model of the learning unit 122 .

The inputted user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color can be input to the learning model and converted into data, and then, race information can be extracted and classified through a filter.

The classified race information may be converted into a vector value to generate seventh vector information.

The race prediction module 156 may predict the user's race by comparing the seventh vector information with the first vector information and matching the first vector information most similar to the seventh vector information.

Here, races can be divided into whites, blacks, and yellows.

The age prediction module 157 includes a face shape prediction module 151, a skin tone prediction module 152, a body shape prediction module 153, a height prediction module 154, a gender prediction module 155, and a race prediction module 156. Based on the prediction result, the age of the user can be predicted.

For example, the age prediction module 157 may predict the user's age by comparing the prediction result of each module with the age information of the database module 120, that is, skin tone, face shape, body type, height, gender, and race information according to age. have.

On the other hand, the age prediction module 157 may predict the age of the user by detecting the user's face contour, skin region, body type, eye shape, nose shape, mouth shape, and hair length from the user image, and inputting it to the learning model.

For example, the age prediction module 157 may detect the user's face contour, skin area, body type, eye shape, nose shape, mouth shape, and hair length from the user image and input it to the learning model. Thereafter, the learning model can be classified by converting the input user's face contour, skin region, body type, eye shape, nose shape, mouth shape, and hair length into data, and extracting age information through a filter.

The classified age information may be converted into a vector value to generate eighth vector information.

The age prediction module 157 may predict the age of the user by comparing the eighth vector information with the first vector information and matching the first vector information most similar to the eighth vector information.

The above-described prediction module 150 may analyze the user image to predict the user's face shape, skin tone, body type, height, gender, race, and age, and generate the result as second body information.

In this case, the second body information may be stored in various forms such as text and images.

On the other hand, the prediction module 150 is based on the algorithm of a plurality of convolutional neural networks (CNN), analyzes the user image to predict the user's face shape, skin tone, body type, height, gender, race, age However, the present invention is not limited thereto, and any method may be used as long as it is a method capable of predicting a user's face shape, skin tone, body type, height, gender, race, and age by analyzing a photo.

In addition, the prediction module 150 is not limited to predicting the user's face type, skin tone, body type, height, gender, race, age, and can extract and classify various features of the user by analyzing the user image. have.

The recommendation module 160 may match the fashion information corresponding to the second body information generated by the prediction module 150 .

Thereafter, the recommendation module 160 may generate second beauty information and second coordination information suitable for the user based on the matched fashion information.

For example, if the content of the second body information is a rectangular face shape, an autumn warm tone with an RGB value of #FFC19E, an hourglass body shape, a height of 150 cm to 155 cm, a female, a yellow race, and a 30s, the recommendation module 160 provides the above-described second body Fashion information corresponding to the information may be found and matched in the fashion information collection module 110 .

In more detail, the recommendation module 160 extracts hair style information, hair length information, etc. that can supplement a square face shape according to the user's second body information content, and skin cosmetics suitable for a warm-toned yellow race in autumn. Information, lip cosmetics information, cheek cosmetics information, hair color information, color lens information, etc. may be extracted to generate the second beauty information.

In addition, it complements the hourglass body shape, is suitable for a height of 150 cm to 155 cm, and extracts clothing information suitable for women in their 30s, bag information, shoe information, socks information, hat information, accessory information, etc. can be extracted together to generate the second coordination information.

For example, the recommendation module 160 converts the second body information predicted by the prediction module 150 and the fashion information stored in the fashion information collection module 110 into text, then divides them into token units, and selects meaningless words. You can perform a stopword removal operation to remove.

Thereafter, the recommendation module 160 may distinguish the tokenized body information and the part-of-speech of each word of the fashion information. Accordingly, since unnecessary noise words are removed, the accuracy of keyword extraction can be improved. Thereafter, the tokenized second body information and the words of fashion information may extract a plurality of keywords having the highest values by the TF-IDF method.

Each extracted keyword can improve keyword extraction performance by using the Text-CNN algorithm.

For example, if the prediction module 150 predicts that the second body information is a woman of a yellow race in her 30s, 150 to 155 cm tall, with an hourglass body shape, with a square face shape, autumn warm skin tone, and an hourglass body shape, the recommendation module 160 will display the above-mentioned text. can recognize keywords such as 'square face shape', 'autumn warm tone', 'hourglass body type', '150 to 155 cm', '30s', 'yellow race', and 'woman'.

After that, the recommendation module 160 matches women's hairstyle information suitable for a square face shape in the fashion information collection module 110, and women's clothing information suitable for autumn warm tones, women's accessories information, women's cosmetics information, and women's hairstyles It is possible to match the fashion information corresponding to the second body information, such as matching information, matching women's clothing information suitable for the hourglass body shape, but matching clothing information suitable for a person with a height of 150 to 150 cm.

In this case, the recommendation module 160 may generate second beauty information by dividing information related to beauty among fashion information corresponding to the second body type information, and may generate second coordination information by classifying information related to coordination. .

Here, the beauty information is information related to beauty and may include skin cosmetics information, lip cosmetics information, cheek cosmetics information, hair style information, nail art information, and color lens information, and the coordination information is information related to clothing and miscellaneous goods, including clothing information. , bag information, shoe information, socks information, hat information, and accessory information.

On the other hand, the recommendation module 160 may generate the third beauty information and the third coordination information by matching fashion information that does not match the second body type information, that is, should be avoided, based on the second body type information.

For example, when the user's skin tone is a warm autumn tone and the user's body type is a diamond body type, the recommendation module 160 provides information on pink lip cosmetics, cheek cosmetics information, and hair color that do not match the autumn warm tone in the fashion information collection module 110 . The third beauty information may be generated by matching the information, and the third coordination information may be generated by matching information on a short top that does not match the diamond body shape, information on a tight top, and information on a short bottom length at the top.

That is, the recommendation module 160 matches and provides fashion information that does not suit the user on the basis of the user's second body information, so that when the user performs shopping or coordination by himself in the future, he or she purchases or coordinates a product that does not suit the user. can be prevented from doing

On the other hand, the recommendation module 160 may generate second beauty information and second coordination information suitable for the season, place, and situation to the user according to the specified season, place, and situation.

A user may set a season, a place, and a situation in the recommendation module 160 through the user terminal 10 . At this time, the season can be designated as one of spring, summer, autumn, and winter, and the location can be designated as one of various places such as a wedding hall, a funeral hall, a company, a house, an airport, a hotel, a restaurant, a court, a hospital, etc. , party, blind date, meeting, interview, athletic meet, travel, visit, funeral, etc. can be designated as one of various situations.

When the user sets summer, restaurant, and banquet through the user terminal 10 , the recommendation module 160 may generate second beauty information and second coordination information corresponding to the second body type information.

For example, the recommendation module 160 includes second beauty information including information on a simple makeup method suitable for a meeting and summer, information on color cosmetics having a color similar to the skin tone, etc., and information on clothes that can be worn in summer. Second coordination information may be generated.

According to the recommendation module 160, the second beauty information and the second coordination information matching the season, place and situation can be appropriately provided to the user, so that the user can easily coordinate according to the situation.

Meanwhile, the recommendation module 160 may determine the location of the user through the user terminal 10 possessed by the user, and may generate second beauty information and second coordination information according to the location characteristics of the user.

Here, the user terminal 10 is provided to receive a GPS signal, and may receive a GPS signal from a plurality of satellites.

The user terminal 10 receives GPS signals from three or more artificial satellites, analyzes the received GPS signals, and measures a pseudorange with each artificial satellite. In addition, the user terminal 10 may generate location information on the current location based on the measured pseudorange.

The generated location information is transmitted to the recommendation module 160 , and the recommendation module 160 may receive information about the weather in the location where the user is located from the external server 2000 based on the location information of the user.

Thereafter, the recommendation module 160 may generate second beauty information and second coordination information corresponding to the weather of the user's location.

For example, when it rains where the user is located, the recommendation module 160 may generate the second coordination information by matching rain-related boots information, rain gear information, and umbrella information. In addition, the second beauty information may be generated by matching the waterproof cosmetic information and hairspray information that is strong against moisture.

According to the recommendation module 160 , the second coordination information and the second beauty information corresponding to the weather of the location where the user is located may be provided to the user.

Meanwhile, the recommendation module 160 may provide the user with vehicle information suitable for the second coordination information based on the generated second beauty information and the second coordination information.

For example, the second body type information is 'a yellow male in his thirties with a height of 170 to 175 cm with an hourglass body shape with an autumn warm skin tone of a square face shape', and the user reports the current season, place, and situation through the user terminal 10 . When winter, hotel, and blind date are designated, the recommendation module 160 may generate second beauty information and second coordination information by matching fashion information corresponding to the second body type information. After that, the recommendation module 160 extracts information on vehicles for mid-size or higher luxury cars preferred by men in their 30s and suitable for hotels and blind dates based on the second coordination information and the season, place, and situation specified by the user to extract vehicle information. can create

The generated second beauty information, second coordination information, and vehicle information may be provided to the user on a screen through the user terminal 10 , and the user may select desired vehicle information through the user terminal 10 .

When the user selects any one vehicle information through the user terminal 10 , the user terminal 10 may access a vehicle sharing reservation server of a vehicle sharing company.

According to this, the user can easily share the vehicle by receiving a recommendation for a vehicle suitable for the user's situation and body type.

Meanwhile, the recommendation module 160 may provide the user with restaurant information suitable for the second coordination information based on the generated second beauty information and the second coordination information.

For example, if the second body type information is 'a woman in her twenties who is 150 to 155 cm tall and has a warm skin tone in autumn with a square face shape and an hourglass body shape,' and the user selects the current season, place, and situation through the user terminal 10 in summer , restaurant, and when designated as Sanggyunrye, the recommendation module 160 generates second beauty information and second coordination information by matching fashion information corresponding to the second body information, and at the same time generates restaurant information suitable for the second coordination information. can do.

In this case, the restaurant information may include the location of the restaurant, the menu of the restaurant, the interior of the restaurant, the menu price of the restaurant, the main age group of the restaurant, the size of the restaurant, the number of employees of the restaurant, and the like.

In addition, the recommendation module 160 may match and provide restaurant information suitable for the second coordination information to the user based on the second coordination information and the restaurant information.

The interface module 170 may configure each generated information (second beauty information, third beauty information, second beauty information, and third beauty information) as a user interface and provide it to the user through the user terminal 10 . .

The customized coordination system 100 using artificial intelligence may further include an electronic transaction server 180 .

The electronic transaction server 180 may mediate a transaction between a user and a product and a company included in the second coordination information and the second beauty information recommended to the user.

More specifically, when the user selects any one of the second coordination information and the second beauty information provided to the user through the user terminal 10, the user terminal 10 may be provided with a server related to the information selected by the user. have.

For example, when the user selects skin cosmetics information among the second beauty information displayed on the screen of the user terminal 10 , the user may access a server selling skin cosmetics. In addition, when the user selects nail art information among the second beauty information displayed on the screen of the user terminal 10 , the user may access the nail art shop treatment reservation server.

On the other hand, when the user selects any one of the second coordination information and the second beauty information, the electronic transaction server 180 may provide information on a place where products or procedures related to the information selected by the user are performed offline. .

In this case, the information on the place where the product is sold may be a seller within 5 km from the user's location. In addition, information on a place where goods are sold may be provided to the user by being arranged in the order of the distance from the user to the nearest.

According to another embodiment of the present invention, the body-tailored coordination system 100 using artificial intelligence may further include a weight module (not shown).

The weight module may identify the user's preferred item and assign a weight to each user's item preference.

According to the present embodiment, the weight module may receive a list of items searched by the user from the user terminal 10 . The list may include product name information, product price information, product shape information, product color information, and the like.

In addition, the weight module may receive an item list of items stored in the shopping cart of the user from a plurality of shopping mall servers to which the user has subscribed.

In this case, the weight module may receive a list of items stored in the shopping cart for less than 3 months from the date of receipt.

Thereafter, the weight module may determine the user's preference for each item by calculating the number of searches and/or storage of the above-described item list.

For example, if the user searches for 'sprite tea' 10 times through the user terminal 10 and stores three 'sprite tea' in the shopping mall shopping cart, the weight module counts the number of searches as 1 point each, and the shopping cart shopping cart By counting the number of storage as 1 point, the user's preference for 'Sprite Tea' can be calculated as 13 points. Thereafter, the weight module may score and store user preferences for each article.

The scored user preference for each item is transmitted to the recommendation module 160 , and the recommendation module 160 may generate second beauty information and second coordination information by preferentially detecting an item having a high user preference.

In the present specification, the fashion information collection module 110, the database module 120, the input module 130, the processing module 140, the prediction module 150, and the recommendation included in the body-customized coordination system 100 using artificial intelligence in the present specification. The module 160 , the interface module 170 , etc. may be processors that execute consecutive execution processes stored in a memory. Alternatively, it may operate as software modules driven and controlled by a processor. Furthermore, the processor may be a hardware device.

In the above, even though it has been described that all the components constituting the embodiment of the present invention operate by being combined or combined into one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, terms such as "include", "comprise" or "have" described above mean that the corresponding component may be inherent, unless otherwise stated, so that other components are excluded. Rather, it should be construed as being able to further include other components. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms commonly used, such as those defined in the dictionary, should be interpreted as being consistent with the contextual meaning of the related art, and are not interpreted in an ideal or excessively formal meaning unless explicitly defined in the present invention.

And, the above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: Body-tailored coordination system using artificial intelligence according to an embodiment of the present invention
110: fashion information collection module
120: database module
130: input module
140: processing module
150: prediction module
160: recommended module
170: interface module

Claims (5)

First coordination information including various clothes information, accessories information, hat information, socks information and shoes information, and fashion information including first beauty information including hair style information, nail art information and cosmetics information from an external server; , a fashion information collection module that collects vehicle information and restaurant information;
A first body including face shape information, race information, skin tone information, age information, body type information, gender information, height information, hair length information, hair color information, hair shape information and nail shape information of a plurality of people from an external server a database module that collects information and generates a learning model by learning each information based on a convolutional neural network;
an input module into which a user image obtained by photographing the user's whole body and personal information of the user is input;
a processing module receiving the user image from the input module and pre-processing the user image;
Based on the information of the database module, by analyzing the user image processed by the processing module, the user's face shape, the user's skin tone, the user's body type, the user's height, the user's gender, the user's race, and the user's age a prediction module for predicting and generating second body information;
By matching the fashion information corresponding to the second body information generated by the prediction module, second beauty information and second coordination information for a user, and third beauty information and third coordination information that do not correspond to the second body information a recommendation module for generating information;
an interface module for receiving the second beauty information, the second coordination information, and the third beauty information and the third coordination information from the recommendation module and providing them to a user;
an electronic transaction server for the transaction of the second beauty information and the product included in the second coordination information; and
A weighting module for receiving the list of items searched by the user and the list of items in the shopping cart to which the user has subscribed from the user terminal possessed by the user, scoring the user preference for each item, and transmitting it to the recommendation module,
The prediction module,
A face that detects a face region in the user image, detects a plurality of feature points in the detected face region, connects the plurality of feature points, and predicts the user's face shape through values calculated by calculating the curvature and length of the connected line type prediction module;
a skin tone prediction module that detects the user's skin from the user image and predicts the user's skin tone by matching the skin tone most similar to the user's skin tone by comparing it with the first body information of the database module;
a body shape prediction module for detecting a user's body in the user image, and predicting the user's body shape by matching the body shape most similar to the user's body shape by comparing it with the first body information of the database module;
a height prediction module for detecting the user's face and the user's body from the user image, and predicting the height of the user by comparing it with the first body information of the database module;
a gender prediction module for detecting the user's face contour, eye shape, nose shape, mouth shape, hair length, body type, and skin color from the user image, and predicting the user's gender by comparing it with the first body information of the database module;
a race prediction module that detects the user's face contour, eye shape, nose shape, mouth shape, hair length, body shape, and skin color from the user image, and compares it with the first body information of the database module to predict the user's race; and
Based on the prediction results of the face shape prediction module, the skin tone prediction module, the body shape prediction module, the height prediction module, the gender prediction module, and the race prediction module, an age prediction module for predicting the age of the user;
The recommended module is
Recommending the second beauty information and the second coordination information matching the season, place and situation set by the user to the user,
receiving the user preference for each item from the weighting module, preferentially detecting an item having a high user preference, and generating the second beauty information and the second coordination information;
Based on the second beauty information and the second coordination information, it is characterized in that the vehicle information and the restaurant information corresponding to the second coordination information are recommended to the user,
The electronic transaction server,
When the user selects at least one of the second beauty information and the second coordination information through the user terminal, it is connected to the electronic transaction server,
Characterized in that the second beauty information selected by the user and information on a place for selling a product related to the second coordinating information and a place for performing a procedure are provided to the user by arranging them in the order of the distance from the user to the nearest. Body-tailored coordination system using artificial intelligence.
delete The method according to claim 1,
The second beauty information,
including skin cosmetics information, lip cosmetics information, cheek cosmetics information, hair style information, nail art information, and color lens information corresponding to the second body information,
The second code information,
The body-tailored coordination system using artificial intelligence, characterized in that it includes clothes information, bag information, shoe information, socks information, hat information, and accessory information corresponding to the second body information.
The method according to claim 1,
The recommended module is
Recommending the second beauty information and the second coordination information matching the weather information by identifying the location of the user through the user terminal possessed by the user, collecting weather information on the location of the user from an external server Body-tailored coordination system using artificial intelligence. delete

Images