KR20230045633A - A computer program for simulating hairstyles - Google Patents

Abstract

According to an embodiment of the present invention, a program for executing a method of operating a hairstyle simulation device in a computer, comprises: a step of performing convolutional processing on pre-collected artificial intelligence learning data to construct a model architecture of a bald head transformation model for bald head transformation so as to construct the bald head transformation model; a step of applying a user image in which hair and background areas are segmented, to the bald head transformation model to generate head information based on a head image in which the hair and background areas are inpainted as a head area; a step of performing a realistic hairstyle simulation based on the head information and user analysis information corresponding to the user image; and a preference score prediction model processing step of evaluating a preference score of a customized hairstyle simulation to provide a recommendation service. Therefore, a realistic hairstyle simulation can be performed by generating head information based on a bald head transformation model.

Description

A computer program for simulating hairstyles

The present invention relates to a hair style simulation computer program. More specifically, the present invention relates to artificial intelligence model-based bald hair transformation and a program for executing a realistic hair style simulation using the same on a computer.

A virtual hair styling service using a virtual hair image has a problem in that it takes a lot of time and money to produce a virtual hair image, and a need to solve this problem has been raised.

In addition, since the virtual hair image extracted from the human head image is extracted based on a simple binary mask or binary mask, the quality of the hair image is remarkably low.

In addition, there is a problem in that the virtual hair image does not naturally overlap with the customer's head or face, causing awkwardness, and there is a need to solve this problem.

The present invention has been made to solve the above problems, and converts a bald head image of a person with hair removed from a person's hair image through an AI algorithm, and a realistic hair style simulation device based thereon and a method of operating the same And, the purpose is to provide a computer program for executing the method on a computer and a computer-readable recording medium on which the program is recorded.

An apparatus according to an embodiment of the present invention for solving the above problems is a hair style simulation device, comprising: a learning model building unit for building a bald head conversion model; a head image generating unit generating head image information based on the bald head transformation model from a user image; and a realistic hair style simulation providing unit that performs a realistic hair style simulation based on the head image information and user analysis information corresponding to the user image.

In addition, a method according to an embodiment of the present invention for solving the above problem, in the hair style simulation method, constructing a bald hair conversion model; generating head image information based on the bald head transformation model from a user image; and performing realistic hair style simulation based on the head image information and user analysis information corresponding to the user image.

In addition, the method according to the embodiment of the present invention for solving the above problems may be implemented as a computer readable recording medium and a computer program for executing the method on a computer.

According to an embodiment of the present invention, a realistic hair style simulation may be performed by constructing a bald hair transformation model and generating head image information based on the bald hair transformation model from a user image.

Accordingly, according to an embodiment of the present invention, it is possible to convert a bald head image of a person whose hair has been removed from a person's hair image through an AI algorithm, and provide a realistic and realistic hair style simulation device based on the AI algorithm and an operation method thereof. .

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.
2 is a block diagram showing a hair simulation device according to an embodiment of the present invention in more detail.
3 to 11 are diagrams for explaining in detail the operation of each element of the hair simulation device according to an embodiment of the present invention.

The following merely illustrates the principles of the present invention. Therefore, those skilled in the art can invent various devices that embody the principles of the present invention and fall within the concept and scope of the present invention, even though not explicitly described or shown herein. In addition, it is to be understood that all conditional terms and embodiments listed herein are, in principle, expressly intended only for the purpose of making the concept of the present invention understood, and not limited to such specifically listed embodiments and conditions. It should be.

Further, it should be understood that all detailed descriptions reciting specific embodiments, as well as principles, aspects and embodiments of the present invention, are intended to encompass structural and functional equivalents of these matters. In addition, it should be understood that such equivalents include not only currently known equivalents but also equivalents developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of exemplary circuits embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc., are meant to be tangibly represented on computer readable media and represent various processes performed by a computer or processor, whether or not the computer or processor is explicitly depicted. It should be.

The functions of various elements shown in the drawings including functional blocks represented by processors or similar concepts may be provided using dedicated hardware as well as hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be construed as exclusively citing hardware capable of executing software, but without limitation, digital signal processor (DSP) hardware, ROM for storing software (ROM), random access memory (RAM) and non-volatile memory. Other hardware for the governor's use may also be included.

In the claims of this specification, components expressed as means for performing the functions described in the detailed description include, for example, a combination of circuit elements performing the functions or all types of software including firmware/microcode, etc. It is intended to include any method that performs the function of performing the function, combined with suitable circuitry for executing the software to perform the function. Since the invention defined by these claims combines the functions provided by the various enumerated means and is combined in the manner required by the claims, any means capable of providing such functions is equivalent to that discerned from this disclosure. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.

The entire system according to an embodiment of the present invention may include a mirror display device 100, a realistic hair simulation device 600, and an artificial intelligence model 700.

The realistic hair simulation device 600 may be connected to each mirror display device 100 and the artificial intelligence model 700 through a wired/wireless network to provide services according to an embodiment of the present invention, and mutual communication is performed. can be done

Here, each network is a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a Personal Area Network (PAN), a mobile communication network ( It can be implemented in all types of wired/wireless networks, such as a mobile radiocommunication network) or a satellite communication network.

The mirror display device 100 is any one of a computer, a mobile phone, a smart phone, a smart pad, a laptop computer, a personal digital assistant (PDA), and a portable media player (PMP). It may be at least one multi-device of an individual device or a common device such as a kiosk or a stationary display device installed in a specific place, and provides user information to the realistic hair simulation device 600, and the realistic hair simulation device ( 600) may receive and output the processed information.

In such a system configuration, the realistic hair simulation device 600 constructs the bald head conversion model as the artificial intelligence model 700, and the head image based on the bald head conversion model is obtained from the user image received from the mirror display device 100. information is generated, and realistic hair style simulation is performed based on the head image information and user analysis information corresponding to the user image.

In addition, the system according to an embodiment of the present invention is a mirror display device 100 installed in a target facility such as a beauty salon and a customer using the mirror display device 100 through a communication network N (hereinafter collectively referred to as "customer"). .) Personal information management server 200 that manages personal information, advertisement providing server 300 that provides advertisement information deemed beneficial to customers, etc., beauty-related products such as shampoo and cosmetic materials provided to customers ( It is provided with a product server 400 that manages (hereinafter collectively referred to as “products”). In addition, the communication network N includes a communication network through which information can be exchanged between the mirror display device 100, the personal information management server 200, the advertisement providing server 300, and the product server 400. A network disclosed in this specification includes, for example, a wireless network, a wired network, a public network such as the Internet, a private network, a Global System for Mobile communication network (GSM) network, a general packet wireless network (General Packet Radio Network (GPRN), Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN), Cellular Network, Public Switched Telephone Network PSTN), Personal Area Network, Bluetooth, Wi-Fi Direct, Near Field communication, Ultra-Wide band, a combination thereof, or any It may be another network of, but is not limited to these.

The personal information management server 200 is constituted by, for example, a computer with high arithmetic processing capability, and realizes server functions by executing a predetermined server program on the computer. Here, the computer constituting the personal information management server 200 does not necessarily have to be one, and may be composed of several computers distributed in the communication network (N). The personal information management server 200 is connected to the mirror display device 100 installed in each facility (assumed to be a hair salon in this embodiment) through a communication network N. The personal information management server 200 includes a personal information DB for managing personal information of customers of each facility and a processor including CPU, ROM, and RAM. The personal information DB includes a customer management table (TA1). In addition, the personal information management server 200 may also store information on a date visited by the customer in the past and information on a hair style performed on the date of the visit. These components may be implemented in hardware or software, or through a combination of hardware and software.

Also, in this embodiment, a state in which the mirror display device 100 is introduced to several beauty salons is exemplified, but the number of beauty salon stores may be one. Of course, the place where the mirror display device 100 is installed is not limited to a beauty salon and can be applied to various facilities (clothing-related stores, health clubs, entertainment facilities, etc.) introducing the mirror display device 100.

The advertisement providing server 300 may select an advertisement suitable for the customer according to a preset advertisement selection algorithm based on the customer's personal information transmitted from the mirror display device 100 . For example, the advertisement providing server 300 displays one or several other advertisement information about the most popular sports, cooking, housing, etc. of the household by using the gender, age, etc. included in the customer's personal information. can choose

And, the advertisement providing server 300 may transmit the selected one or more advertisement information to the mirror display device 100 of the store.

The product server 400 is composed of, for example, a server computer and has a product DB and a processor including CPU, ROM, and RAM. In the product DB, product management that manages the identification ID of each product, inventory status, order status, etc. A table TA2 is registered. Also, the electronic catalog stored in the storage device may be stored in the product DB of the product server 400 and transmitted to the mirror display device 100 .

According to this process, the realistic hair simulation device 600 can provide a realistic hair style simulation through the mirror display device 100 based on the previously built artificial intelligence model 700, and the user's A customized hair style recommendation suitable for the head shape and color change simulation may be provided.

2 is a block diagram showing a hair simulation device according to an embodiment of the present invention in more detail.

Referring to FIG. 2 , the realistic hair simulation device 600 includes a data collection unit, a learning model construction unit, a bald head conversion based head generation unit, a realistic hair style simulation providing unit, a color simulation providing unit, and a style API providing unit. do.

The data collection unit collects data and learning data for operation of the realistic hair simulation device 600 .

And, the learning model construction unit constructs a model architecture of the bald transformation model for the bald transformation by convolutional processing the artificial intelligence learning data collected in advance. The built architecture can be stored and managed through the artificial intelligence model 700.

In addition, the bald head conversion-based head generation unit generates the head image based on the head image in which the hair and background regions are inpainted as head regions by applying the user image in which the hair and background regions are subdivided to the bald head conversion model. .

The realistic hair style simulation providing unit performs a customized hairstyle simulation using at least one of fashion feature information and biometric information analysis information obtained from the user analysis information.

Here, the realistic hair style simulation providing unit may collect hair style search word information for the customized hairstyle simulation.

Furthermore, the realistic hair style simulation providing unit may include a preference score prediction model processing unit that evaluates preference scores of the customized hairstyle simulation and provides a recommendation service.

The style API provider may process API responses by generating realistic hair style simulation information corresponding to API requests from the mirror display device 100 and other various user terminals. For example, the style API provider may provide a virtual dyeing experience API, a hair style information API, and a hair simulation API.

Accordingly, the realistic hair simulation device 600 can provide a more realistic hair simulation through generating a head image based on bald head conversion, and can recommend a style suitable for the user's fashion or biological characteristics, and respond accordingly. It is possible to effectively provide color conversion and product recommendation services.

3 to 11 are diagrams for explaining the operation of each element of the hair simulation device according to an embodiment of the present invention in more detail.

Referring to FIG. 3 , the realistic hair simulation device 600 according to an embodiment of the present invention synthesizes a virtual head image generated by recognizing an object when a portrait photo/video is input and a material image of hair style recommendation based on preference. Accordingly, it is possible to provide realistic hair style simulation, including a hair extraction model processing unit for head generation, a hair region recognition unit, a hair image extraction unit, a hair region removal unit, an image inpainting unit, and a virtual head generation unit. It may include a target recognizing unit for recommendation, a fashion feature analysis model processing unit, a biometric information analysis model processing unit, a preference score prediction model processing unit, a hair style database unit, and a preference-based style recommendation unit.

In addition, the realistic hair simulation device 600 may provide color simulation by driving the hair mask generation unit of the color simulation providing unit and the hair color transfer processing unit, and the hair material analysis information is stored and managed and transmitted to the hair style database. It can be.

4 is a diagram for explaining a style API providing unit of the realistic hair simulation device 600 according to an embodiment of the present invention.

As shown in FIG. 4 , the style API provider may generate realistic hair style simulation information corresponding to API requests from the mirror display device 100 and various other user terminals, and process API responses. For example, the style API provider may provide a virtual dyeing experience API, a hair style information API, and a hair simulation API.

5 is a diagram for explaining a learning model building unit according to an embodiment of the present invention.

Referring to FIG. 5 , the learning model building unit according to an embodiment of the present invention may perform partial convolution processing and gate convolution processing, and classify and combine them to construct a model architecture for shaved head transformation. .

And, FIGS. 6 and 7 show a hair style synthesis process according to hair region recognition and head generation based on bald hair transformation.

First, as shown in FIG. 6 , the bald head transformation-based head generation unit according to an embodiment of the present invention may include a hair and background region segmentation unit, a bald head transformation execution unit, a head generation modeling unit, and a head region inference unit.

)의 선형결합으로 표현할 수 있으므로,

로 표현되고, 이를 이용하여

를 산출할 수 있다. 여기에서, 대상 영상에 대한 바이너리 맵(binary map)을 확장하여 트리 맵(trimap)을 생성할 수 있고, 이를 이용하여 F와 B를 계산할 수 있으며, 각 픽셀의 RGB 값이 3차원 벡터임으로 전술한 픽셀 값(I)의 수식을 변경하여 각 픽셀의 투명도(

)를 계산할 수 있다.The hair and background region subdivision unit may extract the hair region using, for example, a GrabCut algorithm, and finely refine the hair in the hair region by applying alpha matting to the boundary of the extracted hair region. And you can express yourself naturally. Since the hair region has various shapes and is always attached to the face region, it is difficult to accurately extract the hair region only by setting a bounding box, that is, specifying an extraction region. Since hair has very fine and complex boundaries, there are limitations in expressing it only in binary images. Accordingly, the segmentation unit may apply alpha matting that combines the hair of the hair region with the background image based on the alpha value of each pixel in the hair region. When alpha matting is applied, transparency expressed as a gray image of at least 8 bits instead of a binary image can be calculated for the target area, and fine details of the hair can be expressed through this. In general, an image has a foreground (F) and a background (B), and transparency (

) can be expressed as a linear combination of

is expressed as, using this

can be calculated. Here, a tree map can be created by extending a binary map for the target image, F and B can be calculated using this, and since the RGB value of each pixel is a 3-dimensional vector, as described above, By changing the formula of the pixel value (I), the transparency of each pixel (

) can be calculated.

는

이므로 수학식

를 통해서 산출할 수 있다. 여기에서,

는 픽셀 값,

는 전경 영상,

는 배경 영상이다. 따라서, 제어부(530)는 합성 영상의 각 픽셀 값(

)을 수학식

을 통해서 산출할 수 있다. 여기에서,

는 합성할 배경 영상이다.Then, the realistic hair simulation device 600 generates a mask having a value of 0 to 1 using the transparency value of each pixel of the target image, and then, based on the generated mask, the hair region 1250 is created as a new background. A synthesized image synthesized with an image may be generated. transparency value

Is

so the equation

can be calculated through From here,

is the pixel value,

is the foreground image,

is the background image. Therefore, the controller 530 controls each pixel value of the synthesized image (

) to the equation

can be calculated through From here,

is the background image to be synthesized.

Meanwhile, FIGS. 8 to 10 show biological characteristics analysis and fashion characteristics analysis processes for user information customized simulation of the realistic hair simulation device 600 according to an embodiment of the present invention.

As shown in FIG. 8 , the realistic hair simulation device 600 according to an embodiment of the present invention analyzes the user's biological characteristics (face shape, age, skin color, etc.) and fashion characteristics to provide customized hair style simulation with high satisfaction. can be provided with

To this end, the realistic hair simulation device 600 uses face embedding processing using a multi-label classifier based on a CNN model, feature point extraction processing based on regression analysis using a CNN, recommendation information processing using a binary classifier, and image infating technology. It is possible to perform a hair synthesis treatment using

9 shows a biometric characteristic analysis model in more detail, using a face detection technology, a basic information extraction unit for extracting basic information such as race, gender, age, skin, etc., face shape, pupil size, color, etc. It may include a facial feature extraction unit that extracts the biometric features of.

10 shows a fashion feature analysis model in more detail. The wearing clothes extractor can extract worn clothes information based on the human segmentation model, and the fashion feature analyzer can extract and classify feature points from the worn clothes image.

On the other hand, FIG. 11 shows the preference score prediction model of the realistic hair simulation device 600 in more detail. The preference score prediction model determines whether a portrait and hair style match well, and whether a face and fashion style match well. etc. can be calculated by scoring. This can use a model created by performing dataset learning based on preference score measurement through an expert, and a preference prediction model that uses face embedding vectors, facial feature points, and hair styles as input information and scores as output information. It can be created in advance on a learning basis.

The method according to the present invention described above may be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, and magnetic tape. , floppy disks, and optical data storage devices.

The computer-readable recording medium is distributed to computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (5)

As a program for executing the operation method of the hair style simulation device on a computer,
constructing a bald transformation model by constructing a model architecture of the bald transformation model for bald transformation by convolutional processing of pre-collected artificial intelligence training data;
generating the head image information based on a head image in which the hair and background regions are inpainted as head regions by applying a user image in which hair and background regions are subdivided to the bald head conversion model;
performing realistic hair style simulation based on the head image information and user analysis information corresponding to the user image; and
A preference score prediction model processing step of evaluating the preference score of the customized hairstyle simulation and providing a recommendation service
A program for executing the operation method of the hair style simulation device on a computer. According to claim 1,
The above steps are
Performing a customized hairstyle simulation using at least one of fashion feature information or biometric information analysis information obtained from the user analysis information
A program for executing the operation method of the hair style simulation device on a computer. According to claim 2,
The above steps are
Collecting hair style search word information for the customized hairstyle simulation
A program for executing the operation method of the hair style simulation device on a computer. According to claim 1,
The above steps are
A basic information extraction unit that constructs a biometric characteristic analysis model for user information analysis, applies the user's image to the biometric characteristic analysis model, and extracts basic information such as race, gender, age, and skin based on face detection; Extracting biometric features such as shape, pupil size, and color
A program for executing the operation method of the hair style simulation device on a computer. According to claim 1,
The above steps are
For user information analysis, human segmentation model-based wearing clothing information may be extracted, and the fashion feature analysis unit further comprises extracting and classifying feature points from the wearing clothing image
A program for executing the operation method of the hair style simulation device on a computer.

Images