KR20220135526A - System for providing virtual cloth fitting service using three-dimensional avatar - Google Patents

Abstract

A system for providing a try-on service using a 3D avatar is provided. The system comprises: a user terminal that uploads body information including the size of at least one part of the human body, uploads a photo including face information, outputs a 3D avatar generated based on the body information and face information, and outputs a result of extracting clothes having a size corresponding to the body information on the 3D avatar and trying the clothes on; and a try-on service providing server including a storage unit that tags and stores a size and a material in clothes data of at least one category of clothes, a conversion unit that converts into 3D data by performing 3D modeling based on an image and a size in the clothes data, a generation unit that generates a 3D avatar based on the body information and the face information in the case of inputting the body information and the face information in a user terminal, a try-on unit that extracts the 3D data of clothes having a size corresponding to the body information on the 3D avatar and tries the clothes on, and a transmission unit that transmits the result of the try-on to the user terminal. Accordingly, the present invention can provide information that can only be known by actually wearing clothes.

Description

A system for providing try-on service using an avatar {SYSTEM FOR PROVIDING VIRTUAL CLOTH FITTING SERVICE USING THREE-DIMENSIONAL AVATAR}

The present invention relates to a system for providing a try-on service using a 3D avatar, and provides a platform for creating a 3D avatar using body information and face information and then performing a simulation of trying on clothes.

Due to the recent development of graphic technology, virtual clothing is widely used in various fields such as the clothing industry, and this 3D virtual fitting technology provides customized services to individual customers by reflecting the size and taste of individual customers in the mass production system. It is becoming a core technology that can be provided. The types of 3D virtual fitting programs currently in use include 3D Runway Designer, CLO 3D, i-Designer, 3D-Fit, Narcis, V-Stitcher, DC-suit, etc. Among them, CLO 3D and DC-Suit are 3 A dimensional fashion show is possible, and in Korea, preference for CLO 3D, which can be linked with the pattern CAD program YUKA, is increasing. In addition, as an example of the use of a three-dimensional virtual fitting program, recently, fashion startup platform Iparo has been conducting group fashion shows every first half and second half using CLO 3D.

At this time, a method of creating and trying on a three-dimensional avatar using the customer's body information and face information was researched and developed. And in Korean Patent Application Laid-Open No. 2020-0025291 (published on March 10, 2020), in order to generate a body avatar based on body information, receiving at least one body information from a user's weight, height, and foot pressure, and The user's current body condition is calculated based on the information, and the user's body avatar is generated based on the user's current body condition. After extracting features such as , skin texture, and hairstyle, a virtual 3D avatar is created based on the user's face image and input body information in the activated 3D avatar, and matching colors are diagnosed and recommended based on the user's face color. , a configuration of fitting and outputting clothing having a recommended color to a 3D avatar, and performing a purchase and payment process to purchase the fitted clothing are disclosed.

However, in the above configuration, the 3D scanner has to rotate the customer's body 360 degrees and equipping it with expensive equipment such as a 3D scanner is not only expensive, but even if it is equipped, it is Customers have limited access to using it. In addition, even if you use a try-on program, you cannot know how much space there is or how tight it is to the extent that it feels heavy or light because you are not actually trying it on. Accordingly, research and development of a platform capable of generating a 3D avatar with only body information even without a 3D scanner and taking into account various physical forces when wearing clothing is required to produce the same effect as wearing clothes.

According to an embodiment of the present invention, a 3D avatar is created using the user's body information, materials and sizes are databased using tags including at least one clothing and fashion detailed information, and then 3D modeling is performed. In the case of a material that stretches even at the same size by generating clothing data, considering the user's taste while selecting clothing of the size that fits the 3D avatar, and further taking into account physical forces such as tension, gravity, and elasticity, which are the extent to which it is stretched according to the material of the clothing A 3D avatar that can provide information that you have to try on, such as a gap between the buttons or a case in which the button does not fit, if it cannot be applied even though it can be worn with a tight fit, or if there is no room due to the inelastic material. It is possible to provide a method of providing a try-on service using However, the technical task to be achieved by the present embodiment is not limited to the above-described technical task, and other technical tasks may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention uploads body information including the size of at least one part of the human body, uploads a photo including face information, and then uploads the body information and a user terminal that outputs a 3D avatar generated based on face information, extracts clothes having a size corresponding to the body information on the 3D avatar, and outputs a result of trying on, and a size in the clothes data of at least one category of clothes and a storage unit that tags and stores materials, a conversion unit that performs 3D modeling based on images and sizes in clothing data and converts them into 3D data, body information and face information when inputting body information and face information from a user terminal A try-on service comprising a generator that generates a 3D avatar based on Includes server.

According to any one of the above-described problem solving means of the present invention, a 3D avatar is created using the user's body information, and materials and sizes are databased using tags including at least one clothing and fashion detailed information. After that, 3D clothing data is generated by 3D modeling, considering the user's taste while choosing clothing of the size that fits the 3D avatar, and furthermore, by considering physical forces such as tension, gravity, and elasticity, which are the degree of stretching according to the material of the clothing, the same In the case of a material that stretches in size, it cannot be applied even though it can be worn in a tight fit, or if there is no room because it is not elastic at all, it provides information that you can only know when you try it on, such as when there is a gap between the buttons or when it does not fit. can do it

1 is a diagram for explaining a system for providing a try-on service using a 3D avatar according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a try-on service providing server included in the system of FIG. 1 .
3 and 4 are diagrams for explaining an embodiment in which a try-on service using a 3D avatar according to an embodiment of the present invention is implemented.
5 is a flowchart illustrating a method of providing a try-on service using a 3D avatar according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

The terms "about," "substantially," and the like, to the extent used throughout the specification are used in or close to the numerical values when manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of the present invention. To help, precise or absolute figures are used to prevent unfair use by unscrupulous infringers of the stated disclosure. As used throughout the specification of the present invention, the term "step of (to)" or "step of" does not mean "step for".

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

In this specification, some of the operations or functions described as being performed by the terminal, apparatus, or device may be performed instead of in a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the terminal's unique number or personal identification information, which is the identification data of the terminal. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a system for providing a try-on service using a 3D avatar according to an embodiment of the present invention. Referring to FIG. 1 , a try-on service providing system 1 using a 3D avatar may include at least one user terminal 100 , a try-on service providing server 300 , and at least one shopping mall server 400 . However, since the try-on service providing system 1 using the 3D avatar of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to the try-on service providing server 300 through the network 200 . In addition, the try-on service providing server 300 may be connected to at least one user terminal 100 and at least one shopping mall server 400 through the network 200 . Also, the at least one shopping mall server 400 may be connected to the try-on service providing server 300 through the network 200 .

Here, the network refers to a connection structure in which information exchange is possible between each node, such as a plurality of terminals and servers, and an example of such a network includes a local area network (LAN), a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi (Wi-Fi) , Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( Near-Field Communication) networks, satellite broadcast networks, analog broadcast networks, Digital Multimedia Broadcasting (DMB) networks, and the like are included, but are not limited thereto.

In the following, the term at least one is defined as a term including the singular and the plural, and even if at least one term does not exist, each component may exist in the singular or plural, and may mean the singular or plural. it will be self-evident In addition, that each component is provided in singular or plural may be changed according to embodiments.

The at least one user terminal 100 generates a 3D avatar by inputting body information and face information using a try-on service related web page, an app page, a program or an application using the 3D avatar, and the size of the 3D avatar. It may be a terminal that outputs a result of trying on clothes that fit.

Here, the at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop. In this case, the at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smartphone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The try-on service providing server 300 may be a server that provides a try-on service web page, an app page, a program, or an application using a 3D avatar. In addition, the try-on service providing server 300 may be a server that builds a database by tagging each clothing data collected from at least one shopping mall server 400 based on size and material. Also, the try-on service providing server 300 may be a server that models at least one piece of clothing data from 2D data to 3D data. In addition, the try-on service providing server 300 may be a server that activates and displays only categories having clothes that fit the user's size. In addition, when the try-on service providing server 300 is interlocked with a haptic suit in the user terminal 100, the amount of pulling, tightening, sufficient, and surplus is determined when the user wears the clothes through the tactile suit. It may be a server that builds a tactile engine library so that it can grasp the presence and the feeling of the material.

Here, the try-on service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop.

The at least one shopping mall server 400 may be a server of a shopping mall using a try-on service related web page, an app page, a program, or an application using a 3D avatar. In this case, the shopping mall server 400 may be a server that settles a fee to the try-on service providing server 300 when a purchase and payment event of the user terminal 100 occurs in the try-on service providing server 300 . Also, the shopping mall server 400 may be a server that provides images, detailed information, etc. of each garment, or permits access to the database so that the try-on service providing server 300 can build a database.

Here, the at least one shopping mall server 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop. In this case, the at least one shopping mall server 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. The at least one shopping mall server 400 is, for example, a wireless communication device that guarantees portability and mobility, including navigation, Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smartphone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

2 is a block diagram illustrating a try-on service providing server included in the system of FIG. 1, and FIGS. 3 and 4 show an embodiment in which a try-on service using a 3D avatar according to an embodiment of the present invention is implemented. It is a drawing for explanation.

Referring to FIG. 2 , the try-on service providing server 300 includes a storage unit 310 , a conversion unit 311 , a generation unit 313 , a try-on unit 315 , a transmission unit 317 , and an avatar sizing unit 320 . ), face generation unit 330, shopping guide unit 340, personalization unit 350, coordination unit 360, preference storage unit 370, overseas shopping unit 380, overseas direct purchase unit 390 and free A collection unit 391 may be included.

The try-on service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100 and at least one shopping mall server 400 uses a 3D avatar to try on. When transmitting a service application, program, app page, web page, etc., the at least one user terminal 100 and the at least one shopping mall server 400 use a try-on service application, program, app page, and web page using a 3D avatar. The back can be installed or opened. In addition, a service program may be driven in at least one user terminal 100 and at least one shopping mall server 400 by using a script executed in a web browser. Here, the web browser is a program that enables the use of a web (WWW: World Wide Web) service, and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), for example, Netscape. , Explorer, Chrome, and the like. In addition, the application means an application on the terminal, for example, includes an app (App) executed in a mobile terminal (smartphone).

Referring to FIG. 2 , the storage unit 310 may tag and store sizes and materials in clothing data of at least one category of clothing. At this time, each garment is attached with a tag called a tag. Various fabric mixing ratios, washing methods, fabric types, sizes, etc. are described here, and sizes and materials can be tagged using these. At this time, since the actual number is different for each brand and even for the same M size according to each gender, the data recorded as xx cm, which is the actual number, can be stored rather than recording the same M as the M size.

The conversion unit 320 may perform 3D modeling based on the image and size in the clothing data to convert the 3D data into 3D data. Most recently, CAD or 3D MAX is used to produce clothes. In this case, if you receive the CAD or 3D MAX file itself, you can use it as it is without converting it into 3D data. However, there is no actual clothing production file in the case of mass production in a factory by making patterns without actually working with CAD, or in the case of consignment sales that only sell items made by others. Accordingly, 3D modeling must be performed from photos and size information posted in the shopping mall, and the process is as follows.

<Segmentation>

Segmentation is the first step to transform a 2D image into 3D data. Segmentation is to find the object area of 2D image when converting from 2D image obtained from product photo to 3D model and separate the background part and foreground part of the image. Separation of foreground and background of a simple model is possible by a relatively simple method. However, figures with rather complex outlines require somewhat complex calculations when separating foreground and background. There are many different methods for separating the foreground and the background. Since the subject is a pattern, edges are detected to be robust to this, and the object (subject) area is detected using the Waterfront algorithm.

In this case, as a 2D image used for 3D modeling production, an image having a blank space outside the central figure may be used in order to relatively easily separate the foreground image. If there is no blank space, a blank space is created to separate the foreground and the background. Under this assumption, since there is no information in the background part, an edge may be detected to detect a boundary region. Edge detection is a technique widely used in image processing in terms of reducing data size, removing meaningless data, and preserving structural information of images. Most edge extraction masks are sensitive to noise, so even a small noise is considered as an edge and extracted in many cases. A method to compensate for the shortcomings of this part is an edge extraction technique using a Canny mask, which makes it insensitive to noise and enables effective edge extraction.

<Waterfront Algorithm>

The Waterfront algorithm can be regarded as an island if the background image is expressed as a sea, and is an algorithm that detects the boundary between the sea and the island. For example, under the assumption that seawater flows only in the horizontal and vertical axes, when water flows only in the horizontal right and vertical downward directions from the top, the flow of water stops at the part where the edge meets. At this time, the place where the water passes is classified as a background. Next, find the lowest position, not the background, and classify it as a background until it meets an edge when water flows in the horizontal left and vertical directions. This is called one cycle, and the algorithm ends when the background is no longer detected. If most of the background is classified in one cycle operation and only some areas are still classified as foreground, the foreground and background can be accurately classified over several cycles.

<Foreground area segmentation>

The foreground region separated from the background using the Waterfront algorithm is labeled by the foreground region segmentation method. This is to compose the entire 3D model by combining relatively simple models in order to make a complex figure into a 3D model. Foreground segmentation is similar to the Waterfront algorithm. When moving in the horizontal right direction and the vertical right direction at the top of the part to which no labeling is applied, the background area and the area up to the contact point become the same labeling area.

The next is vertex distribution, where 3D models are created separately for each same label, and the final model is composed of the whole by combining them. Using this method, it is possible to construct a model of a complex figure. The distribution of vertices subdivides the labeling area into a certain section. For example, the same number of vertices is assigned to the same horizontal axis with respect to the horizontal axis. When the region is small, the vertex spacing becomes smaller, and when the region is wide, the vertex spacing becomes wider. The higher the number of vertices allocated to the same horizontal axis, the higher the 3D model becomes. Of course, even after completing the 3D model in this way, the process of changing or transforming the 3D model with human intervention may be more necessary. For parts that are not natural or need to be modified, each person in charge may manually proceed with further work.

When the user terminal 100 inputs body information and face information, the generator 313 may generate a 3D avatar based on the body information and face information. The user terminal 100 uploads body information including the size of at least one part of the human body, uploads a photo including face information, and then outputs a 3D avatar generated based on the body information and face information. can

The try-on unit 315 may extract 3D data of clothing having a size corresponding to the body information on the 3D avatar and try on it. The user terminal 100 may extract the clothes having a size corresponding to the body information on the 3D avatar and output a try-on result. An important element of virtual fitting simulation is the user's accurate 3D human body data and natural combination of clothing. and reaction processing techniques.

For example, a method that reflects the physical elastic modulus of the cloth, a method of improving the tension structure model of the clothing image, or a method of deriving the appearance similar to the actual wearing result by reflecting the physical properties of the clothing in the wearing process can be used. have. In order to wear, that is, fit a clothing pattern to the human body data, the target clothing is decomposed into a pattern, and then the decomposed pattern image is overlaid on the human body data to show the result of wearing clothes in a complex form. After that, you can use the method of processing and applying data for uploading online.

Alternatively, a digital clothing image may be separately used for virtual fitting, and a method of extracting the clothing and front and back clothing photographic images as background separation using the actual length and size of the clothing may be used. In this case, when physical properties are given to the extracted front and back clothing images and applied to the human body data, results similar to those applied to the actual human body can be obtained. As described above, the clothing image may be extracted by separating the background using an edge detection method, and at least one feature point of the clothing may be set as a reference point.

In this case, when a mass spring model is used in which the vertices of triangular polygons constituting the clothing image are set as mass points and a spring is used as a line segment, the force acting on the modeled clothing particles is proportional to the spring force and the distance between the particles. It can be defined as the damping force that resists the relative velocity of a particle. Using the particle's position vector, speed vector, spring constant, spring damping constant, actual length between particles, and spring tension vector in the 3-axis coordinate system, even if the actual garment is virtually fitted, how much it stretches as if wearing it. It is possible to give an effect such as whether or not it is stretched. Of course, it will be obvious that various methods that can give a sense of reality can be used without being limited to the above-described method.

The transmitter 317 may transmit the trial-on result to the user terminal 100 . Wearing an avatar is a visual aspect, such as style, design, and color. will make a refund In this case, the transmission unit 317 may further apply a haptic suit so that the user can feel the correct fit or fit when the user actually wears the clothes. Recently, many people shop online for non-face-to-face and non-contact purposes, but even when buying clothes offline, people are often reluctant to try on clothes that other people have actually touched or tried on. Accordingly, when shopping online as well as offline, if you wear a haptic suit and measure the actual fit and then find the size you want to try on or buy, you can minimize the number of times you touch things that others have touched. and minimal contact can be made.

The most important elements of a haptic suit are weight and volume. Since all of the systems currently put to practical use provide visual virtual fitting through a synthesized image, compared to this method, if wearing a haptic top is heavy and inconvenient, its practicality is low. In particular, in the case of clothing in close contact with the body, the wearability may vary greatly depending on the wearer's body type and the characteristics of the cloth. Accordingly, it is possible to adjust the size by the length of the cable using the cable-type haptic suit and to control the stiffness of the motor driving the cable to experience the elasticity of the clothing material and the comfort when moving. . For haptic devices using cables, there are many commercially available devices with a force feedback function, including the Phantom.

For example, if a user has their own haptic suit, the risk of having to try on clothes hanging in a store where the coronavirus may remain because someone else has touched or coughed due to the corona virus, or online You can check how much tension and elasticity it has without trying on it. There are many refunds or returns for clothing because of its fit. If you check the visual aspects such as how long it is through avatar simulation and whether it matches your face color, you can use a haptic suit to determine whether it has elasticity and how much tension it has. You can check the wearability aspect of how much pressure is applied. This can be mainly used in three ways. ① One is a method of using a haptic suit using the tension of a cable to reproduce the size and elasticity of the fabric, and ② the other is using pneumatic, that is, intestinal villi as shown in Fig. 4(d). A protrusion of the same shape is put inside in the form of an airbag, and the contraction and expansion of the protrusion are individually controlled by the pneumatic valve according to the four stages of atmospheric, inlet, hold, and exhaust, so that which part has a feeling of pressure and which part This is to check if it is pulling. ③ Another one can use Vibrotactile, Electrotactile, and Pneumatictactile. Inexpensive and easy to configure.

<Haptic Suit>

In the case of the upper garment, the circumference of the chest and waist is important, and the elasticity for the movement of the upper body is the main concern. The most important factor when raising or extending the upper arm is the size of the armhole of the garment. For example, if the top you want to wear is in close contact with the body, the purpose is to use a cable to feel the circumference length and elasticity according to the height in order to feel the size of the clothes and the elasticity of the fabric when wearing these clothes. to be. To this end, it is necessary to make the length of the cable according to the height the same as the actual size of the clothes, and to adjust the tension according to the elasticity of the fabric. The cable in the circumferential direction when wearing actual clothes and the cable that implements the dimensions of both vacuums are shown. The most ideal way is to weave the clothes with cables as if to make clothes by using a lot of cables for the entire top, but this is unrealistic because it requires a large number of cables and a motor and a controller to control each cable.

For example, it is possible to design a structure in which one cart with a pulley moves up and down along the rail at high speed and sequentially feels the size and elasticity of each part, and independent motors can be used for both vacuum parts. . One rail is fixed on the left and right of the chest, and the cart moves up and down along it. The pulley is fixed to the cart on the right side of the top (in the direction of the wearer), and the cable wound around the pulley fixed to the motor shaft at the waist moves the pulley of the right cart. It is fixed to a cart that moves up and down along the rail on the left. In order to reduce the weight of the haptic top, the two motors for moving the cart and the motor for controlling the cable can both be fixed to the waist belt. In both vacuum sections, each motor controls the length and tension of the cable according to the vacuum size. A total of 5 motors can be used by using two motors to control the upper and lower positions of the cart on the rail, a motor to control the cable, and two motors to control the cables in both vacuum parts.

The upper part is cut in the center to realize various clothes sizes, and the cut part is connected with a cable to determine the size of the clothes worn by the length of the cable. can be designed to The vacuum part is a structure that controls the length and tension of the cable by winding the outside of the jacket with a cable. Two motors on either side of the belt control the up and down movement of each cart, and the central motor controls the cable tension. The cart moves up and down on the rails on the left and right sides of the chest, and the end of the cable can be connected to the left cart with a hook to make it easier to wear the haptic top.

Alternatively, the haptic suit may deliver various tactile sensations to the wearer through a Neuro-Muscular Electrical Stimulation system that is uniformly attached to the entire garment. Therefore, users wearing haptic suits can feel various senses in virtual reality through the tactile system. As an electrical system to stimulate the neuromuscular muscle, methods such as electrical muscle stimulation (EMS) used for training belts for abdominal muscle training or transdermal nerve stimulation (TENS) used for electrical therapy to relieve pain can be applied. In this case, a user's desired environment can be directly built by using the haptic engine library.

The avatar sizing unit 320 builds a human body model for each race, gender, age and body type as a database, and then determines the user's body type based on the race, gender, age and body information input from the user terminal 100 . By extracting and selecting a human body model most similar to the user's body type, the human body model can be transformed according to the size information of the body information. In order to easily compose an avatar, which is an individual character, it is desirable to build a database of various human body models and then select a model close to the individual body type and transform the human body model to fit the size of the individual body type. In this case, in order to freely select clothes suitable for an individual's body type and taste, a system with an easy construction and scalability of a human body model database and a clothes model database is required. Such a system composes a character similar to the individual body type, and creates a fashion fitting based on virtual reality or augmented reality with various clothes on the individual character.

The avatar sizing unit 320 refers to the individual body type data in the steps of the production of the standard model and the clothing model database, the pre-processing of the model for clothing coordination, the composition of the individual character, and the fitting process of the clothing model. A method of configuring a personal character by selecting a model having a similar body type and transforming the selected model may be used. A virtual fit prediction service is possible by fitting a selected clothing model to an individual character. The first step is to divide by gender and body type, create a model that fits the personality of Koreans, build a database, select a human body model with a body shape similar to the individual body type from the database, and select the height and width of the model according to the size of the body type. , thickness, and circumference to form an avatar as a personal character. Then, as a preprocessing process for human body transformation, groups are set for each part, and linear transformation can be used for each part of the human body model as a transformation method. can effectively transform various human body models. A 3D modeling production tool may be used to generate various clothing models, but the production tool is not limited to any one tool. In this case, it is easy to construct a database of the human body model because the human body model already produced for other purposes can be utilized. In addition, because geometrical linear deformation is mainly adopted and the speed is fast, it may be possible to fit the individual sensibility and body type.

For example, for body type, classify men and women into skinny, normal, and obese, and consider the distribution of fat by dividing whether they are upper body obese or lower body obese, and then consider the difference in width, thickness, girth, rise length, and feet at the same height. The knee-to-knee ratio and the knee-to-thigh ratio can be further considered. In other words, the average height, width, thickness, and circumference items are averaged, and the average height, width, thickness, and circumference items corresponding to 10% of the percentile are skinny, and dimensions corresponding to 90%. can be an obese type, but is not limited thereto. Building 3D models of various body types of Koreans as a database is to make individual characters more realistic in a simple way. Rather than constructing avatars of all body types by modifying the normal type, it is easier to construct realistic avatars by selecting a model similar to the individual body type from the database and then slightly modifying the selected model. For example, if you choose a skinny type rather than a normal type for a skinny body and slightly modify it, you can compose a more realistic personal character.

When receiving a photo from the user terminal 100, the face generator 330 may identify and extract a face among subjects included in the photo, and 3D model and generate 3D face data from the extracted 2D face data. . In this case, when making 3D with only 2D face data, various photos such as sides and back may be required because the sense of depth cannot be estimated from a photo with only the front.

When at least one line constituting a face is displayed as a 3D line, it is displayed as a 3D line using a 3D polygon model that creates a mesh with a set of points, lines, and faces constituting the shape of the face. can do. At this time, in an embodiment of the present invention, an intuitive mesh can be automatically constructed based on the extracted feature points using a two-step mesh warping method, and the surrounding vertices (Vertex) are formed by area-based deformation. ) can be configured so that they can be arranged naturally. In order to perform texture mapping, a texture map is constructed based on an input image, and the time required for this process occupies a large part in face modeling. Accordingly, in an embodiment of the present invention, when the face is deformed, the UV coordinates of the general model are also transformed through two-step mesh warping, so that it is not necessary to configure a separate texture map.

<Candidate-3 Face Model & MPEG-4 FDP>

Candidate-3 model is available as a 3D general face model. Since the Candidate-3 face model is modeled based on the facial feature points defined in MPEG-4, face modeling is possible with a relatively small number of vertices. Accordingly, if detailed and sophisticated modeling is desired, a face model composed of M vertices and N triangle polygons can be used by subdividing vertices based on the Candidate-3 face model.

<Extraction of facial feature points>

Next, facial feature information should be extracted from the input face image. At this time, as an algorithm for extracting facial feature points, AdaBoost face detection algorithm based on ASM (Active Shape Model) algorithm, AAM (Active Appearance Model), ACM (Active Contour Model), and Point Distribution Model, etc. exist. In ASM and AAM, feature modeling and optimization based on Principal Component Analysis are commonly applied to parameterize the face model and approach statistically. The difference between the two modeling techniques is that, while AAM applies a Global Appearance model in the process of optimizing model parameters, in ASM, an initial label is given and a new location is found at every label location (Local Region). ) based on It has been shown that ASM outperforms AAM in shape localization and is more robust to lighting effects. However, ASM uses a profile for texture feature modeling as a one-dimensional array, and has a disadvantage in that it is difficult to distinguish feature points of a local area in the area search process.

In order to extract facial feature points based on AAM, it is an algorithm that detects a statistical model, which is information on the shape and texture of an object, from an input image using a statistical modeling method. AAM consists of a modeling step and a fitting step. The former modeling step uses a shape model. The shape model is a model constructed by designating feature points directly by the user for the training image. The face shape model S is obtained by linearly combining Φi(i=1,...,n), n shape eigenvectors corresponding to the larger of the average shape (bar)X and the eigen value. It can be expressed as Equation 1 below.

A texture model refers to a pixel brightness value in an image for a feature region to be extracted. The process of modeling using a set of pixels in the inner space of the average shape reduces the rate of texture change caused by shape differences. With respect to the texture data that has undergone the normalization process, a linear model is obtained as in Equation 2 below through the same PCA process as the shape model.

A texture model g can be obtained by linearly combining the normalized average gray level vector (bar)g, the texture eigenvector T, and the texture parameter c.

<Combination model>

Then, by adjusting the shape parameter bs and the texture parameter bg, various face shapes and features can be expressed. By combining these two parameters with a heat vector and adjusting the appearance parameter c, which is a combination of shape and texture parameters, the shape model S' and the texture model g' can be simultaneously controlled.

<AAM detection>

Fitting is a process of detecting features in an input image based on a learned shape model and a texture model. Find the appearance parameter c that minimizes the square of the error between the texture of the face image modeled by Equation 3 and the texture in which the input detection target image is warped with the average shape (bar)X, and mathematically with the c The shape model expressed by Equation 3 is determined as facial feature points to be found. How to find the appearance parameter c that minimizes the squared error of the model and input data is a matter of optimization.

<3D face and UV transformation through mesh warping>

When facial feature information is extracted through AAM, 3D general face modeling and UV information are transformed based on the extracted feature points. For this, two-step mesh warping can be used, which has a clear correspondence and can reduce processing time. In mesh warping, a mesh is configured by designating control points corresponding to the source image and the target image, and pixel values of the source image are mapped to fit the mesh region of the target image. In order to perform two-step mesh warping, a source mesh and a target mesh are constructed based on the control points of the source image and the control points of the target image. The target mesh consists of facial feature points extracted through AAM from the input image, and the source mesh consists of (x,y) coordinates of the vertices of the 3D general model corresponding to the feature points. For natural warping, you can warp the face shape and face components (eyes, nose, mouth) independently. When warping the face shape, the mesh is configured to include feature points outside the face, and when warping the face component, the mesh is configured to include feature points of the eyes, nose, and mouth. When the mesh is constructed, warping is performed through two-step processing based on the mesh.

The first step is processing for vertical splines, and the x-coordinate of the source mesh control point is moved to the x-coordinate of the destination mesh. Each pixel in the row is interpolated with new values by applying an interpolation function to the x coordinate values of the control points and re-extracting using a resampling method. Interpolation methods range from simple linear interpolation to complex spline functions. In the second step, the processing for the horizontal spline is repeated for the x-coordinate, and each column of the output image is re-extracted from the intermediate image. If the position of the 3D model vertex values is the position of the 3D model vertex values when calculating the transformed position in each step, the transformed corresponding coordinates are stored in the buffer, and the vertex values of the 3D face model are aligned with the coordinates. As a result, the 3D model vertices are smoothly deformed based on the few feature point coordinates extracted with AAM.

<Extraction of depth value based on brightness change>

Finally, in order to derive depth information of a 3D face, a depth value is extracted through a change in brightness value within an image or a shooting screen. This method has the disadvantage that it must be performed under fixed lighting conditions, but has the advantage of being simpler and faster than the Shape-From-X algorithm, which requires a series of images and undergoes complex processing. At this time, based on the lighting estimation and shadow generation technique, the lighting location is placed in a predetermined place, and the virtual lighting is placed in a similar location based on this. The lighting is placed above the face so that the size of the depth value can be easily distinguished by the shadow. It allows the user to adjust the lighting in 3D space so that the brightness distribution at each sample point of the rendered 2D image has a value similar to that of the face brightness distribution at the sample point of the input image. This is an error when transforming the depth value by limiting the value before adjusting the depth value by limiting the value so that the difference in brightness of adjacent sample points can represent the difference in the brightness value between the part entering and the part coming out of the common human face region. It's about reducing the odds.

When the two images have a similar brightness distribution in the face region, the depth value of the feature vertices of the face model is adjusted by comparing the relative brightness of the feature points specified in advance. In the rendered face model image, the brightness value (Mn(n=1,...,9)) of each sample point for an area heavily affected by lighting, for example, the nose, forehead, mouth, and chin, is constant around When the average brightness value of the region is taken, and the average brightness value of the region in the input image is In(n=1,...,9), the vertex corresponding to each sample point is applied by applying Equation 4 below. Find the depth value vn(z) of .

A represents the difference in brightness values between adjacent sample points in the image, and B represents the difference in brightness values between adjacent sample points in the rendered model. When the depth value of the corresponding vertex is obtained, the adjacent vertices connected to the corresponding vertex are extracted, and each vertex is divided into a total step, which is a total step up to the adjacent sample point. At each step, the depth value of the vertices is adjusted with the weight α obtained through Equation 5 below.

Of course, in addition to the above-described method, a method for displaying a 3D line and performing UV mapping may be various, so that the method is not limited to the above-described method.

The shopping guide 340 may generate a shopping list by extracting clothes having a size corresponding to the body information input from the user terminal 100 by category. Korea does not have a wide variety of sizes, so the issue of diversity has been raised for a long time. However, due to inventory management, industry insiders manufacture and sell the most common sizes in order not to leave stock. On the other hand, overseas brands have up to 10 sizes, such as 32, 34, 36, 38, 40, 42, 44, 46, 48, and 50, on the grounds that not selling clothing of various sizes is a violation of equal rights. Even in the fast fashion industry, on average, there are six sizes of sizes: XS, S, M, L, XL, and XXL, but in Korea, there are only three. Therefore, it is very difficult for people who are tall, fat, short or too skinny to buy clothes or shoes that fit them because they are not of average body type. Accordingly, even when these users see pretty clothes, when they enter the site and look at the detailed page, they do not have their own size, so the process of selecting clothes again and looking at the size is repeated. Accordingly, in one embodiment of the present invention, by deleting (inactivating) a category or menu itself containing clothes or shoes that the user cannot buy, the user sees clothes that he cannot buy or does not fit him and reviews the detailed page. It can save you time.

When outputting at least one category, the personalization unit 350 maintains activation of only a category in which clothing having a size corresponding to the body information input from the user terminal 100 exists, and is inputted from the user terminal 100 . At least one category may be personalized by inactivating a category in which clothing having a size corresponding to one body information does not exist.

The coordinator 360 may recommend clothes suitable for the currently tried-on clothes based on Instance Based Learning (IBL). IBL instantiates the intelligence of an expert and prepares it as training data, and the training data is referred to in order to receive a recommendation for the top or top that matches the top or bottom actually worn. Similar clothing search function can be used more than once in instance-based matching processing. This method has a disadvantage in that an expert needs to train directly on the instance. The more training data, the more precisely the expert's intelligence can be expressed, but since the match count increases rapidly, it may take considerable effort until the training data is stable. When one piece of clothing is selected, after decomposing the clothing data for each attribute, an attribute value matrix of an expert is used to obtain the values of the attributes that best match each attribute value. When the attributes with the best match for each attribute are selected, the clothes closest to these attributes are searched for in product data and automatically matched.

The properties of clothing are divided into materials, colors, patterns, and lines. In addition, individual physical characteristics are divided into height, weight, waist size, face size/shape and color, neck length, arm length and thickness, leg length and thickness, chest circumference, hip circumference, upper body to lower body ratio, etc. lose The relationship between these characteristics can be generalized by a coordinating expert and used to recommend clothing suitable for an individual's physical characteristics. That is, if the relationship between the properties of clothing and the individual's physical characteristics is well established, it is possible to create a system for recommending customized clothing suitable for the individual. In this case, the classification of clothing attributes may follow the classification of previous studies. First of all, materials, patterns, and colors can be defined as common attributes of all women's wear. After that, the properties of the top (jacket, blouse, shirt) and bottom (pants, skirt) are classified separately, and the common properties of the top are the overall line, neckline, sleeve length, collar (collar), and button. Separated by number, jackets and blouses can define pockets and sleeve shapes as additional attributes. In the case of bottoms, length and line can be defined as common attributes, and as additional attributes of pants, the upper knee and trousers, the lower knee and trousers, and protruding pockets can be defined. Further, additional attributes of the skirt include a pleat shape and an overall shape (design).

The preference storage unit 370 collects identification codes of at least one pre-owned clothing from the user terminal 100, searches for clothing based on the identification code, and stores the style, brand, and size of the clothing as user preference data. can For example, if user A has a large amount of brand B clothes, user A can say that they prefer clothes from brand B. And, looking at the size that user A owns or purchased, it can be said that brand B prefers size C. Each brand may have a different size, and a user's preferred fit may be different for each brand. For example, suppose that Brand B prefers a perfect fit because it is yoga or sportswear, but Brand C is a brand that sells casual or comfortable clothes such as hoodies and short-sleeved shirts and prefers a loose fit. (Slim Fit)], [C Brand-Fit Type (Loose Fit)] can be saved.

The overseas shopping unit 380 searches the overseas shopping mall to determine the size and size of the clothes in the domestic shopping mall when there is no clothing that satisfies the brand of the user preference data and the clothing having a size corresponding to the body information on the 3D avatar. Clothing that satisfies the brand may be extracted, and the sum of the delivery price and customs duties resulting from the use of a delivery agent may be transmitted to the user terminal 100 . As mentioned above, in Korea, there is little variety of sizes, so if you want to buy the right clothes, you can't help but shop overseas. In addition, since the price of imported brands in Korea is four to five times more expensive, you can buy them at a lower price than Korea, even including overseas shipping and tariffs. Above all, there is no need to cut the pants and mend them because they come in various sizes, and in the case of trousers, not only the waist circumference but also the length are different. For example, in the case of men's trousers, they are divided into Regular, Big, and Tall, and again Regular has waist sizes from 26W to 42W and length sizes from 28L to 34L. When buying pants, you can not only buy based on waist size, but also consider leg length. Accordingly, if a user's desired brand of clothes is not found in Korea, it is possible to search an overseas site that sells them and inform the user of the sum of the customs duties and the shipping cost. Each country has different tariff-addition policies, but in the United States, customs duties are not imposed on clothes up to $200 (the sum of shipping costs), so it may be recommended to buy less than $200.

The overseas direct purchase unit 390 is a 3D avatar with clothes having a size corresponding to the body information and clothes satisfying both brands of user preference data exist in a domestic shopping mall, but in an overseas shopping mall that sells the same clothes. When the sum of the clothing price, shipping cost, and tariff is lower than the price of the domestic shopping mall, the URL of the overseas shopping mall may be transmitted to the user terminal 100 .

When the free collection unit 391 orders and pays for clothes in the user terminal 100 to request delivery, and the user directly requests a return or exchange after try-on, the reason for requesting a return or exchange from the user terminal 100 can be selected and collected for free. As described above, even if the visual aspect is checked through the avatar simulation and the fit is seen using the haptic suit, it cannot replace the actual wearing. Accordingly, the free collection unit 391 enables a free return or exchange if the user has tried on, but does not like the material or other matters after ordering, payment, and delivery. Or, to avoid exchanging because of the size, if you ordered a size M, you can also send the size S or L, that is, below or above it, so that you can purchase only the right size and collect the rest for free.

Hereinafter, an operation process according to the configuration of the try-on service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as an example. However, it will be apparent that the embodiment is only one of various embodiments of the present invention and is not limited thereto.

Referring to FIG. 3 , (a) the try-on service providing server 300 collects body information and face information from the user terminal 100, selects the most similar human body model from the body information, (b) A 3D avatar is generated by adjusting the human body model accordingly, and (c) the face information is generated from the 2D face data extracted from the photo, and 3D face data is generated and synthesized into the human body model. If the face and body are completed through (d), only the category of clothing having the size corresponding to the user's body information is activated and output as shown in FIG. 4 (a). And, (b) if there is no size suitable for the domestic shopping mall or if the overseas shopping mall is cheaper, information is provided, (c) the clothing data is output to the 3D avatar to check the visual, and (d) if there is a haptic suit, the fit so you can feel it too.

The matters not described with respect to the method of providing a try-on service using the 3D avatar of FIGS. 2 to 4 are the same as those described above for the method of providing the try-on service using the 3D avatar through FIG. 1 or easily from the described content. Since it can be inferred, the following description will be omitted.

5 is a diagram illustrating a process in which data is transmitted/received between components included in the system for providing a try-on service using the 3D avatar of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted/received between each component will be described with reference to FIG. 5, but the present application is not limited to such an embodiment, and the example shown in FIG. 5 according to the various embodiments described above will be described. It is apparent to those skilled in the art that the data transmission/reception process may be changed.

Referring to FIG. 5 , the try-on service providing server tags and stores the size and material in clothing data of at least one category of clothing ( S5100 ).

In addition, the try-on service providing server performs 3D modeling based on the image and size in the clothing data to convert it into 3D data (S5200), and when inputting body information and face information from the user terminal, the body information and face information Based on the 3D avatar is generated (S5300).

Also, the try-on service providing server extracts 3D data of clothes having a size corresponding to the body information on the 3D avatar and tries them on (S5400), and transmits the try-on result to the user terminal (S5500).

The order between the above-described steps ( S5100 to S5500 ) is merely an example and is not limited thereto. That is, the order between the above-described steps ( S5100 to S5500 ) may be mutually changed, and some of the steps may be simultaneously executed or deleted.

Matters that have not been described with respect to the method of providing a try-on service using the 3D avatar of FIG. 5 are the same as those described above for the method of providing a try-on service using the 3D avatar through FIGS. Since it can be inferred, the following description will be omitted.

The method of providing a try-on service using a 3D avatar according to an embodiment described with reference to FIG. 5 may be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. have. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described method for providing a try-on service using a 3D avatar according to an embodiment of the present invention may be executed by an application basically installed in a terminal (which may include a program included in a platform or an operating system, etc. basically installed in the terminal). and may be executed by an application (ie, a program) installed directly in the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the corresponding service. In this sense, the above-described method for providing a try-on service using a 3D avatar according to an embodiment of the present invention is implemented as an application (that is, a program) installed by default in a terminal or directly installed by a user and read by a computer such as a terminal. It can be recorded on a recordable medium.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (10)

After uploading body information including the size of at least one part of the human body and uploading a photo including face information, a 3D avatar generated based on the body information and face information is output, and the 3D avatar is displayed on the 3D avatar. a user terminal for extracting clothes having a size corresponding to body information and outputting a try-on result; and
A storage unit that tags and stores sizes and materials in clothing data of at least one category of clothing, a conversion unit that performs 3D modeling based on images and sizes in the clothing data and converts them into 3D data, and the body in the user terminal When information and face information are input, a generator generates a 3D avatar based on the body information and face information, and a try-on unit extracts 3D data of clothing having a size corresponding to the body information on the 3D avatar and tries on it a try-on service providing server including a transmission unit for transmitting the try-on result to the user terminal;
A system for providing a try-on service using a 3D avatar comprising a.
The method of claim 1,
The try-on service providing server,
After building a human body model for each race, gender, age and body type as a database, the user's body type is extracted based on the race, gender, age and body information input from the user terminal, and the body shape most similar to the user's body type is extracted. an avatar sizing unit that selects a human body model and transforms the human body model to fit the size information of the body information;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
a face generator that, when receiving a picture from the user terminal, identifies and extracts a face from among the subjects included in the picture, and generates 3D face data by 3D modeling from the extracted 2D face data;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
a shopping guide unit for generating a shopping list by extracting clothes having a size corresponding to the body information input from the user terminal by category;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
When outputting at least one category, only the category in which clothing having a size corresponding to the body information input from the user terminal exists is kept active, and clothing having a size corresponding to the body information input from the user terminal is maintained. a personalization unit that inactivates non-existent categories to personalize the at least one category;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
a coordinating unit that recommends clothes that match the clothes you are currently trying on based on IBL (Instance Based Learning);
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
a preference storage unit that collects identification codes of at least one pre-owned clothing from the user terminal, searches for clothing based on the identification code, and stores the style, brand, and size of the clothing as user preference data;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
8. The method of claim 7,
The try-on service providing server,
When clothes having a size corresponding to the body information and clothes satisfying both the brand of the user preference data and the clothes having the size corresponding to the body information on the 3D avatar do not exist in a domestic shopping mall, search an overseas shopping mall to find clothes that satisfy the size and brand an overseas shopping unit for extracting and transmitting the sum of the shipping price and customs duties according to the use of a shipping agent to the user terminal;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
8. The method of claim 7,
The try-on service providing server,
On the 3D avatar, clothes having a size corresponding to the body information and clothes satisfying both the brand of the user preference data exist in a domestic shopping mall, but clothing prices, shipping costs, and an overseas direct purchase unit for transmitting the URL of the overseas shopping mall to the user terminal when the sum of tariffs is lower than the price of the domestic shopping mall;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.
The method of claim 1,
The try-on service providing server,
a free collection unit that receives a selection of a reason for requesting a return or exchange from the user terminal and collects it for free;
A try-on service providing system using a 3D avatar, characterized in that it further comprises.

Images