KR20230077646A - Method of providing a realistic fashion metaverse service, apparatus thereof, and computationally non-transitory readable medium for storing a program thereof - Google Patents

Abstract

A method for providing a realistic fashion metaverse service, according to a disclosed embodiment of the present invention comprises the steps of: preparing files corresponding to components constituting the content of a realistic metaverse service; performing a rendering process based on a three-dimensional map representing priority for rendering with respect to the prepared files, wherein the three-dimensional map represents the importance of a vertex according to the position of the vertex, and the importance of the vertex is generated at the boundary point of an item represented by an asset to be higher than at the internal point of the item; and providing a GL transmission format (glTF) file and a pre-glTF file including the three-dimensional map information with respect to the prepared files. The efficiency of a design process and a manufacturing process can increase.

Description

A non-temporary computer-readable recording medium storing a method for providing a realistic fashion metabus service, a device for the same, and a program for performing the method STORING A PROGRAM THEREOF}

The present specification relates to a method and apparatus for providing a realistic fashion metaverse service, and a non-transitory computer-readable recording medium storing a program for performing the method.

As non-face-to-face demand increases after COVID-19, fashion tech-based online sales are rapidly increasing. Accordingly, various attempts are being made to combine offline fashion business and technology by utilizing the metaverse, including the metaverse platform business, such as virtual fashion shows using AI and XR technology. However, conventional metaverse platform businesses have failed to provide a virtual experience that is almost similar to that of a user purchasing clothes at a physical store.

The purpose of the examples disclosed as a solution to the above problem is to measure the user's body size and generate an avatar that reflects the user's appearance almost similarly, thereby providing a realistic virtual fitting experience to the user. It is to provide a method, device and non-transitory computer readable recording medium for providing a fashion metaverse service.

The purpose of the disclosed examples is a realistic fashion metaverse service that can increase the efficiency of the design process and manufacturing process by providing high-quality virtual clothing content that implements texture, color, and shape that are almost similar to real clothing. To provide a method, apparatus and non-transitory computer readable recording medium for providing.

The purpose of the disclosed examples is a method, apparatus, and non-temporary computer-readable method for providing a realistic fashion metaverse service that can be used for various businesses related to fashion, trading and utilizing assets, etc. by providing high-quality virtual content. To provide a recording medium.

The disclosed embodiment includes the steps of preparing a file corresponding to components constituting the contents of an immersive metaverse service; Generating based on a 3D map indicating priority for rendering of the prepared file, wherein the 3D map indicates the importance of the vertex according to the position of the vertex, wherein the importance of the vertex is a border point is created higher than an interior point of the item; and providing a free glTF file including a glTF (GL Transmission Format) file and the 3D (3 dimension) map information as the prepared file.

As for the importance of the vertices, a high level is assigned to vertices representing the outer contour of the item, a medium level is assigned to vertices located on the inner contour of the item, and a low level is assigned to vertices located at other positions of the item. can be assigned.

The 3D map represents a Rendering Priority Index (RPI) map indicating a priority for rendering of the prepared file based on an AI algorithm.

The vertex may be displayed as a point having location information and texture information on the image of the content.

From another point of view, the embodiment includes a database for storing files corresponding to components constituting the contents of the immersive metaverse service; and a processor that executes computer commands, including: a preparation module that provides files corresponding to components constituting contents of a tangible metaverse service; a generation module for generating based on a 3D map indicating a priority for rendering of the prepared file; Here, the 3D map indicates the importance of the vertex according to the position of the vertex, wherein the importance of the vertex is generated when a boundary point of an item represented by an asset is higher than an internal point of the item; and a providing module for providing a glTF (GL Transmission Format) file and a free glTF file including the 3D (3 dimension) map information from the prepared file.

In another aspect, the embodiment includes the steps of: being executed by one or more processors, the processor providing files corresponding to components constituting the contents of the tangible metaverse service; generating based on a 3D map indicating priority for rendering of the prepared file; Here, the 3D map indicates the importance of the vertex according to the position of the vertex, wherein the importance of the vertex is generated when a boundary point of an item represented by an asset is higher than an internal point of the item; and providing a pre glTF file including a glTF (GL Transmission Format) file and the 3D (3 dimension) map information as the prepared file.

According to the disclosed embodiment, it is possible to provide a virtual fitting experience closer to reality to the user by generating an avatar that almost similarly reflects the user's appearance by measuring the user's body size.

According to the disclosed embodiment, it is possible to provide high-quality virtual clothing content that implements almost similar texture, color, shape, etc. to actual clothing, thereby increasing the efficiency of the design process and manufacturing process.

According to the disclosed embodiment, by providing high-quality virtual content, it can be utilized in various businesses related to fashion, property transaction and utilization business, and the like.

1 shows a conceptual diagram of a realistic fashion metaverse service providing system.
2 is an example of a realistic fashion metaverse service providing system.
FIG. 3 shows specific functions of each server described in FIG. 2 .
4 is an example of a flow of a realistic fashion metaverse service.
5 shows an example of a user interface for measuring a user's body size.
6 shows an example of a user interface for virtual fitting.
7 shows an example of a realistic fashion metabus service providing system.
8 shows an example of body size measurement.
9 are graphs showing the pitch value effect.
10 are graphs showing pitch value effects.
11 are graphs showing pitch value effects.
12 shows an example of a chest circumference correction process.
13 shows an example of a chest circumference derivation process.
14 is a block diagram of a realistic fashion metaverse service processing process.
15 shows a process of generating realistic fashion metaverse contents.
16 shows a process of generating WebGL-based immersive fashion metaverse contents.
17 shows an example of a realistic fashion metaverse service providing system.
18 shows an example of pre-glTF (pre-glTF).
19 shows the operation of the optimized glTF generator.
20 shows an example of optimized vertices.
21 is a flowchart illustrating a rendering process based on an RPI 3D map.
22 shows an example of a virtual fitting type.
23 shows a masking process of an avatar masking fitting type.
24 shows a process of avatar masking fitting type.
25 shows a processing procedure of a clothing shader fitting type.
26 shows a processing procedure of a clothing shading fitting type.
27 shows a processing procedure of a clothing collider fitting type.
Fig. 28 shows the processing process of the clothing collider fitting type.
29 is a flow chart illustrating an image seamless texturing algorithm.
30 shows an example of an image seamless texturing algorithm.
31 shows an example of an image seamless texturing algorithm.
32 is a flowchart illustrating a method of providing a realistic fashion metaverse service.

The following embodiments are provided to more fully describe the system and method to those skilled in the art. In addition, the embodiments may be modified in many different forms, and the scope of the present system and method is not limited to the following embodiments. The embodiments are provided to more fully and completely describe and fully convey the system and method to those skilled in the art.

Terms used in the specification are used to describe specific embodiments, and do not limit the systems and methods described by the embodiments. The singular forms used herein include the plural forms unless the context clearly dictates otherwise. Also, the term “and/or” as used herein may include any one and all combinations of one or more of the listed items.

1 shows a conceptual diagram of a realistic fashion metaverse service providing system.

The conceptual diagram 100 of the realistic fashion metaverse service providing system shown in FIG. 1 includes a process of generating realistic virtual fashion contents 110 and a multi-platform optimization process 120 to provide a realistic fashion metaverse service. . The realistic fashion metaverse service includes a body measurement service that measures the user's body, a virtual fitting service that creates a realistic avatar based on the body measurement and performs fitting of realistic virtual items, and a virtual fitting service in a realistic virtual space. It includes a virtual fashion show service configured to move one avatar.

The process of generating immersive virtual fashion contents 110 is a process of generating immersive virtual fashion contents including immersive avatars, immersive items, and immersive virtual spaces, and includes immersive avatar creation 112, virtual fitting 114, and Animation 116 process is included. The realistic avatar is created by realizing the user's body and face similarly by performing image correction based on the measurement of the user's body size and face. In the virtual fitting process 114 , a realistic virtual fitting experience is provided to the user by combining the generated realistic avatar and the realistic item. Realistic items are fashion items such as actual clothes, shoes, and/or accessories, and are created by realizing the shape, texture, color, etc. of each item similarly to reality. The animation process 116 implements animation, which is a motion of an avatar disposed in the created realistic virtual space. The immersive virtual space represents a virtual fashion show and/or showroom contents.

The multi-platform optimization process 120 is a process of implementing platform (eg, app/web platform, etc.) or optimization between platforms so that the sensibility of realistic virtual fashion content is not damaged. The multi-platform optimization process 120 includes an app platform 122, cross platform 124, and web platform 126 implementation process. The app platform 122 provides realistic virtual fashion content, such as realistic avatars and realistic fashion items, based on AR/VR technology. The cross-platform 124 is a process of compressing realistic virtual fashion content provided from the app platform 122 and delivering it to the web platform 124 . The web platform 124 provides immersive virtual fashion contents in a WebGL-based browser. Realistic virtual fashion content provided by the web platform 124 includes a web platform conversion technology (eg, 3D data format) of app platform content, a 3D resource (geometry*, texture) generation optimization technology for 3D expression within the web platform, It is implemented based on AI-based optimal 3D resource automatic generation technology.

The system according to embodiments can provide high-quality realistic virtual fashion content to various fields. Realistic virtual fashion content can be used in the design process, manufacturing process, etc. to improve work efficiency. For example, it can be used for communication channels of consumers, fashion small business owners, fashion service providers, etc.).

2 is an example of a realistic fashion metaverse service providing system.

The realistic fashion metaverse service providing system 200 (hereinafter referred to as a system) includes one or more servers. Each server is a virtual server (or instant) provided in the cloud computing system for creating or storing the contents of the realistic fashion metaverse service, and can be provided to drive the operating system and applications, and to perform each function. It may include a processor for Realistic fashion metaverse contents can be created based on 3D authoring tools (geometry: Unity, Blender, 3D Max, CLO3D, etc., texture: Photoshop, GIMP, Substane Painter, etc.). The servers shown in FIG. 2 are just examples, and the system 200 providing realistic virtual fashion content may further include servers not shown in the figure.

The system 200 includes a service server 210 for communicating with a user's web or app, an avatar production main server 220 for creating realistic avatars, a dedicated server 230 for creating realistic avatar faces, and a storage server 240. ), a database server 250 and a central management and API server 260 may be included. As shown in FIG. 2, the service server 210 is an interface server that users can access through a web or app, and can communicate with the avatar creation main server 220 and the central management and API server 260. The avatar creation main server 220 is a server that comprehensively provides functions for creating realistic avatars described in FIG. 1, and can communicate with the avatar face creation server 230 and the database server 250. The server 230 dedicated to making avatar faces is a server that centrally provides functions for making faces of realistic avatars, and can communicate with the avatar making main server 220 and the storage server 240 . The storage server 240 stores resources, files, etc. for the realistic fashion metaverse service. The database server 250 stores data necessary for a realistic fashion metaverse service. The central management and API server 260 manages computational resources and APIs related to content creation of the realistic fashion metaverse service. Figure 2 shows communication between servers according to the data processing sequence of the realistic fashion metaverse service providing system, and communication between servers is not limited to this example.

FIG. 3 shows specific functions of each server described in FIG. 2 .

The service server 210 may include a virtual space management module, a virtual fitting service module, and an API communication management module. The virtual space management module is a module that manages a WebGL-based web platform (for example, the web platform 126 described in FIG. 1). The virtual space management module includes a pre-glTF generator and interpreter, a glTF generator and interpreter, a vertex optimizer, and a 3D File interpreter. Pre-glTF is a proprietary standard file that includes glTF format data and priority 3D maps for rendering to provide realistic fashion metaverse contents based on webGL. The realistic virtual fashion content module is a module that manages the realistic items described in FIG. 1 .

The avatar production main server 220 comprehensively provides modules related to avatar production. The avatar creation main server 220 includes a body measurement/creation module that measures the user's body based on user information (eg, a photo, etc.), and a realistic avatar that creates a realistic avatar based on the measured user information. It includes a production module and a virtual fitting module that combines a realistic item with a realistic avatar. The server 230 dedicated to creating avatar faces includes a user face extraction module, a virtual makeup setting and application module, and a hairstyle application module.

The storage server 240 stores resources and files required to provide realistic virtual fashion content services. The storage server 240 includes resources and files for creating virtual space (Space (glTF, Textures)), resources and files for creating realistic avatars (Faces (glTF, Textures), Body (glTF, Textures), Avator (glTF, Textures)), resources and files for producing realistic items (3D Garment), resource files for realistic virtual fashion content service (Template), etc. The database server 250 stores realistic virtual fashion content. Store data and information required for service. The database server 250 is an API key (for user authentication), data for creating avatars (Faces, body, Avatar), virtual space and realistic item data (Space, 3D Garment), realistic virtual fashion content service data (Template) keep the back

The central management and API server 260 manages computational resources (eg, body measurement, face creation, etc.) and APIs related to realistic avatar creation. The central management and API server 260 is used to create user authentication and information transmission/reception modules and realistic avatars for APIs designed to provide services provided by web or apps to third parties (eg, external companies). It includes modules for creating, managing, and deleting computational resources (for example, body measurement and face creation resources (used for AI, etc.)) 3 is only an example, and functions performed by each server and modules included in each server are not limited to this example.

4 is an example of a flow of a realistic fashion metaverse service.

The flow 400 shown in FIG. 4 represents a process of measuring a user's body size provided in a realistic fashion metaverse service.

The realistic fashion metaverse service providing system according to the embodiments includes the servers described in FIGS. 1 to 3 and a user terminal capable of communicating with the servers (eg, various devices such as mobile terminals, body measurement devices, and laptops). can do.

The immersive fashion metaverse service providing system (for example, the system 100, the system 200 or the avatar production main server 220) receives at least two photos or images representing the user's body through a communication network (410). . At least two pictures or images may be obtained through a camera sensor included in the system. The at least two photos or images represent a front of the user and a side of the user, respectively. At least two pictures or images are transmitted to the system through a wired communication network or a wireless communication network for transmitting and receiving data from a user's device. Communication networks include wired Internet, local area wireless communication networks such as LAN and Bluetooth, wireless Internet such as WiFi, portable Internet such as WiBro or WiMax, 2G mobile communication networks such as GSM or CDMA, 3G mobile communication networks such as WCDMA or CDMA2000, HSDPA or HSUPA. It may include a 3.5G mobile communication network, a 4G mobile communication network such as an LTE network or an LTE Advanced network, a 5G mobile communication network, a 6G communication network, and the like.

The immersive fashion metaverse service providing system processes the received images based on pose estimation and segmentation techniques (420), and identifies body keypoints from the front and side images of the user. ) are extracted (430). The realistic fashion metaverse service providing system can detect the user's posture based on AI technology from the received images, distinguish the area corresponding to the human body and the area corresponding to the background, and delete the area corresponding to the background. there is. In addition, the immersive fashion metaverse service providing system can output posture information and analyzed images (background deletion, color classification for each body part).

The immersive fashion metabus service providing system converts the analyzed image into data (for example, conversion into coordinate values such as human outlines (silhouettes), etc.), detects the locations of key points to be measured, and converts key points, outline data, etc. output (440).

The immersive fashion metaverse service providing system measures the main length and circumference values using key points, corrects the size value based on the person's posture information, and converts the corrected size value based on the user's height information into a real value. After conversion (450), a body size result (body measurement data) including the user's chest, waist, hip circumference, limbs, and shoulder length is output (460). The body measurement data may include coordinate values, sizes, and the like measured in the user's image. The immersive fashion metaverse service providing system can collect and analyze body size results measured based on the AI process.

5 shows an example of a user interface for measuring a user's body size.

A realistic fashion metabus service providing system (for example, the system described in FIGS. 1 to 4 ) includes an application installed in a user terminal (eg, a mobile terminal) for providing a body size measurement service. 5 shows an example 500 of a user interface for a user body size measurement function provided by an application installed in a user terminal. The user interface includes a menu for receiving data from the user, a result processed based on the received data (for example, a body size measurement result), and a visual output including content received from servers.

When the user drives the application, a home page is provided (510). Although not shown in the drawing, the user may select a body size measurement function through a menu included in the homepage or the like. An interface 520 for measuring body size is provided according to user selection. The interface 520 for body size measurement includes visual output such as one or more images, icons, and menus for selecting a shooting mode. The shooting mode according to the embodiments is to secure front and side images of the user, and the rear shooting mode secures the user's image through a camera installed on the rear of the user terminal and the user's image through a camera installed in the front of the user terminal. It includes a frontal shooting mode to secure the image. When the user's front and side images are secured according to the shooting mode, as described in FIG. 4, the realistic fashion metaverse service providing system (or server) can measure the user's body size and provide the measurement result through the application. . Alternatively, the application may directly measure the user's body size. The application provides an interface 530 indicating the size measurement result. The interface 530 indicating the size measurement result may include a user's body and a visual output (eg, a 2D avatar) indicating the size measurement result. In addition, the application derives an optimal clothing size suitable for the user's body based on the user's body size measurement result and provides an interface 540 indicating the optimal clothing size.

6 shows an example of a user interface for virtual fitting.

FIG. 6 shows an example 600 of a user interface for virtual fitting provided by the system described in FIGS. 4 and 5 through an application.

The realistic fashion metaverse service providing system may provide the user's body size measurement result described in FIG. 5 through an application. Alternatively, the application may directly perform the process of measuring the user's body size described in FIGS. 4 and 5 . The application provides an interface 610 indicating the user's body size measurement result. When the user stores the body size measurement result and selects specific clothing, the application provides an interface 620 that provides a clothing size suitable for the user's body size measurement result. The corresponding interface 620 includes visual output such as an icon for selecting a virtual fitting. When the user selects virtual fitting, the application provides interfaces 630 and 640 including virtual fitting results of the 2D or 3D avatar.

7 shows an example of a realistic fashion metabus service providing system.

7 shows an example of a realistic fashion metaverse service providing system that creates an avatar based on the user's body size measurement described in FIGS. 4 to 6 and provides virtual fitting and clothing sales based on the avatar.

Blocks 700 shown at the top of FIG. 7 represent data input to the realistic fashion metaverse service providing system described in FIGS. 1 to 6 and/or output data provided to the user from the system.

The blocks 710 shown in the middle of FIG. 7 are the size of the user's body driven by the server based on the information (photo, input value, etc.) transmitted from the realistic fashion metaverse service providing system and/or the application installed on the user terminal. It represents the processes of creating avatars based on measurements and providing virtual fitting and apparel sales based on them.

Blocks 720 shown at the bottom of FIG. 7 represent functions provided by the system through each process or modules providing functions.

Upon receiving input data from the user (for example, the user's front and side images and the user's height value described in FIGS. 4 to 6), the application described in FIGS. 4 to 6 is executed in FIGS. Provides an interface for measuring body size as described. The interface for body size measurement includes one or more visual outputs such as one or more images, icons, and menus for selecting a shooting mode created based on a technique for guiding a user's photo-taking.

The immersive fashion metaverse service providing system and/or the application installed on the user terminal measure the user's body size and output the body measurement data (or the user's body size measurement result). The immersive fashion metaverse service providing system and/or applications installed on user terminals are photo correction technology for size extraction, body value and body type extraction, body value correction, additional correction (based on height and/or chest circumference), and body value generation module. may include

A realistic fashion metaverse service providing system and/or an application installed in a user terminal may generate a realistic avatar based on body measurement data. The immersive fashion metaverse service providing system and/or applications installed in the user terminal may include 3D avatar body data, basic avatar body information selection, body value correction, and 3D avatar generation (bone-based) modules.

When the user selects specific clothing, the realistic fashion metaverse service providing system and/or the application installed on the user terminal receive pre-stored clothing modeling data or clothing modeling data received from an external server, etc., and AI-based customized clothing recommendation module. / Using the processor, it is possible to recommend customized clothes suitable for the user's body size based on the body measurement data.

The immersive fashion metaverse service providing system and/or the application installed in the user terminal may perform virtual fitting to output a virtually fitted immersive avatar. The immersive fashion metaverse service providing system and/or the application installed in the user terminal may include an avatar calling module for calling a created and stored avatar, a recommended clothing calling module, and a virtual fitting module for virtual fitting.

The immersive fashion metaverse service providing system and/or the application installed on the user terminal may provide a conversational chatbot according to a customer request signal. The immersive fashion metaverse service providing system and/or the application installed in the user terminal may include an AI-based chatbot engine, a customer order response module, a customer inquiry response module, and a customer complaint response module.

The input/output data, processes, and modules shown in FIG. 7 are merely examples, and the realistic fashion metaverse service providing system may further include additional input/output data, processes, and modules not shown in the drawing.

8 shows an example of body size measurement.

As described in FIGS. 1 to 7 , the realistic fashion metaverse service providing system can measure the user's body size after receiving the user's front and side images and the user's height value.

The immersive fashion metaverse service providing system (or the avatar production main server 220 described above) may measure the body size by generating key points. For example, the system may generate and display a plurality of key points for body parts of the user appearing in the front and side images of the user. Key points are displayed on at least one of the user's eyes, neck, shoulders, chest, waist, hips, elbows, wrists, knees, and ankles. A key point may be displayed at the center of a body part (for example, a shoulder) to which the key point belongs.

In addition, the realistic fashion metaverse service providing system uses image analysis algorithms, such as Harris corner detector, Shi & Tomasi, SIFT (Scale Invariant Feature Transform), SURF, BRIEF, etc., to extract a plurality of key points from front and side images. You can use it yourself or create/use your own algorithm using it.

The immersive fashion metaverse service providing system separates the area representing the user's body image from the background area in the frontal and side images based on the image segmentation algorithm, and secures a boundary line representing the boundary of the area representing the body image to ensure body parts. It is used to calculate the star circumference. In addition, the immersive fashion metaverse service providing system can smoothly transform the boundary of the area representing the body image using OPEN CV's erosion/expansion technology.

The immersive fashion metaverse service providing system may measure the length of an arm or leg by measuring the distance between a plurality of key points, or calculate the circumference of a body part using the equation below.

[Equation 1]

In the equation, C represents the circumference of each body part, a represents the width of the corresponding body part in the front image, and b represents the width of the corresponding body part in the side image.

In the case of the waist circumference, the realistic fashion metaverse service providing system selects two key points closest to the top and bottom of the waist in the frontal image and extracts a point with the smallest width of the body area between the selected key points. A value a in the equation is derived by measuring the width of the body region corresponding to the extracted point, and a value b in the equation is derived by measuring the width of the body region in the side image corresponding to the same height as the extracted point.

In the case of the chest circumference, the realistic fashion metaverse service providing system selects two key points closest to the top and bottom of the chest displayed in the side image, and extracts the inflection point of the body region boundary between the selected key points. The inflection point of the body region boundary represents a point where the slope of the boundary changes from a positive value to a negative value. Therefore, the b value of the equation is derived by measuring the width of the body region at the extracted inflection point, and the a value of the equation is derived by measuring the width of the body region in the frontal image corresponding to the same height as the extracted inflection point. The inflection point according to the embodiments calculates the image weight in the x-direction and the y-direction using Sobel's x-direction filter and y-direction filter at each pixel located at the boundary of the body region, and then calculates the inclination angle at the corresponding pixel It may be extracted by calculating , and the neck, thigh, arm, and leg circumferences may be calculated using a circle circumference formula after measuring the diameter in the front image or side image. The realistic fashion metaverse service providing system compares the measured body size with the received key information to calculate a conversion ratio, and multiplies the calculated conversion ratio to the measured body size to obtain the actual body size. For example, when the key calculated by the realistic fashion metaverse service providing system is A cm and the key received from the user is B cm, the conversion ratio is derived by dividing B by A.

The realistic fashion metaverse service providing system can measure the user's body size only with the key value input from the user and at least two user's front and side images. In addition, in order to minimize the range of error, the realistic fashion metabus service provision system additionally corrects the chest circumference with the largest error in the measured body size according to the height value of the user and the angle of the photo taking, which have a lot of influence on the overall body size measurement. It can accurately measure the user's body size.

In particular, the realistic fashion metaverse service providing system finds the user's key area (in pixel units) from the user's key value input by the user and the user's front and side images, and uses the ratio of the two values to measure all measured values of the user's body. Calculate Therefore, the user's key value is a very important factor. However, the user's key value extracted from the image may change depending on whether the user is wearing shoes or has a hair style. Therefore, the realistic fashion metaverse service providing system designates the user's ankle and eyes as key points in the user image in order to minimize the error range according to the user's shoes or hair style. The user's ankle height and forehead length are derived by applying a predetermined ratio (for example, 3-5%) of the statistically detected ankle-to-eye length. That is, the user's height derived from the user's image becomes a sum (in pixels) of key points, the length between the ankle and the eye, the height of the ankle, and the length from the eye to the head. However, the length between the ankle and the eye, which are key points, may vary depending on the direction of the user's face.

Therefore, the immersive fashion metaverse service providing system considers the rotational elements of the user's face that most affect the user's key value and removes the elements that affect the user's key value.

8(A) shows values of yaw, pitch, and roll, which are rotation factors of the user's face that most affect the user's key value. The yaw value represents the rotation value that occurs around the z-axis when the user rotates the face left and right, the pitch value represents the rotation value that occurs around the y-axis when the user moves the face up and down, and the roll value represents the rotation value that occurs around the y-axis when the user moves the face up and down. represents the rotation value that occurs around the x-axis when tilting left and right. Among them, the pitch value affects the user's key value. The right side of FIG. 8 shows a process in which the realistic fashion metaverse service providing system finds key points of faces shown in three user side images through an AI-based algorithm. The three side images shown on the right side of FIG. 8 have different pitch values (for example, 0, 10, and -10 from the leftmost) according to the position of the user's face.

The realistic fashion metaverse service providing system can measure and data the error rate between the key value input by the user and the key value of the user measured through the image as follows.

Error rate: (input key - restored key) / (input key). *Error: (Input key - Restored key)

In addition, the immersive fashion metaverse service providing system can formulate the degree (unit %) of the change in the height of the user on the pixel of the image according to the pitch value using the AI algorithm as follows.

[Equation 2]

Error rate (dependent variable) ~ pitch **2 + pitch + error (residual)

Therefore, the realistic fashion metaverse service providing system can derive an accurate key value (in pixel units) using the key value input by the user and the pitch value obtained from the front and side images.

9 are graphs showing the pitch value effect.

The graph (A) of FIG. 9 shows the change in user height (cm) according to the type of shoe. The x-axis of the graph represents the index indicating the order of data, the central y-axis value is the restored/model predicted key (cm), and the right y-axis value is the restored key (cm) - model predicted (cm) key value indicate the difference. Depending on the type of shoe, height can be affected, but the effect is not linear.

The graph (B) of FIG. 9 shows the pitch square value, pitch value, roll value, front image (is_front), shoe type (barefoot, slippers, sneakers) as independent variables in the boosting ensemble algorithm XGBoost (Random Forest), As a dependent variable, the estimated height value based on the two key points (eyes and ankles) is inserted, and the graph when overfitting is shown. Lines in the graph represent predicted values, and dots in the graph represent restored key values. The x-axis of the graph represents the data index value, and the y-axis represents the restored/model-predicted height (cm) value.

10 are graphs showing pitch value effects.

The graph (A) of FIG. 10 is a graph showing an error when the restored key value (dots) is subtracted from the predicted value (line) shown in the graph (B) of FIG. 9 . That is, the more variables that can affect the key value are included, the more accurate the key value is. The x-axis of the graph represents the data index value, and the y-axis represents the difference between the restored key (cm) and model-predicted key value (cm).

Graph (B) of FIG. 10 shows the importance of variables input to the boosting ensemble algorithm XGBoost (Random Forest). As shown in the graph, the pitch value and whether it is a front image (is_front) are variables that have the most influence on the key value (pixel). Whether or not it is a front image (is_front) is affected by the distance from the camera, so the pitch value is determined as the variable that has the most influence on the key value.

11 are graphs showing pitch value effects.

The graph (A) of FIG. 11 shows the result of predicting the key value by inputting only the pitch value, the roll value, whether it is a front image (is_front), and whether it is a shoe as independent variables to the boosting ensemble algorithm XGBoost (Random Forest). The graph (B) of FIG. 11 shows the key values affected by only the pitch value in the state of removing the roll value, front image (is_front), and shoes. Lines in each graph represent predicted values, and dots in the graph represent restored key values.

As shown in the two graphs, whether the front image is recognized (is_front) and the roll value do not significantly affect the key prediction value. In addition, whether or not shoes (or the type of shoes) can affect the height prediction value, but the degree of influence is inconsistent. Therefore, it can be seen that the pitch value is the most important variable in the key prediction value.

12 shows an example of a chest circumference correction process.

As described above, the realistic fashion metaverse service providing system selects the two key points closest to the top and bottom of the chest displayed in the side image, extracts the inflection point of the body area boundary between the selected key points, and based on the inflection point Chest circumference can be measured by measuring and deriving the width of specific body regions on the side and front. However, if the user is not standing in the correct lateral posture (rotated 90 degrees from the front posture), an error may occur in the measured chest circumference if the side width (chest width) is over or undermeasured. Therefore, the immersive fashion metaverse service providing system estimates the rotation angle by how much the side of the user shown in the side image is rotated compared to the user's full side (posture rotated 90 degrees from the front to the left or right), and the estimated rotation angle Based on , the chest circumference value can be corrected by correcting the lateral chest width error.

12(A) shows examples of the user's front side 1200, the user's turned side 1210, and the user's full side 1220. As shown in the figure, the chest width of the rotated side 1210 of the user is wider than the chest width of the full side 1220 of the user. Therefore, if the chest circumference value is simply calculated using the chest width of the rotated side surface 1210 of the user, an error may occur.

12(B) is an example of a rotation angle estimation process of a realistic fashion metaverse service providing system.

The immersive fashion metaverse service providing system sets at least one or more key points on the user's front side obtained through the user's frontal image and the rotational side of the user shown in the user's side image. The immersive fashion metaverse service providing system measures the distance 1230 between key points set on the user's front side, and measures the distance 1240 between key points set on the side of the user's rotation, according to the following equation. Estimate rotation angle.

[Equation 3]

rotation angle = arccos((front side length) / (side side length) )

The immersive fashion metaverse service providing system calculates the chest circumference value by calculating the chest width of the full side 1220 shown in (A) based on the estimated rotation angle.

13 shows an example of a chest circumference derivation process.

The realistic fashion metaverse service providing system may measure the chest circumference by measuring the chest thickness in addition to the chest width described in FIG. 11 . The immersive fashion metaverse service providing system can calculate the chest circumference by calculating the elliptic equation with the front (f) and side (s). The realistic fashion metaverse service providing system calculates the chest circumference in consideration of the curvature of the chest when the user is a woman.

13(A) shows a cross-sectional view of a female user's chest. The green ellipse shown in the middle of the sectional view and the line representing the user's body cross-section do not match.

13(B) shows an example of a cross-sectional view of a female user's chest for chest circumference measurement. In this example, it is assumed that the center of the chest is located at ¼ points from each end of the body.

13(C) shows the chest circumference measurement process. Based on the point where the curve of the chest protrudes (1/4 point), two halves of an ellipse are placed on both sides, and the length between the two chests is assumed to be a straight line. Therefore, the length between the breasts corresponds to the length of two straight lines with a length of 1/2f. The realistic fashion metaverse service providing system calculates the chest circumference value as follows.

[Equation 4]

Perimeter of 2 half ellipses (circumference of one oval) + ½ face length (face length)

The measured user body size described in FIGS. 8 to 13 may be stored in the database server (database 250 described in FIGS. 2 to 3 ) of the realistic fashion metaverse service providing system.

14 is a block diagram of a realistic fashion metaverse service processing process.

A block diagram 1400 of FIG. 14 shows a process of processing a realistic fashion metaverse service processed by the realistic fashion metaverse service providing system described in FIGS. 1 to 13 . Each process is performed by the server described in FIGS. 2 and 3 .

The realistic fashion metaverse service providing system receives an avatar creation request signal (login) from the user through the web server (1410). The server 1420 (server 230 dedicated to making avatar faces) of the realistic fashion metaverse service providing system receives a picture of the user's face from the storage server 1430 (storage server 240) and creates a 3D avatar can make the face of In addition, the avatar face creation dedicated server 1420 may create a 3D avatar's face by reflecting the makeup in the 3D face creation process when a makeup selection and/or change signal is received from the user through the web server.

The avatar production main server 1440 (avatar production main server 220) receives a face file from the avatar face production dedicated server 1420 and adds the created face to the basic avatar, and the database server 1450 (database The user's body size stored in the server 250 is received and an avatar suitable for the user's body shape is created. The user's body size may be measured according to the description with reference to FIGS. 1 to 13 and stored in the database server 1450 . The avatar production main server 1440 receives hair style and clothing resource files from the storage server 1430, applies a hair style to the created avatar, fits clothing items, and outputs the avatar file.

The service server 1460 (service server 210) receives the avatar file and outputs a 3D avatar. In addition, when the service server 1460 receives a signal for selecting a user's 3D space, it receives space data from the database server 1450 and resources for space production from the storage server 1430 to create a 3D space. The service server 1460 may receive walking data from the database server 1450 and may generate and output 3D avatar cat walking content by adding a 3D avatar animation to the created 3D space.

15 shows a process of generating realistic fashion metaverse contents.

A block diagram 1500 shown in FIG. 15 shows a process in which the realistic fashion metaverse service providing system described in FIGS. 1 to 14 creates content to provide a virtual fashion show service in which an avatar operates in a virtual space.

The process of creating realistic fashion metaverse contents includes a process of creating realistic virtual fashion contents (realistic avatars, realistic items, and realistic virtual space) (1510), VR production process (1520), VR space modeling and optimization Process 1530 and PC setting process 1540 are included.

The process of generating 3D sources (clothes, avatars, and animations) (1510) includes a process (1511) of creating realistic virtual fashion contents using a 3D authoring tool (Blender or Maya, etc.) to create catwalk animations of avatars. do. Also, in the process of generating realistic virtual fashion content (1510), clothes suitable for the avatar are produced using a dedicated 3D authoring tool (CLO 3D, etc.), and clothes animation suitable for the catwalk animation data of the avatar is produced. Also, in the process of creating VR (1520), clothes and textures (PNG files) and avatar and working animation (FBX) data are received and arranged to generate final animation avatars. The VR space modeling and optimization process 1530 uses Blender to create 3D data formats (FBX and Blender files, etc.) for space apps. In the process of creating VR (1520), a 3D data format for an app for space is received, avatars and clothes are placed in the space, other necessary functions are implemented, or a unity build file is output by implementing space and clothes materials. In the PC setting process 1540, an app for providing VR content is installed or a function setting for pairing with a head mounted display providing VR content is performed.

16 shows a process of generating WebGL-based immersive fashion metaverse contents.

A block diagram 1600 shown in FIG. 16 shows a process in which the realistic fashion metaverse service providing system described in FIGS. 1 to 15 creates and provides WebGL-based realistic fashion metaverse contents. Realistic fashion metaverse contents based on WebGL can be created based on 3D authoring tools. In order to express realistic 3D avatars and items in a web environment with limited computational speed, the realistic fashion metaverse service providing system can utilize and optimize the 3D data format standard for the web (glTF). Optimization of glTF is absolutely necessary to express the detailed expression of offline 3D objects, which cannot be expressed through basic glTF, without slowing down the speed even in the web environment.

The main server 1610 includes a converter that receives a 3D data file for an app for a space created in the avatar production main server 220 and generates it as a glTF file. In addition to the 3D avatar file, the main server 1610 may also convert a virtual clothing file, a 3D space file, an animation file, etc. fitted to the input 3D avatar into a 3D data format standard (glTF) file for the web. The glTF files may be compressed through mesh compression and stored in the database server 1620 (database server 250). In addition, the main server 1610 separates 3D space, clothes, avatar animation data, and the like, and the separated files may be separately stored in the file server 1630.

The avatar face making server 1640 (the avatar face making server 230) creates 3D avatar face data using a user's face picture and makeup information selected by the user.

The web server 1650 receives compressed avatar files (compressed glTF), space, clothing and animation files, and 3D avatar face data (JPEG), and provides final results to the user through the web. The final result is content in which an avatar with selected or designated clothes and a face made up applied in a 3D space performs an animation operation.

17 shows an example of a realistic fashion metaverse service providing system.

A block diagram 1700 of FIG. 17 shows servers that generate WebGL-based realistic fashion metaverse contents included in the realistic fashion metaverse service providing system described in FIGS. 1 to 16 . Servers shown in the drawing are virtual servers (or instant) provided in a cloud computing system, and are provided to drive operating systems and applications. The servers shown in the drawings are merely examples, and the realistic fashion metaverse service providing system may further include servers not shown in the drawings.

The API server 1710 is a server that implements a corresponding function by calling an API for each implemented function (for example, an avatar module, a virtual fitting module, etc. described in FIGS. 2 and 3 ).

An exporter 1720 converts the pre-glTF file into a glTF file format. The exporter 1720 includes a communicator, a pre-glTF interpreter (or pre-glTF decoder), and a glTF generator.

The file server 1730 is the file server 1630 described in FIG. 16 and stores files corresponding to components constituting the contents of the realistic metaverse service, that is, clothes, avatars, 3D space, and avatar animation data.

The core server 1740 is a server that monitors and manages traffic generated in performing functions of the servers and performs load balancing, and includes a communicator, a load balancer, and an asset manager. The asset manager manages Unity assets used to create WebGL-based immersive fashion metaverse contents. Each asset is a 3D file representing an avatar, a clothing item, and the like.

The avatar production main server described in FIGS. 1 to 16 creates a 3D data format (FBX, etc.) file including geometry and texture information of a 3D avatar as an asset format for the avatar.

As described in FIG. 16, the optimized glTF generator 1750 generates pre-glTF files corresponding to components constituting the contents of realistic metaverse services, such as 3D data format (FBX, etc.) files for apps. (pre-glTF). The pre-glTF is generated by including the glTF and a Rendering Priority Index (RPI) 3D map, which is a 3D map of priorities for rendering. The realistic fashion metaverse service providing system depends on the type of realistic fashion metaverse service (eg, body measurement service and virtual fitting service described in FIGS. 4 to 6, sales service, or virtual fashion show service described in FIG. 15). Each asset can be rendered differently. For example, if an asset for a 3D T-shirt is provided to a virtual fitting service, the immersive fashion metaverse service providing system renders the 3D file to look as similar to real clothing as possible. In addition, when assets for 3D T-shirts are provided to the avatar walking animation service, since the expression of the 3D T-shirts changes according to the movement of the avatar, the immersive fashion metaverse service providing system can render only important parts of the corresponding 3D files. The RPI 3D map is a map indicating priorities for rendering of vertices set in a 3D image represented by each asset. Therefore, the realistic fashion metaverse system can process only the minimum vertex and texture data using the RPI 3D map to perform faster data processing and minimize the use of resources (CPU, GPU).

For this purpose, the optimized glTF generator 1750 includes a communicator, a 3D file interpreter, a vertex optimizer module (or geometry generation module), and a Rendering Priority Index (RPI) 3D map module. , a pre-glTF generator (pre-glTF generator or pre-glTF encoder). A 3D file interpreter converts several types of 3D files into usable ones. For example, a 3D file interpreter can convert 3D files into assets and the like. The vertex optimizer module (or geometry generation module) creates optimized vertices for each item represented by each asset based on AI-based deep learning technology. Vertices are points arranged in an image represented by a 3D file, and each vertex has texture information such as location information (geometry value or coordinate value) and color. The vertex optimization module creates optimized vertices by adjusting the number of vertices, arrangement position, etc., based on key points and boundaries (silhouettes) of items represented by each asset. Optimized vertices have importance according to position. The RPI 3D map module creates 3D map data for rendering based on the importance of optimized vertices. RPI 3D maps are provided to minimize texturing according to the importance of each vertex. The pre glTF generator generates a pre glTF including the generated RPI 3D map.

The database server 1760 (for example, the database server 250 of FIG. 2 ) stores data and information required for a realistic virtual fashion content service.

18 shows an example of pre-glTF (pre-glTF).

As described above, realistic fashion metaverse content can be created based on the Unity asset tool. In addition, the avatar production main server described in FIGS. 1 to 17 creates an FBX file including geometry and texture information of a 3D avatar in an asset format for the avatar. These 3D data format (FBX, etc.) files for apps are converted to 3D data format (glTF) files for web as described in FIG. 16 and stored.

The immersive fashion metaverse service providing system described in FIGS. 1 to 17 may create a 3D map of priority for rendering based on an AI algorithm in the process of generating the geometry of a 3D avatar and store it together with glTF. The pre-glTF is generated through the pre-glTF generator described in FIG. 17, including the glTF and the 3D map of priority for rendering.

19 shows the operation of the optimized glTF generator.

As described in FIG. 17 , the optimized glTF generator (for example, the optimized glTF generator 1750) may optimize vertices through AI-based deep learning and generate an RPI 3D map (1910). 3D assets consist of a geometry creation step (create geometry) and a texture creation step (create texture), and the RPI 3D map is created based on an AI algorithm in the geometry creation step and used in the rendering step where the texture is applied.

The realistic fashion metaverse service providing system described in FIGS. 1 to 17 performs rendering by processing textures corresponding to vertices according to importance based on the RPI 3D map (1920).

20 shows an example of optimized vertices.

As described above, the vertex optimization module creates optimized vertices by adjusting the number of vertices, arrangement position, etc., based on key points and boundaries (silhouettes) of items represented by each asset. Optimized vertices have importance according to position. The importance of a vertex may be set to a level from 1 to 5, but is not limited to this example. As shown in FIG. 20, vertices A representing the outer contour correspond to positions with high importance, and thus are assigned levels 4-5. The vertex (B) located in the inner contour is assigned a low level of 3 because it corresponds to the position with the next highest importance. Vertices (C) located in other ranges have low importance and are assigned levels 1 and 2.

21 is a flowchart illustrating a rendering process based on an RPI 3D map.

The immersive fashion metaverse service providing system described in FIGS. 1 to 17 (for example, the pre-glTF encoder described in FIG. 17) generates rendering importance information, that is, an RPI 3D map, according to the vertex position of each item modeled, and glTF and RPI Encode the 3D map to generate a free glTF. The realistic fashion metaverse service providing system may store the generated free glTF and transmit it to the user terminal (2110). The user terminal decodes the free glTF to generate glTF, and performs rendering based on the RPI 3D map (2120).

22 shows an example of a virtual fitting type.

As described in FIGS. 1 to 21 , the immersive fashion metaverse service providing system may provide a virtual fitting service in which clothing items are put on an avatar. The virtual fitting service changes the size of the clothing item according to the body size of the avatar and unconditionally fits the clothing item to the avatar. The first method (every fit method) and the physical properties, gravity friction, It is implemented based on at least one of the second methods (size fit method) in which size change of clothing items is limited according to variables such as resistance.

22 shows three virtual fitting types used by the realistic fashion metaverse service providing system to implement the first method and the second method. The first method and the second method may be implemented through each of the three virtual fitting types or a combination of at least one or more.

22 (A) represents the avatar masking fitting type, (B) represents the clothing shader fitting type (Cloth_shader fitting type), and (C) represents the clothing collider fitting type (Cloth_Collider Fitting Type). .

FIG. 22 shows a technology used in the avatar creation process of the avatar creation server 220 described in FIG. 2 . The avatar masking fitting type is a technique of performing a fitting by performing masking after performing vertices-based mesh modeling. The clothing shader fitting type is a technology that performs fitting by finding a collision point for main textures of a clothing item and adjusting vertex positions using a shader based on this. The clothing collider fitting type is a technique of performing fitting by creating a Unity collider to transform only colliding vertices among vertices.

23 shows a masking process of an avatar masking fitting type.

The immersive fashion metaverse service providing system starts with the form of vertices of a 3D object (avatar) according to the avatar masking fitting type, creates a plane, and displays it on the screen. The realistic fashion metaverse service provision system sets vertices in the basic state of the avatar and generates a line (Ray) that can detect collision in the normal (vertical) direction from the position of each vertex to the inside of the avatar. If the line and the mesh on the clothes collide, the realistic fashion metaverse service providing system may set a masking value so that the triangle (polygons) part of the avatar mesh is not visible.

24 shows a process of avatar masking fitting type.

24(A) shows an example of a part of an avatar that cannot be masked when a collision is detected and masked using only position values of vertices rather than faces. The immersive fashion metaverse service providing system may change the line into a sphere shape in order to detect a non-collision portion, and may perform a masking process on all values included in the sphere shape. 24 (B) shows an example in which avatar mesh processing is performed up to the boundary between the avatar and clothing in the case of a spherical shape (B). The immersive fashion metaverse service providing system may set an exception collision range to be treated as an exception when masking according to the boundary between the clothing mesh and the avatar and the clothing. 24(C) shows an example of an exception collision range.

25 shows a processing procedure of a clothing shader fitting type.

The clothing shader fitting type is different from the avatar masking fitting type in that the collision point is found in the clothing standard. The clothing shader fitting type converts the position of the collision point into UV coordinates and adjusts the position of the vertex in the shader. 25 shows a process of converting a collision point into UV coordinates.

26 shows a processing procedure of a clothing shading fitting type.

26(A) shows an example in which the realistic fashion metaverse service providing system deforms clothing items to fit the avatar, and FIG. 26(B) shows the realistic fashion metaverse service providing system colliding with a line through a line. shows an example of detecting 26(C) shows an example in which the realistic fashion metaverse service providing system secures the positions of vertices corresponding to UV coordinated collision positions and adjusts the shader according to the positions of the vertices. 26(D) shows an example in which the realistic fashion metaverse service providing system outputs a final fitting result.

27 shows a processing procedure of a clothing collider fitting type.

The clothing collider fitting type is a technology that creates a collider at each vertex and performs virtual fitting according to the changed avatar.

27(A) shows an example of generating colliders of the same size for each vertex, and FIG. 27(B) shows an example of creating colliders of different sizes for each vertex.

As in the example of FIG. 27(A), if colliders of the same size are created and used for each vertex, the overall shape of clothing can be changed according to the colliding point.

Therefore, as shown in (B) of FIG. 27, the realistic fashion metaverse service providing system creates a relatively larger collider to give a size change according to the collision for the vertex corresponding to the collision location, and does not correspond to the collision location. For vertices that do not, we create relatively small colliders so that the clothing shape does not change.

Fig. 28 shows the processing process of the clothing collider fitting type.

28(A) shows an example in which the realistic fashion metaverse service providing system deforms a clothing item to fit the avatar, and FIG. 28(B) shows the realistic fashion metaverse service providing system colliding with a line through a line. shows an example of detecting 28(C) shows an example in which the realistic fashion metaverse service providing system secures the positions of vertices corresponding to coordinated collision positions and creates the same or different colliers according to the positions of the vertices. 28(D) shows an example in which the realistic fashion metaverse service providing system outputs a final fitting result.

29 is a flow chart illustrating an image seamless texturing algorithm.

The immersive fashion metaverse service providing system described in FIGS. 1 to 28 may print an image desired by a user on the entire virtual clothing based on an image seamless texturing algorithm. Flow 2900 of FIG. 29 represents an image seamless texturing algorithm. The image seamless texturing algorithm includes preprocessing (2910), intersection boundary processing (2920), point processing (2930), intersection line processing (2940) and matrix transform processing (2950).

Pre-processing 2910 prepares the meshes of the clothing item (e.g., T-shirt) (e.g., a mesh representing the front of the T-shirt, a mesh representing the back of the T-shirt, a mesh representing the right sleeve, and a mesh representing the left sleeve, etc.) It is a process of Here, clothing items are exemplified, but furniture items other than clothing items can also be applied, and this embodiment can be applied to any item that can decorate the exterior of items other than small items, large items such as vehicles or building exteriors, or personal property. there is.

Intersection boundary processing 2920 finds overlapping boundaries (intersection boundaries) between meshes, for example, shoulder boundaries, for seamless texturing. The point processing 2930 finds points indicating a region where a photo or image selected by the user and the mesh overlap each other with respect to at least one mesh (for example, a mesh representing the front of a T-shirt). Intersection line processing 2940 straightens lines that overlap other meshes for that mesh based on the points. The matrix transform processing 2950 performs a matrix operation to output images of other meshes connected to the mesh, rotates or converts the image, and outputs the image to each mesh. Point processing 2930 finds points representing areas where the user-selected photo or image and the mesh overlap each other.

30 shows an example of an image seamless texturing algorithm.

30(A) shows intersection boundary processing (intersection boundary processing in FIG. (2920)).

30(B) shows an example of point processing (point processing 2930) for finding points p1, p2, p3, and p4 representing the overlapping area when the image 3010 selected by the user and the mesh 3020 overlap. indicate Coordinates of points p1, p2, p3 and p4 are derived according to the equations shown in the figure.

31 shows an example of an image seamless texturing algorithm.

(A) of FIG. 31 shows intersection line processing (intersection line processing 2940 in FIG. 29) that makes the boundary overlapping with other meshes into a straight line 3110 for the mesh on which the image selected by the user is output based on the points. shows an example of

31(B) shows matrix transformation processing that rotates or converts an image by performing a matrix operation to output an image of other meshes connected to the mesh and outputs the rotated or converted image to each mesh (FIG. 29 shows an example of matrix transform processing (2950) of

32 is a flowchart illustrating a method of providing a realistic fashion metabus service.

The realistic fashion metaverse service providing system creates files corresponding to components composing the contents of the realistic metaverse service (3210). The files are created by combining the 3D face created based on the user's face picture and the avatar reflecting the user's body shape created based on the user's body size information, and the 3D space constituting the contents of the realistic metaverse service. It includes a 3D space file representing a 3D space file and a clothing file representing a clothing item for virtual fitting to an avatar. A detailed description is the same as that described in FIGS. 1 to 31 .

The immersive fashion metaverse service providing system generates RPI (Rendering Priority Index) 3D map information indicating priority for rendering by performing importance judgment based on AI algorithm for each generated file (3220). ). A detailed description is the same as that described in FIGS. 1 to 31 .

The immersive fashion metaverse service providing system converts each generated file into a glTF file and encodes it into a free glTF file together with RPI 3D map information (3230). A detailed description is the same as that described in FIGS. 1 to 31 .

The user's body size information according to the embodiments measures the error rate between the key value in units of pixels derived from the side image and the front image of the user and the key value input from the user, and the degree of rotation of the user's face appearing in the user's side image. It is measured by measuring the phosphorus pitch value and correcting the measured error rate with the measured pitch value.

The user's body size information according to embodiments includes a chest circumference value, the chest circumference value sets one or more first key points in the side image of the user, measures a distance between the set first key points, and , Set one or more second key points in the user's front image, measure the distance between the set second key points, and based on the distance between the first key points and the distance between the second key points, the side image of the user It is generated based on the measured rotation angle by measuring the rotation angle of the user's body expressed in .

The step of generating RPI (Rendering Priority Index) 3D map information indicating the priority for rendering by performing an importance judgment based on an AI algorithm for each generated file,

Setting one or more vertices in an image represented by the file and generating the RPI 3D map information based on the importance of the one or more set vertices. A vertex is a point having location information and texture information, and each vertex has an importance based on its location.

The realistic fashion metaverse service providing system decodes the free glTF file to obtain RPI 3D map information and glTF files, and renders the glTF file based on the obtained RPI 3D map information to provide the contents of the realistic fashion metaverse service. You can follow the steps provided.

For convenience of description, each drawing has been divided and described, but it is also possible to design to implement a new embodiment by merging the embodiments described in each drawing. And, according to the needs of those skilled in the art, designing a computer-readable recording medium in which programs for executing the previously described embodiments are recorded falls within the scope of the embodiments. The apparatus and method according to the embodiments are not limited to the configuration and method of the embodiments described above, but all or part of each embodiment is selectively combined so that various modifications can be made. may be configured. Although preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and common knowledge in the art to which the present invention pertains without departing from the gist of the embodiments claimed in the claims. Of course, various modifications are possible by those who have, and these modifications should not be individually understood from the technical spirit or prospects of the embodiments. Descriptions of the device and method according to the embodiments may be applied complementary to each other.

The realistic fashion metaverse service providing system and components (eg, encoders, modules, etc.) described in FIGS. 1 to 32 may be configured by hardware, software, firmware, or a combination thereof. Various components of the embodiments may be implemented as one chip, for example, as one hardware circuit. According to embodiments, components may be implemented as separate chips. Depending on the embodiments, at least one or more of the components of the device according to the embodiments may be composed of one or more processors capable of executing one or more programs, and the one or more programs may be executed. Any one or more of the operations/methods according to the examples may be performed or may include instructions for performing the operations/methods. Executable instructions for performing methods/operations of an apparatus according to embodiments may be stored in a non-transitory CRM or other computer program products configured for execution by one or more processors, or may be stored in one or more may be stored in transitory CRM or other computer program products configured for execution by processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (eg, RAM) but also non-volatile memory, flash memory, PROM, and the like. Also, those implemented in the form of a carrier wave, such as transmission through the Internet, may be included. In addition, the processor-readable recording medium is distributed in computer systems connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In this document, “/” and “,” shall be interpreted as “and/or”. For example, “A/B” is interpreted as “A and/or B” and “A, B” is interpreted as “A and/or B”. Additionally, "A/B/C" means "at least one of A, B and/or C". Also, “A, B, C” means “at least one of A, B and/or C”. Additionally, “or” shall be construed as “and/or” in this document. For example, "A or B" could mean 1) only "A", 2) only "B", or 3) "A and B". In other words, “or” in this document may mean “additionally or alternatively”.

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, interpretation of various components according to embodiments should not be limited by the above terms. These terms are only used to distinguish one component from another. only thing For example, a first user input signal may be referred to as a second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. Use of these terms should be construed as not departing from the scope of the various embodiments. Although both the first user input signal and the second user input signal are user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

Terms used to describe the embodiments are used for the purpose of describing specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. and/or expressions are used in a sense that includes all possible combinations between the terms. The expression includes describes that there are features, numbers, steps, elements, and/or components, and does not imply that additional features, numbers, steps, elements, and/or components are not included. . Conditional expressions such as when ~, when, etc., used to describe the embodiments, are not limited to optional cases. When a specific condition is satisfied, a related action is performed in response to the specific condition, or a related definition is intended to be interpreted.

210: 1460: service server
220, 1440: Avatar production main server
230, 1420: Dedicated server for creating avatar faces
240, 1430: storage server
250, 1450: database server
1200: user's front
1210:: Turn side of user
1220: the complete aspect of the user

Claims (10)

Preparing files corresponding to components constituting the contents of the realistic metaverse service;
Generating based on a 3D map indicating a priority for rendering for the prepared file;
Here, the 3D map indicates the importance of the vertex according to the position of the vertex,
Here, the importance of the vertex is that a boundary point of an item represented by an asset is generated higher than an internal point of the item; and
Providing a pre-glTF file including a glTF (GL Transmission Format) file and the 3D (3 dimension) map information as the prepared file. According to claim 1,
The importance of the vertex is
Vertices representing the outer contour of the item are assigned high levels, vertices located on the inner contour of the item are assigned medium levels, and vertices located elsewhere in the item are assigned low levels. How to provide Hyung Fashion Metaverse service. According to claim 1,
The 3D map is a RPI (Rendering Priority Index) map indicating a priority for rendering the prepared file based on an AI algorithm, a realistic fashion metaverse service providing method. According to claim 1,
The vertex is a realistic fashion metaverse service providing method in which a point having location information and texture information is displayed on the image of the content. a database for storing files corresponding to components constituting the contents of the immersive metaverse service; and
It includes a processor that executes computer instructions,
the processor,
A preparation module that provides files corresponding to components constituting the contents of the realistic metaverse service;
a generation module for generating based on a 3D map indicating a priority for rendering of the prepared file;
Here, the 3D map indicates the importance of the vertex according to the position of the vertex,
Here, the importance of the vertex is that a boundary point of an item represented by an asset is generated higher than an internal point of the item; and
A providing module for providing a glTF (GL Transmission Format) file with the prepared file and a free glTF file including the 3D (3 dimension) map information; a realistic fashion metaverse service providing device. According to claim 5,
The importance of the vertex is
Vertices representing the outer contour of the item are assigned high levels, vertices located on the inner contour of the item are assigned medium levels, and vertices located elsewhere in the item are assigned low levels. Hyung fashion Metaverse service providing device. According to claim 5,
The 3D map is an RPI (Rendering Priority Index) map indicating a priority for rendering the prepared file based on an AI algorithm, a realistic fashion metaverse service providing device. According to claim 5,
wherein the vertex displays a point having location information and texture information on the image of the content. executed by one or more processors;
The processor
Providing files corresponding to components constituting the contents of the realistic metaverse service;
generating based on a 3D map indicating priority for rendering of the prepared file;
Here, the 3D map indicates the importance of the vertex according to the position of the vertex,
Here, the importance of the vertex is that a boundary point of an item represented by an asset is generated higher than an internal point of the item; and
A non-transitory computer-readable recording medium storing a program for performing steps including providing the prepared file as a glTF (GL Transmission Format) file and a free glTF file including the 3D (3 dimension) map information. . According to claim 9,
The glTF file,
An avatar file created by combining a 3D face created based on a user's face photo and an avatar reflecting the user's body shape created based on user body size information, 3D representing the 3D space constituting the contents of the realistic metaverse service A non-transitory computer-readable recording medium comprising a spatial file and a clothing file representing an item of clothing for virtual fitting to the avatar.

Images