KR102341866B1 - System for providing artificial intelligence-based platform implementing metaverse digital idol and Method thereof - Google Patents

Abstract

The present invention relates to a system and a method for providing an artificial intelligence-based platform implementing a metaverse digital idol. The system includes: a metaverse implementing server receiving motion data and biometric data from a user and generating metaverse data generated by synchronization between the real-world user and a metaverse digital idol, in which the metaverse data is executed using the biometric data and the motion data; and an electronic device receiving the metaverse data from the metaverse implementing server and displaying the metaverse data. The metaverse implementing server includes: a digital idol database storing digital idol information; a motion change calculation unit generating a character motion sequence defining the motion of the digital idol using the motion data and the digital idol information; a shading change calculation unit generating character shading data defining the appearance of the digital idol using the biometric data and the digital idol information; and a graphic processing unit generating the metaverse data including graphic data on the digital idol synchronized with the user through the character motion sequence and the character shading data.

Description

System for providing artificial intelligence-based platform implementing metaverse digital idol and Method thereof

The present invention relates to an artificial intelligence-based platform providing system and method for implementing a metaverse digital idol. Specifically, the present invention relates to a system and method for generating content by synchronizing a digital idol of a metaverse with a user in the real world.

Digital idol content production is done through a very complicated procedure. In the case of the conventional digital idol content production, the production is made in a linear manner using DCC (Digital Content Creation tools) that deal with 3D graphics. Specifically, the current digital idol content production method sequentially performs a modeling step, an animation step, a rendering step, and a compositing step to generate a final animation image.

In this type of digital idol content production method, various methods are used to bring out the reality of the character. The most representative work is to create a 3D mesh based on the movement of an actor with multiple sensors, and then apply the generated 3D mesh to the character to bring the character to life.

Various characters and backgrounds are used in the digital idol content creation process, and settings and situations for each character and background may be different. However, whenever the settings and circumstances of these characters and backgrounds are changed, time and cost increase when an actor performs a new role and creates and applies a 3D mesh, and it is difficult to immediately apply changes according to new characters and situations was present.

Accordingly, automated additional processing processes for the production of various digital idol contents are being developed in order to preserve the reality of the character and reduce the production cost.

Laid-open Patent Publication No. 10-2004-0096799

An object of the present invention is to provide an artificial intelligence-based platform providing system that implements a metaverse digital idol that synchronizes a user and a digital idol by using the user's biometric data and motion data.

Another object of the present invention is to provide an artificial intelligence-based platform providing method that implements a metaverse digital idol that synchronizes a user and a digital idol using the user's biometric data and motion data.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the appended claims.

In order to solve the above problems, an artificial intelligence-based platform providing system implementing a metaverse digital idol according to some embodiments of the present invention receives motion data and biometric data from a user, and a digital idol of the metaverse and a user in the real world. an electronic device that generates metaverse data generated by synchronizing the metaverse data and receives and displays the metaverse data from a metaverse realization server that is performed using the biometric data and the motion data and the metaverse realization server A motion change that includes an apparatus, wherein the metaverse implementation server generates a digital idol database that stores digital idol information, and a character motion sequence that defines the movement of the digital idol using the motion data and the digital idol information. a calculation unit, a shading change calculation unit generating character shading data defining the appearance of the digital idol by using the biometric data and the digital idol information; and the user synchronized with the user through the character motion sequence and the character shading data and a graphic processing unit that generates the metaverse data including graphic data of a digital idol.

In addition, the metaverse implementation server generates digital idol content through the metaverse data, receives the digital idol content, uploads it to SNS (Social Networking Service), and collects users' preferences to calculate a rating score. The method may further include a server, and a metaverse management server that receives the rating score and generates reward information that allows a reward corresponding to the rating score to be reflected in the user's account in the metaverse.

Also, the metaverse management server may receive purchase information from the user, and the reward information may be generated based on the reward information and the rating score.

In addition, the purchase information may be generated through purchase authentication information obtained from purchased toys and/or purchased contents.

In addition, the purchase authentication information may be generated by recognizing at least one of a barcode, a QR code, and a serial number.

In addition, the motion change calculating unit may generate a clothing motion sequence that defines the movement of the digital idol's clothes by calculating friction between the digital idol's body and the digital idol's clothes through the biometric data.

In addition, the metaverse implementation server further comprises: a motion data receiving unit for generating normal motion data by normalizing the motion data; and the shading change calculating unit may receive the motion normal data and the bionormal data, respectively.

An artificial intelligence-based platform providing method for implementing a metaverse digital idol according to some embodiments of the present invention for solving the above other problems, receives motion data and biometric data from a user, and normalizes the motion data to obtain normal motion data generating, normalizing the biometric data to generate bionormal data, generating a character motion sequence using digital idol information, the motion normal data and the bionormal data, and generating the digital idol information, the motion normal data and the Generating character shading data through normal biometric data, and performing character graphic processing through the character motion sequence and the character shading data to create metaverse data that implements a digital idol of the metaverse synchronized with the user in the real world include that

In addition, digital idol content is generated using the metaverse data, the digital idol content is uploaded to SNS to calculate a rating score, and a reward score is generated according to the rating score to the user's account in the metaverse. It may further include interlocking.

The method may further include generating a clothing motion sequence through the digital idol information, the character motion normal data, the bionormal data, and the character motion sequence.

The artificial intelligence-based platform providing system and method for implementing the metaverse digital idol of the present invention synchronize the digital idol of the metaverse with a user in the real world so that the user can experience the metaverse from the position of a digital idol.

In addition, in order to maximize the reality of the experience, the user's motion data and biometric data can be applied to the digital idol of the metaverse to perform precise synchronization.

Furthermore, users can induce participation and competition by uploading digital idol contents created by users to SNS (Social Network Service) and providing rewards in the metaverse based on rating scores based on the preferences of multiple users. .

The specific effects of the present invention in addition to the above will be described together while explaining the specific details for carrying out the invention below.

1 is a conceptual diagram for explaining an artificial intelligence-based platform providing system that implements a metaverse digital idol according to some embodiments of the present invention.
FIG. 2 is a conceptual diagram for explaining the biometric data of FIG. 1 .
3 is a diagram for explaining the synchronization of a digital idol in the metaverse with a user in the real world.
FIG. 4 is a block diagram for explaining the metaverse implementation server of FIG. 1 in detail.
FIG. 5 is a block diagram for explaining in detail the motion change calculator of FIG. 4 .
FIG. 6 is a diagram for explaining character motion information, background information, and character setting information of FIG. 5 .
7 is a conceptual diagram for explaining the parameter calculation flow of FIG. 5 .
8 is an exemplary diagram for explaining a change in a character motion sequence according to a heart rate.
9 is a block diagram for explaining in detail the motion change calculation module of FIG. 5 .
10 is a flowchart for explaining the operation of the artificial intelligence training module of FIG.
11 is a block diagram for explaining in detail the clothing motion change calculation module of FIG. 5 .
12 is a block diagram for explaining in detail the shading change calculating unit of FIG. 4 .
13 is a block diagram for explaining in detail the shading change calculation module of FIG. 12 .
14 is a conceptual diagram for explaining a parameter calculation flow of FIG. 12 .
15 is a conceptual diagram for explaining a process of acquiring purchase information of FIG. 1 .
16 is a flowchart illustrating a method of providing an artificial intelligence-based platform for implementing a metaverse digital idol according to some embodiments of the present invention.
17 is a flowchart for explaining in detail the steps of generating the character motion sequence and the clothing motion sequence of FIG. 16 .
FIG. 18 is a flowchart for explaining in detail an example of a step of deriving the amount of sweat from each part of the body of FIG. 17 .
FIG. 19 is a flowchart for explaining in detail another example of a step of deriving the amount of sweat from each part of the body of FIG. 17 .
FIG. 20 is a flowchart for explaining in detail a step of generating the clothing motion sequence of FIG. 17 .
21 is a flowchart for explaining in detail a step of generating the character shading data of FIG. 16 .
22 is a flowchart for explaining in detail a step of generating metaverse data of FIG. 16 .

Terms or words used in this specification and claims should not be construed as being limited to a general or dictionary meaning. In accordance with the principle that the inventor can define a term or concept of a word in order to best describe his/her invention, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. In addition, since the embodiments described in this specification and the configurations shown in the drawings are only one embodiment in which the present invention is realized, and do not represent all the technical spirit of the present invention, they can be substituted at the time of the present application. It should be understood that there may be various equivalents and modifications and applicable examples.

Terms such as first, second, A, B, etc. used in this specification and claims may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term 'and/or' includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Terms used in this specification and claims are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. It should be understood that terms such as “comprise” or “have” in the present application do not preclude the possibility of addition or existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification in advance. .

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not technically contradict each other.

Hereinafter, an artificial intelligence-based platform providing system for implementing a metaverse digital idol according to some embodiments of the present invention will be described with reference to FIGS. 1 to 15 .

1 is a conceptual diagram for explaining an artificial intelligence-based platform providing system that implements a metaverse digital idol according to some embodiments of the present invention.

Referring to FIG. 1 , an artificial intelligence-based platform providing system implementing a metaverse digital idol according to some embodiments of the present invention provides motion data (D_mot), biometric data (D_bio) and purchase information (Ipur) from a user 100 . ) can be provided. A system for providing an artificial intelligence-based platform for implementing a metaverse digital idol according to some embodiments of the present invention includes a metaverse implementation server 200 , an electronic device 300 , an SNS server 400 , and a metaverse management server 500 . includes

The user 100 may be a person who uses an artificial intelligence-based platform providing system that implements the metaverse digital idol of the present invention. The user 100 may transmit his/her motion data (D_mot) and biometric data (D_bio) to the metaverse implementation server 200 through various sensors. In addition, the user 100 may purchase related toys or contents and transmit purchase information Ipur to the metaverse management server 500 . This will be described in more detail later.

The motion data D_mot may be information about the movement of the user 100 . The motion data D_mot may be information on a posture or motion taken by the user 100 . The motion data D_mot may be, for example, information about when the user 100 dances.

FIG. 2 is a conceptual diagram for explaining the biometric data of FIG. 1 .

1 and 2 , the biometric data D_bio may be data collected from the body of the user 100 . For example, the biometric data D_bio may include at least one of heart rate information Ihr_u, pace image information If, blood flow information Ibf, respiration amount information Ir, and sweat discharge information Isw.

In this case, the heart rate information Ihr_u may be collected by a sensor worn by the user 100 . The heart rate information Ihr_u may be used as basic data for calculating other biometric data D_bio, that is, blood flow information Ibf, respiration amount information Ir, and sweat discharge information Isw. However, the present embodiment is not limited thereto.

That is, the blood flow information Ibf is collected by the sensor worn by the user 100 , and the heart rate information Ihr_u may also be calculated through the blood flow information Ibf. The biometric data D_bio may be measured and calculated in various ways.

The face image information If may be collected by an image sensor such as a camera. Unlike the large motion of the motion data D_mot, the face image information If may be data for collecting facial motions or expressions. For example, the face image information If may include time series information of eye blinks or mouth shapes.

3 is a diagram for explaining the synchronization of a digital idol in the metaverse with a user in the real world.

1 and 3, the metaverse implementation server 200 receives the motion data (D_mot) and biometric data (D_bio) of the user 100, the user 100 in the real world (Rw) and the metaverse Metaverse data D_mv can be generated by synchronizing the digital idol 100a of (MV). The metaverse data D_mv is data that implements an online world corresponding to the real world. In the metaverse MV, the user 100 can experience the surroundings as a digital idol 100a.

The metaverse implementation server 200 may transmit the generated metaverse data D_mv to the electronic device 300 so that the user 100 can experience it. The electronic device 300 may include at least one of a display, virtual reality implementation equipment, and augmented reality implementation equipment. The user 100 can sense the metaverse MV and the digital idol 100a in the metaverse MV through the electronic device 300 .

The metaverse implementation server 200 may generate digital idol content (Cmv) through metaverse data (D_mv). The digital idol content Cmv may be an image or video in which the movement of the digital idol 100a in which the movement and facial expression of the user 100 are reflected. That is, the digital idol content (Cmv) may be an image or video material generated by the user 100 as the digital idol 100a, such as a dance or a pose, into content. Accordingly, digital idol contents Cmv may be generated for each user 100 . That is, when a plurality of user accounts exist in the metaverse MV, digital idol contents Cmv may be generated for each user account. The metaverse implementation server 200 may transmit digital idol content (Cmv) to the SNS server 400 .

The SNS server 400 may be a server operating an SNS. The SNS server 400 may receive the digital idol content (Cmv) and upload it on the SNS. The SNS server 400 may collect a rating score (S_Rat) for the digital idol content (Cmv). The rating score (S_Rat) may be calculated using at least one of the number of views of the digital idol content (Cmv), the preference score, and the number of subscriptions to the account of the user 100 on the SNS. The rating score (S_Rat) can be defined as a preference score for digital idol content (Cmv) evaluated by several other users. The SNS server 400 may transmit the rating score (S_Rat) to the metaverse management server 500 .

The metaverse management server 500 may receive a rating score (S_Rat) from the SNS server 400 . The metaverse management server 500 may receive purchase information Ipur from the user 100 . The purchase information Ipur may be information about when the user 100 purchases a product related to the digital idol 100a.

The metaverse management server 500 may generate the reward information Irw using the rating score S_Rat and/or the purchase information Ipur. The reward information Irw may be information about a reward in the metaverse MV for the user 100 . That is, the higher the rating score (S_Rat) of the digital idol content (Cmv), the greater the reward in the metaverse (MV). Also, when the user 100 purchases a product for the digital idol 100a, a reward in the metaverse MV may be given. Through this, the metaverse management server 500 may induce participation and competition of the user 100 and may promote product sales.

In this case, the metaverse implementation server 200 , the SNS server 400 , and the metaverse management server 500 are a workstation, a data center, an internet data center (IDC), and a direct attached storage (DAS). ) system, a storage area network (SAN) system, a network attached storage (NAS) system, and a RAID (redundant array of inexpensive disks, or redundant array of independent disks) system may be implemented as at least one, but the present embodiment is limited thereto no.

The user 100 , the electronic device 300 , the metaverse implementation server 200 , the SNS server 400 , and the metaverse management server 500 may transmit data through a network. The network may include a network based on a wired Internet technology, a wireless Internet technology, and a short-range communication technology. Wired Internet technology may include, for example, at least one of a local area network (LAN) and a wide area network (WAN).

Wireless Internet technologies are, for example, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet) Access), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS) and 5G New Radio (NR) technology. However, the present embodiment is not limited thereto.

Short-range communication technologies include, for example, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, and Near Field Communication: At least one of NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and 5G NR (New Radio) may include However, the present embodiment is not limited thereto.

The user 100 , the electronic device 300 , the metaverse implementation server 200 , the SNS server 400 , and the metaverse management server 500 communicating through the network complies with the technical standards and standard communication methods for mobile communication. can do. For example, standard communication methods include Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), and Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO). , at least one of Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTEA), and 5G New Radio (NR) may include However, the present embodiment is not limited thereto.

FIG. 4 is a block diagram for explaining the metaverse implementation server of FIG. 1 in detail.

3 and 4 , the metaverse implementation server 200 includes a motion data receiver 210 , a biometric data receiver 220 , a digital idol database 230 , a motion change calculator 240 , and a shading change calculator 250 . ), an account information management unit 260 and a graphic processing unit 270 .

The motion data receiver 210 may receive the motion data D_mot and normalize it. In this case, normalization may refer to an operation of matching data collected from various sensor devices and transmitted in various formats into one consistent format. The motion data receiver 210 may generate the motion normal data Dn_mot by normalizing the motion data D_mot.

The biometric data receiver 220 may receive the biometric data D_bio and normalize it. The biometric data receiver 220 may normalize the biometric data D_bio to generate the normal biometric data Dn_bio.

The digital idol database 230 may store information about the digital idol 100a, that is, digital idol information Did. The digital idol information (Did) may include all detailed information about the digital idol 100a, such as the appearance, clothes, movement, voice, and pose of the digital idol 100a.

The motion change operation unit 240 may receive the motion normal data (Dn_mot), the normal biometric data (Dn_bio), and the digital idol information (Did), and generate a character motion sequence (Scm) and a clothing motion sequence (Sclm). . The character motion sequence (Scm) may be data on the movement of the digital idol 100a, and the clothing motion sequence (Sclm) may be data on the movement of the clothes of the digital idol 100a.

The shading change calculating unit 250 may receive the motion normal data Dn_mot, the normal biometric data Dn_bio, and the digital idol information Did, and generate character shading data Dcs. The character shading data Dcs may be data for shading of the digital idol 100a.

The account information management unit 260 may receive the reward information Irw and reflect it in the user's 100 account. The account information management unit 260 may update the goods information Dc of the user 100's account according to the compensation information Irw.

The graphic processing unit 270 may receive the character motion sequence Scm, the clothing motion sequence Sclm, the character shading data Dcs, and the goods information Dc to generate metaverse data D_mv. The graphic processing unit 270 includes the movement of the digital idol 100a, the movement of the clothes of the digital idol 100a, the appearance of the digital idol 100a, and goods that the user 100 can use in the metaverse (MV). Metaverse data (D_mv) can be generated through graphic processing for .

FIG. 5 is a block diagram for explaining in detail the motion change calculator of FIG. 4 .

Referring to FIG. 5 , the motion change operation unit 240 may include a first input information operation module 241 , a motion change operation module 242 , and a clothing motion change operation module 243 .

The first input information calculation module 241 may receive motion normal data Dn_mot, bionormal data Dn_bio, and digital idol information Did. The first input information calculation module 241 provides character motion information (MI), background information (BI) and character setting information ( CI) can be created.

FIG. 6 is a diagram for explaining character motion information, background information, and character setting information of FIG. 5 .

1, 5 and 6, the character motion information (MI) may include information on the heart rate (Ihr), posture (I1), fatigue (I2), and emotion (I3) of the digital idol 100a. can Each of the information Ihr, I1, I2, and I3 may be composed of time-series data. That is, the character motion information MI may include changes in heart rate, posture, fatigue, and emotions of the digital idol 100a over time.

Such character motion information MI may be generated through the user 100's motion normal data Dn_mot, bioregular data Dn_bio, and digital idol information Did. For example, the normal biometric data Dn_bio of the user 100 may be used as the character motion information MI of the digital idol 100a through synchronization. That is, the heart rate information Ihr_u of the user 100 may be used as the heart rate Ihr of the digital idol 100a as it is. However, the present embodiment is not limited thereto, and the heart rate information Ihr_u of the user 100 may be used as the heart rate Ihr through a certain conversion according to the digital idol information Did.

The background information BI may include a temperature B1, a humidity B2, and a time zone B3 of the background. The type of character motion may be classified based on at least one parameter of heart rate, posture, fatigue, and emotion included in the character motion information MI. For example, the state of the character may be divided into an excited state, a normal state, and a resting state based on the character's heart rate. In addition, the posture of the character may be divided into a standing posture, a sitting posture, an attack posture, a defense posture, and the like. Fatigue may be classified into vitality, normality, and fatigue state using a predetermined reference value, and emotions may be classified into joy, sadness, excitement, depression, emotion, and the like. However, the present embodiment is not limited thereto.

The character setting information CI may be information included in the digital idol information Did. Specifically, the character setting information (CI) includes the character's gender (C1), age (C2), body type (C3), size (C4), skin color (C5), and clothing (C6) (for example, clothes you are wearing). However, it goes without saying that each item is arbitrary, and each item may be modified and used differently.

7 is a conceptual diagram for explaining the parameter calculation flow of FIG. 5 .

5 and 7 , the motion change calculation module 242 may receive the heart rate Ihr of the character included in the character motion information MI. Then, the motion change calculation module 242 derives the lung volume change Vl according to the heart rate Ihr. Here, the lung volume change Vl means a time-series change in the lung volume of the character.

Next, the motion change calculation module 242 derives the posture change amount Vp according to the heart rate. Here, the posture change amount Vp means a time-series change amount with respect to the character's pose. The posture change amount Vp may be divided into upper body motion change amount, shoulder motion change amount, arm motion change amount, etc. for each body part of the character. In addition, the posture change amount Vp may be expressed as changes of a plurality of points expressing the movement of the character. However, the present invention is not limited thereto.

Next, the motion change calculation module 242 derives the shape change Vs of each body part according to the heart rate. Here, the shape change amount Vs means a time-series change amount with respect to the body shape of the character. The shape change Vs may be divided into a chest shape change amount and an abdominal shape change amount for each body part of the character.

For example, when the heart rate is relatively high, since the character mainly performs chest breathing, the chest shape change amount may be set to be larger than the abdominal shape change amount. On the other hand, when the heart rate is relatively low, since the character mainly breathes abdominally, the chest shape change amount may be set smaller than the abdominal shape change amount. That is, the character's chest breathing and abdominal breathing can be divided and expressed based on the shape change amount (Vs). However, the present invention is not limited thereto.

Next, the motion change calculation module 242 derives the sweat amount Vw of each body part of the character according to the heart rate. The sweat amount Vw may be used to generate a clothing motion sequence Sclm that expresses the movement of the character's clothing in a later step.

Specifically, in the case of a portion having a relatively large amount of derived sweat discharge Vw, the movement of the clothing may be expressed such that the dependence of the character on the corresponding body portion is set to be high. For example, a portion with a large amount of sweat Vw may be expressed as moving clothing by being in close contact with a body portion of the character.

On the other hand, in the case of a portion having a relatively small amount of sweat output (Vw), the movement of the clothing may be expressed such that the dependence on the movement of the corresponding body portion of the character is set to be low. For example, the movement of clothing may be expressed in a region with a small amount of sweat Vw so that the dependence on gravity or inertia is high separately from the movement of the body part of the character. However, the present invention is not limited thereto.

The dependence between the character's body and the clothing can be converted to a numerical value through the clothing friction force Ff. The clothing friction force Ff may be reflected in calculating the clothing motion variation Vclm. A detailed description thereof will be provided below.

Then, the motion change calculation module 242 corrects the values (Vl, Vp, Vs, Vw) derived based on the heart rate based on the character motion information (MI), the background information (BI), and the character setting information (CI) do. For example, each change amount (Vl, Vp, Vs) of the character may be changed according to the posture (I1), fatigue (I2), and emotion (I3) of the character. Each change amount (Vl, Vp, Vs) of the character may be changed according to the gender (C1), age (C2), body type (C3), and size (C4) of the character. In addition, each change amount (Vl, Vp, Vs) of the character may be changed according to the temperature (B1), humidity (B2), and time period (B3) of the background space.

In this case, the ratio at which the character motion information MI, the background information BI, and the character setting information CI are reflected in the respective variation amounts Vl, Vp, and Vs may be determined by a predetermined weight. For example, the character motion information MI and the character setting information CI may be reflected in the respective variation amounts Vl, Vp, and Vs with a higher weight than the background information BI.

Then, the motion change calculation module 242 generates a character motion sequence Scm by using each of the corrected variation amounts Vl, Vp, and Vs. The generated character motion sequence (Scm) may be used for layout work and animation work.

8 is an exemplary diagram for explaining a change in a character motion sequence according to a heart rate.

Referring to FIG. 8 , the lung volume change Vl according to the heart rate may be represented as shown in the graph SC of FIG. 8 . In this case, the lung volume change Vl may be corrected based on factors such as the type and gender of the character and expressed as a time-series graph SC.

In addition, the lung volume change (Vl) may be reflected in the posture change amount (Vp) and shape change amount (Vs), and based on each change amount (Vl, Vp, Vs), the character's chest motion and abdominal movement (Stomach Motion) can be expressed in detail. In addition, it goes without saying that an overall motion that combines them can be expressed.

9 is a block diagram for explaining in detail the motion change calculation module of FIG. 5 .

Referring to FIG. 9 , the motion change calculation module 242 may include an artificial intelligence module (AIM1), an artificial intelligence training module (ATM), and a motion compensation module (MAM).

The artificial intelligence module (AIM1) receives the heart rate (Ihr). The artificial intelligence module (AIM1) receives heart rate (Ihr) as an input, and outputs the character's lung volume change (Vl), posture change (Vp), shape change of each body part according to heart rate (Vs), and sweat output. (Vw) is provided. In this case, the artificial intelligence module AIM1 may be a deep learning module. The artificial intelligence module AIM1 may have already been trained using training data. In this case, the artificial intelligence module AIM1 may use supervised learning. However, the present embodiment is not limited thereto, and the artificial intelligence module AIM1 may use unsupervised learning.

Artificial intelligence module (AIM1) is, for example, DFN (Deep Feedforward Network), CNN (Convolutional Neural Network), GNN (Graph Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), SVM (Support vector) machine), Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), Deep Residual Network (DRN), Generative Adversarial Network (GAN), Graph Convolutional Network (GCN), and Spiking (SNN) Neural Network) may be included. However, the present embodiment is not limited thereto.

As another example, the artificial intelligence module AIM1 may receive the character motion information MI, the background information BI, and the character setting information CI along with the heart rate Ihr. The artificial intelligence module (AIM1) receives heart rate (Ihr), character motion information (MI), background information (BI), and character setting information (CI), and outputs the character's lung volume change (Vl), posture change amount (Vp), shape change amount (Vs) of each part of the body, and sweat output (Vw) can be output.

At this time, the artificial intelligence module (AIM1) includes an input layer using heart rate (Ihr), character motion information (MI), background information (BI), and character setting information (CI) as input nodes, lung volume change (Vl), posture an output layer having an amount of change (Vp), a shape change (Vs), and a sweat amount (Vw) as output nodes, and at least one hidden layer disposed between the input layer and the output layer, wherein the input node and the output The weights of layers between nodes may be updated by a learning process.

Motion compensation module (MAM) based on the character motion information (MI), background information (BI) and character setting information (CI), each change amount (Vl, Vp, Vs) output from the artificial intelligence module (AIM1) Correct. The motion compensation module MAM outputs a character motion sequence Scm including each of the corrected values Vl, Vp, Vs, and Vw.

10 is a flowchart for explaining the operation of the artificial intelligence training module of FIG.

9 and 10 , the artificial intelligence training module (ATM) generates training data for training the artificial intelligence module (AIM1), and provides the generated training data to the artificial intelligence module (AIM1).

Specifically, referring to FIG. 10 , the artificial intelligence training module (ATM) receives a plurality of 3D meshes scanned by a breathing object (eg, a performer with a plurality of sensors) ( S1100 ).

Next, a plurality of 3D meshes are grouped for each parameter included in the character setting information CI ( S1200 ). For example, a plurality of 3D meshes having similar gender (C1), age (C2), body type (C3), and size (C4) are grouped.

Next, the grouped 3D mesh is aligned using the pre-stored template mesh (S1300). This corresponds to a pre-processing process to obtain an average value of a plurality of different 3D meshes.

Then, for each grouped 3D mesh, average values of lung volume change (Vl), posture change (Vp), shape change (Vs), and sweat output (Vw) according to heart rate (Ihr) are derived (S1400). In addition, the average value of each parameter of the character motion sequence (Scm) of the grouped 3D mesh is also derived.

Next, the artificial intelligence module AIM1 is trained using the average values of the derived values Vl, Vp, Vs, and Vw (S1500). At this time, each average value of the derived values (Vl, Vp, Vs, Vw) and heart rate (Ihr) is applied to the input node of the artificial intelligence module (AIM1), and the corresponding character motion sequence (Scm) is artificial intelligence It is applied to the output node of module AIM1. That is, the training data may include an average value of each of the derived values (Vl, Vp, Vs, Vw) and a heart rate (Ihr), and an average value of a character motion sequence (Scm) corresponding thereto. The artificial intelligence training module (ATM) may train the artificial intelligence module (AIM1) using the generated training data.

For example, training data includes, for a group of a plurality of 3D meshes having the same gender (C1) and age (C2), heart rate (Ihr), lung volume change (Vl), posture change amount (Vp), shape change amount (Vs) , may include an average value of the sweat discharge (Vw). In addition, the training data may include an average value of the aforementioned data (Vl, Vp, Vs, Vw, Ihr) for a group of a plurality of 3D meshes having a gender (C1) and a body shape (C3) similar to the reference value. That is, the training data may be generated based on different combinations of parameters included in the character setting information CI.

In addition, the present invention is not limited thereto, and training data may be generated based on different combinations of parameters included in character motion information (MI) or background information (BI) to train the artificial intelligence module (AIM1). is of course

Again, referring to FIG. 9 , the artificial intelligence module AIM1 estimates the respiratory rate change pattern according to the heart rate Ihr of the digital idol 100a. Here, the respiratory volume change pattern represents a change in the time-series respiratory volume generated based on the heart rate. The respiratory volume change pattern may be generated based on the relationship between the previously measured amount of respiratory volume change and the heart rate. For example, the respiratory volume change pattern may be shown as in the graph SC disclosed in FIG. Also, the respiratory volume change pattern may be generated based on the lung volume change amount generated based on the heart rate.

Next, the artificial intelligence module (AIM1) derives the amount of change of posture (Vp) for each part of the character's body based on the pattern of change of respiration rate and the character setting information (CI). In this case, the character setting information CI may include the character's gender (C1), age (C2), body type (C3), and size (C4). The artificial intelligence module (AIM1) may derive the posture change amount (Vp) through this.

The artificial intelligence module AIM1 may operate by first deriving a reference value of the posture change amount Vp based on the heart rate Ihr and correcting detailed numerical values using the character setting information CI. For example, the first correction value of the first character for a fat male with a height of 180 cm at the age of 30 may be greater than the second correction value of the second character for a skinny woman with a height of 160 cm at the age of 60. have.

In addition, the artificial intelligence module (AIM1) derives the shape change (Vs) for each part of the character's body based on the respiratory rate change pattern and the character setting information (CI). The animation module 220 may derive the shape change amount Vs using the previously learned artificial intelligence module AIM1. The shape change Vs may be divided into a chest shape change amount and an abdominal shape change amount for each body part of the character. For example, when the heart rate is relatively high, since the character mainly performs chest breathing, the chest shape change amount may be set to be larger than the abdominal shape change amount. On the other hand, when the heart rate is relatively low, since the character mainly breathes abdominally, the chest shape change amount may be set smaller than the abdominal shape change amount.

In addition, the artificial intelligence module (AIM1) derives the sweat amount (Vw) for each part of the character's body based on the respiratory rate change pattern and character setting information (CI). In this case, it goes without saying that the calculation order of the posture change amount Vp, the shape change amount Vs, and the sweat amount Vw may be changed and performed.

Then, the motion compensation module MAM corrects the derived posture change amount Vp, shape change amount Vs, and sweat amount Vw based on the background information BI. The background information BI may include a temperature B1, a humidity B2, and a time zone B3 of the background BG. For example, in the case of the first character in the daytime with a relatively high temperature, the correction amount of posture change (Vp), shape change (Vs), and sweat output (Vw) compared to the second character in the night time period when the temperature is relatively low This can be large. However, this is only an example and the present invention is not limited thereto.

Hereinafter, another embodiment of the method will be described.

The artificial intelligence module (AIM1) estimates the respiratory volume change pattern according to the heart rate (Ihr) of the character to be laid out.

Next, the artificial intelligence module (AIM1) derives the amount of change of posture (Vp) for each part of the character's body based on the respiratory rate change pattern and the character motion information (MI). In this case, the character motion information CI may include information about the character's posture I1 , fatigue I2 , and emotion I3 . The artificial intelligence module (AIM1) may operate by first deriving a reference value of the posture change amount (Vp) based on the respiratory rate change pattern, and then correcting detailed numerical values using the character motion information (MI). For example, the first correction value of the first character in the happy emotional state full of vitality in the standing posture may be greater than the second correction value of the second character in the tired and depressed state in the sitting position.

In addition, the artificial intelligence module (AIM1) derives the shape change (Vs) for each part of the character's body based on the respiratory rate change pattern and the character motion information (MI). In addition, the artificial intelligence module (AIM1) derives the sweat amount (Vw) for each part of the character's body based on the respiratory rate change pattern and the character motion information (MI). In this case, it goes without saying that the calculation order of the posture change amount Vp, the shape change amount Vs, and the sweat amount Vw may be changed and performed.

Then, the motion compensation module MAM corrects the derived posture change amount Vp, shape change amount Vs, and sweat amount Vw based on the background information BI. In summary, the artificial intelligence module (AIM1) derives the amount of change in posture (Vp), change in shape (Vs), and amount of sweat (Vw) for each part of the character's body based on the heart rate (Ihr). Then, the motion compensation module (MAM) uses some of the plurality of parameters included in the character setting information (CI), the character motion information (MI), or the background information (BI), the derived posture change amount (Vp), the shape A change amount (Vs) and a sweat amount (Vw) may be corrected.

11 is a block diagram for explaining in detail the clothing motion change calculation module of FIG. 5 .

5 and 11 , the clothing motion change calculation module 243 includes a clothing motion calculation module (CMM) and a clothing motion compensation module (CMAM).

The clothing motion calculation module (CMM) generates and outputs the pre-clothing motion sequence (Sclmp) based on the character motion sequence (Scm) indicating the movement of the character. Here, the pre-clothing motion sequence (Sclmp) means data representing time-series movement of clothing worn by the character. In this case, the clothing motion calculation module CMM may generate the pre-clothing motion sequence Sclmp with reference to the character setting information CI and the character motion sequence Scm.

For example, the clothing motion calculation module (CMM) performs a character motion sequence ( Scm) may be used to generate a pre-clothing motion sequence (Sclmp) indicating the movement of clothing.

The clothing motion compensation module (CMAM) performs a function of correcting the pre-clothing motion sequence (Sclmp) output from the clothing motion calculation module (CMM) in order to add realistic movement of the character. The clothing motion compensation module (CMAM) can output the clothing motion sequence (Sclmp) by correcting the pre-clothing motion sequence (Sclmp) based on the sweat output (Vw) output from the aforementioned artificial intelligence module (AIM1). have.

Specifically, the clothing motion compensation module CMAM calculates the frictional force (ie, the clothing frictional force Ff) between the character's body and the clothing C6 based on the character's sweat discharge Vw.

Next, the clothing motion compensation module CMAM outputs a clothing motion sequence Sclm obtained by correcting the pre-clothing motion sequence Sclmp based on the calculated clothing friction force Ff. The clothing motion sequence Sclm includes a clothing motion variation Vclm of the character.

Specifically, in the case of a part of the character's body that has a relatively large amount of sweat (Vw), the clothing motion variation (Vclm) has a high dependence on the character's posture variation (Vp) and shape variation (Vs) for the corresponding body part. It can be expressed to be set.

On the other hand, in the case of a part of the character's body that has a relatively small amount of sweat (Vw), the clothing motion variation (Vclm) has a low dependence on the character's posture variation (Vp) and shape variation (Vs) for the corresponding body part. can be expressed as possible. For example, the clothing motion variation Vclm may be set so that a portion with a small amount of sweat Vw has a high dependence on gravity or inertia separately from the movement of a body part of the character. However, this is only an example, and the present invention is not limited thereto.

The clothing motion compensation module (CMAM) may convert the dependence between the movement of the character and the clothing into a numerical value through the clothing friction force (Ff). That is, the clothing motion compensation module (CMAM) calculates the clothing motion change amount (Vclm) based on the clothing friction force (Ff) and the character's posture change amount (Vp) and shape change amount (Vs) by calculating the clothing motion sequence (Sclm) ) can be created.

Through this, the artificial intelligence-based platform providing system for implementing the metaverse digital idol according to some embodiments of the present invention can maximize reality by expressing the change in detailed movement related to the breathing of the character based on the heart rate. In particular, by controlling not only the movement of the body of the digital idol 100a but also the movement of the clothing (clothes), the user 100's experience can be felt more realistically.

12 is a block diagram for explaining in detail the shading change calculating unit of FIG. 4 .

12 , the shading change calculation module 252 may include a second input information calculation module 251 and a shading change calculation module 252 .

The second input information calculation module 251 may receive motion normal data Dn_mot, bionormal data Dn_bio, and digital idol information Did. The second input information calculation module 251 provides a heart rate (Ihr), background information (BI) and character setting information (CI) based on motion normal data (Dn_mot), bionormal data (Dn_bio), and digital idol information (Did). can create

13 is a block diagram for explaining in detail the shading change calculation module of FIG. 12 .

Referring to FIG. 13 , the shading change calculation module 252 may include an artificial intelligence module AIM2 and a shader module SM.

The artificial intelligence module (AIM2) receives the heart rate (Ihr). The artificial intelligence module (AIM2) receives the heart rate (Ihr) and provides the sweat output (Vw) and blood flow (Vbf) for each body part of the character as outputs. The artificial intelligence module (AIM2) may be a deep learning module. The artificial intelligence module AIM2 may have already been trained using training data. In this case, the artificial intelligence module (AIM2) may use supervised learning. However, the present embodiment is not limited thereto, and the artificial intelligence module AIM2 may use unsupervised learning.

In this case, the blood flow Vbf may be calculated based on the heart rate or may be preset based on previously measured blood flow information Ibf of the user 100 .

Artificial intelligence module (AIM2) is, for example, DFN (Deep Feedforward Network), CNN (Convolutional Neural Network), GNN (Graph Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), SVM (Support vector) machine), Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), Deep Residual Network (DRN), Generative Adversarial Network (GAN), Graph Convolutional Network (GCN), and Spiking (SNN) Neural Network) may be included. However, the present embodiment is not limited thereto.

As another example, the artificial intelligence module AIM2 may receive the character motion information MI, the background information BI, and the character setting information CI along with the heart rate Ihr. The artificial intelligence module (AIM2) receives heart rate (Ihr), character motion information (MI), background information (BI), and character setting information (CI), and outputs sweat output (Vw) from each part of the character's body. and a blood flow Vbf.

At this time, the artificial intelligence module (AIM2) has an input layer using heart rate (Ihr), character motion information (MI), background information (BI), and character setting information (CI) as input nodes, and sweat output (Vw) and blood flow ( Vbf) as an output node, and at least one hidden layer disposed between the input layer and the output layer, wherein the weight of the layer between the input node and the output node may be updated by a learning process. have.

The shader module (SM) uses the background information (BI) and character setting information (CI) based on the sweat output (Vw) and blood flow (Vbf) output from the artificial intelligence module (AIM2). Generates character shading data (Dcs) for performing shading.

Here, shading refers to the operation of changing the illuminance of the object surface according to the distance and angle between the object and the light source. In the shading stage, the surface of each polygon constituting the character is shaded according to the position, brightness, and color of the light source, thereby expressing three-dimensionality and realism of the object.

14 is a conceptual diagram for explaining a parameter calculation flow of FIG. 12 .

13 and 14 , the shader module SM may perform skin shading, Ÿ‡ shading, and glitter shading.

Here, the skin shading means shading to produce an effect close to human skin. In particular, human skin has a unique and complex structure in terms of color and texture, as well as reflection and transmission. For the translucent effect of the skin, the SSS (Sub-Surface Scattering) technique can be used, and the SSS technique can express a phenomenon that occurs when light hits the surface of a translucent object such as skin or paper and is scattered without being transmitted. In addition, as the blood flow (Vbf) is higher, the red tone is reflected through the skin, and thus the overall skin tone may be changed.

Ÿ‡ Shading refers to shading to emphasize the realism of the skin by expressing sweat, moisture, etc. formed on the skin of the person and expressing the moisture of the skin. Glitter shading refers to shading to create a reflective or shiny effect due to the moisture of the skin.

The shader module SM may perform at least one of skin shading, Ÿ‡ shading, and glitter shading based on sweat discharge (Vw) and blood flow (Vbf). Skin shading, Ÿ‡ shading, and glitter shading can be partially integrated or divided into separate steps and performed by the shader module (SM). Also, it goes without saying that the shader module SM may perform only a part of the above-described shading or additional shading.

The artificial intelligence module (AIM2) calculates sweat discharge (Vw) and blood flow (Vbf) according to the character's heart rate (Ihr). Next, the artificial intelligence module AIM2 calculates light source information including the position of the light source and the amount of light based on the information about the time zone B3 included in the background information BI. Additionally, the artificial intelligence module (AIM2) refines the light source information through a predetermined calculation formula based on the area information (eg, latitude, longitude, sunshine, weather, cloudiness, etc.) included in the background information (BI). can be derived In addition, the light source information may be divided into predetermined artificial light and natural light and included.

Next, the shader module SM generates character shading data DCs based on the calculated light source information, character setting information CI, sweat discharge Vw, and blood flow Vbf. The character shading data Dcs means data for performing at least one of the aforementioned skin shading, Ÿ‡ shading, and glitter shading.

That is, the present invention uses data such as sweat output (Vw) and blood flow (Vbf) calculated based on the character's heart rate (Ihr) to detect changes in sweat, glitter, or red tone of blood flow expressed on the character's skin. It can be expressed in detail in the rendering stage. Rather than manually defining and setting the detailed changes of these characters, the creator can express the detailed changes in the character's appearance just by adjusting the setting values of the character or background, maximizing the character's reality, digital It is possible to increase the convenience of creating idol contents and the speed of work.

In addition, the present invention derives the amount of change in posture, change in shape, and amount of sweat for each part of the character's body based on biometric information, and uses some of a plurality of parameters included in character setting information, character motion information, or background information. By correcting the derived data, it is possible to maximize the reality of the movement of the character and the movement of the character's clothing.

15 is a conceptual diagram for explaining a process of acquiring purchase information of FIG. 1 .

1, 3, and 15 , the user 100 may purchase a product related to the digital idol 100a of the metaverse MV. The product may include, for example, a toy of the digital idol 100a of the metaverse MV, or content such as music or video.

The user 100 may purchase the purchased toy Toy_m as a real thing. In this case, the user 100 may obtain the purchase authentication information Iv from the purchased toy Toy_m. The purchase authentication information Iv may be obtained through an authentication code attached to the purchased toy Toy_m. The authentication code may include, for example, at least one of a barcode, a QR code, and a serial number. However, the present embodiment is not limited thereto.

Similarly, the user 100 may purchase the purchased content Con_m. In this case, the user 100 may obtain the purchase authentication information Iv from the purchase content Con_m.

The user 100 may transmit the purchase information Ipur generated by integrating all the purchase authentication information Iv to the metaverse management server 500 . Accordingly, the metaverse management server 500 may generate the corresponding compensation information Irw. That is, the metaverse management server 500 may provide a reward related to goods in the metaverse MV according to the user 100's purchase of a real or content product. Accordingly, the user 100 may have a motivation for purchasing the related product, and thus the sale of the product may be promoted.

In this embodiment, the user 100 has a higher sense of unity with the digital idol 100a of the metaverse MV, and the user 100 has a vivid experience to increase the time and frequency of service use. can

Furthermore, since the motion data D_mot and the biometric data D_bio of the user 100 are reflected and expressed in the digital idol 100a as they are, the reality felt by the user 100 can be maximized.

Hereinafter, an artificial intelligence-based platform providing method for implementing a metaverse digital idol according to some embodiments of the present invention will be described with reference to FIGS. 1 to 22 . Parts overlapping with the above-described embodiment will be simplified or omitted.

16 is a flowchart for explaining a method for providing an artificial intelligence-based platform for implementing a metaverse digital idol according to some embodiments of the present invention, and FIG. 17 is a step of generating the character motion sequence and clothing motion sequence of FIG. 16 It is a flowchart for explaining in detail. 18 is a flowchart for explaining in detail an example of the step of deriving the sweat amount of each body part of FIG. 17 , and FIG. 19 is another example of the step of deriving the sweat amount of each body part of FIG. 17 in detail It is a flowchart for explanation. FIG. 20 is a flowchart for explaining in detail the step of generating the clothing motion sequence of FIG. 17 , and FIG. 21 is a flowchart for explaining in detail the step of generating the character shading data of FIG. 16 . 22 is a flowchart for explaining in detail a step of generating metaverse data of FIG. 16 .

First, referring to FIG. 16 , motion data and biometric data are received ( S100 ).

Specifically, referring to FIGS. 1 and 2 , the metaverse implementation server 200 may receive motion data D_mot and biometric data D_bio of the user 100 . The motion data D_mot may be information about the movement of the user 100 . The motion data D_mot may be information on a posture or motion taken by the user 100 . The biometric data D_bio may be data collected from the body of the user 100 . For example, the biometric data D_bio may include at least one of heart rate information Ihr_u, pace image information If, blood flow information Ibf, respiration amount information Ir, and sweat discharge information Isw.

Again, referring to FIG. 16 , motion normal data and bionormal data are generated ( S200 ).

Specifically, referring to FIG. 4 , the motion data receiver 210 may receive the motion data D_mot and normalize it. The motion data receiver 210 may generate the motion normal data Dn_mot by normalizing the motion data D_mot. The biometric data receiver 220 may receive the biometric data D_bio and normalize it. The biometric data receiver 220 may normalize the biometric data D_bio to generate the normal biometric data Dn_bio.

Again, referring to FIG. 16 , a character motion sequence and a clothing motion sequence are generated ( S300 ).

Referring to FIG. 17 in detail, the heart rate of the character is received (S310).

Specifically, referring to FIGS. 5 and 7 , the motion change calculation module 242 may receive the heart rate Ihr of the character included in the character motion information MI.

Again, referring to FIG. 17 , a change in lung volume according to the heart rate is derived ( S320 ).

Specifically, referring to FIGS. 5 and 7 , the motion change calculation module 242 derives the lung volume change Vl according to the heart rate Ihr. Here, the lung volume change Vl means a time-series change amount with respect to the size of the character's lungs.

Again, referring to FIG. 17 , an amount of change in posture according to the heart rate is derived ( S330 ).

Specifically, referring to FIGS. 5 and 7 , the motion change calculation module 242 derives a posture change amount Vp according to a heart rate. Here, the posture change amount Vp means a time-series change amount with respect to the posture of the character. The posture change amount Vp may be divided into upper body motion change amount, shoulder motion change amount, arm motion change amount, etc. for each body part of the character. In addition, the posture change amount Vp may be expressed as changes of a plurality of points expressing the movement of the character. However, the present invention is not limited thereto.

Again, referring to FIG. 17 , the amount of change in shape of each part according to the heart rate is derived ( S340 ).

Specifically, referring to FIGS. 5 and 7 , the motion change calculation module 242 derives the shape change Vs of each body part according to the heart rate. Here, the shape change amount Vs means a time-series change amount with respect to the body shape of the character. The shape change Vs may be divided into a chest shape change amount and an abdominal shape change amount for each body part of the character.

Again, referring to FIG. 17 , the sweat amount of each part according to the heart rate is derived ( S350 ).

In detail, referring to FIG. 18 , a pattern of change in respiration volume is estimated according to the heart rate ( S351 ).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 estimates the respiratory rate change pattern according to the heart rate Ihr of the digital idol 100a. Here, the respiratory volume change pattern represents a change in the time-series respiratory volume generated based on the heart rate.

Again, referring to FIG. 18 , the amount of change in posture for each part is derived based on the respiratory rate change pattern and character setting information including age, gender, body type, and size ( S352 ).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 derives the amount of change of posture Vp for each part of the character's body based on the respiratory rate change pattern and the character setting information CI. In this case, the character setting information CI may include the character's gender (C1), age (C2), body type (C3), and size (C4). The artificial intelligence module (AIM1) may derive the posture change amount (Vp) through this.

The artificial intelligence module AIM1 may operate by first deriving a reference value of the posture change amount Vp based on the heart rate Ihr and correcting detailed numerical values using the character setting information CI.

Again, referring to FIG. 18 , the shape change amount for each part is derived based on the respiratory amount change pattern and character setting information ( S353 ).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 derives the shape change Vs for each part of the character's body based on the respiratory rate change pattern and the character setting information CI. The animation module 220 may derive the shape change amount Vs using the previously learned artificial intelligence module AIM1. The shape change Vs may be divided into a chest shape change amount and an abdominal shape change amount for each body part of the character.

Again, referring to FIG. 18 , the sweat amount for each part is derived based on the respiratory rate change pattern and character setting information (S354).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 derives the sweat amount Vw for each part of the character's body based on the respiratory rate change pattern and the character setting information CI.

In this case, the order of steps S352, S353, and S354 above may be changed at any time.

Again, referring to FIG. 18 , the derived attitude change amount, shape change amount, and sweat amount are corrected based on background information including temperature, humidity, and time period ( S355 ).

Specifically, referring to FIG. 9 , the motion compensation module MAM corrects the derived posture change amount Vp, shape change amount Vs, and sweat amount Vw based on the background information BI. The background information BI may include a temperature B1, a humidity B2, and a time zone B3 of the background BG.

Hereinafter, an embodiment different from that of FIG. 18 will be described with reference to FIG. 19 . 19 and S350 of FIG. 17 will be described in detail.

Referring to FIG. 19 , a pattern of change in respiration volume is estimated according to the heart rate (S351a).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 estimates the respiratory rate change pattern according to the heart rate Ihr of the digital idol 100a.

Again, referring to FIG. 19 , the amount of change in posture for each part is derived based on the character motion information including the respiratory rate change pattern and posture, fatigue, and emotion ( S352a ).

Specifically, referring to FIG. 9 , the artificial intelligence module (AIM1) derives the amount of change of posture (Vp) for each part of the character's body based on the respiratory rate change pattern and the character motion information (MI). In this case, the character motion information CI may include information about the character's posture I1 , fatigue I2 , and emotion I3 . The artificial intelligence module (AIM1) may operate by first deriving a reference value of the posture change amount (Vp) based on the respiratory rate change pattern, and then correcting detailed numerical values using the character motion information (MI).

Again, referring to FIG. 19 , the shape change amount for each part is derived based on the respiratory amount change pattern and character motion information (S353a).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 derives the shape change Vs for each part of the character's body based on the respiratory rate change pattern and the character motion information MI.

Again, referring to FIG. 19 , the sweat amount for each part is derived based on the respiratory rate change pattern and character motion information (S354a).

Specifically, referring to FIG. 9 , the artificial intelligence module AIM1 derives the sweat amount Vw for each part of the character's body based on the respiratory rate change pattern and the character motion information MI.

In this case, the order of the steps S352a, S353a, and S354a above may be changed freely.

Again, referring to FIG. 19 , the derived attitude change amount, shape change amount, and sweat amount are corrected based on background information including temperature, humidity, and time period (S355a).

Specifically, referring to FIG. 9 , the motion compensation module MAM corrects the derived posture change amount Vp, shape change amount Vs, and sweat amount Vw based on the background information BI.

Again, referring to FIG. 17 , a value derived based on the heart rate is corrected based on character motion information, background information, or character setting information ( S360 ).

Specifically, referring to FIG. 9 , the motion compensation module (MAM) performs each of the output from the artificial intelligence module (AIM1) based on the character motion information (MI), the background information (BI) and the character setting information (CI). The change amount (Vl, Vp, Vs) is corrected. The motion compensation module MAM outputs a character motion sequence Scm including each of the corrected values Vl, Vp, Vs, and Vw. Steps S355 and S355a above may be partially included in this step.

Again, referring to FIG. 17, a character motion sequence is generated based on the corrected values (S370).

Specifically, referring to FIG. 9 , the motion compensation module MAM outputs a character motion sequence Scm including each of the corrected values Vl, Vp, Vs, and Vw.

Again, referring to FIG. 17, a clothing motion sequence is generated based on the character motion sequence (S380).

Referring to FIG. 20 in detail, a pre-clothing motion sequence of the character is calculated based on the character motion sequence (S381).

Specifically, referring to FIG. 11 , the clothing motion calculation module (CMM) generates and outputs the pre-clothing motion sequence Sclmp based on the character motion sequence Scm indicating the movement of the character. Here, the pre-clothing motion sequence (Sclmp) means data representing time-series movement of clothing worn by the character. In this case, the clothing motion calculation module CMM may generate the pre-clothing motion sequence Sclmp with reference to the character setting information CI and the character motion sequence Scm.

Again, referring to FIG. 20 , the frictional force between the character's body and the clothing is calculated based on the character's sweat amount (S382).

Specifically, referring to FIG. 11 , the clothing motion compensation module (CMAM) calculates the frictional force between the character's body and the clothing C6 based on the character's sweat output (Vw) (ie, the clothing frictional force (Ff)). to calculate

Again, referring to FIG. 20, the pre-clothing motion sequence is corrected based on the calculated friction force (S382).

Specifically, referring to FIG. 11 , the clothing motion compensation module CMAM outputs a clothing motion sequence Sclm obtained by correcting the pre-clothing motion sequence Sclmp based on the calculated clothing friction force Ff. . The clothing motion sequence Sclm includes a clothing motion variation Vclm of the character.

Again, referring to FIG. 16 , character shading data is generated ( S400 ).

In detail, referring to FIG. 21 , the amount of sweat and blood flow according to the heart rate of the character is calculated ( S410 ).

Specifically, referring to FIGS. 13 and 14 , the artificial intelligence module AIM2 may receive the character motion information MI, the background information BI, and the character setting information CI along with the heart rate Ihr. . The artificial intelligence module (AIM2) receives heart rate (Ihr), character motion information (MI), background information (BI), and character setting information (CI), and outputs sweat output (Vw) from each part of the character's body. and a blood flow Vbf. In this case, the blood flow Vbf may be calculated based on the heart rate or may be preset based on previously measured blood flow information Ibf of the user 100 .

Again, referring to FIG. 21 , light source information including the amount of light and the light source position is calculated based on the time period included in the background information ( S420 ).

Specifically, referring to FIG. 13 , the artificial intelligence module AIM2 calculates light source information including the position of the light source and the amount of light based on the information about the time zone B3 included in the background information BI. Additionally, the artificial intelligence module (AIM2) refines the light source information through a predetermined calculation formula based on the area information (eg, latitude, longitude, sunshine, weather, cloudiness, etc.) included in the background information (BI). can be derived In addition, the light source information may be divided into predetermined artificial light and natural light and included.

Again, referring to FIG. 21 , character shading data is generated based on the calculated light source information, character setting information, sweat amount and blood flow ( S430 ).

Specifically, referring to FIG. 13 , the shader module SM generates character shading data (Dcs) based on the calculated light source information, character setting information (CI), sweat discharge (Vw), and blood flow (Vbf). do. The character shading data Dcs means data for performing at least one of the aforementioned skin shading, Ÿ‡ shading, and glitter shading.

Again, referring to FIG. 16 , metaverse data is generated through character graphic processing ( S500 ).

In detail, referring to FIG. 22 , character graphics are processed through a character motion sequence, a clothing motion sequence, and character shading data ( S510 ).

Specifically, referring to FIG. 4 , the graphic processing unit 270 may receive a character motion sequence (Scm), a clothing motion sequence (Sclm), and character shading data (Dcs) to generate metaverse data (D_mv). . The graphic processing unit 270 may perform graphic processing on the movement of the digital idol 100a, the movement of the clothes of the digital idol 100a, and the appearance of the digital idol 100a.

Again, referring to FIG. 22 , reward information is generated using the purchase information and the rating score ( S520 ).

Specifically, referring to FIG. 1 , the metaverse management server 500 may receive a rating score (S_Rat) from the SNS server 400 . The metaverse management server 500 may receive purchase information Ipur from the user 100 . The purchase information Ipur may be information about when the user 100 purchases a product related to the digital idol 100a.

The metaverse management server 500 may generate the reward information Irw using the rating score S_Rat and/or the purchase information Ipur. The reward information Irw may be information about a reward in the metaverse MV for the user 100 . That is, the higher the rating score (S_Rat) of the digital idol content (Cmv), the greater the reward in the metaverse (MV). Also, when the user 100 purchases a product for the digital idol 100a, a reward in the metaverse MV may be given. Through this, the metaverse management server 500 may induce participation and competition of the user 100 and may promote product sales.

Again, referring to FIG. 22, the reward information is reflected in the user account to generate the goods information (S530).

Specifically, referring to FIG. 4 , the account information management unit 260 may receive the reward information Irw and reflect it in the user's 100 account. The account information management unit 260 may update the goods information Dc of the user 100's account according to the compensation information Irw.

Again, referring to FIG. 22 , the metaverse data is completed by reflecting the character graphic and the goods information ( S540 ).

Specifically, referring to FIG. 4 , the graphic processing unit 270 performs the movement of the digital idol 100a, the movement of the clothes of the digital idol 100a, the appearance of the digital idol 100a, and the metaverse of the user 100 Metaverse data (D_mv) can be generated through graphic processing for goods that can be used in (MV).

The above description is merely illustrative of the technical idea of this embodiment, and a person skilled in the art to which this embodiment belongs may make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present embodiment.

Claims (10)

It receives motion data and biometric data from a user, and generates metaverse data generated by synchronizing a digital idol of a metaverse with a user in the real world, and the metaverse data is performed using the biometric data and the motion data. metaverse implementation server; and
an electronic device for receiving and displaying the metaverse data from the metaverse implementation server;
The metaverse implementation server,
a digital idol database that stores digital idol information;
a motion change calculator for generating a character motion sequence defining the movement of the digital idol by using the motion data and the digital idol information;
a shading change calculating unit for generating character shading data defining an appearance of the digital idol by using the biometric data and the digital idol information;
a graphic processing unit for generating the metaverse data including graphic data of the digital idol synchronized with the user through the character motion sequence and the character shading data;
The metaverse implementation server,
a motion data receiving unit normalizing the motion data to generate motion normal data;
Further comprising a biometric data receiving unit for normalizing the biometric data to generate the biometric data,
The motion change calculating unit and the shading change calculating unit receive the motion normal data and the bionormal data, respectively,
The motion change calculating unit,
A first input information calculation module for generating character motion information, background information, and character setting information based on the motion regular data, the biometric data, and the digital idol information, wherein the character motion information includes heart rate information of the digital idol a first input information calculation module;
a motion change calculation module for deriving the amount of change in lung volume, the amount of change in posture, the amount of change in shape of each body part, and the amount of sweat output of the digital idol according to the heart rate, and generating a character motion sequence;
A pre-clothing motion sequence is generated using the character motion sequence, a friction force between the body and clothing of the character is calculated based on the sweat amount, and the pre-clothing motion sequence is corrected based on the friction force. A clothing motion change calculation module for outputting a clothing motion sequence,
The lung volume change amount means a time-series change amount with respect to the size of the character's lungs, the posture change amount means a time-series change amount of a plurality of points expressing the movement of the character, and the shape change amount means the character's chest including the amount of change in shape and the amount of change in the shape of the abdomen;
When the heart rate is higher than a predetermined reference value, the chest shape change amount is set to be larger than the abdominal shape change amount,
When the heart rate is lower than a predetermined reference value, the chest shape change amount is set to be smaller than the abdominal shape change amount,
The motion change calculation module,
an artificial intelligence module that receives the heart rate and provides the change amount of the lung volume, the posture change amount, and the shape change amount of each body part as an output;
Motion for outputting the character motion sequence by correcting the amount of change in lung volume, the amount of change in posture, and the amount of change in shape of each body part output from the artificial intelligence module based on the character motion information, the background information, and the character setting information a calibration module;
Group a plurality of pre-scanned 3D meshes by any one of parameters included in the character setting information, align the grouped plurality of 3D meshes using a pre-stored template mesh, and waste volume of the aligned plurality of 3D meshes An artificial intelligence training module that derives the average value of the change amount, the posture change amount, and the shape change amount of each body part, and generates training data for training the artificial intelligence using the derived average value,
The character motion information includes at least one of a posture, fatigue, and emotion of the character, and a heart rate of the character,
The background information includes at least one of temperature, humidity, and time zone of the background,
The character setting information is an artificial intelligence-based platform providing system for implementing a metaverse digital idol including at least one of gender, age, body type, and size of the character.
According to claim 1,
The metaverse implementation server generates digital idol content through the metaverse data,
an SNS server that receives the digital idol content, uploads it to a social networking service (SNS), collects users' preferences, and calculates a rating score;
AI-based for implementing a metaverse digital idol further comprising a metaverse management server that receives the rating score and generates reward information for reflecting the reward corresponding to the rating score on the user's account in the metaverse platform delivery system.
3. The method of claim 2,
The metaverse management server receives purchase information from the user,
The reward information is an artificial intelligence-based platform providing system that implements a metaverse digital idol generated based on the purchase information and the rating score.
4. The method of claim 3,
The purchase information is an artificial intelligence-based platform providing system that implements a metaverse digital idol generated through purchase authentication information obtained from purchased toys and/or purchased contents.
5. The method of claim 4,
The purchase authentication information is an artificial intelligence-based platform providing system that implements a metaverse digital idol generated by recognizing at least one of a barcode, a QR code, and a serial number.
delete delete Receive motion data and biometric data from a user,
Normalizing the motion data to generate motion normal data,
Normalizing the biometric data to generate bionormal data,
A character motion sequence is generated using digital idol information, the motion normal data, and the normal biometric data,
generating character shading data using the digital idol information, the motion normal data, and the normal biometric data;
and generating metaverse data for implementing a digital idol of a metaverse synchronized with the user in the real world by processing character graphics through the character motion sequence and the character shading data,
Generating the character motion sequence comprises:
generate character motion information, background information, and character setting information based on the motion regular data, the biometric data, and the digital idol information, wherein the character motion information includes heart rate information of the digital idol;
The digital idol's lung volume change amount, posture change amount, shape change amount of each body part, and sweat amount are derived by the artificial intelligence module according to the heart rate,
generating a character motion sequence by correcting the amount of change in lung volume, posture, and shape of each body part of the digital idol based on the character motion information, the background information, and the character setting information;
generating a pre-clothing motion sequence using the character motion sequence;
Calculate the friction force between the character's body and clothing based on the sweat amount,
Compensating the pre-clothing motion sequence based on the friction force to generate a clothing motion sequence,
The lung volume change amount means a time-series change amount with respect to the size of the character's lungs, the posture change amount means a time-series change amount of a plurality of points expressing the movement of the character, and the shape change amount means the character's chest Including the amount of change in shape and the amount of change in the shape of the abdomen,
When the heart rate is higher than a predetermined reference value, the chest shape change amount is set to be larger than the abdominal shape change amount,
When the heart rate is lower than a predetermined reference value, the chest shape change amount is set to be smaller than the abdominal shape change amount,
The artificial intelligence module is trained by an artificial intelligence training module,
The artificial intelligence training module groups a plurality of pre-scanned 3D meshes by any one of parameters included in the character setting information, aligns the grouped plurality of 3D meshes using a pre-stored template mesh, and the aligned plurality of 3D meshes Derive the average value of the amount of change in lung volume of the 3D mesh, the amount of change in posture, and the amount of change in the shape of each body part, and train the artificial intelligence module using the derived average value,
The character motion information includes at least one of a posture, fatigue, and emotion of the character, and a heart rate of the character,
The background information includes at least one of temperature, humidity, and time zone of the background,
The character setting information is an artificial intelligence-based platform providing method for implementing a metaverse digital idol including at least one of gender, age, body type, and size of the character.
9. The method of claim 8,
Create digital idol contents using the metaverse data,
By uploading the digital idol contents to SNS, the rating score is calculated,
The method of providing an artificial intelligence-based platform for implementing a metaverse digital idol further comprising generating a reward score according to the rating score and linking it to the user's account in the metaverse.
9. The method of claim 8,
The method for providing an artificial intelligence-based platform for implementing a metaverse digital idol further comprising generating a clothing motion sequence through the digital idol information, the motion normal data, the bionormal data, and the character motion sequence.

Images