KR102610267B1 - Method for analyzing status of specific user corresponding to specific avatar by referring to interactions between the specific avatar and other avatars in the metaverse world and providing service to the specific user and device using the same - Google Patents

Abstract

The present invention relates to a method for analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to interactions between avatars in the metaverse, wherein (a) a computing device sends a first message to at least some of the specific avatar and other avatars; Triggering vectors to the mth triggering vectors are input respectively, so that (i) at least a portion of the first to mth response vector sets and (ii) at least a portion of the first to mth normal response vector ranges recording in a database; and (b) the computing device provides at least a portion of the first to the mth triggering vectors to a specific avatar, and at least a portion of the specific first to mth reaction vectors corresponding to the vectors to which the specific avatar reacted. Obtaining and comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific first response vector to the specific mth response vector to form a first response vector gap to the mth response vector. Obtaining at least some of the gaps and recording them in a database.

Description

Method and device for analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to the interaction between avatars in the metaverse AVATARS IN THE METAVERSE WORLD AND PROVIDING SERVICE TO THE SPECIFIC USER AND DEVICE USING THE SAME}

The present invention relates to a method and device for analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to interactions between avatars in the metaverse.

According to a press release from the Ministry of Health and Welfare in 2021, the number of patients using medical services due to mental illness and psychiatric problems increased by about 4.2% on average per year from about 2.07 million in 2009 to about 3.12 million in 2019. In addition, according to the 2021 Ministry of Health and Welfare's 'COVID-19 National Mental Health Survey', the risk group for depression increased 6 times and the rate of suicidal thoughts increased 3.5 times in 2021 compared to 2018. Therefore, non-face-to-face mental health care is needed to resolve psychiatric problems caused by COVID-19.

However, in Korea, the mental health service utilization rate is only 22.2% due to resistance to psychiatric treatment (Ministry of Health and Welfare, National Mental Health Status Report 2020, 2021), so it is necessary to reduce resistance to psychiatric treatment and actively encourage service use. There is a need for the development of non-face-to-face mental health care services.

In this situation, non-face-to-face mental health management services have been developed that provide simple information through smartphone applications or allow users to enter simple information values, but these smartphone application-based services are difficult to personalize. , there is a problem that patients are unable to continue treatment for a long time and have a high dropout rate.

Therefore, methods and devices are needed to perform more systematic interactions with patients using the metaverse and provide precise solutions to patients through analysis of data generated during the interaction process.

Korean Patent Publication No. 10-2022-0125766

The purpose of the present invention is to solve the above-mentioned problems.

The present invention provides, among the first normal response vector range to the mth normal response vector range determined with reference to at least a part of the first to mth triggering vectors and at least a part of the first to mth response vector sets. By comparing at least a part of the specific first reaction vector to at least a part of the specific m-th reaction vector, at least a part of the first reaction vector gap to the m-th reaction vector gap is obtained and recorded in the database to provide more information to the user. Another purpose is to provide accurate services.

In order to achieve the purpose of the present invention as described above and realize the characteristic effects of the present invention described later, the characteristic configuration of the present invention is as follows.

According to one aspect of the present invention, in a method of analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to the interaction between avatars in the metaverse, (a) a computing device, the first metadata triggering avatar By causing at least one triggering avatar corresponding to at least some of the group to the nth metadata triggering avatar group to input the first to the mth triggering vectors to at least some of the specific avatar and other avatars, respectively, (i ) At least a portion of the first reaction vector set to the mth reaction vector set corresponding to vectors in which the specific avatar and the other avatar reacted to the first triggering vector to the mth triggering vector, and (ii) the first reaction recording in a database at least a portion of a first normal response vector range to an mth normal response vector range determined with reference to at least a portion of the vector set or the mth response vector set; and (b) the computing device provides at least a portion of the first triggering vector to the mth triggering vector to the specific avatar, and provides a specific first response vector to a specific vector corresponding to a vector to which the specific avatar reacts. Obtaining at least a portion of the mth response vector, and comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific first response vector to the specific mth response vector, A method comprising obtaining at least a portion of the first to mth reaction vector gaps and recording them in the database is disclosed.

As an example, each vector is: (i) facial expression vector data corresponding to the facial expression of a subject expressing each vector - the subject is a random avatar included in the specific avatar, the triggering avatar, and the other avatar - The facial expression vector data has as components a first facial expression component to a p facial expression component for representing each of the first emotional category to p th emotional category - and (ii) behavior vector data corresponding to the subject's behavior - the behavior A method is disclosed, wherein vector data is recorded including a first behavioral component to a p-th behavioral component as components, respectively, for representing each of the p-th emotional categories within the first emotional category.

As an example, the computing device includes a learning device, and before performing step (b), (b0) the computing device: It further includes calculating a first to mth predicted reaction vector predicted to react, and in step (b), when the specific first to mth specific reaction vectors are obtained, The computing device compares the specific first response vector to the specific mth response vector with the first predicted response vector to the mth predicted response vector, and produces a first response loss to an mth response for the specific avatar. A method is disclosed, characterized in that obtaining a loss and performing backpropagation using the first reaction loss to the mth reaction loss.

As an example, in step (b), when the computing device causes the triggering avatar to input the first to the mth triggering vectors to the specific avatar, the specific user for each triggering vector Obtaining at least some of the specific user biometric data, the specific user image data captured by the specific user, and the specific user voice data recorded the reaction of the specific user, respectively, the specific user biometric data, the specific user image data, and Determining at least some of the specific adjusted first response vector to the specific adjusted mth response vector by adjusting at least some of the specific first to the specific mth response vectors with reference to at least some of the specific user voice data, respectively; , Comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific adjusted first response vector to the specific adjusted mth response vector, to determine the first response vector gap to the A method is disclosed, characterized in that obtaining at least some of the mth response vector gaps and recording them in the database.

As an example, the computing device further obtains a specific avatar ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific avatar, and obtains a specific user ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific user. In a state in which a specific user digital pinotype is additionally acquired, before step (b), (b1) the computing device, (i) the specific user ecological instantaneous assessment, the specific user digital pinotype, and the specific user biometric mode of the specific user and the specific avatar, with reference to at least some of the data, the specific user video data and the specific user voice data, and (ii) the specific avatar ecological instantaneous assessment and the specific avatar digital phenotype. Further comprising the step of obtaining a matching rate, wherein in step (b), with reference to the mode matching rate, the weight of at least a portion of the specific user video data and the specific user voice data is adjusted to obtain the specific adjusted first response. A method is disclosed, characterized in that determining the vector to the specific adjusted mth reaction vector.

As an example, a first virtual expert module to an m-th virtual expert module corresponding to each of the first to the m-th reaction vector gap - each of the first to the m-th virtual expert modules is the first to the m-th virtual expert module, Configured to provide counseling content corresponding to each of the first response vector gap to the mth response vector gap - with each being matched and recorded in the database, after step (b), (c) the first response vector When the k-th reaction vector gap among the gap to the m-th reaction vector gap exceeds a first threshold, the computing device selects a response vector gap corresponding to the k-th reaction vector gap among the first virtual expert module to the m-th virtual expert module. The k-th virtual expert module provides the k-th consultation content corresponding to the k-th response vector gap to the specific avatar, and provides the k-th specific avatar satisfaction with the k-th consultation content input by the specific avatar. A method is disclosed, further comprising: recording in the database.

As an example, before step (b), (b2) the computing device determines at least one similar avatar as a similar avatar among the other avatars, the similarity of which is higher than a second threshold with the specific avatar. steps; (b3) the computing device determines that the kth similar avatar reaction vector gap among the first similar avatar reaction vector gaps to the mth similar avatar reaction vector gaps recorded in the database corresponding to the similar avatar exceeds a third threshold. Obtaining information about a predetermined similar avatar; and (b4) the computing device provides the k-th similar avatar counseling content that was provided to the predetermined similar avatar to the specific avatar, and the k-th similar avatar inputs satisfaction with the k-th similar avatar counseling content. A method is disclosed, further comprising providing similar avatar satisfaction.

As an example, in step (b), the computing device extracts at least a portion of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set from the database, and At least a portion of a 1_1 specific reaction vector sequence to a 1_z specific reaction vector sequence, which is a sequence of the first specific reaction vector, is extracted from a specific reaction vector, and in step (c), the computing device, the first reaction vector By comparing at least a part of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the set with at least a part of the 1_1 specific reaction vector sequence to the 1_z specific vector event sequence, a first vector sequence gap or Disclosed is a method of obtaining a zth vector sequence gap and transmitting the first vector sequence gap to the zth vector sequence gap to the virtual expert module, thereby assisting the virtual expert module to adjust the kth consultation content. do.

According to another aspect of the present invention, an apparatus for analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to interactions between avatars in the metaverse, comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions, wherein (I) the processor is configured to use at least one triggering avatar corresponding to at least some of the first metadata triggering avatar group to the nth metadata triggering avatar group. By causing at least some of the specific avatar and other avatars to input a first to the mth triggering vector, respectively, (i) the specific avatar and the other avatar with respect to the first to the mth triggering vectors at least a portion of the first to mth reaction vector sets corresponding to the vectors reacted, and (ii) a first normal reaction determined with reference to at least a portion of the first to mth reaction vector sets. A process of recording at least a portion of the vector range to the mth normal response vector range into a database; and (II) providing at least a portion of the first to the mth triggering vectors to the specific avatar, among the specific first to mth reaction vectors corresponding to vectors to which the specific avatar reacted. obtain at least a portion, and compare at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific first response vector to the specific mth response vector to form a first response vector gap. An apparatus is disclosed that performs a process of obtaining at least some of the to mth response vector gaps and recording them in the database.

As an example, each vector is: (i) facial expression vector data corresponding to the facial expression of a subject expressing each vector - the subject is a random avatar included in the specific avatar, the triggering avatar, and the other avatar - The facial expression vector data has as components a first facial expression component to a p facial expression component for representing each of the first emotional category to p th emotional category - and (ii) behavior vector data corresponding to the subject's behavior - the behavior An apparatus is disclosed, wherein vector data is recorded including a first behavioral component to a p-th behavioral component as components, respectively, for representing each of the pth emotional categories within the first emotional category.

As an example, the computing device includes a learning device, and before performing the process (II), (II0) the processor determines whether the specific avatar is A process of calculating a first predicted reaction vector to an m-th predicted reaction vector predicted to react is further performed, and in the process (II), when the specific first to m-th predicted reaction vectors are obtained, Compare a specific first to the mth reaction vector with the first to the mth predicted reaction vector to obtain first to mth reaction losses for the specific avatar, An apparatus is disclosed, characterized in that backpropagation is performed using a first reaction loss to the mth reaction loss.

As an example, in the process (II), when the processor causes the triggering avatar to input the first to the mth triggering vectors to the specific avatar, for each triggering vector, the specific user's Obtaining at least some of specific user biometric data, specific user image data captured by the specific user, and specific user voice data recorded by recording a reaction of the specific user, respectively, the specific user biometric data, the specific user image data, and the specific user voice data Determining at least some of the specific first adjusted response vector to the specific mth adjusted response vector by adjusting at least some of the specific first to the specific mth response vectors with reference to at least some of the user voice data, respectively; By comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific adjusted first response vector to the specific adjusted mth response vector, the first response vector gap to the An apparatus is disclosed, characterized in that it obtains at least some of the m response vector gaps and records them in the database.

As an example, the processor further obtains a specific avatar ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific avatar, and obtains a specific avatar ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific user. In the state in which the user digital pinotype is additionally acquired, before the process (II), (II1) (i) the specific user ecological momentary assessment, the specific user digital pinotype, the specific user biometric data, and the specific user image A process of obtaining a mode matching rate between the specific user and the specific avatar by referring to at least some of the data and the specific user voice data, and (ii) the specific avatar ecological momentary assessment and the specific avatar digital phenotype. Further, in the process (II), with reference to the mode matching rate, the weight of at least some of the specific user video data and the specific user voice data is adjusted to adjust the specific adjustment first response vector to the specific adjustment second response vector. An apparatus is disclosed, characterized in that it determines m response vectors.

As an example, a first virtual expert module to an m-th virtual expert module corresponding to each of the first to the m-th reaction vector gap - each of the first to the m-th virtual expert modules is the first to the m-th virtual expert module, configured to provide counseling content corresponding to each of the first response vector gap to the mth response vector gap - with each being matched and recorded in the database, after the process (II), (III) the first response vector If the kth reaction vector gap among the gap to the mth response vector gap exceeds a first threshold, the kth virtual expert module corresponding to the kth response vector gap among the first virtual expert module to the mth virtual expert module Providing the k-th counseling content corresponding to the k-th response vector gap to the specific avatar, and recording the k-th specific avatar satisfaction input by the specific avatar with the k-th counseling content in the database An apparatus for further performing a process is disclosed.

As an example, before the process (II), (II2) a process of determining, among the other avatars, a predetermined avatar whose similarity with the specific avatar is higher than a second threshold as at least one similar avatar; (II3) for a predetermined similar avatar whose kth similar avatar response vector gap exceeds the third threshold among the first to mth similar avatar response vector gaps recorded in correspondence with the similar avatar from the database; The process of obtaining information; and (II4) providing the specific avatar with the k-th similar avatar counseling content that was provided to the predetermined similar avatar, and providing the k-th similar avatar satisfaction in which the predetermined similar avatar inputs the satisfaction with the k-th similar avatar counseling content. A device for further performing a process is disclosed.

As an example, in the process (II), at least a portion of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set is extracted from the database, and the first specific reaction vector is extracted from the first specific reaction vector. At least a portion of a 1_1 specific reaction vector sequence to a 1_z specific reaction vector sequence, which is a sequence of a first specific reaction vector, is extracted, and in the process (III), a 1_1 reaction vector sequence set included in the first reaction vector set By comparing at least a portion of the 1_z reaction vector sequence set with at least a portion of the 1_1 specific reaction vector sequence to the 1_z specific vector event sequence, a first vector sequence gap to a z-th vector sequence gap are obtained, An apparatus is disclosed, which supports the virtual expert module to adjust the k-th consultation content by transmitting the first to the z-th vector sequence gap to a virtual expert module.

The present invention provides, among the first normal response vector range to the mth normal response vector range determined with reference to at least a part of the first to mth triggering vectors and at least a part of the first to mth response vector sets. By comparing at least a part of the specific first reaction vector to at least a part of the specific m-th reaction vector, at least a part of the first reaction vector gap to the m-th reaction vector gap is obtained and recorded in the database to provide more information to the user. It is effective in providing accurate service.

The following drawings attached for use in explaining embodiments of the present invention are only some of the embodiments of the present invention, and to those skilled in the art (hereinafter “those skilled in the art”), the invention Other drawings can be obtained based on these drawings without further work being done.
1 is a diagram schematically showing a device for implementing first to third embodiments of the present invention.
Figure 2 shows a process performed in a method for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the first embodiment of the present invention. This is a flow chart schematically showing.
Figure 3 shows a process performed in a method for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the second embodiment of the present invention. This is a flow chart schematically showing.
FIG. 4 is a diagram illustrating an example of a triggering vector generated from a triggering avatar and a specific response vector generated from a specific avatar in a state in which an interaction between a triggering avatar and a specific avatar occurs, according to a second embodiment of the present invention.
Figure 5 shows a process performed in a method for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the third embodiment of the present invention. This is a flow chart schematically showing.
Figure 6 is a diagram schematically showing a specific avatar observing a model response vector within the metaverse, according to a third embodiment of the present invention.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced to make clear the objectives, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

Additionally, throughout the description and claims, the word “comprise” and variations thereof are not intended to exclude other technical features, attachments, components or steps. Other objects, advantages and features of the invention will appear to those skilled in the art, partly from this description and partly from practice of the invention. The examples and drawings below are provided by way of example and are not intended to limit the invention.

Moreover, the present invention encompasses all possible combinations of the embodiments shown herein. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described. Similar reference numbers in the drawings refer to identical or similar functions across various aspects.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 schematically shows a device for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the present invention.

Referring to FIG. 1, a computing device 100 is shown for providing specific content for the first to third embodiments of the present invention, and the computing device 100 includes a memory 101 and a processor 102. may include. At this time, the computing device 100 may additionally include a display 103, but it may also be configured separately rather than included in the computing device 100.

The memory 101 of the computing device 100 is operated by the processor 102 to perform a more systematic interaction with the patient using the metaverse and to provide a precise solution to the patient through analysis of data generated during the interaction process. Instructions may be stored. Specifically, the instructions are codes generated for the purpose of enabling the computing device 100 to provide content to function in a specific manner, and may be directed to computers or other programmable data processing equipment. It may be stored in computer-usable or computer-readable memory. Instructions may perform processes to execute functions described in the specification of the present invention.

In the present invention, content will be explained assuming content implemented in virtual reality, but it is of course not limited thereto.

In addition, the processor 102 of the computing device 100 for providing content has a hardware configuration such as a Micro Processing Unit (MPU) or Central Processing Unit (CPU), cache memory, and data bus. It can be included. Additionally, the computing device 100 may further include an operating system and a software component of an application that performs a specific purpose.

Additionally, the computing device 100 to provide specific content may be linked to the database 110. Here, the database 110 includes a flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory), At least one of RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (ReadOnly Memory), EEPROM (Electrically Erasable Programmable ReadOnly Memory), PROM (Programmable ReadOnly Memory), magnetic memory, magnetic disk, and optical disk. It may include one type of storage medium, but is not limited to this and may include all media capable of storing data. In addition, the database 110 may be installed separately from the computing device 100 for providing content, or alternatively, may be installed inside the computing device 100 for providing content to transmit data or record received data. Alternatively, as shown, it may be implemented separately into two or more parts, which may vary depending on the implementation conditions of the invention.

Additionally, the display 103 may be separate from the memory 101 and processor 102 in the computing device 100 as a separate device.

Additionally, the computing device 100 may be a user terminal such as an HMD, or may be a server for providing a virtual reality service. In the latter case, the server may be linked to a user terminal (not shown) such as a separate HMD.

First, the method according to the first embodiment of the present invention will be described with reference to FIG. 2.

Figure 2 shows a process performed in a method for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the first embodiment of the present invention. This is a flow chart schematically showing.

Referring to FIG. 2, in the first step, the computing device 100 generates data corresponding to the response of at least some of the specific avatar and other avatars to at least some of the first to nth triggering events of the triggering avatar. With at least a portion of the first reaction event set to the nth reaction event set stored in the database, a specific virtual ecological momentary assessment (EMA) and a specific virtual digital phenotype are generated from a specific avatar in the metaverse. At least some of them can be obtained (S201).

Here, the triggering avatar is an avatar that is separate from the specific avatar, and the computing device 100 may support inputting a predetermined triggering event to at least some of the specific avatar and other avatars within the metaverse through the triggering avatar. At this time, the triggering avatar may be an avatar created by the computing device 100 or an avatar operated by a user separate from the specific user. A 'triggering event' may be a wide range of events that can elicit a reaction event from a specific avatar. For example, it may be an event that applies psychological pressure to a specific avatar by asking them to leave their seat, but it is not limited to this. Additionally, when a 'triggering event' is provided by a triggering avatar created by the computing device 100, it may be programmed to be provided as a predetermined vector value by the computing device 100. Specific examples of vectors are described in detail in the second embodiment, but it goes without saying that they can also be applied to the first embodiment. Of course, the various events of the first embodiment do not have to be implemented as vector values.

Additionally, 'ecological instantaneous assessment' may be data that repeatedly collects the subject's behavior and experience in real time in everyday situations. For example, it may be data that surveys subjects on 'how they currently feel' at a set time every day, or it may be data that surveys subjects' repeated actions at a set time every day, but is not limited to this. In addition, a specific virtual ecological instantaneous evaluation is an ecological instantaneous evaluation of a specific avatar in the metaverse, which may be different from the ecological instantaneous evaluation of a specific actual user corresponding to a specific avatar. For example, a specific avatar within the metaverse may be found to have little stress when asked to take a survey at 6 PM every day, while a specific user outside the metaverse may be found to be highly stressed when asked to take a survey at the same time. There will be.

In addition, the 'digital pinotype' may be information including the usage pattern of online services (location of use, diversity of time, time spent at home, regularity, distance traveled while using online services, etc.). Here, the specific virtual digital pinotype of a specific avatar within the metaverse may be different from the specific actual digital pinotype of a specific real user outside the metaverse. For example, within the metaverse, a specific avatar actively moves around and uses online services (chat services within the metaverse, etc.), but outside of the metaverse, they rarely move from home and use online services (SNS services, phone calls, etc.) It may be investigated as using .

Thereafter, the computing device acquires at least some of a specific first reaction event to a specific n-th reaction event corresponding to data in which a specific avatar reacted to at least some of the first triggering event to the n-th triggering event, and generates a first reaction By comparing at least a portion of the first normal response range to the nth normal response range determined with reference to at least a portion of the event set to the nth reaction event set, and at least a portion of the specific first reaction event to the specific nth reaction event, At least a portion of the first to n-th reaction gaps may be obtained, and a specific avatar risk may be calculated with reference to at least a portion of the first to n-th reaction gaps (S202).

Here, the specific first reaction event to the specific n-th reaction event refers to (i) a specific virtual ecological momentary assessment, (ii) a specific virtual digital phenotype, and (iii) the first to n-th triggering event of a specific avatar. It may be determined by referring to at least some of a specific first real-time reaction to a specific n-th real-time reaction that a specific avatar reacted to in real time. Here, the specific first real-time reaction to the specific n-th real-time reaction that a specific avatar reacted to in real time may be obtained by analyzing the vector value on the metaverse output by the specific avatar, but is not limited to this. Accordingly, it may be obtained through analysis of the facial expressions, gestures, and voices of users operating a specific avatar.

Thereafter, when the specific avatar risk exceeds the first threshold, the computing device asks at least a portion of the first to nth reaction gaps a first set of questions corresponding to each of the first to nth reaction gaps. An adjustment question set in which the weights of each of the through to nth question sets are changed may be determined, and the adjustment question set may be provided to a specific avatar through a virtual expert (S203).

At this time, the virtual expert may exist in the form of an avatar within the metaverse controlled by a computing device, and the virtual expert may be trained to an intellectually knowledgeable level in a specific field. Here, the virtual expert may have been trained or may be able to perform additional learning using data input and output through at least some of the processes of the first to third embodiments.

Returning to step S203, if the risk of a specific avatar exceeds the first threshold, a virtual expert within the metaverse may approach the specific avatar and provide a set of adjustment questions.

Next, the specific avatar risk is adjusted with reference to the specific avatar's specific avatar answer set to the adjustment question set, and if the specific avatar risk exceeds the second threshold, the specific avatar is supported to interact with a real expert (S204). can do.

For example, if the risk of a specific avatar does not decrease at all or actually increases as a result of referring to a specific avatar answer set, interaction with an actual expert can be supported. At this time, the computing device can send an emergency signal to an actual expert to quickly perform an interaction with a specific avatar.

In the method of the present invention, before step S201, each of the specific avatar and the other avatar may be included in at least one category among the first to mth categories, for example, the specific avatar may be included in the kth category among the first to mth categories. It can be assumed that it is included in the category. Here, each category can be divided by various classification criteria such as age, gender, and occupation, and each avatar may correspond to multiple categories.

At this time, each of the k_1th reaction event set to the k_nth reaction event set corresponding to the data to which at least some of the kth other avatars belonging to the kth category among the specific avatar and other avatars responded to the first reaction event set to the first reaction event set. Each nth reaction event set can be selected and stored in a database. That is, the vector values of each of the k_1th reaction event set to the k_nth reaction event set (generated as a result of responding to the first to nth triggering events) of many avatars belonging to the kth category are the behavior of the corresponding avatar, It may be calculated from facial expressions, etc., but it may also be calculated with reference to information included in virtual ecological instantaneous evaluation, virtual digital pinotype, etc., and in some cases, actual ecological instantaneous evaluation of actual users controlling the avatars, actual digital pinotype, etc. It may be calculated with reference to at least some of the type, biometric data, etc.

Thereafter, in step S202, at least a portion of the k_1th normal reaction range to the k_nth normal reaction range determined with reference to at least a portion of the k_1th reaction event set to the k_nth reaction event set, and a specific first reaction event to a specific nth reaction event set. By comparing at least a portion of the reaction events, at least a portion of the k_1th reaction gap to the k_nth reaction gap may be obtained, and a specific avatar risk may be calculated with reference to at least a portion of the k_1th reaction gap to the k_nth reaction gap. That is, with reference to the k_1th reaction event set to the k_nth reaction event set (generated as a result of responding to the first to nth triggering events) of many avatars belonging to the kth category, It can be assumed that each of the k_1th normal reaction range to the k_nth normal reaction range of many avatars is obtained and recorded in a database, and the specific user's specific (generated as a result of responding to the first to nth triggering event) The purpose is to determine how much each of the first reaction event or the specific nth reaction event deviates from the k_1th normal reaction range to the k_nth normal reaction range recorded in the database.

Next, in step S203, if the specific avatar risk exceeds the first threshold, the computing device corresponds to each of the k_1th reaction gap to the k_nth reaction gap with reference to at least a portion of the k_1th reaction gap to the k_nth reaction gap. The adjusted question set can be determined by changing the weight of each of the first to nth question sets as much as possible, and the adjusted question set can be provided to a specific avatar through a virtual expert. In other words, for a set of reaction events that deviate significantly from the normal range, an additional set of questions is assigned to perform a detailed analysis.

Meanwhile, in the following, the device of the present invention includes a learning device (not shown), and an example of a learning process using the learning device will be described.

First, before performing step S202, the computing device 100 generates a first to nth predicted reaction event in which a specific avatar is predicted to react to at least some of the first to nth triggering events. It can be calculated. For this purpose, the learning device includes interactions between various past avatars already recorded in the database, such as provision information of the first to nth triggering events and the corresponding first to nth reaction event set information, The attributes of each of the various past avatars may be input as learning data and may have been learned to some extent.

Thereafter, in step S202, when a specific first reaction event to a specific n-th reaction event is obtained, the computing device 100 generates a specific first reaction event to a specific n-th reaction event and a first predicted reaction event to an n-th prediction. By comparing reaction events, the first to nth reaction losses can be obtained for a specific avatar, and backpropagation can be performed using the first to nth reaction losses. The process of performing backpropagation can be repeated, and by repeating backpropagation, the learning device will be able to more accurately calculate the predicted response event.

Next, we will explain how to adjust a specific reaction event (more precisely, a vector value included in a specific reaction event) using data obtained from a specific actual user and data obtained from a specific avatar.

In a state in which the computing device 100 additionally acquires specific real user biometric data, a specific real ecological momentary assessment, and a specific real digital pinotype from a specific real user, before step S202, the computing device 100, (i ) with reference to at least some of a specific virtual eco-moment assessment and a specific virtual digital phenotype, and (ii) at least some of a specific real-world eco-moment assessment, a specific real-world digital phenotype and a specific real-world user biometric data, with reference to a specific real-world user and a specific avatar; A mode matching rate of . Here, the specific actual user biometric data may include, but is not limited to, the specific actual user's heart rate, heart rate variability, and breathing rate.

Thereafter, in step S202, a specific first reaction event to a specific n-th reaction event may be determined by adjusting the weights of at least some of the specific virtual ecological instantaneous assessment and the specific virtual digital pinotype with reference to the mode coincidence rate. That is, when the mode match rate is low (when the data obtained for a specific avatar and the data obtained for a specific real user are significantly different), a specific virtual ecological momentary evaluation reflected in a specific first reaction event to a specific nth reaction event, and The weight of a specific virtual digital pinotype can be lowered to better reflect the data obtained for a specific real user.

Additionally, the device according to the present invention can provide services that were provided to similar avatars to a specific avatar while obtaining information about similar avatars similar to a specific avatar in a database. The process is explained in detail below.

First, in the method according to the present invention, before step S201, a certain other avatar whose similarity to a specific avatar among other avatars is higher than the third threshold can be set as a similar avatar.

Thereafter, at least a portion of each of the first to nth reaction event sets corresponding to data on the reaction of at least some of the specific avatar and other avatars to at least some of the first to nth triggering events of the triggering avatar. With stored in the database, the computing device 100 may obtain at least some of the similar avatar virtual ecological momentary assessment and the similar avatar virtual digital pinotype from the similar avatar in the metaverse.

Next, the computing device 100 generates at least some of the first to nth similar avatar reaction events corresponding to data in which the similar avatar reacted to at least some of the first to nth triggering events. Obtaining, and at least a portion of the first normal reaction range to the nth normal reaction range determined with reference to at least a portion of the first to nth reaction event sets, and the first to nth similar avatar reaction events, Compare at least some of the events to obtain at least a portion of the first similar avatar response gap to the nth similar avatar response gap, and use at least a portion of the first similar avatar response gap to the nth similar avatar response gap to obtain a similar avatar response gap. The risk can be calculated. This is done by analyzing information on which avatars similar to the relevant avatar output response events with which vector values to which stimuli and calculating the normal range specialized for other avatars with the same or similar tendencies as the relevant avatar. This means that rather than using a set normal range as the object of comparison, the normal range calculated for similar tendencies can be used as the object of comparison. Here, the first to nth similar avatar reaction events are (i) the similar avatar virtual ecological instantaneous evaluation, (ii) the similar avatar virtual digital phenotype, and (iii) the first to nth similar avatar triggering events. The decision may be made by referring to at least some of the first similar avatar real-time response to the nth similar avatar real-time response to the triggering event.

Thereafter, when the similar avatar risk exceeds the first threshold, the computing device 100 determines the first similar avatar reaction gap to the nth similar avatar response gap with reference to at least a portion of the first similar avatar reaction gap to the nth similar avatar reaction gap. A similar avatar adjustment question set is determined by changing the weight of each of the first to nth similar avatar question sets corresponding to each avatar response gap, and the similar avatar adjustment question set is provided to the similar avatar through a virtual expert. You can. In other words, for event sets that deviate significantly from the normal range, a detailed analysis can be performed by assigning an additional set of questions.

Next, the similar avatar risk is adjusted with reference to the similar avatar's answer set to the adjustment question set, and if the similar avatar risk exceeds the second threshold, the similar avatar can be supported to interact with a real expert. Thereafter, the similar avatar adjustment question set, the similar avatar answer set, and the previous similar avatar's interaction with the actual expert may be stored in the database.

At this time, all steps required to provide interaction between a similar avatar and an actual expert may be performed before step S201. That is, before the triggering avatar inputs the first to nth triggering event for a specific avatar, all steps required to provide interaction between the similar avatar and the actual expert may have been performed.

Referring to FIG. 2 again, in step S204, as a result of referring to the specific avatar answer set calculated through interaction with the virtual expert, the specific avatar risk is reduced by the first improvement degree, and the specific avatar is provided with interaction with the real expert. As a result of the application, when the risk of a specific avatar is reduced by the second improvement degree, and at least one of the first improvement degree and the second improvement degree is lower than the fourth threshold, the similar avatar adjustment question set, the similar avatar answer set, and the similar avatar Treatment can be maximized by providing at least some of the information about interactions with actual experts to a specific avatar for reference.

That is, if the specific avatar risk after the specific avatar interacts with the virtual expert and the specific avatar risk after the specific avatar interacts with the real expert are both lower than the fourth threshold, the interaction with the virtual expert and The therapeutic effect of interaction with an actual expert can be considered insufficient. In this case, a set of similar avatar adjustment questions previously experienced by the similar avatar, a set of similar avatar answers, and information about the similar avatar's interaction with an actual expert may be provided to the specific avatar, thereby making up for the insufficient treatment effect.

Next, the steps for supporting the virtual expert to determine the adjusted question set by adjusting the first to nth question sets will be described in detail.

In step S203, when a virtual expert is selected with reference to at least a portion of the first reaction gap to the nth reaction gap, the computing device 100 selects a specific first reaction event to a specific nth reaction event of a specific avatar to the virtual expert. , it is possible to support the virtual expert in determining an adjusted question set by changing the weights of each of the first to nth question sets corresponding to the first to nth response gaps, respectively. That is, in order to determine which virtual expert to assign among a plurality of virtual experts, how large is the gap size for at least some of the first to nth reaction gaps (i.e., how much does a reaction gap deviate from normal)? By judging the degree of severity with reference to, for example, a virtual expert specialized in severe cases may be assigned, and which response gap (so-called specific response gap) among the first response gap to the nth response gap is particularly deviant from normal can be determined to determine a specific response gap. It would also be possible to assign a trained virtual expert to the response gap. Here, the specific response gap, as can be inferred from the above description, refers to a specific symptom related to a specific triggering event and a specific reaction event when a specific reaction event is output for a specific triggering event among the first to nth triggering events. Since this may mean that the degree of deviation from normal is large, it is possible to assign a specific virtual expert who has specialized training in relation to a specific symptom. For example, in a specific first reaction event, the stress is greater than the normal response range and the first response gap is measured to be large, and in a specific second response event, the stress is similar to the normal response range and the second response gap is measured to be small. In this case, a virtual expert corresponding to the first response gap may be selected.

Additionally, in step S203, the computing device 100 may extract at least a portion of a 1_1 specific reaction event sequence to a 1_z specific reaction event sequence, which is a sequence of the first specific reaction event, from the first specific reaction event. Here, ‘event sequence’ means dividing the reaction event into several stages in time series. For example, if the first specific reaction event is output as 'the specific avatar gets angry at the triggering avatar with the highest intensity' for the triggering event 'the triggering avatar gets angry at the specific avatar', then 'the specific avatar' There may be multiple 'response event sequences', such as 'approaches the triggering avatar', 'a specific avatar makes eye contact with the triggering avatar', and 'a specific avatar makes a loud sound at maximum volume to the triggering avatar'.

Thereafter, the computing device 100, at least some of the 1_1 reaction sequence set to 1_z reaction sequence set included in the first reaction event set, obtained from the database, and the 1_1 specific reaction event sequence to 1_z specific reaction event Compare at least some of the sequences to obtain first to z-th sequence gaps, and pass the first to z-th sequence gaps to the virtual expert, so that the virtual expert asks the first to n-th question sets, respectively. It can assist in determining a set of adjustment questions that change the weights of .

In addition, similar to the above-mentioned, the method of the present invention, prior to step S202, computes specific real-user biometric data, specific real-world ecological momentary assessment, and specific real-world digital pinotypes from a specific real-life user. Device 100 refers to (i) at least some of a specific virtual eco-momentum assessment and a specific virtual digital pinotype, and (ii) at least some of a specific real-world biometric assessment, a specific real-world digital pinotype, and specific real-world user biometric data. Thus, it may further include the step of obtaining a mode matching rate between a specific real user and a specific avatar, and then, in step S204, (i) a specific virtual ecological moment evaluation and a specific virtual digital phenotype, (ii) a specific real ecological moment Evaluation, with reference to at least some of the specific real digital pinotype and specific real user biometric data, (iii) the mode match rate of the specific real user and the specific avatar, and (iv) the first sequence gap to the z sequence gap, with real experts. It may also support interaction.

Next, the method according to the second embodiment of the present invention will be described with reference to FIG. 3.

Figure 3 shows a process performed in a method for performing a more systematic interaction with a patient using the metaverse and providing a precise solution to the patient through analysis of data calculated during the interaction process, according to the second embodiment of the present invention. This is a flow chart schematically showing. Hereinafter, definitions and descriptions of words similar to those in the first embodiment will be replaced with definitions and descriptions in the first embodiment.

Referring to FIG. 3, the computing device 100 causes at least one triggering avatar corresponding to at least some of the first metadata triggering avatar group to the nth metadata triggering avatar group to trigger at least some of the specific avatar and other avatars. to input the first to mth triggering vectors respectively, (i) a set of first to mth reaction vectors corresponding to the vectors in which a specific avatar and other avatars reacted to the first to mth triggering vectors; At least a portion of the reaction vector set and (ii) at least a portion of the first to mth normal reaction vector ranges determined with reference to the first to mth reaction vector sets are recorded in the database ( S301) You can. Here, each of the first metadata triggering avatar group to the nth metadata triggering avatar group corresponds to one metadata, and at this time, the metadata includes various metadata such as gender, age, disposition, occupation, physical characteristics, and mental characteristics. can do.

Thereafter, the computing device 100 provides at least some of the first to mth triggering vectors to a specific avatar, and the specific first to mth reaction vectors corresponding to the vectors to which the specific avatar reacted. Obtaining at least a part of the first normal response vector range to the mth normal response vector range and comparing at least a part of the specific first response vector to the specific mth reaction vector to form a first response vector gap to the mth normal response vector range. At least some of the m response vector gaps may be obtained and recorded in the database (S302).

Here, the 'vector' concept will be explained with reference to FIG. 4.

Figure 4 shows a vector input from the triggering avatar (the so-called stimulus vector) and a vector output from the specific avatar (the so-called response vector) in a state in which an interaction between the triggering avatar and a specific avatar occurs, according to an embodiment of the present invention. This is a drawing showing an example.

First, each vector may include a first to a p-th vector component. That is, any triggering vector may include a first triggering vector component to a p-th triggering vector component, and any specific response vector may include a specific first response vector component to a specific p-th response vector component.

Referring to FIG. 4, the computing device 100, in a state where an interaction between a triggering avatar (left avatar) and a specific avatar (right avatar) occurs, first to pth triggering vector components input from the triggering avatar. In response to , a specific first response vector component to a specific pth response vector component output from a specific avatar can be obtained. In FIG. 4, the case where p is 7 is exemplarily explained. Of course, here, the specific first reaction vector component or the specific pth reaction vector component output from a specific avatar may be obtained by quantifying the actions, facial expressions, virtual ecological instantaneous evaluation, virtual digital pinotype, etc. performed by the specific avatar. It may also be obtained by quantifying the analysis of facial expressions, gestures, and voices of users operating a specific avatar.

As shown in FIG. 4 , the computing device 100 may determine first to seventh triggering vector component values corresponding to the triggering avatar. At this time, the computing device 100 may determine the first to seventh triggering vector component values to accurately implement the state of the triggering avatar interacting with a specific avatar, and in some cases, these values may be adjusted according to time. The rate of change may also be determined. As described in the first embodiment, the triggering avatar may be an avatar created by the computing device 100 or an avatar operated by a user separate from the specific user. However, the example below assumes the former case. I will do it.

Referring again to FIG. 4, the first to seventh triggering vector components correspond to vector component values for “happiness, anger, anger, sadness, and desire.” In other words, it corresponds to the vector component values for each of joy, anger, sadness, joy, love, hate, and greed.

As shown in Figure 4, when (joy, anger, sadness, joy, love, hate, greed) = (0, 0.6, 0.2, 0, 0, 0.1, 0.2) is input as the triggering vector component value (i.e., a specific avatar (when stimulated), the computing device 100 is (joy, anger, sadness, joy, love, hate, greed) expressed by the triggering avatar = (0, 0.6, 0.2, 0, 0, 0.1, 0.2) A specific first reaction vector component to a specific seventh reaction vector component of a specific avatar that reacts can be obtained. Referring to FIG. 4, the specific first reaction vector component value to the specific seventh reaction vector component value determined therefrom is (joy, anger, sadness, joy, love, hate, greed) = (0.1, 0.3, 0.3, 0.1) , 0.1, 0.8, 0.1).

At this time, each vector is (i) facial expression vector data corresponding to the facial expression of the subject expressing each vector (the subject is a specific avatar, a triggering avatar, and an arbitrary avatar included in other avatars) and (ii) the subject's facial expression. Each action may be recorded including action vector data corresponding to the action. Expression vector data and action vector data may be values included in a vector component. In some cases, when it is difficult to identify the avatar's expressions and actions, the virtual ecological instantaneous evaluation of the specific avatar, virtual digital phenotype, etc. can be quantified and used as a reference, and the facial expressions, gestures, and Analysis of voices, etc. can be quantified and used as a reference.

Here, the facial expression vector data may have as components a first facial expression component to a p-th facial expression component for representing each of the first to p-th emotional categories, and the action vector data may have components for each of the p-th emotional categories in the first emotional category. It may have as components a first action component to a p-th action component to represent .

Meanwhile, learning can also be performed on the method of the present invention according to the second embodiment.

The computing device 100 of the present invention according to the second embodiment includes a learning device, and before performing step S302, the computing device 100 specifies at least some of the first to mth triggering vectors. The method may further include calculating a first to mth predicted response vector through which the avatar is predicted to react.

Thereafter, in step S302, when the specific first reaction vector to the specific m-th reaction vector is obtained, the computing device 100 generates the specific first reaction vector to the specific m-th reaction vector and the first predicted reaction vector to the m-th prediction. By comparing reaction vectors, the first to mth reaction losses can be obtained for a specific avatar, and backpropagation can be performed using the first to mth reaction losses. The process of performing backpropagation can be repeated, and by repeating backpropagation, the learning device will be able to more accurately calculate the predicted response vector.

In addition, as briefly mentioned above, when the computing device 100 causes the triggering avatar to input the first to mth triggering vectors to a specific avatar in step S302, a specific user of a specific user for each triggering vector. At least some of biometric data, specific user video data captured by a specific user, and specific user voice data recorded by recording the reaction of a specific user may be obtained. By adjusting at least some of the specific first to specific m-th reaction vectors with reference to at least some of the audio data, at least some of the specific first to specific m-th reaction vectors can be determined, Comparing at least a part of the normal reaction vector range to the mth normal reaction vector range with at least a part of the specific adjusted first reaction vector to the specific adjusted mth reaction vector, and at least a portion of the first to mth reaction vector gaps can be obtained and recorded in the database.

Here, each of the first reaction vector gap to the mth reaction vector gap may have the first facial expression component gap to the pth facial expression component gap as a component, and may have the first behavioral component gap to the pth behavioral component gap as a component. There will be. That is, the first reaction vector gap may have as components a 1_1 facial expression component gap to a 1_p facial expression component gap and a 1_1 behavioral component gap to 1_p behavioral component gap. Each of the second to mth reaction vector gaps may also have the same facial expression component gap and action component gap as components. Of course, it is not limited to this, and it goes without saying that the response vector gap can be managed through the facial expression component and the behavioral component.

In addition, as mentioned above, the computing device 100 additionally acquires a specific avatar ecological instantaneous assessment and a specific avatar digital pinotype from a specific avatar, and additionally obtains a specific user ecological instantaneous assessment and a specific user digital pinotype from a specific user. Using this, you can obtain the mode coincidence rate through the following process.

That is, before step S302, the computing device 100, (i) at least some of the specific user ecological instantaneous assessment, specific user digital pinotype, specific user biometric data, specific user image data, and specific user voice data, (ii) ) The mode matching rate of a specific user and a specific avatar can be obtained by referring to at least some of the specific avatar ecological instantaneous evaluation and the specific avatar digital pinotype, and in step S302, with reference to the mode matching rate, specific user video data and specific avatar A specific adjusted first response vector to a specific adjusted mth response vector may be determined by adjusting the weight of at least some of the user voice data.

Additionally, each of the first to mth virtual expert modules corresponding to the first to mth response vector gaps may be matched and recorded in the database. Here, each of the first to mth virtual expert modules is configured to provide consultation content corresponding to each of the first to mth response vector gaps. In addition, it goes without saying that what was mentioned regarding the virtual expert in the first embodiment can also be applied to the second embodiment, etc., even if there is no special mention.

In this state, after step S302, if the kth reaction vector gap among the first to mth reaction vector gaps exceeds the first threshold, the computing device 100 selects the first to mth virtual expert modules. Among the expert modules, the kth virtual expert module corresponding to the kth response vector gap provides the kth consultation content corresponding to the kth response vector gap to a specific avatar, and the specific avatar inputs satisfaction with the kth consultation content. As long as k, the satisfaction level of a specific avatar can be recorded in the database.

Here, when the k-th specific avatar satisfaction is recorded above a certain threshold, it may be configured to provide the k-th counseling content to a predetermined similar avatar similar to the specific avatar, and when the k-th specific avatar satisfaction is recorded below a certain threshold. In this case, counseling content that is determined to have satisfied similar avatars above a threshold may be provided to a specific avatar. The latter will be explained in detail below.

In the method of the present invention according to the second embodiment, before step S302, the computing device 100 selects at least one other avatar whose similarity to a specific avatar among other avatars is higher than the second threshold. You can decide with your avatar.

Thereafter, the computing device 100 determines that the kth similar avatar reaction vector gap among the first similar avatar reaction vector gaps to the mth similar avatar reaction vector gap recorded in correspondence with the similar avatar from the database exceeds the third threshold. You can obtain information about similar avatars.

Next, the computing device 100 provides the k-th similar avatar counseling content that was provided to the predetermined similar avatar to a specific avatar, and the k-th similar avatar satisfaction level in which the predetermined similar avatar inputs the satisfaction with the k-th similar avatar counseling content. Information about can also be provided to a specific avatar. Accordingly, a specific avatar can simultaneously receive satisfaction with the counseling content of similar avatars and counseling content that has already been conducted by similar avatars similar to the avatar.

In addition, in the method of the present invention according to the second embodiment, in step S302, the computing device 100 selects at least a portion of the 1_1 response vector sequence set to the 1_z response vector sequence set included in the first response vector set. It is possible to extract from the database, and extract at least a portion of the 1_1 specific reaction vector sequence to the 1_z specific reaction vector sequence, which is the sequence of the first specific reaction vector, from the first specific reaction vector.

As mentioned in the first embodiment, 'vector sequence' means dividing the reaction vector into several steps. For example, if a certain response vector is output for a certain triggering vector, there may be multiple 'reaction vector sequences' such as a first response vector sequence, a second response vector sequence, and a third response vector sequence. . At this time, the sum of the first reaction vector sequence, the second reaction vector sequence, and the third reaction vector sequence may be equal to the reaction vector.

Thereafter, in step S303, the computing device 100, at least a portion of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set, the 1_1 specific reaction vector sequence to the 1_z specific vector Compare at least some of the event sequences to obtain the first vector sequence gap to the z-th vector sequence gap, and transmit the first vector sequence gap to the z-th vector sequence gap to the virtual expert module, so that the virtual expert module provides the k-th consultation. We can help you coordinate content.

Next, the method according to the third embodiment of the present invention will be described with reference to FIG. 5.

Figure 5 is a flowchart schematically showing the process performed in the method according to the third embodiment of the present invention.

Referring to FIG. 5, the computing device 100 causes at least one triggering avatar corresponding to at least some of the first metadata triggering avatar group to the nth metadata triggering avatar group to trigger at least some of the specific avatar and other avatars. by inputting the first to the mth triggering vectors respectively, (i) a set of first reaction vectors corresponding to vectors to which at least some of the specific avatar and other avatars reacted to the first to mth triggering vectors; at least a portion of the to mth reaction vector sets, and (ii) at least a portion of the first to mth stress level reduction rate sets, which are stress level reduction rate sets after outputting each of the first to mth reaction vector sets. can be recorded in the database (S501). This can be viewed as a database construction stage to provide services beyond the S502 stage.

Thereafter, when the stress level of a specific user in the real world exceeds the first threshold due to a real-world interaction with the triggering user, the computing device 100 refers to the real-world interaction and creates a stress level to be applied to the specific avatar in the virtual world. The k triggering vector may be determined from the first to mth triggering vectors (S502). In other words, among the first to mth triggering vectors input to the database, a simulation will be performed to determine the triggering vector most similar to the real world interaction as the kth triggering vector. For example, using the definition of vectors described above, we can say that the vectors of real-world interactions are (happy, angry, sad, joy, love, hate, greed) = (0, 0.6, 0.2, 0, 0, 0.1, 0.2) When, the first triggering vector is (0.3, 0.3, 0.5, 0.2, 0.2, 0.1, 0.2), the second triggering vector is (0, 0.6, 0.3, 0.1, 0, 0.3, 0.2), and the third triggering vector is If is (0.7, 0.4, 0.5, 0, 0.5, 0, 0), the second triggering vector with the smallest error from the real-world interaction vector can be determined as the kth triggering vector. Here, rather than simply determining this as the kth triggering vector, it can be determined as the kth triggering vector by selecting the one with the lowest stress level reduction rate or lower than the threshold among the kth triggering vector candidates recorded in the database. will be.

Next, the computing device 100 causes the triggering avatar to input the kth triggering vector to a specific avatar, and for the kth triggering vector, (i) a specific kth reaction vector corresponding to the vector to which the specific avatar reacted; and (ii) the reduction rate of the specific avatar's stress level after the specific avatar outputs the specific kth response vector can be obtained (S503).

Additionally, in the method of the present invention according to the third embodiment, in step S504 (not shown), when the specific avatar stress level reduction rate is lower than the second threshold, the computing device 100 selects the first to mth response vector sets. Among the reaction vector sets, the kth reaction vector set corresponding to the kth triggering vector is extracted from the database, the kth stress level reduction rate set corresponding to the kth reaction vector set is extracted from the database, and the kth stress level reduction rate set is extracted from the database. 2 Obtain at least one k_x stress level reduction rate component and a k_x response vector component corresponding to the k_x stress level reduction rate component higher than the threshold, and provide the k_x response vector component and the k_x stress level reduction rate component to the specific avatar. You can.

In other words, if the stress level reduction rate of a specific avatar is not sufficient (i.e., the stress level is not sufficiently reduced), a response vector component with a sufficient stress level reduction rate is obtained from the database, and (i) corresponds to the kth triggering vector. (ii) a k_x response vector component whose stress reduction rate is higher than the second threshold; and (ii) a k_x stress level reduction rate component corresponding to the k_x response vector component.

For example, when there are a total of three specific avatars and other avatars, for the first triggering vector, the first reaction vector set (including the 1_1 reaction vector component to the 1_3 reaction vector component) and the corresponding first reaction vector set 1 set of stress level reduction rates (including the 1_1st stress level reduction rate component to the 1_3rd stress level reduction rate component) may be extracted from the database. At this time, if the 1_1 stress level reduction rate component is 30, the 1_2 stress level reduction rate component is 70, the 1_3 stress level reduction rate component is 10, and the stress level reduction rate of a specific avatar is 40, the 1_2 stress level reduction rate component and the 1_2 A response vector component may be provided.

Additionally, in the method according to the third embodiment of the present invention, in step S505 (not shown), after the specific avatar is provided with the k_x response vector component and the k_x stress level reduction rate component, the specific avatar stress level reduction rate is set to the third threshold. In higher cases, the computing device 100 may provide content that induces a specific avatar to output a k_x response vector component for the kth triggering vector.

For example, after confirming that the stress level reduction rate of a specific avatar was higher than the third threshold, a UI was created on the front of the specific avatar that provided an acceptance button along with a message such as “Would you like to print the k_x response vector component?” It will be possible to display it.

Next, with reference to FIG. 6, a method of allowing a specific avatar to observe a model response avatar will be described.

Figure 6 is a diagram schematically showing a specific avatar observing a model response vector within the metaverse, according to an embodiment of the present invention.

Referring to FIG. 6, it is shown that a specific avatar does not output a response vector, but receives information from the interaction between the exemplary response avatar and the triggering avatar.

To this end, in step S501 described in FIG. 5, a model score may be additionally recorded in the database for each model response avatar located in the metaverse.

Here, the exemplary response avatar may include at least one first type exemplary response avatar that is linked to the actual exemplary user and at least one second type exemplary response avatar that is not linked to the actual exemplary user. At this time, the second type exemplary response avatar may be created and controlled by the computing device 100, but is not limited thereto.

In addition, the exemplar score is, for the first to mth triggering vectors, the first to mth exemplar reaction vectors corresponding to the vectors to which any exemplar reaction avatar reacted, and the first exemplar reaction stress corresponding thereto. It may be the rate or number of times that the level reduction rate to the mth exemplary reaction stress level reduction rate is obtained, and the exemplary reaction stress level reduction rate higher than the fourth threshold among the first exemplary reaction stress level reduction rate to the mth exemplary reaction stress level reduction rate is obtained. In other words, the higher the exemplary response avatar has, the higher the probability that it will show an exemplary response with a high stress level reduction rate.

In addition, the computing device 100 provides an exemplary response acquisition rate for each of the at least one exemplary response avatar, supports a specific avatar to select a specific exemplary response avatar among the at least one exemplary response avatar, and supports the specific exemplary response avatar selected by the specific avatar. A specific reaction vector component and a specific stress level reduction rate component corresponding to the reaction avatar may be provided as the k_x reaction vector component and the k_x stress level reduction rate component. That is, in 'information provision' shown in FIG. 6, the information provided to a specific avatar may be 'k_x response vector component' and 'k_x stress level reduction rate component'.

Additionally, in the method according to the third embodiment of the present invention, in step S503, when the computing device 100 causes the triggering avatar to input the kth triggering vector to a specific avatar, the specific user's biometric data, It is natural that at least some of the gyroscope data that detects the motion of a specific user, the specific user video data that records the specific user, and the specific user voice data that records the specific user's reaction can be obtained, and in all embodiments, It will be applied.

Thereafter, a specific adjusted k-th response vector may be determined by adjusting the specific k-th response vector by referring to at least some of the specific user biometric data, gyroscope data, specific user image data, and specific user voice data. Here, the specific user biometric data may include information such as the specific user's heart rate, respiratory rate, and blood pressure, but is not limited thereto. In other words, the specific k-th reaction vector can be determined by considering not only the reaction of a specific avatar in the metaverse but also the real-world reaction output by a specific user to a certain extent.

In addition, in the method according to the third embodiment of the present invention, the computing device 100 additionally obtains a specific avatar ecological instantaneous evaluation and a specific avatar digital phenotype from a specific avatar, and obtains a specific user ecological instantaneous evaluation and a specific avatar digital phenotype from a specific user. You can additionally acquire specific user digital pinotypes.

In this state, in step S503, the computing device 100, (i) at least one of the specific user ecological instantaneous assessment, the specific user digital pinotype, the specific user biometric data, the gyroscope data, the specific user image data, and the specific user voice data; and (ii) obtaining a mode matching rate between a specific user and a specific avatar by referring to at least some of the specific avatar ecological instantaneous evaluation and a specific avatar digital pinotype, and using the mode matching rate as a reference to obtain specific user video data and a specific user A specific adjusted kth response vector can be determined by adjusting the weight of at least some of the voice data.

In addition, in the method according to the third embodiment of the present invention, each of the first to mth triggering vectors and each reaction vector included in the first to mth reaction vector sets are: (i) the first (ii) an expression vector corresponding to the facial expression of the subject expressing each of the triggering vectors to the mth triggering vectors and each of the reaction vectors included in the first to mth reaction vector sets, and (ii) an action vector corresponding to the subject's behavior. Data can be recorded, including each.

At this time, each vector is (i) facial expression vector data corresponding to the facial expression of the subject expressing each vector (the subject is a specific avatar, a triggering avatar, and an arbitrary avatar included in other avatars) and (ii) the subject's facial expression. Each action can be recorded including action vector data corresponding to the action. Expression vector data and action vector data may be values included in a vector component.

Here, the facial expression vector data may have as components a first facial expression component to a p-th facial expression component for representing each of the first to p-th emotional categories, and the action vector data may have components for each of the p-th emotional categories in the first emotional category. It may have as components a first action component to a p-th action component to represent .

Since all descriptions related to facial expressions and actions can refer to the content mentioned in the second embodiment, descriptions are omitted.

Meanwhile, in the method according to the third embodiment of the present invention, the computing device 100 includes a learning device, and before step S503, the computing device 100: (i) the first metadata triggering avatar group to the first metadata triggering avatar group n metadata At least some of the first to mth triggering vectors input by at least some of the avatar group to a specific avatar and at least some of the other avatars, (ii) a specific avatar with respect to the first to mth triggering vectors; and at least some of the first to mth reaction vector sets corresponding to vectors to which at least some of the other avatars reacted, and (iii) the stress level reduction rate after outputting each of the first to mth reaction vector sets. Learning may be performed by inputting at least a portion of the first stress level reduction rate set to the mth stress level reduction rate set into the learning device.

Thereafter, in step S503, the learning device calculates a predicted k-th reaction vector for the k-th triggering vector of a specific avatar, calculates a predicted stress level reduction rate corresponding to the predicted k-th reaction vector, and calculates a predicted k-th reaction vector. Backproper using at least a portion of the k-th reaction vector loss determined with reference to the difference between the k-th reaction vector and the stress level reduction rate loss determined with reference to the difference between the predicted stress level reduction rate and the specific avatar stress level reduction rate Gating can be performed. The process of performing backpropagation can be repeated, and by repeating backpropagation, the learning device will be able to more accurately calculate the predicted response vector and predicted stress level reduction rate.

Additionally, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. A computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. , and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. A hardware device may be configured to operate as one or more software modules to perform processing according to the invention, and vice versa.

In the above, the present invention has been described with specific details such as specific components and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the embodiments. Anyone skilled in the art to which the present invention pertains can make various modifications and variations from this description.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and the scope of the patent claims described below as well as all modifications equivalent to or equivalent to the scope of the claims fall within the scope of the spirit of the present invention. something to do.

Claims (16)

In a method of analyzing the status of a specific user corresponding to a specific avatar and providing a service by referring to the interaction between avatars in the metaverse,
(a) The computing device causes at least one triggering avatar corresponding to at least some of the first metadata triggering avatar group to the nth metadata triggering avatar group to send a first triggering vector to at least some of the specific avatar and other avatars. By inputting the through mth triggering vectors, respectively, (i) a set of first through mth reaction vectors corresponding to vectors in which the specific avatar and the other avatar reacted to the first through mth triggering vectors; Recording at least a portion of the vector set and (ii) at least a portion of the first normal response vector range to the mth normal response vector range determined with reference to at least a portion of the first to mth response vector sets in the database. steps; and
(b) The computing device provides at least a portion of the first triggering vector to the mth triggering vector to the specific avatar, and a specific first response vector to a specific first response vector corresponding to the vector to which the specific avatar reacts. Obtaining at least a portion of the m response vectors, and comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific first response vector to the specific mth response vector, Obtaining at least a portion of the first to mth reaction vector gaps and recording them in the database;
Method, including. According to paragraph 1,
Each vector is: (i) expression vector data corresponding to the expression of the subject expressing each vector - the subject is an arbitrary avatar included in the specific avatar, the triggering avatar, and the other avatar - the expression vector The data has as components a first to a p-th facial expression component to represent each of the first to p-th emotional categories - and (ii) action vector data corresponding to the subject's behavior - the action vector data is The method is recorded to include, respectively, a first behavioral component to a p-th behavioral component for representing each of the p-th emotional categories within the first emotional category. According to paragraph 1,
The computing device includes a learning device,
Before performing step (b) above,
(b0) calculating, by the computing device, a first to mth predicted response vector in which the specific avatar is predicted to react to at least some of the first to the mth triggering vectors. Contains,
In step (b) above,
When the specific first response vector to the specific m-th reaction vector is obtained, the computing device, the specific first response vector to the specific m-th response vector, and the first predicted response vector to the m-th predicted response vector. By comparing , the first to mth reaction losses are obtained for the specific avatar, and backpropagation is performed using the first to mth reaction losses. According to paragraph 1,
In step (b) above,
When the computing device causes the triggering avatar to input the first to the mth triggering vectors to the specific avatar, it collects specific user biometric data of the specific user and the specific user for each triggering vector. Obtaining at least some of the captured specific user video data and the specific user voice data recording the specific user's reaction, respectively,
By adjusting at least some of the specific first response vector to the specific mth response vector with reference to at least some of the specific user biometric data, the specific user video data, and the specific user voice data, respectively, a specific adjusted first response vector to determine at least some of the specific coordination m reaction vectors,
By comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific adjusted first response vector to the specific adjusted mth response vector, the first response vector gap to the Method, characterized in that obtaining at least some of the m response vector gaps and recording them in the database. According to paragraph 4,
The computing device further acquires a specific avatar ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific avatar, and obtains a specific user ecological momentary assessment (EMA) and a specific user digital phenotype from the specific user. With additional Pinotypes acquired,
Before step (b) above,
(b1) the computing device, (i) at least some of the specific user ecological instantaneous assessment, the specific user digital pinotype, the specific user biometric data, the specific user video data, and the specific user voice data, and (ii) Referring to at least some of the specific avatar ecological instantaneous evaluation and the specific avatar digital phenotype, it further includes obtaining a mode matching rate between the specific user and the specific avatar,
In step (b) above,
Characterized in that, with reference to the mode coincidence rate, the weight of at least some of the specific user video data and the specific user voice data is adjusted to determine the specific adjusted first response vector to the specific adjusted mth response vector, method. According to paragraph 1,
A first virtual expert module to an m-th virtual expert module corresponding to each of the first response vector gap to the m-th response vector gap - each of the first virtual expert module to the m-th virtual expert module has the first response vector. Configured to provide counseling content corresponding to each gap to the mth response vector gap - with each being matched and recorded in the database,
After step (b) above,
(c) When the kth response vector gap of the first to the mth reaction vector gaps exceeds a first threshold, the computing device, among the first to the mth virtual expert modules, The k-th virtual expert module corresponding to the k-th response vector gap provides the k-th consultation content corresponding to the k-th response vector gap to the specific avatar, and determines the satisfaction of the specific avatar with the k-th consultation content. The method further comprising: recording the input kth specific avatar satisfaction in the database. According to clause 6,
Before step (b) above,
(b2) determining, by the computing device, a predetermined other avatar whose similarity to the specific avatar among the other avatars is higher than a second threshold as at least one similar avatar;
(b3) the computing device determines that the kth similar avatar reaction vector gap among the first similar avatar reaction vector gaps to the mth similar avatar reaction vector gaps recorded in the database corresponding to the similar avatar exceeds a third threshold. Obtaining information about a predetermined similar avatar; and
(b4) the computing device provides the k-th similar avatar counseling content that was provided to the predetermined similar avatar to the specific avatar, and the k-th similar avatar inputs satisfaction with the k-th similar avatar counseling content; A method further comprising providing avatar satisfaction. According to clause 6,
In step (b) above,
The computing device extracts at least a portion of a 1_1 reaction vector sequence set to a 1_z reaction vector sequence set included in the first reaction vector set from the database, and extracts the specific first reaction vector from the specific first reaction vector. Extracting at least a portion of the 1_1 specific reaction vector sequence to the 1_z specific reaction vector sequence, which is a sequence of,
In step (c) above,
The computing device, at least a portion of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set, and at least a portion of the 1_1 specific reaction vector sequence to the 1_z specific vector event sequence By comparing, obtain the first vector sequence gap to the z-th vector sequence gap, and transmit the first vector sequence gap to the z-th vector sequence gap to the virtual expert module, so that the virtual expert module provides the k-th consultation. Methods to help you moderate content. A computing device that analyzes the status of a specific user corresponding to a specific avatar and provides services by referring to interactions between avatars in the metaverse,
at least one memory storing instructions; and
At least one processor configured to execute the instructions,
(I) The processor causes at least one triggering avatar corresponding to at least some of the first metadata triggering avatar group to the nth metadata triggering avatar group to send a first triggering vector to at least some of the specific avatar and other avatars. By inputting the mth triggering vector, respectively, (i) a set of first to mth reaction vectors corresponding to vectors in which the specific avatar and the other avatar reacted to the first to mth triggering vectors; at least a portion of the set and (ii) recording in a database at least a portion of the first normal response vector range to the mth normal response vector range determined with reference to at least a portion of the first to the mth response vector set. process; and (II) providing at least a portion of the first to the mth triggering vectors to the specific avatar, among the specific first to mth reaction vectors corresponding to vectors to which the specific avatar reacted. obtain at least a portion, and compare at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific first response vector to the specific mth response vector to form a first response vector gap. A computing device that performs a process of obtaining at least some of the to mth response vector gaps and recording them in the database. According to clause 9,
Each vector is: (i) expression vector data corresponding to the expression of the subject expressing each vector - the subject is an arbitrary avatar included in the specific avatar, the triggering avatar, and the other avatar - the expression vector The data has as components a first to a p-th facial expression component to represent each of the first to p-th emotional categories - and (ii) action vector data corresponding to the subject's behavior - the action vector data is A computing device, each recorded including a first behavioral component to a p-th behavioral component for representing each of the p-th emotional categories within the first emotional category. According to clause 9,
The computing device includes a learning device,
Before carrying out process (II) above,
(II0) The processor further performs a process of calculating first to mth predicted response vectors in which the specific avatar is predicted to react to at least some of the first to mth triggering vectors. do,
In process (II) above,
When the specific first reaction vector to the specific m-th reaction vector is obtained, the specific first reaction vector to the m-th reaction vector is compared with the first predicted reaction vector to the m-th predicted reaction vector, A computing device, characterized in that: obtaining first to mth reaction losses for a specific avatar, and performing backpropagation using the first to mth reaction losses. According to clause 9,
In process (II) above,
When the processor causes the triggering avatar to input the first to mth triggering vectors to the specific avatar, the processor captures the specific user's biometric data and the specific user for each triggering vector. Obtaining at least a portion of video data for a specific user and voice data for a specific user recording a reaction of the specific user,
By adjusting at least some of the specific first response vector to the specific mth response vector with reference to at least some of the specific user biometric data, the specific user video data, and the specific user voice data, respectively, a specific adjusted first response vector to determine at least some of the specific coordination m reaction vectors,
By comparing at least a portion of the first normal response vector range to the mth normal response vector range with at least a portion of the specific adjusted first response vector to the specific adjusted mth response vector, the first response vector gap to the Computing device, characterized in that obtaining at least some of the m response vector gaps and recording them in the database. According to clause 12,
The processor further acquires a specific avatar ecological momentary assessment (EMA) and a specific avatar digital phenotype from the specific avatar, and obtains a specific user ecological momentary assessment (EMA) and a specific user digital phenotype from the specific user. With additional types acquired,
Prior to process (II) above,
(II1) (i) at least a portion of the specific user eco-moment assessment, the specific user digital phenotype, the specific user biometric data, the specific user video data, and the specific user voice data, and (ii) the specific avatar eco-moment Further performing an evaluation process of obtaining a mode matching rate between the specific user and the specific avatar by referring to at least some of the specific avatar digital phenotype,
In process (II) above,
Characterized in that, with reference to the mode coincidence rate, the weight of at least some of the specific user video data and the specific user voice data is adjusted to determine the specific adjusted first response vector to the specific adjusted mth response vector, Computing device. According to clause 9,
A first virtual expert module to an m-th virtual expert module corresponding to each of the first response vector gap to the m-th response vector gap - each of the first virtual expert module to the m-th virtual expert module has the first response vector. Configured to provide counseling content corresponding to each gap to the mth response vector gap - with each being matched and recorded in the database,
After process (II) above,
(III) If the k-th reaction vector gap among the first response vector gap to the m-th response vector gap exceeds a first threshold, the k-th response vector gap among the first virtual expert module to the m-th virtual expert module The kth virtual expert module corresponding to provides the kth consultation content corresponding to the kth response vector gap to the specific avatar, and the kth specific avatar inputs satisfaction with the kth consultation content. A computing device further performing a process of recording avatar satisfaction in the database. According to clause 14,
Prior to process (II) above,
(II2) a process of determining, among the other avatars, a predetermined other avatar whose similarity with the specific avatar is higher than a second threshold as at least one similar avatar;
(II3) for a predetermined similar avatar whose kth similar avatar response vector gap exceeds the third threshold among the first to mth similar avatar response vector gaps recorded in correspondence with the similar avatar from the database; The process of obtaining information; and
(II4) Providing to the specific avatar the k-th similar avatar counseling content that was provided to the predetermined similar avatar, and providing satisfaction with the k-th similar avatar where the predetermined similar avatar inputs satisfaction with the k-th similar avatar counseling content. A computing device that further performs a process; According to clause 14,
In process (II) above,
At least a portion of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set is extracted from the database, and a 1_1 sequence of the specific first reaction vector is extracted from the specific first reaction vector. Extracting at least a portion of a specific reaction vector sequence or a 1_z specific reaction vector sequence,
In process (III) above,
By comparing at least a part of the 1_1 reaction vector sequence set to the 1_z reaction vector sequence set included in the first reaction vector set with at least a part of the 1_1 specific reaction vector sequence to the 1_z specific vector event sequence, Obtaining the 1st vector sequence gap to the zth vector sequence gap, and transmitting the first vector sequence gap to the zth vector sequence gap to the virtual expert module to support the virtual expert module to adjust the kth consultation content. A computing device that does.