KR102617431B1 - A System And Method For Calculating The Optimum Temperature For A User-Customized Immersive Content Test Bed Performing Content By Creating An Avatar - Google Patents

Abstract

The present invention relates to a system and method for calculating the optimal temperature of a user-customized realistic content testbed that creates an avatar and performs content, and receives the user's biometric information, activity information, and medical information through a VR device, and Based on this, an avatar can be created in virtual reality, information about the avatar is provided to an external metaverse platform, and when the user performs content provided by the external metaverse platform, a plurality of avatars are created according to the content. The test bed temperature is controlled by deriving the preliminary temperature, and while the test bed temperature is controlled to the preliminary temperature, the expression score and brain wave score are calculated for each of the plurality of preliminary temperatures to calculate the sum score, and the optimal score is calculated based on the sum score. This relates to a system and method for deriving temperature and calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar to perform content.

Description

A system and method for calculating the optimal temperature of a user-customized immersive content test bed that creates an avatar to perform content {A System And Method For Calculating The Optimum Temperature For A User-Customized Immersive Content Test Bed Performing Content By Creating An Avatar}

The present invention relates to a system and method for calculating the optimal temperature of a user-customized realistic content testbed that creates an avatar and performs content, and receives the user's biometric information, activity information, and medical information through a VR device, and Based on this, an avatar can be created in virtual reality, information about the avatar is provided to an external metaverse platform, and when the user performs content provided by the external metaverse platform, a plurality of avatars are created according to the content. The test bed temperature is controlled by deriving the preliminary temperature, and while the test bed temperature is controlled to the preliminary temperature, the expression score and brain wave score are calculated for each of the plurality of preliminary temperatures to calculate the sum score, and the optimal score is calculated based on the sum score. This relates to a system and method for deriving temperature and calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar to perform content.

Realistic content refers to digital content that applies realistic technology such as virtual reality, augmented reality, and mixed reality that maximizes human five senses and provides experiences similar to reality. According to global consulting firm PWC, virtual convergence technology (XR), which encompasses VR and AR as a realistic content-based technology, is expected to create an economic ripple effect worth $476.4 billion (approximately KRW 534 trillion) in 2025.

Virtual reality is a technology that allows users to experience a virtual space implemented on a computer by wearing an input/output device such as an HMD (Head Mounted Display) device. Users can experience temporal, It is a technology that allows you to experience a specific situation without spatial restrictions. It was predicted that this virtual reality technology would be difficult to become popular in the near future due to expensive equipment and a lack of virtual reality content, but recently, with the spread of VR and AR devices and the increasing demand for non-face-to-face content, virtual reality technology It is also a situation that is being reexamined.

Metaverse technology can be said to be a four-dimensional virtual space and time that continues economic, social, cultural, and political activities through avatars, virtual egos. The metaverse technology is a concept that includes virtual reality, augmented reality, mirror world, and lifelogging. This metaverse technology is being utilized in conjunction with various technological fields such as games, education, shopping, medical care, and tourism. Specifically, platforms such as games, education, shopping, medical care, and tourism implemented in the metaverse can be experienced through avatars. You can.

Users need a space to perform realistic content in the metaverse using virtual reality. Depending on the environment provided by the space, the user's concentration may change, and the results of realistic content performed may vary. Accordingly, when performing the content provided by Metaverse, it is expected that if the environment in the test bed where the user exists can be optimally created for the content, results can be obtained excluding variables other than content performance.

As a method to build a healthcare environment to improve the user's health, Korean Patent Publication No. 10-2020-0022776 creates a 4D avatar for the user and induces the user to perform specific actions, thereby providing the user with a medical experience without having to visit the hospital. A method for performing rehabilitation exercises has been presented. However, the conventional technology simply estimates the user's movement information based on the user's joint information and creates a user avatar that can experience virtual reality. It provides content by considering the user's biometric characteristics or creates an avatar similar to the user. There has been no disclosure regarding a method for generating user immersion and improving user immersion. Additionally, the above-described conventional technologies do not provide the effect of optimizing the temperature within the test bed, which is the space where users perform content.

Therefore, an avatar similar to the user's appearance and physical characteristics can be created, information about the avatar can be provided to an external metaverse platform, and when performing content provided by the external metaverse platform, the user exists. There is an emerging need to develop a system that maximizes content performance results by controlling the temperature within the test bed to an optimal temperature.

Republic of Korea Patent No. 10-1943585 (2019.01.23.) Republic of Korea Patent Publication No. 10-2130750 (2020.06.30.)

The present invention relates to a system and method for calculating the optimal temperature of a user-customized realistic content testbed that creates an avatar and performs content, and receives the user's biometric information, activity information, and medical information through a VR device, and Based on this, an avatar can be created in virtual reality, information about the avatar is provided to an external metaverse platform, and when the user performs content provided by the external metaverse platform, a plurality of avatars are created according to the content. The test bed temperature is controlled by deriving the preliminary temperature, and while the test bed temperature is controlled to the preliminary temperature, the expression score and brain wave score are calculated for each of the plurality of preliminary temperatures to calculate the sum score, and the optimal score is calculated based on the sum score. This relates to a system and method for deriving temperature and calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar to perform content.

In order to solve the above problems, it is a system for calculating the optimal temperature of a user-customized realistic content test bed that creates an avatar and performs content. It is installed on the user and provides images and sounds for virtual reality, and the user's HMD that detects voices; A camera device that acquires a user image including the user's facial expression and derives image information of the user by photographing the user's appearance; A brain wave measuring device attached to the user's head and capable of measuring the user's brain waves; A pulse meter attached to the user's wrist and capable of measuring the user's pulse; and a motion sensing device that measures the user's movements; a VR device including a; A heating and cooling device that measures and controls the test bed temperature within the test bed; a VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, creates an avatar for the user, and provides a plurality of virtual reality contents to the user; an avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about a user's interaction using the avatar in the external metaverse platform, and performs an update on the avatar; and a machine-learned inference model, and the test bed based on the user image including the user's facial expression received from the camera device, the user's EEG information received from the EEG meter, and the machine-learned inference model. An optimal temperature deriving unit that derives an optimal temperature among a plurality of preliminary temperatures and controls the air conditioning and heating device based on the optimal temperature; wherein the VR service unit is available only to the user, and the plurality of A virtual reality implementation unit that implements virtual reality in which virtual reality content is provided; a VR communication unit that communicates with the VR device to exchange information; User's biometric information received from the VR device; and a user information input unit that receives user information including activity information and medical information received from the user's smartphone; an avatar creation unit that creates an avatar for the user based on the user information in the virtual reality implemented by the virtual reality implementation unit; and a VR content provider capable of providing the plurality of contents to the avatar created in the virtual reality and updating the appearance of the avatar each time this is performed. The biometric information is provided through the camera device. Input image information; The user's brain wave information measured by the brain wave measuring device; and pulse information of the user measured by the pulse meter. It provides a system for calculating the optimal temperature of the test bed, including.

In one embodiment of the present invention, the optimal temperature deriving unit may derive the plurality of preliminary temperatures based on content provided by the external metaverse platform.

In one embodiment of the present invention, the optimal temperature deriving unit includes a preliminary temperature maintaining step of controlling the cooling and heating device to maintain the test bed temperature at one of the plurality of preliminary temperatures for a preset time; While the test bed temperature is maintained at the preliminary temperature, an expression score calculation step of receiving a user image including a user's expression from the camera device and calculating an expression score based on the user's expression; While the test bed temperature is maintained at the preliminary temperature, an EEG score calculation step of receiving the user's EEG information from the EEG measuring device and calculating an EEG score based on the EEG information; and a summation score calculation step of calculating a sum score by adding up the expression score and the brain wave score. The preliminary temperature maintenance step to the sum score calculation step may be performed for each of the plurality of preliminary temperatures.

In one embodiment of the present invention, the optimal temperature deriving unit further performs an optimal temperature deriving step, and the optimal temperature deriving step corresponds to the highest summed score among the summed scores calculated for each of the plurality of preliminary temperatures. The preliminary temperature can be derived as the optimal temperature, and the cooling and heating device can be controlled based on the optimal temperature.

In one embodiment of the present invention, the facial expression score calculation step inputs the user image received from the camera device into a learned inference model to derive a user facial expression including mouth opening and the corners of the mouth. The score is calculated according to the state, and the expression score can be calculated by assigning weight as the angle at which the corner of the mouth rises and the mouth opening become larger.

In one embodiment of the present invention, the EEG score calculation step calculates a score by determining which waveform among beta waves, theta waves, mid-alpha waves, and SMR waves includes the user's EEG information received from the EEG measuring device. , when the user's brain wave frequency corresponds to an SMR wave of 12 to 13 Hz or a mid-alpha wave of 10 to 12 Hz, weighting is applied to calculate the brain wave score, and the optimal test bed temperature can be calculated.

In one embodiment of the present invention, the avatar update unit includes an avatar information providing step of providing information about the avatar to an external metaverse platform; An interaction information receiving step of receiving interaction information corresponding to the avatar from the external metaverse platform; and an avatar update step of extracting information related to the avatar from the interaction information and updating the avatar based on the information related to the avatar.

In one embodiment of the present invention, the avatar generator derives each external feature information from a plurality of the image information, averages the derived plurality of external feature information to derive a character determination parameter in the form of a one-dimensional vector, , a basic appearance determination unit that determines the basic appearance of the avatar based on the character determination parameters; an aging information determination unit that inputs the biometric information into one or more artificial neural network models to determine aging information of the avatar, and adds a first graphic effect corresponding to the aging information to the basic appearance; A movement speed determination unit that determines the movement speed of the avatar by increasing the movement speed as the movement distance or number of steps increases, based on the movement distance or number of steps for a preset time derived from the user's smartphone; And determining the disease information of the avatar based on medical information including one or more medical histories entered by the user, and adding a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added. It may include a disease information determination unit.

In one embodiment of the present invention, the VR content provider is a first content provider that determines the level of difficulty based on the aging information of the user and provides content that can perform training to strengthen the user's cognitive ability and concentration. study; a second content provider that determines the level of difficulty based on the activity information of the user and provides content that allows the user to perform physical activity strengthening training; A third content provider that determines the type of content based on the disease information of the user and provides content that can provide training for body parts or cognitive abilities that require rehabilitation training; Performance results according to the performance of the content provided by the first content provider, the second content provider, and the third content provider, EEG information measured by the EEG measurement device during the performance period for the content, and an aging information control unit that calculates one or more scores based on pulse information measured by the pulse meter and adjusts the aging information according to the one or more scores; It may include an avatar appearance update unit that updates the appearance of the user's avatar by adjusting the first graphic effect according to the adjusted aging information by the aging information adjusting unit.

In order to solve the above problems, as a method for calculating the optimal temperature of the test bed implemented in a system for calculating the optimal temperature of a user-customized realistic content test bed that creates an avatar and performs content, the system provides the user with An HMD that is mounted to provide images and sounds for virtual reality and detects the user's voice; A camera device that acquires a user image including the user's facial expression and derives image information of the user by photographing the user's appearance; A brain wave measuring device attached to the user's head and capable of measuring the user's brain waves; A pulse meter attached to the user's wrist and capable of measuring the user's pulse; and a motion sensing device that measures the user's movements; a VR device including a; A heating and cooling device that measures and controls the test bed temperature within the test bed; a VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, creates an avatar for the user, and provides a plurality of virtual reality contents to the user; an avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about a user's interaction using the avatar in the external metaverse platform, and performs an update on the avatar; and a machine-learned inference model, and the test bed based on the user image including the user's facial expression received from the camera device, the user's EEG information received from the EEG meter, and the machine-learned inference model. An optimal temperature deriving unit that derives an optimal temperature among a plurality of preliminary temperatures and controls the air conditioning and heating device based on the optimal temperature, wherein the method is used only by the user by the VR service unit. A virtual reality implementation step of implementing a virtual reality where possible and providing a plurality of virtual reality contents; A VR communication step of exchanging information by communicating with the VR device by the VR service unit; The user's biometric information received from the VR device by the VR service unit; and a user information input step of receiving user information including activity information and medical information received from the user's smartphone; An avatar creation step of generating, by the VR service unit, an avatar of the user based on the user information in a virtual reality implemented by the virtual reality implementation unit; And a VR content providing step of providing, by the VR service unit, the plurality of contents to an avatar created in the virtual reality, and updating the appearance of the avatar each time this is performed; including, the biometric information is image information input through the camera device; The user's brain wave information measured by the brain wave measuring device; and pulse information of the user measured by the pulse meter. It provides a method for calculating the optimal temperature of the test bed, including.

Figure 1 schematically shows the configuration of a system for creating an avatar and providing a plurality of contents in virtual reality according to an embodiment of the present invention.
Figure 2 schematically shows the configuration of a VR service unit according to an embodiment of the present invention.
Figure 3 schematically shows a VR device worn by a user using virtual reality and its configuration according to an embodiment of the present invention.
Figure 4 schematically shows the process of deriving a plurality of external feature information for creating a user's avatar according to an embodiment of the present invention.
Figure 5 schematically shows the operation of the avatar creation unit according to an embodiment of the present invention.
Figure 6 schematically shows the detailed configuration of the aging information determination unit according to an embodiment of the present invention.
Figure 7 schematically shows a user's avatar and the avatar's abilities according to an embodiment of the present invention.
Figure 8 schematically shows the configuration of a VR content providing unit according to an embodiment of the present invention.
Figure 9 schematically shows execution steps in the VR content unit according to an embodiment of the present invention.
Figure 10 schematically shows the aging information reduction step according to an embodiment of the present invention.
Figure 11 schematically shows a process for changing the appearance of an avatar according to an embodiment of the present invention.
Figure 12 schematically shows an execution screen of the first content according to an embodiment of the present invention.
Figure 13 schematically shows a virtual reality interface shown to the user when the concentration score and activity score are less than the first reference value according to an embodiment of the present invention.
Figure 14 schematically shows the steps performed in the avatar update unit according to an embodiment of the present invention.
Figure 15 schematically shows interaction information according to an embodiment of the present invention.
Figure 16 schematically shows the avatar update step based on aging information according to an embodiment of the present invention.
Figure 17 schematically shows an avatar update step based on movement speed according to an embodiment of the present invention.
Figure 18 schematically shows the process of deriving the optimal temperature and controlling the heating and cooling system according to an embodiment of the present invention.
Figure 19 schematically shows derivation of the preliminary temperature in the optimal temperature derivation unit according to an embodiment of the present invention.
Figure 20 schematically shows steps performed by the optimal temperature deriving unit according to an embodiment of the present invention.
Figure 21 schematically shows the optimal temperature derivation step according to an embodiment of the present invention.
Figure 22 schematically shows the facial expression score calculation step and the brainwave score calculation step according to an embodiment of the present invention.
Figure 23 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that this aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain example aspects of one or more aspects. However, these aspects are illustrative and some of the various methods in the principles of the various aspects may be utilized, and the written description is intended to encompass all such aspects and their equivalents.

Additionally, various aspects and features may be presented by a system that may include multiple devices, components and/or modules, etc. It is also understood that various systems may include additional devices, components and/or modules, etc. and/or may not include all of the devices, components, modules, etc. discussed in connection with the drawings. It must be understood and recognized.

As used herein, “embodiments,” “examples,” “aspects,” “examples,” etc. may not be construed to mean that any aspect or design described is better or advantageous over other aspects or designs. . The terms '~part', 'component', 'module', 'system', 'interface', etc. used below generally refer to computer-related entities, such as hardware, hardware, etc. A combination of and software, it can mean software.

Additionally, the terms “comprise” and/or “comprising” mean that the feature and/or element is present, but exclude the presence or addition of one or more other features, elements and/or groups thereof. It should be understood as not doing so.

Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the embodiments of the present invention, have an ideal or excessively formal meaning. It is not interpreted as

The virtual reality rehabilitation system implemented in the social server of the present invention can be implemented in a form that creates an avatar that mimics the user's appearance and state, and the user manipulates the avatar to perform a plurality of contents. Below, we will describe a system for creating an avatar and providing multiple contents in virtual reality.

1. A system that creates an avatar and provides multiple contents in virtual reality

Figure 1 schematically shows the configuration of a system for creating an avatar and providing a plurality of contents in virtual reality according to an embodiment of the present invention.

As shown in Figure 1, a system for creating an avatar and providing a plurality of contents in virtual reality includes an HMD mounted on a user, providing images and sounds for virtual reality, and detecting the user's voice; A camera device that acquires a user image including the user's facial expression and derives image information of the user by photographing the user's appearance; A brain wave measuring device attached to the user's head and capable of measuring the user's brain waves; A pulse meter attached to the user's wrist and capable of measuring the user's pulse; A motion sensing device that measures the user's movements; and an eye tracker that detects the user's eye movements; a VR device 1000 including a; and communicate with the VR device 1000 worn by the user, provide information about virtual reality to the VR device 1000, and create an avatar for the user to provide a plurality of virtual reality contents to the user. A VR service unit that includes; The VR service unit includes a virtual reality implementation unit 2100 that is available only to the user and implements a virtual reality in which the plurality of virtual reality contents are provided; a VR communication unit (2200) that communicates with the VR device (1000) to exchange information; User's biometric information received from the VR device 1000; and a user information input unit 2300 that receives user information including activity information and medical information received from the user's smartphone; an avatar creation unit 2400 that creates an avatar for the user based on the user information in the virtual reality implemented by the virtual reality implementation unit 2100; and a VR content provider 2500 capable of providing the plurality of contents to the avatar created in the virtual reality and updating the appearance of the avatar each time this is performed. The biometric information is provided by the camera. Image information received through the device; The user's brain wave information measured by the brain wave measuring device; Pulse information of the user measured with the pulse meter; and eye tracking information of the user measured by the eye tracker.

Briefly, the system for creating an avatar and providing a plurality of contents in virtual reality includes the VR device 1000 and the VR service unit, and the VR service unit is included in the computing system 10000.

Specifically, the VR device 1000 includes one or more components that sense information that can determine the appearance and abilities of the avatar generated by the VR service unit by being mounted on the user or photographing the user's appearance, As described above, the components include the HMD (Head Mounted Display), the camera device, the EEG meter, and the pulse meter; It includes the motion sensing device and the eye tracker. In addition, the VR device 1000 stores the user's biometric information including the image information, the brain wave information, the pulse information, and the eye tracking information sensed through each component of the VR device 1000. It can be forwarded to the VR Service Department. A more detailed description of the VR device 1000 will be provided later with reference to FIG. 3.

The biometric information sensed through the VR device 1000 is transmitted to the computing system 10000. The VR service unit of the computing system 10000 implements virtual reality, creates an avatar for the user in the virtual reality based on the biometric information, and provides a plurality of contents to the user through the avatar.

Meanwhile, as an embodiment of the present invention, the computing system 10000 can communicate with the social server 3000, as shown in FIG. 1, and the user can communicate with the social server 3000 through the avatar created by the VR service unit. You can communicate with one or more other users within the social server (3000). In addition, as another embodiment of the present invention, the computing system 10000 may include an additional content unit 4000 that provides content other than the service provided by the VR service unit, preferably in the virtual reality. A system that creates an avatar and provides a plurality of contents may simultaneously include the social server 3000 and the additional content unit 4000.

Figure 2 schematically shows the configuration of a VR service unit according to an embodiment of the present invention.

As shown in FIG. 2, the VR service unit includes a virtual reality implementation unit 2100 that implements a virtual reality that is available only to the user and in which the plurality of virtual reality contents are provided; a VR communication unit (2200) that communicates with the VR device (1000) to exchange information; User's biometric information received from the VR device 1000; and a user information input unit 2300 that receives user information including activity information and medical information received from the user's smartphone; an avatar creation unit 2400 that creates an avatar for the user based on the user information in the virtual reality implemented by the virtual reality implementation unit 2100; and a VR content providing unit 2500 that provides the plurality of contents to the avatar created in the virtual reality and updates the appearance of the avatar each time this is performed.

Specifically, the virtual reality implementation unit 2100 can implement virtual reality in the computing system 10000 and transmit information about the virtual reality to the user through the VR communication unit 2200. The information about the virtual reality includes visual elements such as the virtual reality interface and display; and auditory elements such as sound effects, and the audio-visual elements may be delivered to the user through the HMD of the VR device 1000.

As described above, the VR communication unit 2200 delivers information about the virtual reality implemented by the virtual reality implementation unit 2100 to the user, and motion information and the biometric information sensed by the VR device 1000. is delivered to the computing system (10000). The motion information and the biometric information are used to create the user's avatar or perform a plurality of contents within the virtual reality.

The biometric information received through the VR device 1000 is transmitted to the biometric information reception unit 2310 of the user information input unit 2300. Meanwhile, as shown in FIG. 2, the user information input unit 2300 further includes an activity information receiver 2320 and a medical information receiver 2330 in addition to the biometric information receiver 2310. The activity information receiving unit 2320 and the medical information receiving unit 2330 each receive the user's activity information and the user's medical information from the user's smartphone.

The activity information corresponds to exercise records recorded at a preset period recorded on the user's smartphone, life information recorded at a preset period, etc., for example, the number of steps per day, steps per day, and drinks consumed per day. This includes the amount and frequency of water, bedtime and wake-up time, etc. According to one embodiment of the present invention, the activity information may be received through a wearable device capable of recording the activity information. Additionally, the medical information includes information on one or more medical histories entered by the user, and, as an embodiment of the present invention, may be generated using medical records provided by a medical institution.

A plurality of image information generated by capturing the user's face image and body image through the camera device of the VR device 1000 is received by the biometric information receiving unit 2310 through the VR communication unit 2200, and the avatar is received. The creation unit 2400 creates the user's avatar in the virtual reality based on the plurality of image information.

The avatar creation unit 2400 includes a basic appearance determination unit 2410 that determines the basic appearance of the avatar, determines aging information of the user based on a plurality of image information, and determines aging information to change part of the appearance of the avatar. wealth (2420); a movement speed determination unit 2430 that determines the movement speed of the avatar based on the user's activity information; and a disease information determination unit 2440 that changes part of the appearance of the avatar based on the medical information. That is, the avatar creation unit 2400 sets the appearance and capabilities of the avatar based on the user's biometric information, activity information, and medical information, and implements the avatar in the virtual reality.

The VR content provider 2500 provides a plurality of content to the avatar within the virtual reality. The plurality of contents include content capable of performing training to strengthen the user's cognitive ability and concentration; Content that allows the user to perform physical activity enhancement training; and content that can perform training on body parts or cognitive abilities that require rehabilitation training for the user. A more detailed description of the VR content providing unit 2500 will be described later.

Figure 3 schematically shows the VR device 1000 worn by a user using virtual reality and its configuration according to an embodiment of the present invention.

Schematically, Figure 3 (a) shows an embodiment in which the user wears the VR device 1000, and Figure 3 (b) shows the HMD (1100) mounted on the user's head. ) and the configuration of the brain wave measuring device 1300, and Figure 3 (c) shows the configuration of the VR device 1000.

As shown in FIG. 3, the VR device 1000 includes an HMD 1100 mounted on the user to provide images and sounds for virtual reality and detect the user's voice; A camera device 1200 that captures the user's appearance and derives image information about the user; A brain wave measuring device (1300) attached to the user's head and capable of measuring the user's brain waves; A pulse meter (1400) attached to the user's wrist and capable of measuring the user's pulse; A motion sensing device 1500 that measures the user's movements; and an eye tracker 1600 that detects the user's eye movements.

Specifically, Figure 3(a) corresponds to an embodiment in which the user wears the VR device 1000, and the camera device 1200 is not shown separately. It is preferable that the user captures the user's face image and body image through the camera device 1200 before the user installs the VR device 1000 shown in (a) of FIG. 3. The camera device 1200 can transmit the face image and the body shape image to the biometric information receiver 2310.

As shown in (a) of FIG. 3, the pulse meter 1400 is attached to the user's wrist and can measure the user's pulse. That is, the pulse meter 1400 can measure the user's pulse, which is one of the biometric information, and the user's pulse measured in this way is input to the VR service unit to analyze the user's condition or to It can be used to understand the psychology of the user so that the user can smoothly perform content, or it can be used to create content provided in virtual reality. Additionally, the pulse meter 1400 includes an electromyography sensor and can sense the user's pulse information and the user's electromyography information and transmit them to the VR service.

As shown in (a) of FIG. 3, the motion sensing device 1500 can measure the user's motion information within the preset space by measuring the user's movement in the preset space. The user's motion information measured in this way can be input to the VR service unit and become the basis for determining a plurality of contents provided by the VR service unit, or can be used for the user to experience the plurality of contents.

As shown in (a) of FIG. 3, the motion sensing device 1500 includes a sensor module that the user holds in both hands and can sense movement in the preset space, and the present invention As an example, it may include a sensor module, such as an infrared sensor or camcorder, that can measure the user's movement within the preset space as an image. Meanwhile, according to another embodiment of the present invention, the motion sensing device 1500 is attached to the user's body, such as an acceleration sensor or a gyro sensor, and senses the user's movement within the preset space. can do. Preferably, the preset space corresponds to a space equipped with a position sensor module that measures the coordinates of the motion sensing device 1500.

Figure 3(b) shows the detailed configuration of the HMD 1100. As shown in (b) of FIG. 3, the HMD 1100 is connected to the EEG meter 1300, and the EEG meter 1300 can specify the user's EEG, which is one of the biometric information. . The user's brain waves measured in this way are input to the VR service unit and used to analyze the user's state or understand the user's psychology to enable the user to smoothly perform content, or in virtual reality. It can be used to create provided content.

The HMD (1100) includes a visual providing unit (1110) that provides visual elements of virtual reality to the user; An auditory provision unit 1120 that provides auditory elements of the virtual reality to the user; and a microphone unit (not shown) that detects the user's voice.

The VR device 1000 according to an embodiment of the present invention provides information about the virtual reality to the user through the visual provider 1110 and the auditory provider 1120, so that the user can enjoy the virtual reality. It allows an experience as if existing in a virtual space, and receives the user's voice and behavior through the voice input unit and the motion sensing device 1500, so that the user's avatar implemented in the virtual reality moves or speaks. You can do it.

As shown in (b) of FIG. 3, the vision providing unit 1110 provides visual information to provide a three-dimensional effect to both eyeballs of the user, and additionally detects eye movements of the user. Includes an eye tracker (1600). In one embodiment of the present invention, the user's eye movements can be detected and used for diagnosis of the user. In addition, the eye tracker 1600 is preferably located at the edge of the vision providing unit 1110 so as not to interfere with the visual information providing unit 1110 providing visual information to the user.

As shown in (c) of FIG. 3, the VR device 1000 includes the HMD 1100, the camera device 1200, the EEG meter 1300, and the pulse meter 1400, as described above. , the motion sensing device 1500, and the eye tracker 1600.

Figure 4 schematically shows the operation of the basic shape determination unit 2410 according to an embodiment of the present invention.

As shown in FIG. 4, the basic appearance determination unit 2410 derives each appearance feature information from a plurality of the image information and averages the derived plurality of appearance feature information to create a character in the form of a one-dimensional vector. Decision parameters are derived, and based on the character decision parameters, the basic appearance of the avatar is determined.

Specifically, the VR service unit that communicates with the VR device 1000 includes the avatar creation unit 2400, and the avatar creation unit 2400 is, as described above in FIG. 2, the basic appearance determination unit. (2410), the aging information determination unit 2420, the movement speed determination unit 2430, and the disease information determination unit 2440. As shown in FIG. 4, the basic appearance determination unit 2410 receives the user's image information input through the camera device 1200 of the VR device 1000 and determines the user's image based on the user's image information. A face image and a body image are extracted, and a plurality of external feature information is derived from the extracted face image and body image.

The image information may be a dynamic image of the user including changes in facial expression that can recognize the user's emotions, or a plurality of static images of the user recorded continuously at a certain cycle. In particular, the image information should be interpreted in the broadest sense, including having two or more image frames. Accordingly, the basic appearance determination unit 2410 classifies the received image information into a plurality of frames and extracts a face image and a body shape image from each frame. Afterwards, external feature information can be derived based on the extracted facial image. The external feature information includes characteristic information about unique characteristics that can recognize facial expressions, such as eye slope, eyebrow slope, mouth shape, and mouth slope, and additionally, skin condition information and hair condition of the user. information, and the degree of bending of the skeleton.

Meanwhile, Figure 4 shows only the process of extracting the face image from the plurality of frames as an embodiment of the present invention, and the basic appearance determination unit 2410 performs the process in the same manner as shown in Figure 4. The body shape image can be extracted from a plurality of frames. At this time, preferably, the VR service unit can strictly determine the state of the user's skeleton by requesting various postures from the user.

Figure 5 schematically shows the operation of the avatar creation unit 2400 according to an embodiment of the present invention.

As shown in FIG. 5, the avatar creation unit 2400 derives each external feature information from a plurality of the image information and averages the derived plurality of external feature information to create a character determination parameter in the form of a one-dimensional vector. a basic appearance determination unit 2410 that derives and determines the basic appearance of the avatar based on the character determination parameters; an aging information determination unit 2420 that determines aging information of the avatar by inputting the biometric information into one or more artificial neural network models, and adds a first graphic effect corresponding to the aging information to the basic appearance; A movement speed determination unit 2430 that determines the movement speed of the avatar by increasing the movement speed as the movement distance or number of steps increases, based on the movement distance or number of steps for a preset time derived from the user's smartphone; And determining the disease information of the avatar based on medical information including one or more medical histories entered by the user, and adding a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added. Includes a disease information determination unit (2440),

The external feature information includes the user's skin condition information, hair condition information, and the degree of curvature of the skeleton.

Specifically, as described above in FIG. 4, the basic appearance determination unit 2410 derives a plurality of external feature information of the user and averages the derived plurality of external feature information to determine a character in the form of a one-dimensional vector. Parameters can be derived. The character determination parameter is a parameter that sets the appearance information of one or more avatars with respect to the basic appearance of the avatar, for example, the height of the avatar, the skeleton or the degree of curvature of the skeleton, skin color, skin condition, hair color, and It is a parameter that includes the condition of the hair, etc. In addition, as described above, it further includes characteristic information about unique characteristics that can recognize facial expressions, such as the slope of the eyes, the slope of the eyebrows, the shape of the mouth, and the slope of the mouth. can do. The basic appearance determination unit 2410 can implement the basic appearance of an avatar similar to the face and body shape of an actual user in virtual reality through the character determination parameters as described above.

When the basic appearance of the avatar is determined in the basic appearance determination unit 2410, the aging information determination unit 2420 of the avatar creation unit 2400 determines the user's aging information and reflects it in the appearance graphics of the avatar. do. The aging information determination unit 2420 includes one or more artificial neural networks, and can determine the user's aging information by inputting the biometric information into each artificial neural network model. When the aging information is determined, the aging information determination unit 2420 derives a first graphic effect corresponding to the aging information and adds the first graphic effect to the basic appearance of the avatar. A more detailed description of the aging information determination unit 2420 will be provided later.

The disease information determination unit 2440 adds a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added. The disease information is generated based on the medical information received from the user's smartphone. The disease information determination unit 2440 receives the medical information through the medical information reception unit 2330 and generates the disease information based on the medical information. Afterwards, a second graphic effect corresponding to the disease information is calculated. For example, a graphic effect that makes the left shoulder red may be added to the avatar of a user with adhesive capsulitis in the left shoulder, or a graphic effect that makes the right knee red to the avatar of a user with rheumatoid arthritis in the right knee. Graphic effects may be added. Meanwhile, the disease information can affect not only the appearance of the avatar but also the abilities of the avatar, which will be described later. In this way, the avatar creation unit 2400 implements the user's avatar in virtual reality based on the user's biometric information and disease information.

Figure 6 schematically shows the detailed configuration of the aging information determination unit 2420 according to an embodiment of the present invention.

As shown in FIG. 6, the aging information determination unit 2420 includes a first artificial neural network 2421 that receives image information input through the camera device 1200 and outputs first aging information; a second artificial neural network (2422) that receives the user's brain wave information measured by the brain wave measuring device (1300) and outputs second aging information; A third artificial neural network (2423) that receives the user's pulse information measured by the pulse meter (1400) and outputs third aging information; and a comprehensive judgment module 2424 that adds the first aging information, the second aging information, and the third aging information to derive final aging information.

Specifically, the aging information determination unit 2420 according to an embodiment of the present invention may determine the user's aging information based on one or more of the user's image information, brain wave information, and pulse information. At this time, the aging information determination unit 2420 may determine the user's aging information by analyzing the user's image information, brain wave information, and pulse information through one or more artificial neural networks.

As shown in FIG. 6, the first artificial neural network 2421 to the third artificial neural network 2423 each receive image information, brain wave information, and pulse information of the user, perform analysis, and synthesize the analysis results. It is transmitted to a diagnosis module, and the comprehensive diagnosis module receives the analysis result, determines the user's aging information, and outputs it. Based on the aging information output in this way, and with reference to the description of FIG. 5, the aging information determination unit 2420 derives a first graphic effect corresponding to the aging information, and adds the first graphic effect to the basic appearance of the avatar. Add the first graphic effect.

The first artificial neural network 2421 analyzes the user's skin condition information, hair condition information, and the degree of curvature of the skeleton among the plurality of external characteristic information derived from the basic appearance determination unit 2410 to generate first aging information. Derive. Additionally, the VR service department may analyze aging information on the user's appearance based on the performance results of various postures requested from the user, and through this, the first aging information may be derived.

The second artificial neural network 2422 analyzes brain wave information derived from the brain wave measuring device 1300 of the VR device 1000 to derive second aging information. In one embodiment of the present invention, the second aging information is derived by analyzing the brain wave information for stability of the brain waves and concentration of the user.

The third artificial neural network 2423 analyzes pulse information derived from the pulse meter 1400 of the VR device 1000 to derive third aging information. The pulse information includes the user's pulse rate per minute, pulse regularity, and pulse pressure. The third artificial neural network 2423 can calculate the user's expected age and cardiovascular age based on the pulse information.

The comprehensive judgment module 2424 may calculate final aging information by adding up the first aging information, the second aging information, and the third aging information. Meanwhile, the aging information, as an embodiment of the present invention, is given a score such that the aging information when the user's aging degree is the lowest is set to 1, and the aging information when the user's aging degree is the highest is set to 10. Or it can be derived as a level. Preferably, the comprehensive diagnosis module can be configured to automatically adjust how much weight to derive aging information for the image information, brain wave information, and pulse information, and learns about such weights. You can configure a model to do this.

As a result, the aging information determination unit 2420 calculates the final aging information and selects the first graphic effect corresponding to the aging information from among the plurality of first graphic effects pre-stored in the computing system 10000 to the avatar. It can be added to the basic appearance.

Figure 7 schematically shows a user's avatar and the avatar's abilities according to an embodiment of the present invention.

As shown in FIG. 7, the avatar creation unit 2400 increases the moving speed as the moving distance or number of steps increases based on the moving distance or number of steps during a preset time derived from the user's smartphone. It includes a movement speed determination unit 2430 that determines the movement speed of the avatar. Meanwhile, the abilities of the avatar shown in FIG. 7 are one embodiment of the present invention, and the actual abilities of the avatar of the present invention are not limited to those shown in FIG. 7.

Specifically, the avatar creation unit 2400 may set the capabilities of the avatar based on the user's activity information and medical information received from the user's smartphone. As shown in FIG. 7, the avatar's abilities include the avatar's movement speed, muscle mass, and cognitive ability.

The avatar's movement speed can be calculated based on the user's activity information and medical information. For example, data showing that the user walked an average of 3 km per day for 2 hours during a preset time is recorded on the user's smartphone, and the data can be transmitted to the computing system 10000. The movement speed determination unit 2430 may calculate the movement speed of the avatar based on the data. If, in the above-described example, exercise records for a preset time are not regularly input to the smartphone, or data of walking a short distance for a long time are input to the smartphone, the movement speed determination unit 2430 Through the data, the movement speed of the avatar can be set low. As a contrary example, when a regular exercise record is input to the smartphone or a large amount of exercise is recorded on the smartphone, the movement speed determination unit (2430) can set the movement speed of the avatar to be high.

Additionally, the movement speed of the avatar can be determined by referring to disease information generated based on the user's medical information. For example, if the user has a disease in the lower body, the movement speed of the avatar is set low, and the degree of the movement speed set low is determined according to the severity of the disease.

By the avatar creation unit 2400, the muscle mass of the avatar can be determined based on the medical information of the user. As an embodiment of the present invention, the muscle mass of the avatar may be determined based on the user's actual body composition data included in the user's medical information, and as another embodiment of the present invention, the muscle mass of the avatar may be determined based on the user's actual body composition data included in the user's medical information. Referring to the description, the muscle mass of the avatar can be set through the electromyography sensor included in the pulse meter 1400.

By the avatar generator 2400, the cognitive ability of the avatar can be calculated based on the user's brain wave information sensed by the brain wave measuring device 1300 mounted on the user. More specifically, the user's It may be calculated based on brain wave information measured when performing a plurality of contents, which will be described later, and additionally, when the user performs the plurality of contents through the avatar, the brain wave information; and the execution process and results of the plurality of contents.

Figure 8 schematically shows the configuration of the VR content providing unit 2500 according to an embodiment of the present invention.

As shown in FIG. 8, the VR content provider 2500 determines the level of difficulty based on the aging information of the user and provides content that can perform training to strengthen the user's cognitive ability and concentration. , first content provision department (2510); A second content provider 2520 that determines the level of difficulty based on the activity information of the user and provides content that can perform physical activity strengthening training for the user; A third content provider 2530 that determines the type of content based on the disease information of the user and provides content that can train the user's cognitive abilities or body parts requiring rehabilitation training. ; Performance results according to content performance for the content provided by the first content provider 2510, the second content provider 2520, and the third content provider 2530, and the Aging information that calculates one or more scores based on the brain wave information measured by the brain wave measuring device 1300 and the pulse information measured by the pulse measuring device 1400, and adjusts the aging information according to the one or more scores. Control unit 2540; An avatar appearance update unit 2550 that updates the appearance of the user's avatar by adjusting the first graphic effect according to the aging information adjusted by the aging information control unit 2540,

The first content provided to the user from the first content provider 2510 is content to enhance the user's cognitive ability and concentration, including finding differences, arithmetic, and puzzle solving, and the second content is The second content provided to the user in study 2520 is content that can perform training to strengthen the user's physical activity, including walking, stretching, and squatting, and the third content provider 2530 provides the user with The third content provided to the user is intensive training on the body part where the user is injured or intensive training on strengthening the user's cognitive abilities, including memory, judgment, language ability, and the ability to understand space and time. This corresponds to content for training areas in which the user is weak.

Specifically, the first content provided to the user from the first content providing unit 2510 is content provided to enable the user to adapt well to virtual reality, and the user can enjoy multiple content within virtual reality through the present invention. It can be provided as practice content for performing the content of. That is, the user can learn how to manipulate the user's avatar in the virtual reality through the first content.

In addition, the VR content provider 2500 can evaluate the basic cognitive ability and concentration of the user by providing such first content to the user, and further, the user can provide the first content By performing this, the user's cognitive ability and concentration can be strengthened. Meanwhile, as an embodiment of the present invention, the user's sense of immersion can be improved by displaying the user's brain wave information measured through the brain wave measuring device 1300 on a separate interface within the virtual reality.

The second content provided to the user from the second content provider 2520 is content provided to strengthen the user's insufficient physical abilities, and provides information on how to operate the VR device 1000 through the first content. It corresponds to content that a learned user can perform through certain physical activities. Since the user must actually use the user's body to perform the second content in the virtual reality, it can be effective in performing physical activities for the treatment of the user's disease or rehabilitation training.

The VR content provider 2500 can evaluate the user's basic physical ability by providing such second content to the user. As an embodiment of the present invention, the user's physical ability evaluated at this time is preferably referred to in determining the difficulty level and type of the second content provided next.

The third content provided to the user from the third content provider 2530 is content provided to the user to strengthen the user's body parts or cognitive abilities that require rehabilitation training, and includes the user's waist, legs, and arms. Content that allows users who are injured in a body part or have a specific disease to perform training to strengthen that body part; Or, it corresponds to content that can provide training to strengthen the abilities of users who lack memory, judgment, language ability, and the ability to understand space and time.

Meanwhile, the above-described first to third contents each include one or more contents, and as an embodiment of the present invention, it is desirable to provide different contents to the user every day.

The aging information control unit 2540 adjusts the aging information of the avatar based on data that can be obtained by the user performing the first content, the second content, and the third content, and the avatar The appearance update unit 2550 updates the appearance of the avatar by adjusting the first graphic effect according to the aging information adjusted by the aging information adjustment unit 2540. A more detailed description of the operations of the aging information control unit 2540 and the avatar appearance update unit 2550 will be described later.

Figure 9 schematically shows the steps performed in the VR content unit according to an embodiment of the present invention, and Figure 10 schematically shows the aging information reduction step according to an embodiment of the present invention.

As shown in Figures 9 and 10, the aging information control unit 2540 calculates a concentration score based on the user's brain wave information measured while the user performs the first content. step; An activity score calculation step of calculating an activity score based on the user's pulse information and electromyography information measured while the user is performing the second content; A performance result score calculation step of calculating a first score, a second score, and a third score based on the performance result of the first content, the performance result of the second content, and the performance result of the third content, respectively; And performing an aging information reduction step of reducing aging information based on the sum of the first score, the second score, the third score, the concentration score, and the activity score,

In the concentration score calculation step, the score is calculated by determining which of the beta waves, theta waves, mid-alpha waves, and SMR waves the user's brain waves received from the brain wave measuring device 1300 contain, The concentration score is calculated by assigning weights when the frequency corresponds to an SMR wave of 12 to 13 Hz or a mid-alpha wave of 10 to 12 Hz, and the activity score calculation step is performed using the The activity score is calculated by determining the user's activity level based on the deviation value of the user's pulse or the EMG change amount received from the EMG sensor, where the pulse deviation value is greater than a preset standard or the EMG change amount is 216 to 450 When Hz, weighting is applied to calculate the activity score.

Specifically, as shown in FIG. 9 and with reference to the description of FIG. 8, the first content provider 2510 provides cognitive ability and concentration enhancement training content to the user, and the second content provider 2510 provides cognitive ability and concentration enhancement training content to the user. 2520 provides physical activity strengthening training content to the user, and the third content provider 2530 provides training content for body parts or cognitive abilities that require rehabilitation training to the user.

The aging information control unit 2540 calculates a first score based on the performance result of the user performing the first content, and calculates a second score based on the performance result of the user performing the second content. Calculate and perform a performance result score calculation step of calculating a third score based on the performance result of the user performing the third content.

The aging information control unit 2540 performs a concentration score calculation step of calculating a concentration score based on the user's brain wave information measured through the brain wave measuring device 1300 while the user is performing the first content, While performing the second content, an activity score calculation step is performed to calculate an activity score based on the user's pulse information and electromyography information measured through the pulse meter 1400.

In one embodiment of the present invention, in the concentration score calculation step, it is determined whether the user's brain wave is included in one of the beta waves, theta waves, mid-alpha waves, and SMR waves, and The concentration score can be calculated based on a concentration calculation formula that decreases as beta waves and theta waves increase, and increases as the mid-alpha waves and SMR waves increase.

In addition, the aging information control unit 2540 uses the user's pulse information measured through the pulse meter 1400 while the user is performing the second content and the electromyography sensor included in the pulse meter 1400. The activity score calculation step is performed to calculate the activity score based on the user's electromyography information measured through the user. Preferably, the activity score calculation step calculates the activity score by assigning weight when the deviation value of the user's pulse is greater than a preset standard or when the change in electromyography of the user is 216 to 450 Hz.

As shown in FIG. 10, the aging information control unit 2540 is configured to calculate the concentration score, the activity score, and the concentration score calculated through the concentration score calculation step, the activity score calculation step, and the performance result score calculation step. An aging information reduction step of reducing the user's aging information is performed based on the sum of the first score, the second score, and the third score.

The aging information reduced through the aging information reduction step is transmitted to the avatar appearance update unit 2550, and with reference to the description of FIG. 8, the avatar appearance update unit 2550 updates the first image of the avatar. The appearance of the avatar can be updated by adjusting the graphic effects.

Figure 11 schematically shows a process for changing the appearance of an avatar according to an embodiment of the present invention.

Schematically, as shown in FIG. 11, it shows the process of changing the appearance of the avatar according to the aging information reduction step shown in FIG. 10 and the aging information adjusted through the aging information reduction step. Meanwhile, for the sake of explanation, the process shown in FIG. 11 shows only the process of calculating the first score among the performance result score calculation steps, and in the performance result score calculation step and the aging information reduction step, as described above. , the first score, the second score, the third score, the concentration score, and the activity score are all calculated.

Specifically, after the user creates an avatar in virtual reality, the first content providing unit 2510 of the VR content providing unit 2500 provides first content to the avatar (S100). As described above, the first content corresponds to content intended to enhance the user's cognitive ability and concentration. The user performs the first content through the avatar within the virtual reality (S200).

When the aging degree control unit receives the performance result of the avatar performing the first content (S300), the aging degree control unit calculates a first score based on the performance result (S400). Perform.

The regular aging degree control unit adjusts the user's aging information with reference to the first score (S500). Preferably, step S500, as described above, is a sum of not only the first score, but also the first score, the second score, the third score, the concentration score, and the activity score. Based on this, the user's aging information is adjusted.

The aging degree control unit transmits the adjusted aging information to the avatar appearance update unit 2550 (S600), and the avatar appearance update unit 2550 updates the basic appearance of the avatar according to the received adjusted aging information. Adjust the applied first graphic effect (S700). Referring to the description of FIG. 6, the first graphic effect is the aging effect among a plurality of first graphic effects pre-stored in the computing system 10000 based on the aging information calculated by the aging information determination unit 2420. It corresponds to the first graphic effect corresponding to the aging information calculated by the information determination unit 2420, and when the avatar appearance update unit 2550 receives the adjusted aging information, the avatar appearance update unit 2550 The appearance of the avatar can be changed to a graphic effect corresponding to the adjusted aging information among a plurality of first graphic effects pre-stored in the computing system 10000 (S800).

Figure 12 schematically shows an execution screen of the first content according to an embodiment of the present invention.

Schematically, Figure 12 (a) shows an execution screen shown to the user when the first content is provided to the user, and Figure 12 (b) shows the execution screen after the user performs the first content. , shows an execution screen where the execution results are shown to the user. Meanwhile, FIG. 12 shows an execution screen of the first content shown to the user as an embodiment of the present invention, but FIG. 12 also shows an execution screen of the second content, the third content, or the unexpected content described later. An execution screen as shown in may be provided to the user through the HMD (1100).

Specifically, as shown in FIG. 12, the VR service unit may provide information to the user through a floating-interface. Figure 12(a) shows an interface provided through the HMD 1100 mounted on the user when the first content is provided to the user by the first content provider 2510. The user can select “Yes” or “No” in the floating interface shown in (a) of FIG. 12 by manipulating the VR device 1000. If the user selects “Yes,” the user The first content can be played by manipulating the VR device 1000 mounted on the user.

When the user performs the first content, the performance result can be displayed to the user through a floating interface as shown in (b) of FIG. 12. Meanwhile, the performance result item shown in (b) of FIG. 12 is an embodiment of the present invention, and the performance result item may differ depending on the type of content. For example, when the content provided to the user is walking content of the second content, walking time, distance, and moving speed may be specified as performance result items in the floating interface.

Figure 13 schematically shows a virtual reality interface shown to the user when the concentration score and activity score are less than the first reference value according to an embodiment of the present invention.

Schematically, Figure 12 (a) shows that the clarity of objects existing in the virtual reality or the field of view of the avatar is adjusted, and Figure 12 (b) shows an execution screen when unexpected content is provided to the user. shows.

As shown in FIG. 13, the VR content provider 2500, when the detailed sum value of the concentration score and the activity score during a preset section is less than a preset first reference value, the object existing in the virtual reality The clarity or the field of view of the avatar is adjusted, and when the detailed sum value of the concentration score and the activity score during a preset period is less than the first reference value, the first content provider 2510 and the second content provider (2520) Or, by providing unexpected content while the content is being provided by the third content provider 2530, when the user performs the unexpected content, the clarity of objects existing in the virtual reality or the field of view of the avatar returns to default.

Specifically, the VR content provider 2500 determines that the user's concentration has decreased when the detailed sum of the concentration score and the activity score during the preset period is calculated to be less than the preset first reference value. , the user can be notified in the following manner. As shown in the left drawing of FIG. 13 (a), the clarity of objects existing in the virtual reality provided through the HMD (1100) installed by the user is lowered, or as shown in the right drawing of FIG. 13 (a) As shown, the avatar's field of view can be adjusted through an opaque floating interface.

The VR content provider 2500 may provide unexpected content to the user when the detailed sum value is calculated to be less than the first reference value. The unexpected content is content provided to the user in order to improve the user's concentration on the VR content, and preferably provides content of relatively easy difficulty, such as the first content.

As shown in (b) of FIG. 13, when the unexpected content is provided to the user, referring to the description of FIG. 12, like the first to third contents, a floating interface is provided in virtual reality. The unexpected content may be provided to the user. If the detailed sum of the concentration score and activity score calculated when the user performs the unexpected content is calculated to be higher than the preset first reference value, the clarity of the object existing in the virtual reality is restored to the default value, and the opaque floating The interface disappears.

As an embodiment of the present invention, when the detailed sum of the concentration score and activity score when performing the unexpected content is less than the first reference value, the VR content provider 2500 displays the floating interface recommending rest. It can be provided to the user.

2. A system that creates an avatar, provides it to an external metaverse platform, and updates the avatar.

As described above, the system for generating the avatar of the present invention, providing it to the external metaverse platform (5000), and updating the avatar generates the avatar based on user information and based on performance results, concentration, and activity level according to content performance. You can update the appearance of your avatar. Meanwhile, the system for updating the avatar provides information about the avatar to an external metaverse platform 5000. Hereinafter, information about the avatar is provided to the external metaverse platform 5000 to enable interaction. The system for receiving information and updating the avatar will be described in detail.

The external metaverse platform (5000) often uses a method of providing a specific character to provide an avatar to the user. However, if you use an external metaverse platform (5000) with an avatar that reflects the user's information, you may feel more interested in the feeling of being in the external metaverse platform (5000). Additionally, the avatar and the user can feel a sense of unity within the external metaverse platform (5000). Therefore, we propose a method of updating and providing an avatar suitable for the external metaverse platform (5000) based on the avatar created in the present invention.

Figure 14 schematically shows the steps performed in the avatar update unit according to an embodiment of the present invention.

As shown in FIG. 14, information about the avatar is provided to the external metaverse platform 5000, and interaction information that the user interacts with using the avatar is received from the external metaverse platform 5000. , may include an avatar update unit that performs an update on the avatar, and the avatar update unit provides information about the avatar to an external metaverse platform 5000, and the external metaverse platform 5000 An avatar information providing step (S900) of receiving interaction information that the user has interacted with using the avatar, performing an update on the avatar, and providing information about the avatar to an external metaverse platform (5000); An interaction information reception step (S920) of receiving interaction information corresponding to the avatar from the external metaverse platform (5000); and an avatar update step (S930) of extracting information related to the avatar from the interaction information and updating the avatar based on the information related to the avatar.

Specifically, the avatar information provision step is a step of providing information about the avatar created in the present invention to the outside. As described above, the avatar is created based on the user's biometric information, activity information, and medical information received through the VR device, and some or all of the information about the avatar is provided to the external metaverse platform (5000). can do. The user's medical information reflected in the avatar may not be transmitted depending on security.

Next, in the external metaverse platform 5000 that has received the avatar information, the user uses the avatar, and interaction information is generated according to the user's use of the avatar. Specifically, interaction information may mean information that can update an avatar according to the usage situation of the external metaverse platform (5000), and may vary depending on the service provided by the external metaverse platform (5000). For example, interaction information may be different when the external metaverse platform 5000 provides medical services and when the external metaverse platform 5000 provides games.

According to an embodiment of the present invention, the avatar used by the user in the external metaverse platform 5000 is the avatar created in the present invention or the information about the avatar received by the external metaverse platform 5000 is reflected. It may correspond to an avatar provided by an external metaverse platform (5000).

The computing system that receives interaction information from the external metaverse platform 5000 performs an avatar update based on the interaction information. Specifically, the interaction information includes all information generated while the user uses the avatar within the external metaverse platform 5000, and only information related to the avatar is extracted from the interaction information and used. The avatar can be updated based on the extracted interaction information related to the avatar. According to one embodiment of the present invention, the information related to the avatar in the interaction information may vary depending on the service provided by the external metaverse platform (5000), so that the avatar is selected according to the characteristics of the external metaverse platform (5000). It can have the effect of updating .

Below, a case where the external metaverse platform 5000 provides a game will be described.

Figure 15 schematically shows interaction information according to an embodiment of the present invention.

As shown in FIG. 15, when the game is provided by the external metaverse platform 5000, the interaction information includes usage time, level-up information, number of collected items, number of worn items, number of acquired skills, and quests. The number of clears may be included.

Specifically, all information generated while a user plays a game using an avatar within the external metaverse platform (5000) is stored as interaction information, and when the external metaverse platform (5000) provides a game, , usage time when the user uses the avatar, level up information about the level of the game when the user uses the avatar, number of items collected by the user using the avatar, number of items worn by the user on the avatar, number of items worn by the user on the avatar, The number of acquired skills acquired for the avatar and the number of quests cleared by the user using the avatar in the external metaverse platform (5000) may be included in the interaction information, and the interaction information may be converted into information related to the avatar. Usage time, level-up information, number of collected items, number of worn items, number of skills acquired, and number of quests cleared can be extracted.

Figure 16 schematically shows the avatar update step (S930) based on aging information according to an embodiment of the present invention.

As shown in FIG. 16, the avatar update step (S930), when the game is provided by the external metaverse platform 5000, updates the aging based on level-up information, number of collected items, and number of quest clears. Information can be adjusted, and the avatar can be updated with respect to the avatar's skin condition information, hair condition information, degree of curvature of the skeleton, eye inclination, and mouth shape based on the aging information.

Specifically, in the present invention, aging information is generated based on some or all of the user's plural image information, the user's brain wave information, and the user's pulse information. In order to update the avatar suitable for the game provided by the external metaverse platform 5000, the aging information is adjusted based on some of the interaction information. Preferably, the information corresponding to the interaction information that adjusts the aging information may be the level-up information, the number of collected items, and the number of quests cleared.

The avatar is updated based on aging information adjusted to suit the game provided by the external metaverse platform 5000. All elements that can express aging of the avatar can be updated, but preferably, the avatar's skin condition information, hair condition information, degree of skeletal curvature, eye tilt, and mouth shape can be updated.

Figure 17 schematically shows an avatar update step (S930) based on movement speed according to an embodiment of the present invention.

As shown in FIG. 17, in the avatar update step (S930), when the game is provided by the external metaverse platform 5000, the movement speed is adjusted based on the usage time, number of acquired skills, and number of worn items. can be adjusted, and the avatar can be updated based on the movement speed.

Specifically, in the present invention, the moving speed of the avatar is determined by increasing the moving speed as the moving distance or number of steps increases, based on the moving distance or number of steps during a preset time derived from the user's smartphone. In order to update the avatar suitable for the game provided by the external metaverse platform 5000, the movement speed is adjusted based on some of the interaction information. Preferably, the avatar is updated based on usage time, number of acquired skills, and number of worn items.

Meanwhile, the movement speed adjusted based on interaction information can be increased as the usage time increases and the number of acquired skills increases, and the movement speed can be lowered as the number of worn items increases. Therefore, as shown in Figure 17, the movement speed has increased by '3' compared to before, which is due to some or all of the following: increased use time, increased number of acquired skills, or decreased number of worn items. It can mean.

The abilities of the avatar shown in FIG. 17 are an example of the present invention, and the actual abilities of the avatar of the present invention are not limited to those shown in FIG. 17.

3. A system for calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar to perform content

As described above, the system for calculating the optimal temperature of a user-customized realistic content test bed that creates an avatar and performs content according to the present invention generates an avatar based on user information and determines the performance results, concentration, and activity level according to content performance. Based on this, the appearance of the avatar can be updated, information about the avatar is provided to the external metaverse platform, and interaction information that the avatar interacts with within the external metaverse platform is received, and the avatar is updated based on this. do.

Meanwhile, the system for calculating the optimal temperature of the test bed provides the optimal environment by calculating the optimal temperature when the user performs the content, and determines the temperature of the test bed, which is the environment in which the user is performing the content. The system that allows control of the optimal temperature will be explained in detail.

In order to obtain the best results by executing the content provided by the external metaverse platform, it is necessary to provide the best environment to the user. In the test bed where the user performs the content, there may be elements that disturb the user, and the user's concentration may be low due to a disease.

For example, the user may feel uncomfortable with the HMD worn to perform content and may lose concentration, and the user may perform content with low concentration due to diseases such as mild cognitive impairment or ADHD. Therefore, we present a system for calculating the optimal test bed temperature that creates an optimal test bed environment for performing content so that the user can focus solely on the content.

Figure 18 schematically shows the process of deriving the optimal temperature and controlling the cooling and heating device 8000 according to an embodiment of the present invention.

As shown in FIG. 18, the system is for calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar and performs content, comprising: a camera device 1200 that captures the user's appearance to derive image information about the user; A brain wave measuring device (1300) attached to the user's head and capable of measuring the user's brain waves; The user image may include a heating and cooling device (8000) that measures and controls the test bed temperature within the test bed, includes a machine-learned inference model, and includes the user's facial expression received from the camera device (1200), Based on the user's brain wave information received from the brain wave measuring device 1300 and the machine learned inference model, an optimal temperature among a plurality of preliminary temperatures for the test bed temperature is derived, and based on the optimal temperature, the cooling and heating device ( 8000) may include an optimal temperature deriving unit that controls the temperature.

Specifically, when the user performs content provided by the external metaverse platform using the avatar or an avatar provided by the external metaverse platform in which information about the avatar is reflected, the optimal temperature deriving unit By controlling the temperature of the test bed where the user is present to the derived optimal temperature, it is possible to create an optimal environment.

More specifically, the optimal temperature derivation unit includes a machine-learned inference model, communicates with the camera device 1200 and the EEG meter 1300 to receive user images and EEG information, and derives the optimal temperature, The test bed temperature can be controlled based on the optimal temperature by controlling the cooling and heating device 8000.

The camera device 1200 is included in the VR device and can derive a user image by photographing the user's appearance while the user is performing content, and the user image can include the user's facial expression. The user's facial expression may refer to the facial expression shown by the user's lower jaw considering the user wearing the HMD, and may correspond to the user's mouth shape, preferably, the angle at which the corner of the user's mouth is raised and the degree of mouth opening. .

The brain wave measuring device is included in the VR device and can measure the user's brain wave information while the user is performing content, and the brain wave information may include beta waves, theta waves, alpha waves, and SMR waves, Alpha waves can be classified as mid-alpha waves depending on their frequency.

The machine learned inference model is included in the optimal temperature derivation unit and can calculate an expression score from the user image. The detailed process will be described later. The machine learned inference model according to an embodiment of the present invention can be implemented in a form that includes one or more of artificial neural network models such as convolutional neural network model (CNN), capsule network (CapsNet), and rule-based feature information extraction model. It can be learned through photos that include mouth shapes, such as photos of a person's face or mouth.

The optimal temperature deriving unit may derive an expression score from the user image, derive an EEG score from the EEG information, and calculate a sum score by adding the expression score and the EEG score to derive the optimal temperature. The optimal temperature may refer to one temperature among a plurality of preliminary temperatures, and the detailed process will be described later.

Subsequently, the optimal temperature deriving unit may communicate with the cooling and heating device 8000 and control the cooling and heating device 8000 based on the optimal temperature. The cooling and heating device 8000 is installed in the test bed and can measure and control the test bed temperature within the test bed.

According to an embodiment of the present invention, the cooling and heating device 8000 may refer to any device that can perform cooling and heating. For example, it may include a system air conditioner that can perform both cooling and heating with one device, or it may include both a heater that performs heating and a cooler that performs cooling, and the test bed temperature is adjusted at preset intervals. It can be measured, and it can be measured in real time.

Figure 19 schematically shows derivation of the preliminary temperature in the optimal temperature derivation unit according to an embodiment of the present invention.

As shown in FIG. 19, the optimal temperature deriving unit can derive the plurality of preliminary temperatures based on content provided by the external metaverse platform.

Specifically, the external metaverse platform may provide A content or/and B content, and the A content and the B content may correspond to different contents.

The optimal temperature deriving unit may derive preliminary temperature #1 to preliminary temperature #3 based on the A content. That is, the preliminary temperatures #1 to #3 were derived by reflecting the characteristics of the A content, and the preliminary temperatures #1 to #3 may correspond to ascending or descending order.

Additionally, for the B content, the optimal temperature derivation unit can derive preliminary temperature #4 to preliminary temperature #7 based on the B content. That is, the preliminary temperatures #4 to #7 were derived by reflecting the characteristics of the B content, and the preliminary temperatures #4 to #7 may correspond to ascending or descending order.

When comparing the A content and the B content, if the A content is a content that requires more movement than the B content, the preliminary temperature #1 to the preliminary temperature #3 are the preliminary temperature #4 to the preliminary temperature. It may correspond to a temperature lower than that of temperature #7.

More specifically, external metaverse platforms can provide users with various types of content, such as consultation content and game content. While the user is performing the consultation content or the game content, the optimal test bed temperature required by the user may vary. For example, if the game content is content that requires a lot of movement, the optimal environment can be created by maintaining a test bed temperature different from that of counseling content that does not require a lot of movement. Therefore, in the present invention, a plurality of preliminary temperatures corresponding to each of the different contents provided by the external metaverse platform can be derived.

According to one embodiment of the present invention, the preliminary temperature can be derived by deriving a preset number or by varying the number based on the content, and can be derived by using a preset rule or the machine learned inference model depending on the type of content. It can be derived using

According to an embodiment of the present invention, even if the preliminary temperature is the same content, if the characteristics are different depending on the content stage, the preliminary temperature can be derived differently for each stage. For example, when content A includes step A-1 (not shown) and step A-2 (not shown), step A-1 requires more movement than step A-2. In this case, the plurality of preliminary temperatures derived for step A-1 and step A-2 may be different from each other.

Figure 20 schematically shows steps performed by the optimal temperature deriving unit according to an embodiment of the present invention.

As shown in FIG. 20, the optimal temperature deriving unit controls the cooling and heating device 8000 to maintain the test bed temperature at one of the plurality of preliminary temperatures for a preset time (a preliminary temperature maintenance step ( S1000); While the test bed temperature is maintained at the preliminary temperature, a user image including the user's facial expression is received from the camera device 1200, and an facial expression score is calculated based on the user's facial expression (S1010). ; An EEG score calculation step of receiving the user's EEG information from the EEG meter 1300 and calculating an EEG score based on the EEG information while the test bed temperature is maintained at the preliminary temperature; And a summation score calculation step (S1020) of calculating a sum score by adding up the facial expression score and the brain wave score, and performing the preliminary temperature maintenance step (S1000) to the sum score calculation step for each of the plurality of preliminary temperatures. (S1030); can be performed.

Specifically, in the preliminary temperature maintenance step (S1000), one of the plurality of preliminary temperatures provided by the external metaverse platform and derived based on the content performed by the user may be maintained for a preset time.

The plurality of preliminary temperatures may be applied in descending or ascending order, and the preliminary temperature applied first may be selected in consideration of the energy efficiency of the cooling and heating device 8000 when compared to the remaining preliminary temperatures. Preferably, When the plurality of preliminary temperatures are arranged in descending or ascending order, the preliminary temperature to be applied first can be selected as the preliminary temperature arranged first or last.

According to an embodiment of the present invention, by maintaining the preset time, the total score can be calculated while the user is sufficiently influenced by the preliminary temperature while performing the content.

While the test bed temperature is maintained at the preliminary temperature, the optimal temperature calculation unit may perform the facial expression score calculation step (S1010) and the brainwave score calculation step (S1020).

In the facial expression score calculation step (S1010), a user image of the user is received from the camera device 1200, and the user image may include the user's facial expression. The user's facial expression may correspond to the facial expression of the user's mouth, and the optimal temperature derivation unit may input the user image into the machine-learned inference model to calculate an facial expression score. The detailed process will be described later.

The EEG score calculation step (S1020) receives the user's EEG information from the EEG measuring device 1300, and the EEG information may correspond to the frequency of the EEG and be classified according to frequency. The optimal temperature deriving unit may calculate an EEG score based on the EEG information. The detailed process will be described later.

Subsequently, the optimal temperature deriving unit may perform a summation score calculation step (S1030) in which the expression score and the brain wave score are added to calculate the summation score for the corresponding preliminary temperature.

According to an embodiment of the present invention, by calculating the expression score and the brain wave score and calculating the sum score, it is possible to derive the optimal temperature considering both the user's facial expression and brain wave information.

In addition, the optimal temperature deriving unit performs the preliminary temperature maintaining step (S1000) to the summed score calculating step (S1030) for each of the plurality of preliminary temperatures to derive a summed score corresponding to each of the plurality of preliminary temperatures. You can. In other words, there may be as many summed scores as the number of preliminary temperatures.

Figure 21 schematically shows the optimal temperature derivation step according to an embodiment of the present invention.

Figure 21 (A) corresponds to a diagram showing the total score derived from each of a plurality of preliminary temperatures.

As shown in (A) of FIG. 21, the optimal temperature deriving unit may perform the preliminary temperature maintenance step (S1000) to the sum score calculation step (S1030) for each of the plurality of preliminary temperatures.

Specifically, when a plurality of preliminary temperatures derived based on content provided by an external metaverse platform correspond to preliminary temperature #1 to preliminary temperature #N, the optimal temperature deriving unit determines the preliminary temperature #1 to preliminary temperature #N. For each temperature #N, the preliminary temperature maintenance step (S1000) to the sum score calculation step (S1030) can be performed to calculate sum score #1 to sum score #N. The summed score corresponding to the preliminary temperature #1 is the summed score #1, the summed score corresponding to the preliminary temperature #2 is the summed score #2, and the summed score corresponding to the preliminary temperature #N is the summed score #N. .

More specifically, while the optimal temperature deriving unit controls the cooling and heating device 8000 based on the preliminary temperature #1 to maintain the test bed temperature at the preliminary temperature #1 for a preset time, the brain wave measuring device 1300 ), calculate the EEG score #1, receive the user image #1 from the camera device 1200, and input the user image #1 into the machine-learned inference model to determine the facial expression. Score #1 can be calculated, and a total score #1 can be derived by adding the brain wave information #1 and the expression score #1.

Thereafter, while maintaining the temperature of the test bed temperature, which is the preliminary temperature #1, at the preliminary temperature #2, the optimal temperature deriving unit controls the heating and cooling device 8000 based on the preliminary temperature #2 to perform the test. While maintaining the bed temperature at the preliminary temperature #2 for a preset time, the brain wave information #2 is received from the brain wave measuring device 1300, the brain wave score #2 is calculated, and the user image is received from the camera device 1200. Upon receiving #2, the user image #2 is input into the machine-learned inference model to calculate the expression score #2, and the brain wave information #2 and the expression score #2 are added to derive the sum score #2. can do.

By performing the above process up to the preliminary temperature #N, the sum score #1 to sum score #N can be derived.

Figure 21 (B) corresponds to a diagram showing the optimal temperature derivation step.

As shown in (B) of FIG. 21, the optimal temperature deriving unit further performs an optimal temperature deriving step, and the optimal temperature deriving step is the highest among the summed scores calculated for each of the plurality of preliminary temperatures. The preliminary temperature corresponding to the sum of points can be derived as the optimal temperature, and the cooling and heating device 8000 can be controlled based on the optimal temperature.

Preliminary temperature #2 corresponding to the total score #2, which is the largest value among the total score #1 to the total score #N, can be derived as the optimal temperature.

Specifically, when the total score #1 to the total score #N are sorted in descending order, the total score #2 is at the front, that is, the total score #, which is the highest total score among the total score #1 to the total score #N. By deriving the preliminary temperature #2 used to derive 2 as the optimal temperature, the cooling and heating device 8000 can be controlled to maintain the test bed temperature at the preliminary temperature #2.

Figure 22 schematically shows the facial expression score calculation step and the brainwave score calculation step according to an embodiment of the present invention.

Figure 22 (A) corresponds to a diagram showing the expression score calculation step.

As shown in (A) of FIG. 22, the facial expression score calculation step inputs the user image received from the camera device 1200 into a learned inference model to derive user facial expressions including mouth opening and mouth corners, A score is calculated according to the state of the mouth opening and the mouth corner, and the expression score can be calculated by assigning weight as the angle at which the mouth corner rises and the mouth opening become larger.

Specifically, the user image received from the camera device 1200 may include the user's face, and the user's face may include the user's facial expression. The user's facial expression may include various facial expressions that can indicate the user's emotions, but preferably, considering the user wearing the HMD in the present invention, it may correspond to the facial expression that appears from the user's mouth. By inputting the user image into the machine-learned inference model, information about the user's mouth shape is obtained, and the mouth shape (open mouth or corner of the mouth) corresponding to the four types (A to D) shown in FIG. 22 is obtained. can be derived.

In (A) of Figure 22, A may correspond to a flat corner of the mouth, B may correspond to a lowered corner of the mouth, C may correspond to a raised corner of the mouth, and D may correspond to an open mouth.

Considering the tendency of the mouth to open when concentrating and the tendency for the corners of the mouth to rise when in a good mood, the expression score can be calculated by giving weight to C (raised corners of the mouth) and D (open mouth).

More specifically, by analyzing the mouth corner angle of C (raised mouth corner) compared to the mouth corner angle of A (flat mouth corner), greater weight can be assigned as the mouth corner angle of C is larger, and D (open mouth corner) By analyzing, the larger the mouth opening size, the greater weight can be assigned.

The user's facial expression shown in FIG. 22 is an embodiment of the present invention, and is not limited to the user's facial expression shown in FIG. 22.

Figure 22 (B) corresponds to a diagram showing the EEG score calculation step.

As shown in (B) of FIG. 22, in the EEG score calculation step, the user's EEG information received from the EEG measuring device 1300 is included in any of the waveforms of beta wave, theta wave, mid-alpha wave, and SMR wave. The score can be calculated by determining whether the user's brain wave frequency is an SMR wave of 12 to 13 Hz or a mid-alpha wave of 10 to 12 Hz, and the brain wave score can be calculated by applying weight.

Specifically, the brain wave information received from the brain wave measuring device 1300 may include theta waves, alpha waves, SMR waves, and beta waves, and the alpha waves may include mid-alpha waves depending on the frequency. In the present invention, considering that 10 to 12 Hz included in the mid-alpha wave and 12 to 13 Hz included in the SMR are brain waves that appear when concentration is exerted, 10 to 12 Hz included in the mid-alpha wave and 12 included in the SMR are The EEG score can be calculated by giving weights to 13 Hz.

According to an embodiment of the present invention, the user image and brain wave information that are the basis for calculating the facial expression score calculation step and the EEG score calculation step may be information acquired at the same time. The brain wave information may correspond to brain wave information measured for a preset time while the test bed temperature is maintained at the preliminary temperature, and the user image may be multiple or one user image acquired while the brain wave information is measured. It may apply to

According to another embodiment of the present invention, the user image acquires a plurality of images during the preset time and uses the average value of a plurality of expression scores calculated by inputting each of the plurality of images into the machine learned inference model. You can.

Figure 23 exemplarily shows the internal configuration of a computing device according to an embodiment of the present invention.

As shown in FIG. 23, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, and an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600). At this time, the internal configuration of the computing device 11000 may include the computing system shown in FIG. 1, or the computing device 11000 may correspond to an embodiment of the computing system shown in FIG. 1, and the input /It can be connected to the VR service unit 2000 and the additional content unit 4000 by the output subsystem 11400. The processor 11100 may be comprised of a plurality of processors and may include the first processor module and the second processor module described above.

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. . The memory 11200 may include software modules, instruction sets, or other various data necessary for the operation of the computing device 11000.

At this time, access to the memory 11200 from other components such as the processor 11100 or the peripheral device interface 11300 may be controlled by the processor 11100.

The peripheral interface 11300 may couple input and/or output peripherals of the computing device 11000 to the processor 11100 and the memory 11200. The processor 11100 may execute a software module or set of instructions stored in the memory 11200 to perform various functions for the computing device 11000 and process data.

The input/output subsystem can couple various input/output peripherals to the peripheral interface 11300. For example, the input/output subsystem may include a controller for coupling peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touch screen or sensor to the peripheral device interface 11300. According to another aspect, input/output peripherals may be coupled to the peripheral interface 11300 without going through the input/output subsystem.

Power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or a power source. It may contain arbitrary other components for creation, management, and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, if necessary, the communication circuit 11600 may include an RF circuit to transmit and receive RF signals, also known as electromagnetic signals, to enable communication with other computing devices.

This embodiment of FIG. 23 is only an example of the computing device 11000, and the computing device 11000 omits some components shown in FIG. 23, further includes additional components not shown in FIG. 23, or 2 It may have a configuration or arrangement that combines more than one component. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or sensor in addition to the components shown in FIG. 23, and the communication circuit 11600 may be equipped with various communication methods (WiFi, 3G, LTE). , Bluetooth, NFC, Zigbee, etc.) may also include a circuit for RF communication. Components that can be included in the computing device 11000 may be implemented as hardware, software, or a combination of both hardware and software, including an integrated circuit specialized for one or more signal processing or applications.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to this embodiment may be composed of a PC-based program or a mobile terminal-specific application. The application to which the present invention is applied can be installed on the computing device 11000 through a file provided by a file distribution system. As an example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request from the computing device 11000.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used by any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent. Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (10)

A system for calculating the optimal temperature for a user-customized realistic content test bed that creates an avatar to perform content,
An HMD installed on the user to provide images and sounds for virtual reality and detect the user's voice; A camera device that acquires a user image including the user's facial expression and derives image information of the user by photographing the user's appearance; A brain wave measuring device attached to the user's head and capable of measuring the user's brain waves; A pulse meter attached to the user's wrist and capable of measuring the user's pulse; and a motion sensing device that measures the user's movements; a VR device including a;
A heating and cooling device that measures and controls the test bed temperature within the test bed;
a VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, creates an avatar for the user, and provides a plurality of virtual reality contents to the user;
an avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about a user's interaction using the avatar in the external metaverse platform, and performs an update on the avatar; and
The test bed temperature is based on the user image, including a machine-learned inference model, and the user's facial expression received from the camera device, the user's EEG information received from the EEG meter, and the machine-learned inference model. An optimal temperature deriving unit that derives an optimal temperature among a plurality of preliminary temperatures and controls the air conditioning and heating device based on the optimal temperature,
The VR service department,
a virtual reality implementation unit that implements a virtual reality that is available only to the user and in which the plurality of virtual reality contents are provided;
a VR communication unit that communicates with the VR device to exchange information;
User's biometric information received from the VR device; and a user information input unit that receives user information including activity information and medical information received from the user's smartphone;
an avatar creation unit that creates an avatar for the user based on the user information in the virtual reality implemented by the virtual reality implementation unit; and
A VR content provider capable of providing the plurality of contents to the avatar created in the virtual reality and updating the appearance of the avatar each time this is performed,
The biometric information is,
Image information received through the camera device; The user's brain wave information measured by the brain wave measuring device; And the user's pulse information measured with the pulse meter;
The optimal temperature deriving part,
A preliminary temperature maintenance step of controlling the air conditioning and heating device to maintain the test bed temperature at one of the plurality of preliminary temperatures for a preset time;
While the test bed temperature is maintained at the preliminary temperature, an expression score calculation step of receiving a user image including a user's expression from the camera device and calculating an expression score based on the user's expression;
While the test bed temperature is maintained at the preliminary temperature, an EEG score calculation step of receiving the user's EEG information from the EEG measuring device and calculating an EEG score based on the EEG information; and
Performing a sum score calculation step of calculating a sum score by adding up the expression score and the brain wave score,
Performing the preliminary temperature maintenance step to the sum score calculation step for each of the plurality of preliminary temperatures,
The expression score calculation step is,
The user image received from the camera device is input to the learned inference model to derive user facial expressions including mouth opening and the corners of the mouth, and a score is calculated according to the state of the mouth opening and the corners of the mouth, including the angle at which the corners of the mouth are raised and the A system for calculating the optimal test bed temperature that calculates the expression score by assigning weight to the larger the mouth opening.
In claim 1,
The optimal temperature deriving part,
A system for calculating the optimal test bed temperature, which derives the plurality of preliminary temperatures based on content provided by the external metaverse platform.
delete In claim 1,
The optimal temperature deriving unit further performs an optimal temperature deriving step,
The optimal temperature derivation step is,
Derive the preliminary temperature corresponding to the highest summed score among the summed scores calculated for each of the plurality of preliminary temperatures as the optimal temperature, and calculate the test bed optimal temperature for controlling the air conditioning and heating device based on the optimal temperature. system for.
delete In claim 1,
The EEG score calculation step is,
The score is calculated by determining which waveform of the user's brain wave information received from the brain wave measuring device includes beta waves, theta waves, mid-alpha waves, and SMR waves, where the frequency of the user's brain waves is 12 to 13 Hz. A system for calculating the optimal temperature of the test bed, which calculates the brain wave score by assigning weights when it corresponds to mid-alpha waves of 10 to 12 Hz.
In claim 1,
The avatar update unit,
An avatar information provision step of providing information about the avatar to an external metaverse platform;
An interaction information receiving step of receiving interaction information corresponding to the avatar from the external metaverse platform; and
A system for calculating the optimal temperature of a test bed, which performs an avatar update step of extracting information related to the avatar from the interaction information and updating the avatar based on the information related to the avatar.
In claim 1,
The avatar creation unit,
Each external feature information is derived from a plurality of the image information, a character determination parameter in the form of a one-dimensional vector is derived by averaging the plurality of derived external feature information, and based on the character determination parameter, the basic appearance of the avatar A basic appearance determination unit that determines;
an aging information determination unit that inputs the biometric information into one or more artificial neural network models to determine aging information of the avatar, and adds a first graphic effect corresponding to the aging information to the basic appearance;
A movement speed determination unit that determines the movement speed of the avatar by increasing the movement speed as the movement distance or number of steps increases, based on the movement distance or number of steps for a preset time derived from the user's smartphone; and
A disease that determines the disease information of the avatar based on medical information including at least one medical history entered by the user, and adds a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added. A system for calculating the optimal temperature of the test bed, including an information determination unit.
In claim 8,
The VR content provision department,
A first content provider that determines the level of difficulty based on the aging information of the user and provides content that can perform training to strengthen the user's cognitive ability and concentration;
a second content provider that determines the level of difficulty based on the activity information of the user and provides content that allows the user to perform physical activity strengthening training;
A third content provider that determines the type of content based on the disease information of the user and provides content that can provide training for body parts or cognitive abilities that require rehabilitation training;
Performance results according to the performance of the content provided by the first content provider, the second content provider, and the third content provider, EEG information measured by the EEG measurement device during the performance period for the content, and an aging information control unit that calculates one or more scores based on pulse information measured by the pulse meter and adjusts the aging information according to the one or more scores; and
An avatar appearance update unit that updates the appearance of the user's avatar by adjusting the first graphic effect according to the aging information adjusted by the aging information control unit.
As a method for calculating the optimal temperature of a test bed implemented in a system for calculating the optimal temperature of a user-customized realistic content test bed that creates an avatar and performs content,
The system is,
An HMD installed on the user to provide images and sounds for virtual reality and detect the user's voice; A camera device that acquires a user image including the user's facial expression and derives image information of the user by photographing the user's appearance; A brain wave measuring device attached to the user's head and capable of measuring the user's brain waves; A pulse meter attached to the user's wrist and capable of measuring the user's pulse; and a motion sensing device that measures the user's movements; a VR device including a;
A heating and cooling device that measures and controls the test bed temperature within the test bed;
a VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, creates an avatar for the user, and provides a plurality of virtual reality contents to the user;
an avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about a user's interaction using the avatar in the external metaverse platform, and performs an update on the avatar; and
The test bed temperature is based on the user image, including a machine-learned inference model, and the user's facial expression received from the camera device, the user's EEG information received from the EEG meter, and the machine-learned inference model. An optimal temperature deriving unit that derives an optimal temperature among a plurality of preliminary temperatures and controls the air conditioning and heating device based on the optimal temperature,
The above method is,
A virtual reality implementation step of implementing a virtual reality that is available only to the user and in which a plurality of virtual reality contents are provided by the VR service department;
A VR communication step of exchanging information by communicating with the VR device by the VR service unit;
The user's biometric information received from the VR device by the VR service unit; and a user information input step of receiving user information including activity information and medical information received from the user's smartphone;
An avatar creation step of generating, by the VR service unit, an avatar of the user based on the user information in the virtual reality implemented by the virtual reality implementation step; and
A VR content providing step of providing, by the VR service unit, the plurality of contents to an avatar created in the virtual reality, and updating the appearance of the avatar each time this is performed,
The biometric information is,
Image information received through the camera device; The user's brain wave information measured by the brain wave measuring device; And the user's pulse information measured with the pulse meter;
The optimal temperature deriving part,
A preliminary temperature maintenance step of controlling the air conditioning and heating device to maintain the test bed temperature at one of the plurality of preliminary temperatures for a preset time;
While the test bed temperature is maintained at the preliminary temperature, an expression score calculation step of receiving a user image including a user's expression from the camera device and calculating an expression score based on the user's expression;
While the test bed temperature is maintained at the preliminary temperature, an EEG score calculation step of receiving the user's EEG information from the EEG measuring device and calculating an EEG score based on the EEG information; and
Performing a sum score calculation step of calculating a sum score by adding up the expression score and the brain wave score,
Performing the preliminary temperature maintenance step to the sum score calculation step for each of the plurality of preliminary temperatures,
The expression score calculation step is,
The user image received from the camera device is input to the learned inference model to derive user facial expressions including mouth opening and the corners of the mouth, and a score is calculated according to the state of the mouth opening and the corners of the mouth, including the angle at which the corners of the mouth are raised and the A method for calculating the optimal test bed temperature that calculates the expression score by assigning weight to the larger the mouth opening.

Images