KR102835552B1 - System And Method For Sensory Effects In Which The Humidity Of The Space Where The User Exists Is Controlled By The Humidity Of The Content Environment Performed By The Generated Avatar - Google Patents

Abstract

The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by an avatar created by generating the humidity, wherein the system receives the user's biometric information, activity information, and medical information through a VR device, and creates an avatar in virtual reality based on the biometric information, activity information, and medical information, and provides information about the avatar to an external metaverse platform, and when the user performs content provided by the external metaverse platform, the humidity of the environment in the content is derived as the content humidity, and the real humidity of the real space in which the user exists is controlled based on the content humidity. The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by a generated avatar.

Description

{System and method for sensory effects in which the humidity of the space where the user exists is controlled by the humidity of the content environment performed by the generated avatar}

The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by an avatar created by generating the humidity, wherein the system receives the user's biometric information, activity information, and medical information through a VR device, and creates an avatar in virtual reality based on the biometric information, activity information, and medical information, and provides information about the avatar to an external metaverse platform, and when the user performs content provided by the external metaverse platform, the humidity of the environment in the content is derived as the content humidity, and the real humidity of the real space in which the user exists is controlled based on the content humidity. The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by a generated avatar.

Realistic content refers to digital content that applies realistic technology to virtual reality, augmented reality, and mixed reality to maximize the five senses of humans and provide experiences similar to reality. According to global consulting firm PWC, XR, a virtual fusion technology that encompasses VR, AR, etc. as a realistic content-based technology, is expected to create an economic ripple effect of 476.4 billion dollars (approximately 534 trillion won) in 2025.

Virtual reality is a technology that allows users to experience a virtual space implemented on a computer by wearing an input/output device such as an HMD (Head Mounted Display). It is a technology that allows users to experience specific situations in a virtual space implemented similarly to the real world without temporal or spatial constraints. It was predicted that this virtual reality technology would have difficulty becoming popular in the near future due to the expensive equipment and lack of virtual reality content. However, recently, with the spread of VR and AR devices and the increase in demand for non-face-to-face content, virtual reality technology is also being re-examined.

Metaverse technology is a four-dimensional virtual space-time that can continue economic, social, cultural, and political activities through a virtual self, an avatar. The above metaverse technology is a concept that includes virtual reality, augmented reality, a mirror world, and life logging. This metaverse technology is being utilized in various technological fields such as games, education, shopping, medical care, and tourism, and specifically, platforms such as games, education, shopping, medical care, and tourism implemented in the metaverse can be experienced through avatars.

Accordingly, it is expected that the use of real-time technology in the content provided by the metaverse will maximize the user experience by breaking down the constraints of time and space in the physical world.

As a method for building a healthcare environment for promoting the health of users, Korean Patent Publication No. 10-2020-0022776 proposes a method for creating a 4D avatar for the user and inducing the performance of specific movements so that the user can perform rehabilitation exercises without visiting a hospital. However, the conventional technology simply estimates the user's movement information based on the user's joint information and creates a user avatar that can experience virtual reality, and does not disclose a method for providing content by considering the user's biometric characteristics or creating an avatar similar to the user to enhance the user's immersion. In addition, the above-mentioned conventional technologies do not provide a realistic effect between the environment within the provided content and the actual environment in which the user exists.

Accordingly, there is an emerging need to develop a system that can create an avatar similar to the user's appearance and physical characteristics, provide information about the avatar to an external metaverse platform, and control the space in which the user exists with the humidity of the environment within the content to achieve a realistic effect when performing content provided by the external metaverse platform.

Republic of Korea Patent No. 10-1943585 (January 23, 2019) Republic of Korea Publication Patent No. 10-2130750 (June 30, 2020)

The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by an avatar created by generating the humidity, wherein the system receives the user's biometric information, activity information, and medical information through a VR device, and creates an avatar in virtual reality based on the biometric information, activity information, and medical information, and provides information about the avatar to an external metaverse platform, and when the user performs content provided by the external metaverse platform, the humidity of the environment in the content is derived as the content humidity, and the real humidity of the real space in which the user exists is controlled based on the content humidity. The present invention relates to a system and method for a realistic effect in which the humidity of a space in which a user exists is controlled by the humidity of an environment in which a content is performed by a generated avatar.

In order to solve the above-mentioned problem, a system for a realistic effect that controls the humidity of a content environment in which a user exists by creating an avatar, the system comprising: a VR device including: an HMD that is worn by a user and provides images and sounds of virtual reality and detects the user's voice; a camera device that photographs the user's appearance to derive image information of the user; an electroencephalograph that is attached to the user's head and can measure brain waves of the user; a pulse meter that is attached to the user's wrist and can measure the user's pulse; and a motion sensing device that measures the user's movements; a humidity control device that measures and controls the humidity of a real environment in which the user exists; a VR service unit that performs communication with a VR device worn by the user, provides information about virtual reality to the VR device, and creates an avatar for the user to provide a plurality of virtual reality contents to the user; an avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information on an interaction that the user has made using the avatar on the external metaverse platform, and performs an update on the avatar; And a humidity control unit including a machine-learned inference model, and deriving content humidity of a content environment in the content image based on the machine-learned inference model and the content image of the content provided by the external metaverse platform, and controlling the humidity control device based on the content humidity to control the real humidity of the real environment in which the user exists; and the VR service unit includes a virtual reality implementation unit that implements a virtual reality that only the user can use and in which the plurality of virtual reality contents are provided; a VR communication unit that communicates with the VR device to exchange information; a user information input unit that receives user information including the user's biometric information received from the VR device; and activity information and medical information received from the user's smartphone; an avatar creation unit that creates an avatar of the user based on the user information in the virtual reality implemented by the virtual reality implementation unit; and a VR content provision unit that provides the plurality of contents to the avatar created in the virtual reality and can update the appearance of the avatar whenever this is performed; and the biometric information includes image information input through the camera device; brainwave information of the user measured by the brainwave measuring device; And the system for realistic effect is provided, including the user's pulse information measured by the pulse meter.

In one embodiment of the present invention, the inference model can extract a plurality of frames from the input content image according to a preset rule, and derive the content humidity of each frame from each of the plurality of frames.

In one embodiment of the present invention, the humidity control unit includes a first humidity control unit, and the first humidity control unit can perform a first content image receiving step of receiving the content image from an external metaverse platform; a first content humidity derivation step of inputting the content image into the machine-learned inference model to derive a plurality of content humidity and deriving an average value of the plurality of content humidity as the first content humidity; a first reality humidity receiving step of receiving a reality humidity measured in a real environment in which the user exists from the humidity control device; and a first reality humidity control step of controlling the humidity control device so that the reality humidity reaches the first content humidity.

In one embodiment of the present invention, when performing content that moves from a specific location to another location, the first reality humidity control step may stop the operation of the humidity control device for reaching the first content humidity of the specific location when the content progress rate at the specific location reaches a preset progress rate.

In one embodiment of the present invention, the humidity control unit includes a second humidity control unit, and the second humidity control unit can perform a second content image receiving step of receiving the content image from an external metaverse platform; a second content humidity deriving step of inputting the content image into the machine-learned inference model to derive a plurality of content humidity, and deriving the highest humidity as the second content humidity and the lowest humidity as the third content humidity among the plurality of content humidity; a second reality humidity receiving step of receiving the reality humidity measured in the real environment in which the user exists from the humidity control device; and a second reality humidity control step of controlling the humidity control device so that the reality humidity is maintained between the second content humidity and the third content humidity.

In one embodiment of the present invention, when performing content that moves from a specific location to another location, the second reality humidity control step may stop the operation of the humidity control device to maintain the humidity of the specific location between the second content humidity and the third content humidity when the content progress rate at the specific location reaches a preset progress rate.

In one embodiment of the present invention, the avatar update unit may perform an avatar information providing step of providing information about the avatar to an external metaverse platform; an interaction information receiving step of receiving interaction information corresponding to the avatar from the external metaverse platform; and an avatar update step of extracting information related to the avatar from the interaction information and updating the avatar based on the information related to the avatar.

In one embodiment of the present invention, the avatar generation unit may include a basic appearance determination unit that derives appearance feature information from each of a plurality of the image information, derives a character determination parameter in the form of a one-dimensional vector by averaging the derived plurality of appearance feature information, and determines a basic appearance of the avatar based on the character determination parameter; an aging information determination unit that inputs the biometric information into one or more artificial neural network models to determine aging information of the avatar, and adds a first graphic effect corresponding to the aging information to the basic appearance; a movement speed determination unit that determines the movement speed of the avatar in a manner that the movement speed increases as the movement distance or the number of steps increases based on a movement distance or the number of steps derived from a smartphone of the user for a preset period of time; and a disease information determination unit that determines disease information of the avatar based on medical information including one or more medical histories input by the user, and adds a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added.

In one embodiment of the present invention, the VR content providing unit may include: a first content providing unit, which provides content whose difficulty is determined based on the aging information of the corresponding user and which enables the user to perform cognitive ability and concentration enhancement training; a second content providing unit, which provides content whose difficulty is determined based on the activity information of the corresponding user and which enables the user to perform physical activity enhancement training; a third content providing unit, which provides content whose type is determined based on the disease information of the corresponding user and which enables the user to perform training for a body part requiring rehabilitation training or cognitive ability; an aging information controlling unit, which calculates one or more scores based on performance results according to performance of content provided by the first content providing unit, the second content providing unit, and the third content providing unit, brainwave information measured by the brainwave measuring device during the performance period of the content, and pulse information measured by the pulse measuring device, and adjusts the aging information according to the one or more scores; and an avatar appearance updating unit, which updates the appearance of the avatar of the corresponding user by adjusting the first graphic effect according to the aging information adjusted by the aging information controlling unit.

In order to solve the above-mentioned problem, a method for a realistic effect implemented in a system for a realistic effect in which humidity of a content environment is controlled by the humidity of a space in which a user exists, which is created by an avatar, is provided, the system comprising: a VR device including an HMD which is worn by a user and provides images and sounds of virtual reality and detects the user's voice; a camera device which photographs the user's appearance and derives image information of the user; an electroencephalograph which is attached to the user's head and can measure brain waves of the user; a pulse meter which is attached to the user's wrist and can measure the user's pulse; and a motion sensing device which measures the user's movements; a humidity control device which measures and controls the humidity of a real environment in which the user exists; a VR service unit which performs communication with the VR device worn by the user, provides information on virtual reality to the VR device, and generates an avatar of the user to provide a plurality of virtual reality contents to the user; an avatar update unit which provides information on the avatar to an external metaverse platform, receives interaction information on an interaction in which the user interacts using the avatar on the external metaverse platform, and performs an update on the avatar; And a humidity control unit including a machine-learned inference model, and deriving content humidity of a content environment in the content image based on the machine-learned inference model and the content image of the content provided by the external metaverse platform, and controlling the real humidity of the real environment in which the user exists based on the content humidity; the method comprises: a virtual reality implementation step of implementing a virtual reality in which only the user can use and a plurality of virtual reality contents are provided by the VR service unit; a VR communication step of communicating with the VR device and exchanging information by the VR service unit; a user information input step of receiving, by the VR service unit, user information including the user's biometric information received from the VR device; and activity information and medical information received from the user's smartphone; an avatar creation step of creating, by the VR service unit, an avatar of the user based on the user information in the virtual reality implemented by the virtual reality implementation unit; And a VR content providing step for providing the plurality of contents to an avatar created in the virtual reality by the VR service unit, and updating the appearance of the avatar each time the same is performed; Including; The method for a realistic effect is provided, wherein the biometric information includes: image information input through the camera device; brainwave information of the user measured by the brainwave measuring device; and pulse information of the user measured by the pulse measuring device.

FIG. 1 schematically illustrates the configuration of a system for creating an avatar and providing multiple contents in virtual reality according to one embodiment of the present invention.
Figure 2 schematically illustrates the configuration of a VR service unit according to one embodiment of the present invention.
FIG. 3 schematically illustrates a VR device worn by a user using virtual reality and its configuration according to one embodiment of the present invention.
FIG. 4 schematically illustrates a process of deriving multiple external feature information for creating a user's avatar according to one embodiment of the present invention.
Figure 5 schematically illustrates the operation of an avatar generation unit according to one embodiment of the present invention.
Figure 6 schematically illustrates a detailed configuration of an aging information determination unit according to one embodiment of the present invention.
Figure 7 schematically illustrates a user's avatar and the avatar's abilities according to one embodiment of the present invention.
Figure 8 schematically illustrates the configuration of a VR content providing unit according to one embodiment of the present invention.
Figure 9 schematically illustrates the execution steps in the VR content section according to one embodiment of the present invention.
Figure 10 schematically illustrates an aging information reduction step according to one embodiment of the present invention.
Figure 11 schematically illustrates a process for changing the appearance of an avatar according to one embodiment of the present invention.
Figure 12 schematically illustrates an execution screen of the first content according to one embodiment of the present invention.
FIG. 13 schematically illustrates a virtual reality interface shown to a user when the concentration score and activity score are below a first reference value according to one embodiment of the present invention.
Figure 14 schematically illustrates the execution steps in an avatar update unit according to one embodiment of the present invention.
Figure 15 schematically illustrates interaction information according to one embodiment of the present invention.
Figure 16 schematically illustrates an avatar update step based on aging information according to one embodiment of the present invention.
Figure 17 schematically illustrates an avatar update step based on movement speed according to one embodiment of the present invention.
Figure 18 schematically illustrates a humidity control unit and a humidity control device according to one embodiment of the present invention.
Figure 19 schematically illustrates derivation of content humidity of a machine-learned inference model according to one embodiment of the present invention.
Figure 20 schematically illustrates steps performed by a first humidity control unit according to one embodiment of the present invention.
Figure 21 schematically illustrates steps performed by a second humidity control unit according to one embodiment of the present invention.
FIG. 22 schematically illustrates a humidity control method when moving a location within content according to one embodiment of the present invention.
FIG. 23 exemplarily illustrates the internal configuration of a computing device according to one embodiment of the present invention.

Hereinafter, various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more of the aspects. It will be recognized, however, by one skilled in the art that such aspect(s) may be practiced without these specific details. The following description and the annexed drawings set forth specific exemplary aspects of one or more of the aspects in detail. It should be understood, however, that these aspects are exemplary and that any of the various methods of the principles of the various aspects may be utilized, and the description is intended to encompass all such aspects and their equivalents.

Additionally, various aspects and features will be presented by means of systems that may include a number of devices, components, and/or modules, etc. It is also to be understood and appreciated that various systems may include additional devices, components, and/or modules, etc., and/or may not include all of the devices, components, modules, etc. discussed in connection with the drawings.

The terms "embodiment," "example," "aspect," and "example" as used herein are not to be construed as implying that any aspect or design described is better or advantageous over other aspects or designs. The terms "part," "component," "module," "system," and "interface," as used below, generally refer to computer-related entities, and can refer to, for example, hardware, a combination of hardware and software, or software.

Additionally, it should be understood that the terms "comprises" and/or "comprising" imply the presence of the features and/or components, but do not preclude the presence or addition of one or more other features, components and/or groups thereof.

Also, terms including ordinal numbers such as first, second, etc. may be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and/or includes a combination of a plurality of related described items or any item among a plurality of related described items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant technology, and shall not be interpreted in an ideal or overly formal meaning unless explicitly defined in the embodiments of the present invention.

The virtual reality rehabilitation system performed in the social server of the present invention can be implemented in a form in which an avatar is created that mimics the user's appearance and status, and the user controls the avatar to perform multiple contents. Hereinafter, a system for creating an avatar in virtual reality and providing multiple contents will be described.

1. A system that creates avatars in virtual reality and provides multiple contents.

FIG. 1 schematically illustrates the configuration of a system for creating an avatar and providing multiple contents in virtual reality according to one embodiment of the present invention.

As illustrated in FIG. 1, a system for creating an avatar in virtual reality and providing a plurality of contents includes: a VR device (1000) including: an HMD that is worn by a user and provides images and sounds of virtual reality and detects the user's voice; a camera device that photographs the user's appearance to derive image information of the user; an electroencephalograph that is attached to the user's head and can measure brain waves of the user; a pulse meter that is attached to the user's wrist and can measure the user's pulse; a motion sensing device that measures the user's movement; and an eye tracker that detects the user's eye movement; and a VR service unit that performs communication with the VR device (1000) worn by the user, provides information about virtual reality to the VR device (1000), and creates an avatar of the user and provides a plurality of virtual reality contents to the user; wherein the VR service unit comprises: a virtual reality implementation unit (2100) that implements a virtual reality that only the user can use and in which the plurality of virtual reality contents are provided; a VR communication unit (2200) that communicates with the VR device (1000) to exchange information; A user information input unit (2300) for receiving user information including the user's biometric information received from the VR device (1000); and activity information and medical information received from the user's smartphone; an avatar generation unit (2400) for generating an avatar of the user based on the user information in a virtual reality implemented by the virtual reality implementation unit (2100); and a VR content provision unit (2500) for providing the plurality of contents to the avatar generated in the virtual reality and updating the appearance of the avatar whenever the same is performed; wherein the biometric information includes image information input through the camera device; the user's brainwave information measured by the brainwave measuring device; the user's pulse information measured by the pulse measuring device; and the user's eye tracking information measured by the eye tracker.

In brief, the system for creating an avatar and providing multiple contents in the virtual reality includes the VR device (1000) and the VR service unit, and the VR service unit is included in the computing system (10000).

Specifically, the VR device (1000) includes one or more components that sense information that can determine the appearance and ability of an avatar created in the VR service unit by being worn by a user or by photographing the appearance of the user, and the components include, as described above, the HMD (Head Mounted Display), the camera device, the EEG measuring device, the pulse measuring device, the motion sensing device, and the eye tracker. In addition, the VR device (1000) can transmit the user's biometric information, including the image information, the EEG information, the pulse information, and the eye tracking information sensed through each component of the VR device (1000), to the VR service unit. A more detailed description of the VR device (1000) will be described later with reference to FIG. 3.

The biometric information sensed through the VR device (1000) is transmitted to the computing system (10000). The VR service unit of the computing system (10000) implements virtual reality, creates an avatar of the user within the virtual reality based on the biometric information, and provides the user with multiple contents through the avatar.

Meanwhile, as an embodiment of the present invention, the computing system (10000) can communicate with a social server (3000), as illustrated in FIG. 1, and the user can communicate with one or more other users within the social server (3000) through an avatar created in the VR service unit. In addition, as another embodiment of the present invention, the computing system (10000) can include an additional content unit (4000) that provides other content in addition to the service provided by the VR service unit, and preferably, the system for creating an avatar in the virtual reality and providing a plurality of contents can include the social server (3000) and the additional content unit (4000) at the same time.

Figure 2 schematically illustrates the configuration of a VR service unit according to one embodiment of the present invention.

As illustrated in FIG. 2, the VR service unit includes a virtual reality implementation unit (2100) that implements a virtual reality that is available only to the user and in which the plurality of virtual reality contents are provided; a VR communication unit (2200) that communicates with the VR device (1000) to exchange information; a user information input unit (2300) that receives user information including the user's biometric information received from the VR device (1000); and activity information and medical information received from the user's smartphone; an avatar creation unit (2400) that creates an avatar of the user based on the user information in the virtual reality implemented by the virtual reality implementation unit (2100); and a VR content provision unit (2500) that provides the plurality of contents to the avatar created in the virtual reality and can update the appearance of the avatar whenever this is performed.

Specifically, the virtual reality implementation unit (2100) can implement virtual reality within the computing system (10000) and transmit information about the virtual reality to the user through the VR communication unit (2200). The information about the virtual reality includes visual elements such as the interface and display of the virtual reality; and auditory elements such as sound effects; and the audiovisual elements can be transmitted to the user through the HMD of the VR device (1000).

As described above, the VR communication unit (2200) transmits information about the virtual reality implemented by the virtual reality implementation unit (2100) to the user, and receives motion information and biometric information sensed by the VR device (1000) to the computing system (10000). The motion information and biometric information are used to create the user's avatar or perform multiple contents within the virtual reality.

The biometric information received through the VR device (1000) is transmitted to the biometric information receiving unit (2310) of the user information input unit (2300). Meanwhile, the user information input unit (2300), as illustrated in Fig. 2, further includes an activity information receiving unit (2320) and a medical information receiving unit (2330) in addition to the biometric information receiving unit (2310). The activity information receiving unit (2320) and the medical information receiving unit (2330) each receive the user's activity information and the user's medical information from the user's smartphone, respectively.

The above activity information corresponds to exercise records recorded at preset intervals in the user's smartphone, lifestyle information recorded at preset intervals, etc., and includes, for example, the number of steps per day, the number of steps per day, the amount and number of times of water drunk per day, bedtime and wake-up time, etc. According to one embodiment of the present invention, the activity information can be received through a wearable device capable of recording the activity information. In addition, the medical information includes information on one or more medical records input by the user, and as one embodiment of the present invention, can be generated using a medical record provided by a medical institution.

A plurality of image information created by capturing the user's face image and body image through the camera device of the VR device (1000) is received by the biometric information receiving unit (2310) through the VR communication unit (2200), and the avatar generating unit (2400) creates the user's avatar in the virtual reality based on the plurality of image information.

The above avatar generation unit (2400) includes a basic appearance determination unit (2410) that determines the basic appearance of the avatar, an aging information determination unit (2420) that determines aging information of the user based on a plurality of image information and changes a part of the appearance of the avatar, a movement speed determination unit (2430) that determines the movement speed of the avatar based on the activity information of the user, and a disease information determination unit (2440) that changes a part of the appearance of the avatar based on the medical information. That is, the avatar generation unit (2400) sets the appearance and ability values of the avatar based on the user's biometric information, activity information, and medical information, and implements the avatar in the virtual reality.

The above VR content provider (2500) provides a plurality of contents to the avatar within the virtual reality. The plurality of contents include: contents capable of performing cognitive ability and concentration enhancement training for the corresponding user; contents capable of performing physical activity enhancement training for the corresponding user; and contents capable of performing training for a body part or cognitive ability requiring rehabilitation training for the corresponding user. A more detailed description of the VR content provider (2500) will be described later.

FIG. 3 schematically illustrates a VR device (1000) worn by a user using virtual reality according to one embodiment of the present invention and its configuration.

Schematically, (a) of FIG. 3 illustrates an example in which the user wears the VR device (1000), (b) of FIG. 3 illustrates the configuration of the HMD (1100) and the EEG measuring device (1300) mounted on the head of the user, and (c) of FIG. 3 illustrates the configuration of the VR device (1000).

As illustrated in FIG. 3, the VR device (1000) includes an HMD (1100) that is worn by a user to provide images and sounds of virtual reality and detect the user's voice; a camera device (1200) that photographs the user's appearance and derives image information of the user; an electroencephalograph (1300) that is attached to the user's head and can measure the user's brain waves; a pulse meter (1400) that is attached to the user's wrist and can measure the user's pulse; a motion sensing device (1500) that measures the user's movements; and an eye tracker (1600) that detects the user's eye movements.

Specifically, (a) of FIG. 3 corresponds to an embodiment in which the user wears the VR device (1000), and the camera device (1200) is not illustrated separately. It is preferable for the user to capture a face image and a body image of the user through the camera device (1200) before the user wears the VR device (1000) illustrated in (a) of FIG. 3, and the camera device (1200) can transmit the face image and the body image to the biometric information receiving unit (2310).

As illustrated in (a) of FIG. 3, the pulse meter (1400) can be attached to the user's wrist to measure the user's pulse. That is, the pulse meter (1400) can measure the user's pulse, which is one of the bio-information, and the user's pulse measured in this manner can be input into the VR service unit and used to analyze the user's condition or understand the user's psychology so that the user can smoothly perform content, or can be used to generate content provided in virtual reality. Additionally, the pulse meter (1400) can include an electromyography sensor to sense the user's pulse information and the user's electromyography information and transmit them to the VR service.

As shown in (a) of Fig. 3, the motion sensing device (1500) can measure the motion information of the user within the preset space by measuring the movement of the user within the preset space. The motion information of the user measured in this way is input to the VR service unit and can be used as a basis for determining multiple contents provided by the VR service unit or can be used for the user to experience the multiple contents.

The above motion sensing device (1500), as illustrated in (a) of FIG. 3, includes a sensor module that can sense the movement of the user in the preset space by holding it in each of the user's two hands, and as an embodiment of the present invention, may include a sensor module that can measure the movement of the user in the preset space as an image, such as an infrared sensor or a camcorder. Meanwhile, according to another embodiment of the present invention, the motion sensing device (1500) is in a form that is attached to the user's body, such as an acceleration sensor or a gyro sensor, and can sense the movement of the user in the preset space. Preferably, the preset space corresponds to a space in which a position sensor module that measures the coordinates of the motion sensing device (1500) is provided.

Fig. 3 (b) illustrates a detailed configuration of the HMD (1100). As illustrated in Fig. 3 (b), the HMD (1100) is connected to the brainwave measuring device (1300), and the brainwave measuring device (1300) can identify the user's brainwave, which is one of the bio-information. The user's brainwave measured in this manner is input to the VR service unit, and can be used to analyze the user's state or understand the user's psychology so that the user can smoothly perform the content, or can be used to generate the content provided in virtual reality.

The HMD (1100) includes a visual provision unit (1110) that provides visual elements of the virtual reality to the user; an auditory provision unit (1120) that provides auditory elements of the virtual reality to the user; and a microphone unit (not shown) that detects the user's voice.

A VR device (1000) according to one embodiment of the present invention provides information about the virtual reality to the user through the visual providing unit (1110) and the auditory providing unit (1120), thereby allowing the user to have an experience similar to existing in the virtual space of the virtual reality, and receives the user's voice and actions through the voice input unit and the motion sensing device (1500), thereby allowing the user's avatar implemented in the virtual reality to move or speak.

The above-described visual provision unit (1110) provides visual information to each of the user's eyes to provide a three-dimensional effect, as shown in (b) of FIG. 3, and additionally includes an eye tracker (1600) that detects eye movements of the user. In one embodiment of the present invention, the eye movements of the user can be detected and utilized for diagnosis of the user. In addition, it is preferable that the eye tracker (1600) be located on the edge of the visual provision unit (1110) so as not to interfere with the visual provision unit (1110) from providing visual information to the user.

As shown in (c) of FIG. 3, the VR device (1000) includes, as described above, the HMD (1100), the camera device (1200), the electroencephalograph (1300), the pulse meter (1400), the motion sensing device (1500), and the eye tracker (1600).

Figure 4 schematically illustrates the operation of the basic external shape determination unit (2410) according to one embodiment of the present invention.

As illustrated in FIG. 4, the basic appearance determination unit (2410) derives appearance feature information from each of the plurality of image information, averages the derived plurality of appearance feature information to derive a character determination parameter in the form of a one-dimensional vector, and determines the basic appearance of the avatar based on the character determination parameter.

Specifically, the VR service unit communicating with the VR device (1000) includes the avatar generation unit (2400), and the avatar generation unit (2400) includes the basic appearance determination unit (2410), the aging information determination unit (2420), the movement speed determination unit (2430), and the disease information determination unit (2440), as described above with reference to FIG. 2. As illustrated in FIG. 4, the basic appearance determination unit (2410) receives the user's image information input through the camera device (1200) of the VR device (1000), extracts a face image and a body image based on the user's image information, and derives a plurality of pieces of appearance feature information from the extracted face image and the body image, respectively.

The above image information may be a dynamic image of the user including facial expression changes that can recognize the user's emotions, or a plurality of static images of the user continuously recorded at a certain cycle. In particular, the image information should be interpreted in the broadest sense to include having two or more image frames. Accordingly, the basic appearance determination unit (2410) classifies the received image information into a plurality of frames, and extracts a face image and a body image from each frame. Thereafter, appearance feature information can be derived based on the extracted face image. The appearance feature information includes feature information on unique characteristics that can recognize facial expressions, such as the inclination of the eyes, the inclination of the eyebrows, the shape of the mouth, and the inclination of the mouth, and additionally includes the user's skin condition information, hair condition information, and the degree of bone curvature.

Meanwhile, Fig. 4, as an embodiment of the present invention, only illustrates the process of extracting the face image from the plurality of frames, and the basic appearance determination unit (2410) can extract the body image from the plurality of frames in the same manner as illustrated in Fig. 4. At this time, preferably, the VR service unit can strictly determine the state of the user's skeleton by requesting various postures from the user.

Figure 5 schematically illustrates the operation of an avatar generation unit (2400) according to one embodiment of the present invention.

As illustrated in FIG. 5, the avatar generation unit (2400) includes a basic appearance determination unit (2410) that derives appearance characteristic information from each of the plurality of image information, derives a character determination parameter in the form of a one-dimensional vector by averaging the derived plurality of appearance characteristic information, and determines a basic appearance of the avatar based on the character determination parameter; an aging information determination unit (2420) that inputs the biometric information into one or more artificial neural network models to determine aging information of the avatar, and adds a first graphic effect corresponding to the aging information to the basic appearance; a movement speed determination unit (2430) that determines the movement speed of the avatar in a manner that the movement speed increases as the movement distance or the number of steps increases based on a movement distance or the number of steps derived from a smartphone of the user for a preset time; and a disease information determination unit (2440) that determines disease information of the avatar based on medical information including one or more medical histories input by the user, and adds a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added.

The above-mentioned external feature information includes the user's skin condition information, hair condition information, and degree of bone curvature.

Specifically, the basic appearance determination unit (2410) can derive multiple appearance feature information of the user as described above with reference to FIG. 4, and derive a character determination parameter in the form of a one-dimensional vector by averaging the derived multiple appearance feature information. The character determination parameter is a parameter that sets one or more appearance information of the avatar for the basic appearance of the avatar, and is a parameter including, for example, the height of the avatar, the skeleton or the degree of skeleton bending, skin color, skin condition, hair color, and hair condition of the avatar, and additionally, as described above, can further include feature information on unique characteristics that can recognize facial expressions, such as the inclination of the eyes, the inclination of the eyebrows, the shape of the mouth, and the inclination of the mouth. The basic appearance determination unit (2410) can implement the basic appearance of the avatar similar to the face and body shape of an actual user in virtual reality through the character determination parameter as described above.

When the basic appearance of the avatar is determined in the basic appearance determination unit (2410), the aging information determination unit (2420) of the avatar generation unit (2400) determines the aging information of the user and reflects it in the appearance graphic of the avatar. The aging information determination unit (2420) includes one or more artificial neural networks, and can determine the aging information of the user by inputting the biometric information into each artificial neural network model. When the aging information is determined, the aging information determination unit (2420) derives a first graphic effect corresponding to the aging information and adds the first graphic effect to the basic appearance of the avatar. A more detailed description of the aging information determination unit (2420) will be described later.

The disease information determination unit (2440) adds a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added. The disease information is generated based on the medical information received from the user's smartphone. The disease information determination unit (2440) receives the medical information through the medical information receiving unit (2330) and generates the disease information based on the medical information. Thereafter, the second graphic effect corresponding to the disease information is generated. For example, a graphic effect in which the left shoulder is processed in red may be added to the avatar of a user who has adhesive capsulitis in the left shoulder, or a graphic effect in which the right knee is processed in red may be added to the avatar of a user who has rheumatoid arthritis in the right knee. Meanwhile, the disease information may affect not only the appearance of the avatar but also the abilities of the avatar, which will be described later. In this way, the avatar generation unit (2400) implements the user's avatar in virtual reality based on the user's biometric information and disease information.

Figure 6 schematically illustrates the detailed configuration of an aging information determination unit (2420) according to one embodiment of the present invention.

As illustrated in FIG. 6, the aging information determination unit (2420) includes a first artificial neural network (2421) that receives image information input through the camera device (1200) and outputs first aging information; a second artificial neural network (2422) that receives brainwave information of the user measured by the brainwave measuring device (1300) and outputs second aging information; a third artificial neural network (2423) that receives pulse information of the user measured by the pulse measuring device (1400) and outputs third aging information; and a comprehensive judgment module (2424) that derives final aging information by adding the first aging information, the second aging information, and the third aging information.

Specifically, the aging information determination unit (2420) according to one embodiment of the present invention can determine the aging information of the user based on at least one of the user's image information, brain wave information, and pulse information. At this time, the aging information determination unit (2420) can determine the user's aging information by analyzing the user's image information, brain wave information, and pulse information through at least one artificial neural network.

As illustrated in FIG. 6, the first artificial neural network (2421) to the third artificial neural network (2423) respectively receive the user's image information, brain wave information, and pulse information, perform analysis, transmit the analysis results to the comprehensive diagnosis module, and the comprehensive diagnosis module receives the analysis results to determine and output the user's aging information. Based on the aging information output in this manner, with reference to the description of FIG. 5, the aging information determination unit (2420) derives a first graphic effect corresponding to the aging information, and adds the first graphic effect to the basic appearance of the avatar.

In the above first artificial neural network (2421), the user's skin condition information, hair condition information, and degree of bone curvature among the plurality of external appearance characteristic information derived from the basic external appearance determination unit (2410) are analyzed to derive the first aging information. In addition, the user's external appearance-related aging information can be analyzed based on the performance results for various postures requested from the user by the VR service unit, and the first aging information is derived through this.

The second artificial neural network (2422) above analyzes brainwave information derived from the brainwave measuring device (1300) of the VR device (1000) to derive second aging information. In one embodiment of the present invention, the second aging information is derived by analyzing the brainwave information based on the stability of the brainwaves and the concentration of the user.

The third artificial neural network (2423) analyzes pulse information derived from the pulse meter (1400) of the VR device (1000) to derive third aging information. The pulse information includes the user's pulse rate per minute, pulse regularity, and pulse pressure. The third artificial neural network (2423) can calculate the user's expected age and cardiovascular age based on the pulse information.

The above comprehensive judgment module (2424) can calculate the final aging information by adding up the first aging information, the second aging information, and the third aging information. Meanwhile, the aging information can be derived as a score or level, such as, as an embodiment of the present invention, setting the aging information when the user's aging level is the lowest as 1 and setting the aging information when the user's aging level is the highest as 10. Preferably, the comprehensive diagnosis module can be configured to automatically adjust the weighting for the image information, the brain wave information, and the pulse information to derive the aging information, and can configure a model to learn about such weighting.

Accordingly, the aging information determination unit (2420) can calculate the final aging information and add a first graphic effect corresponding to the aging information among a plurality of first graphic effects pre-stored in the computing system (10000) to the basic appearance of the avatar.

Figure 7 schematically illustrates a user's avatar and the avatar's abilities according to one embodiment of the present invention.

As illustrated in FIG. 7, the avatar generation unit (2400) includes a movement speed determination unit (2430) that determines the movement speed of the avatar in such a way that the movement speed increases as the movement distance or number of steps increases based on the movement distance or number of steps for a preset time derived from the smartphone of the user. Meanwhile, the ability value of the avatar illustrated in FIG. 7 is an embodiment of the present invention, and the ability value of the actual avatar of the present invention is not limited to that illustrated in FIG. 7.

Specifically, the avatar generation unit (2400) can set the avatar's ability based on the user's activity information and medical information received from the user's smartphone. As shown in Fig. 7, the avatar's ability includes the avatar's movement speed, muscle mass, and cognitive ability.

The movement speed of the avatar can be calculated based on the activity information and medical information of the user. For example, data indicating that the user walked an average of 3 km per day for 2 hours during a preset period of time is recorded on the user's smartphone, and the data can be transmitted to the computing system (10000). The movement speed determination unit (2430) can calculate the movement speed of the avatar based on the data. If, in the above-described example, an exercise record for a preset period of time is not regularly input into the smartphone, or data indicating that a short distance was walked for a long period of time is input into the smartphone, the movement speed determination unit (2430) can set the movement speed of the avatar to be low based on the data, and as an opposite example, if a regular exercise record is input into the smartphone, or a large amount of exercise is recorded into the smartphone, the movement speed determination unit (2430) can set the movement speed of the corresponding avatar to be high.

In addition, the movement speed of the avatar can be determined by referring to disease information generated based on the medical information of the user. For example, if the user has a disease in the lower body, the movement speed of the avatar is set low, and the degree of the movement speed set low is determined according to the severity of the disease.

By the above avatar generation unit (2400), the muscle mass of the avatar can be determined based on the medical information of the corresponding user. As one embodiment of the present invention, the muscle mass of the avatar can be determined based on the actual body composition data of the user included in the medical information of the user, and as another embodiment of the present invention, with reference to the description of (a) of FIG. 3, the muscle mass of the avatar can be set through the electromyography sensor included in the pulse measuring device (1400).

By the avatar generation unit (2400), the cognitive ability of the avatar can be calculated based on the user's brainwave information sensed by the brainwave measuring device (1300) equipped on the user, and more specifically, can be calculated based on the brainwave information measured when the user performs the plurality of contents described below, and additionally, when the user performs the plurality of contents through the avatar, can be calculated with reference to the brainwave information; and the performance process and results of the plurality of contents.

Figure 8 schematically illustrates the configuration of a VR content provider (2500) according to one embodiment of the present invention.

As illustrated in FIG. 8, the VR content provider (2500) comprises: a first content provider (2510) which provides content whose difficulty is determined based on the aging information of the user and which enables the user to perform cognitive ability and concentration enhancement training; a second content provider (2520) which provides content whose difficulty is determined based on the activity information of the user and which enables the user to perform physical activity enhancement training; a third content provider (2530) which provides content whose type is determined based on the disease information of the user and which enables the user to perform training for a body part requiring rehabilitation training or the user's cognitive ability. An aging information control unit (2540) that calculates a score of 1 or more based on the performance result according to content performance for the content provided by the first content provider (2510), the second content provider (2520), and the third content provider (2530), the brainwave information measured by the brainwave measuring device (1300) during the performance period for the content, and the pulse information measured by the pulse measuring device (1400), and adjusts the aging information according to the score of 1 or more; An avatar appearance update unit (2550) that updates the appearance of the avatar of the corresponding user by adjusting the first graphic effect according to the aging information adjusted by the aging information control unit (2540);

The first content provided to the user in the first content provider (2510) is content for enhancing the user's cognitive ability and concentration, including finding the difference, arithmetic, and solving puzzles; the second content provided to the user in the second content provider (2520) is content for enabling the user to perform physical activity strengthening training, including walking, stretching, and squats; and the third content provided to the user in the third content provider (2530) is content for enabling the user to perform intensive training on a body part that has been injured, or for enabling the user to perform intensive training for enhancing cognitive ability, including the user's memory, judgment, language ability, and ability to perceive space and time, to train areas in which the user is vulnerable.

Specifically, the first content provided to the user from the first content provider (2510) is content provided so that the user can adapt well to virtual reality, and can be provided as practice content for the user to perform multiple contents within the virtual reality through the present invention. That is, the user can learn how to operate the user's avatar within the virtual reality through the first content.

In addition, the VR content provider (2500) can evaluate the user's basic cognitive ability and concentration level by providing the first content to the user, and furthermore, the user can enhance the user's cognitive ability and concentration level by performing the first content. Meanwhile, as an embodiment of the present invention, the user's brainwave information measured by the brainwave measuring device (1300) can be displayed as a separate interface within the virtual reality to enhance the user's sense of immersion.

The second content provided to the user in the second content provider (2520) is content provided to strengthen the user's insufficient physical ability, and corresponds to content that the user who learned how to operate the VR device (1000) through the first content can perform through a predetermined physical activity. Since the user must actually use the user's body to perform the second content within the virtual reality, it can have the effect of performing physical activity for the user's disease treatment or rehabilitation training.

The VR content provider (2500) can evaluate the user's basic physical ability by providing the second content to the user. In this case, it is preferable that the user's physical ability evaluated is used as a reference for determining the difficulty and type of the second content provided next, as one embodiment of the present invention.

The third content provided to the user by the third content provider (2530) above is content provided to the user to strengthen a body part or cognitive ability that requires rehabilitation training, such as content that allows a user who has an injury to a body part including the waist, legs, and arms or a specific disease to perform training to strengthen the body part; or content that allows a user who has a deficiency in memory, judgment, language ability, and the ability to perceive time and space to perform training to strengthen the ability.

Meanwhile, the first to third contents described above each include one or more contents, and as an embodiment of the present invention, it is preferable to provide different contents to the user every day.

The above aging information control unit (2540) controls the aging information of the avatar based on data that the user can obtain by performing the first content, the second content, and the third content, and the avatar appearance update unit (2550) controls the first graphic effect according to the aging information controlled by the aging information control unit (2540) to update the appearance of the avatar. A more detailed description of the operations of the aging information control unit (2540) and the avatar appearance update unit (2550) will be provided later.

FIG. 9 schematically illustrates an execution step in a VR content section according to one embodiment of the present invention, and FIG. 10 schematically illustrates an aging information reduction step according to one embodiment of the present invention.

As shown in FIGS. 9 and 10, the aging information control unit (2540) performs a concentration score calculation step for calculating a concentration score based on the user's brainwave information measured while the user performs the first content; an activity score calculation step for calculating an activity score based on the user's pulse information and electromyogram information measured while the user performs the second content; a performance result score calculation step for calculating a first score, a second score, and a third score based on the performance results of the first content, the performance results of the second content, and the performance results of the third content, respectively; and an aging information reduction step for reducing aging information based on the sum of the first score, the second score, the third score, the concentration score, and the activity score.

The above concentration score calculation step calculates a score by determining which waveform of beta wave, theta wave, mid-alpha wave, and SMR wave the user's brain waves received from the brainwave measuring device (1300) belong to, and when the user's brain waves correspond to SMR waves having a frequency of 12 to 13 Hz or mid-alpha waves having a frequency of 10 to 12 Hz, a weight is assigned to calculate the concentration score, and the activity score calculation step determines the user's activity based on the deviation value of the user's pulse received from the pulse measuring device (1400) or the amount of electromyography (EMG) change received from the electromyography (EMG) sensor, and calculates the activity score, and when the deviation value of the pulse is equal to or greater than a preset standard or the amount of electromyography change is 216 to 450 Hz, a weight is assigned to calculate the activity score.

Specifically, as illustrated in FIG. 9, referring to the description of FIG. 8, the first content provider (2510) provides cognitive ability and concentration enhancement training content to the user, the second content provider (2520) provides physical activity enhancement training content to the user, and the third content provider (2530) provides training content for a body part or cognitive ability requiring rehabilitation training to the user.

The above aging information control unit (2540) performs a performance result score calculation step of calculating a first score based on the performance result of the user performing the first content, calculating a second score based on the performance result of the user performing the second content, and calculating a third score based on the performance result of the user performing the third content.

The above-mentioned aging information control unit (2540) performs a concentration score calculation step of calculating a concentration score based on the user's brainwave information measured by the brainwave measuring device (1300) while the user performs the first content, and performs an activity score calculation step of calculating an activity score based on the user's pulse information and electromyography information measured by the pulse measuring device (1400) while the user performs the second content.

In one embodiment of the present invention, in the concentration score calculation step, it is determined whether the user's brain waves are included in any of the beta waves, theta waves, mid-alpha waves, and SMR waves, and the concentration score can be calculated based on a concentration calculation formula in which the brain waves decrease as the beta waves and theta waves increase, and increase as the mid-alpha waves and the SMR waves increase.

In addition, the aging information control unit (2540) performs the activity score calculation step of calculating an activity score based on the pulse information of the user measured through the pulse meter (1400) while the user is performing the second content and the electromyography information of the user measured through the electromyography sensor included in the pulse meter (1400). Preferably, the activity score calculation step calculates the activity score by assigning a weight when the deviation value of the pulse of the user is equal to or greater than a preset standard or when the electromyography change amount of the user is between 216 and 450 Hz.

As illustrated in FIG. 10, the aging information control unit (2540) performs an aging information reduction step for reducing the aging information of the user based on the sum of the concentration score, the activity score, the first score, the second score, and the third score calculated through the concentration score calculation step, the activity score calculation step, and the performance result score calculation step.

The aging information reduced through the above aging information reduction step is transmitted to the avatar appearance update unit (2550), and with reference to the description of FIG. 8, the avatar's appearance can be updated by adjusting the first graphic effect of the avatar through the avatar appearance update unit (2550).

Figure 11 schematically illustrates a process for changing the appearance of an avatar according to one embodiment of the present invention.

Schematically, as illustrated in FIG. 11, the process of changing the appearance of the avatar according to the aging information reduction step illustrated in FIG. 10 and the aging information adjusted through the aging information reduction step is illustrated. Meanwhile, the process illustrated in FIG. 11 illustrates only the process of calculating the first score among the performance result score calculation steps for explanation, and in the performance result score calculation step and the aging information reduction step, as described above, the first score, the second score, the third score, the concentration score, and the activity score are all calculated.

Specifically, after the user creates an avatar in the virtual reality, the first content provider (2510) of the VR content provider (2500) provides the first content to the avatar (S100). The first content corresponds to content for enhancing the user's cognitive ability and concentration, as described above. The user performs the first content through the avatar in the virtual reality (S200).

When the aging level control unit receives the performance result of the above-mentioned avatar performing the above-mentioned first content (S300), the aging level control unit performs the performance result score calculation step of calculating the first score (S400) based on the performance result.

The constant aging degree control unit controls the aging information of the user with reference to the first score (S500). Preferably, the step S500 controls the aging information of the user based on the sum of not only the first score, but also the first score, the second score, the third score, the concentration score, and the activity score, as described above.

The above-mentioned aging degree control unit transmits the controlled aging information to the above-mentioned avatar appearance update unit (2550) (S600), and the above-mentioned avatar appearance update unit (2550) controls the first graphic effect applied to the basic appearance of the above-mentioned avatar according to the received controlled aging information (S700). The above first graphic effect, with reference to the description of FIG. 6, corresponds to a first graphic effect corresponding to the aging information calculated by the aging information determination unit (2420) among a plurality of first graphic effects pre-stored in the computing system (10000) based on the aging information calculated by the aging information determination unit (2420), and when the avatar appearance update unit (2550) receives the adjusted aging information, the avatar appearance update unit (2550) can change the appearance of the avatar to a graphic effect corresponding to the adjusted aging information among a plurality of first graphic effects pre-stored in the computing system (10000) (S800).

Figure 12 schematically illustrates an execution screen of the first content according to one embodiment of the present invention.

In outline, (a) of FIG. 12 illustrates an execution screen shown to a user when the first content is provided to the user, and (b) of FIG. 12 illustrates an execution screen in which the result of execution is shown to the user after the user executes the first content. Meanwhile, FIG. 12 illustrates an execution screen of the first content shown to the user as an embodiment of the present invention, but the execution screens of the second content, the third content, or the surprise content described below may also be provided to the user through the HMD (1100) as an execution screen as illustrated in FIG. 12.

Specifically, as illustrated in FIG. 12, the VR service unit can provide information to the user through a floating interface. FIG. 12 (a) illustrates an interface provided through an HMD (1100) worn by the user when the first content is provided to the user by the first content provider (2510). The user can select “Yes” or “No” in the floating interface illustrated in FIG. 12 (a) by operating the VR device (1000), and if the user selects “Yes,” the user can perform the first content by operating the VR device (1000) worn by the user.

When the above user performs the first content, the performance result can be specified to the user through a floating interface as shown in (b) of Fig. 12. Meanwhile, the performance result items shown in (b) of Fig. 12, as an embodiment of the present invention, may differ depending on the type of content. For example, when the content provided to the user is walking content of the second content, the walking time, distance, and movement speed can be specified as performance result items in the floating interface.

FIG. 13 schematically illustrates a virtual reality interface shown to a user when the concentration score and activity score are below a first reference value according to one embodiment of the present invention.

In outline, (a) of Fig. 12 illustrates that the clarity of an object existing in the virtual reality or the field of view of the avatar is adjusted, and (b) of Fig. 12 illustrates an execution screen when sudden content is provided to the user.

As illustrated in FIG. 13, the VR content provider (2500) adjusts the clarity of an object existing in the virtual reality or the field of view of an avatar when the detailed sum of the concentration score and the activity score during a preset section is less than a first reference value, and provides a burst content while content is being provided by the first content provider (2510), the second content provider (2520), or the third content provider (2530), so that when the user performs the burst content, the clarity of an object existing in the virtual reality or the field of view of an avatar is returned to a default value.

Specifically, if the sum of the concentration score and the activity score during the preset section is calculated to be less than the preset first reference value, the VR content provider (2500) may determine that the user's concentration has decreased and may notify the user in the following manner. As illustrated in the left drawing of (a) of Fig. 13, the clarity of an object existing in the virtual reality provided through the HMD (1100) worn by the user may be lowered, or, as illustrated in the right drawing of (a) of Fig. 13, the field of view of the avatar may be adjusted through an opaque floating interface.

The VR content provider (2500) may provide sudden content to the user when the detailed sum is calculated to be less than the first reference value. The sudden content is content provided to the user to improve the user's concentration on the VR content, and preferably, like the first content, provides content with a relatively easy difficulty level.

As illustrated in (b) of Fig. 13, when the sudden content is provided to the user, the sudden content can be provided to the user by providing a floating interface in virtual reality, like the first to third contents, with reference to the description of Fig. 12. When the user performs the sudden content, if the calculated detailed sum of the concentration score and the activity score is calculated to be equal to or higher than the preset first reference value, the clarity of the object existing in the virtual reality is restored to the default value, and the opaque floating interface disappears.

As one embodiment of the present invention, when the detailed sum of the concentration score and activity score when performing the sudden content is less than the first reference value, the VR content provider (2500) can provide the user with a floating interface recommending rest.

2. A system that creates avatars, provides them to external metaverse platforms, and updates the avatars.

As described above, the system for generating an avatar of the present invention, providing it to an external metaverse platform (5000), and updating the avatar can generate an avatar based on user information, and update the appearance of the avatar based on the performance result, concentration, and activity level according to content performance. Meanwhile, the system for updating the avatar provides information about the avatar to an external metaverse platform (5000). Hereinafter, a system for providing information about the avatar to an external metaverse platform (5000) to receive interaction information and update the avatar will be described in detail.

In order to provide an avatar to a user on an external metaverse platform (5000), a method of providing a specific character is often used. However, if an external metaverse platform (5000) is used with an avatar reflecting the user's information, the user can feel more interest in the feeling of being in the external metaverse platform (5000). In addition, the user can feel a sense of unity with the avatar within the external metaverse platform (5000). Therefore, a method of updating and providing an avatar suitable for the external metaverse platform (5000) based on the avatar created in the present invention is proposed.

Figure 14 schematically illustrates the execution steps in an avatar update unit according to one embodiment of the present invention.

As illustrated in FIG. 14, the method may include: providing information about the avatar to an external metaverse platform (5000), receiving interaction information that a user interacts with using the avatar on the external metaverse platform (5000), and performing an update on the avatar; and including an avatar information providing step (S900) that provides information about the avatar to the external metaverse platform (5000), receiving interaction information that a user interacts with using the avatar on the external metaverse platform (5000), performing an update on the avatar, and providing information about the avatar to the external metaverse platform (5000); an interaction information receiving step (S920) that receives interaction information corresponding to the avatar on the external metaverse platform (5000); And an avatar update step (S930) can be performed to extract information related to the avatar from the interaction information and update the avatar based on the information related to the avatar.

Specifically, the avatar information provision step is a step of providing information about the avatar created in the present invention to the outside. As described above, the avatar receives the user's biometric information, activity information, and medical information through the VR device, and is created based on this, and some or all of the information about the avatar can be provided to an external metaverse platform (5000). In the case of the user's medical information reflected in the avatar, it may not be transmitted depending on security.

Next, in the external metaverse platform (5000) that has received the avatar information, the user uses the avatar, and interaction information is generated according to the user's use of the avatar. Specifically, the interaction information may mean information that can update the avatar according to the usage situation of the external metaverse platform (5000), and may vary depending on the service provided by the external metaverse platform (5000). For example, the interaction information may be different when the external metaverse platform (5000) provides a medical service and when the external metaverse platform (5000) provides a game.

According to an embodiment of the present invention, the avatar used by the user in the external metaverse platform (5000) may correspond to an avatar created in the present invention or an avatar provided by the external metaverse platform (5000) in which information about the avatar received by the external metaverse platform (5000) is reflected.

A computing system that receives interaction information from the external metaverse platform (5000) performs an avatar update based on the interaction information. Specifically, the interaction information includes all information generated while the user uses the avatar within the external metaverse platform (5000), and thus only information related to the avatar is extracted and used from the interaction information. The avatar can be updated based on the extracted interaction information related to the avatar. According to one embodiment of the present invention, information related to the avatar in the interaction information can vary depending on the service provided by the external metaverse platform (5000), and thus the effect of updating the avatar according to the characteristics of the external metaverse platform (5000) can be exerted.

Below, we describe a case where the external metaverse platform (5000) provides a game.

Figure 15 schematically illustrates interaction information according to one embodiment of the present invention.

As illustrated in Figure 15, the interaction information may include usage time, level-up information, number of collected items, number of worn items, number of acquired skills, and number of quests cleared when the game is provided on the external metaverse platform (5000).

Specifically, all information generated when a user plays a game using an avatar within the external metaverse platform (5000) is stored as interaction information, and when the external metaverse platform (5000) provides a game, the user's usage time using the avatar, level-up information about the level of the game using the avatar, the number of collected items collected by the user using the avatar, the number of worn items worn by the user on the avatar, the number of acquired skills acquired by the user for the avatar, and the number of quests cleared by the user using the avatar within the external metaverse platform (5000) may be included in the interaction information, and the usage time, level-up information, number of collected items, number of worn items, number of acquired skills, and number of quests cleared by the user using the avatar may be extracted as information related to the avatar from the interaction information.

Figure 16 schematically illustrates an avatar update step (S930) based on aging information according to one embodiment of the present invention.

As illustrated in FIG. 16, the avatar update step (S930) can adjust the aging information based on the level-up information, the number of collected items, and the number of quests cleared, when the game is provided on the external metaverse platform (5000), and update the avatar with respect to the skin condition information, hair condition information, degree of bone curvature, eye inclination, and mouth shape based on the aging information.

Specifically, in the present invention, aging information is generated based on some or all of the user's multiple image information, the user's brainwave information, and the user's pulse information. In order to update the avatar suitable for the game provided by the external metaverse platform (5000), the aging information is adjusted based on some of the interaction information. Preferably, the information corresponding to the interaction information that adjusts the aging information may be the level-up information, the number of collected items, and the number of quests cleared.

The avatar is updated based on aging information that is adjusted to suit the game provided by the external metaverse platform (5000). All elements that can express the aging of the avatar can be updated, but preferably, the skin condition information, hair condition information, degree of bone curvature, eye inclination, and mouth shape of the avatar can be updated.

Figure 17 schematically illustrates an avatar update step (S930) based on movement speed according to one embodiment of the present invention.

As illustrated in FIG. 17, the avatar update step (S930) can adjust the movement speed based on the usage time, the number of skills acquired, and the number of items worn, and update the avatar based on the movement speed when the game is provided on the external metaverse platform (5000).

Specifically, in the present invention, the movement speed of the avatar is determined in such a way that the movement speed increases as the movement distance or the number of steps increases based on the movement distance or the number of steps derived from the user's smartphone for a preset time. In order to update the avatar suitable for the game provided by the external metaverse platform (5000), the movement speed is adjusted based on some of the interaction information. Preferably, the avatar is updated based on the usage time, the number of acquired skills, and the number of worn items.

Meanwhile, the movement speed adjusted based on the interaction information can increase as the usage time increases and the number of acquired skills increases, and can decrease as the number of worn items increases. Accordingly, as shown in Fig. 17, the movement speed has increased by ‘3’ compared to before, which can mean some or all of the following: an increase in usage time, an increase in the number of acquired skills, or a decrease in the number of worn items.

The ability values of the avatar illustrated in FIG. 17 are an embodiment of the present invention, and the ability values of the actual avatar of the present invention are not limited to those illustrated in FIG. 17.

3. A system for realistic effects that controls the humidity of the content environment performed by the avatar that created the humidity of the space where the user exists.

As described above, the system for controlling the humidity of a content environment in which a user of the present invention performs by creating an avatar that creates a humidity of a space in which the user exists for a realistic effect can create an avatar based on user information, update the appearance of the avatar based on the performance result, concentration, and activity level according to the content performance, provide information about the avatar to an external metaverse platform (5000), and receive interaction information about the avatar's interaction within the external metaverse platform and update the avatar based on the information.

Meanwhile, the system for the above-mentioned realism effect provides a realism effect by providing an environment in which the user exists similarly to the environment of the content. Below, a system that derives the humidity of the environment in the content provided by an external metaverse platform (5000) and controls the humidity of the environment in which the user exists based on the environmental humidity in the content will be described in detail.

In order to provide an optimal experience with the content provided by the external metaverse platform (5000), interaction between the content and the user is required. The content provided by the external metaverse platform (5000) only provides visual and auditory effects and does not provide tactile effects. In other words, the humidity of the environment inside the content provided by the external metaverse platform (5000) cannot be felt by the user, which may reduce the sense of immersion in the content.

For example, if the content environment is winter but the actual season where the user exists is summer, the humidity of the environment shown by the content and experienced by the user may be low, but the humidity felt by the user may be high, which may reduce the sense of presence of the content. Therefore, in order to provide a better level of content experience, a method is proposed to control the humidity of the actual space with the environmental humidity in the content by creating an avatar and performing it.

Figure 18 schematically illustrates a humidity control unit (7000) and a humidity control device (8000) according to one embodiment of the present invention.

As illustrated in FIG. 18, a system for a realistic effect that controls the humidity of a content environment in which a user exists by creating an avatar that generates humidity in a space in which the user exists may include a humidity control device (8000) that measures and controls the humidity of a real environment in which the user exists, and may include a humidity control unit (7000) that includes a machine-learned inference model and derives content humidity of a content environment in the content image based on the machine-learned inference model and a content image of the content provided by the external metaverse platform, and controls the humidity control device (8000) based on the content humidity to control the real humidity of the real environment in which the user exists.

Specifically, the humidity control unit (7000) includes a first humidity control unit (7100) and a second humidity control unit (7200), receives the actual humidity, which is the humidity of a space where a user exists, from a humidity control device (8000), and controls the humidity control device (8000) to control the actual humidity based on the content humidity.

More specifically, when the user performs content provided by the external metaverse platform (5000) using the avatar or the avatar provided by the external metaverse platform (5000) in which information about the avatar is reflected, the content image can be input into a machine-learned inference model to derive multiple content sensitivities. The details of deriving the multiple content sensitivities will be described later. The content image can be an image displayed on an HMD worn by the user and showing the user performing content using the avatar.

The above plural content humidity is derived as the first content humidity by the first humidity control unit (7100), and as the second content humidity and the third content humidity by the second humidity control unit (7200). The first humidity control unit (7100) can control the humidity control device (8000) based on the first content humidity to control the actual humidity, and the second humidity control unit (7200) can control the humidity control device (8000) based on the second content humidity and the third content humidity to control the actual humidity. The details of the first humidity control unit (7100) and the second humidity control unit (7200) will be described later.

According to one embodiment of the present invention, the user can control the actual humidity by selecting one of the first humidity control unit (7100) and the second humidity control unit (7200).

The above humidity control device (8000) may refer to any device capable of performing dehumidification and humidification. For example, it may include an integrated device capable of performing both dehumidification and humidification with a single device, or it may include both a humidifier performing humidification and a dehumidifier performing dehumidification, and it may measure the actual humidity at preset intervals and in real time.

Figure 19 schematically illustrates derivation of content humidity of a machine-learned inference model according to one embodiment of the present invention.

As illustrated in Figure 19, the inference model can extract multiple frames from the input content image according to preset rules, and derive the content humidity of each frame from each of the multiple frames.

Specifically, a content image of content provided by the external metaverse platform (5000) can be input into the machine-learned inference model. The content image can mean an image displayed on an HMD worn by a user, in which a user performs content provided by the external metaverse platform (5000) using an avatar. A content image for a preset period of time can be input into the machine-learned inference model. For example, the preset period of time can mean a part or the entire period of time during which a location within the external metaverse platform (5000) where the content is performed does not change.

The above machine-learned inference model can extract multiple frames from the input content video with the above preset rule. The above preset rule can be set so that the number of frames extracted varies depending on the time length of the input content video, or can be fixed. For example, one frame can be extracted every two seconds from the input content video, or 50 frames can be extracted regardless of the time length of the content video.

Next, the machine-learned inference model can obtain multiple content humidity by deriving content humidity for each of the multiple frames. The content humidity can correspond to the humidity of the environment (background) in which the avatar exists in the content video.

The above multiple content humidity derived by the machine-learned inference model can be derived as the first content humidity, the second content humidity, and the third content humidity by the humidity control unit (7000), and details will be described later.

A machine-learned inference model according to an embodiment of the present invention can be implemented in a form that includes at least one of an artificial neural network model such as a convolutional neural network model (CNN), a capsule network (CapsNet), and a rule-based feature information extraction model, and can be learned through image information such as CCTV in which location and time are recorded, and humidity data of a corresponding region and time of the corresponding image.

Figure 20 schematically illustrates steps performed by a first humidity control unit (7100) according to one embodiment of the present invention.

As illustrated in FIG. 20, the humidity control unit (7000) includes a first humidity control unit (7100), and the first humidity control unit (7100) can perform a first content image receiving step (S1000) of receiving the content image from an external metaverse platform (5000); a first content humidity deriving step (S1010) of inputting the content image into the machine-learned inference model to derive a plurality of content humidity and deriving an average value of the plurality of content humidity as the first content humidity; a first reality humidity receiving step (S1020) of receiving a reality humidity measured in a real environment in which the user exists from the humidity control device (8000); and a first reality humidity control step (S1030) of controlling the humidity control device (8000) so that the reality humidity reaches the first content humidity.

Specifically, in the first content image reception step (S1000), the content image for the content being performed by the user can be received, which is provided by the external metaverse platform (5000), and the first humidity control unit (7100) can input the content image into a machine-learned inference model included in the humidity control unit to derive a plurality of content humidity for the frame by the method described above in FIG. 19, and perform the first content humidity derivation step (S1010) of deriving an average value of the plurality of content humidity as the first content humidity.

According to an embodiment of the present invention, multiple content humidities can be derived for frames of the content video for a preset period of time.

Thereafter, the first humidity control unit (7100) can perform a step (S1020) of receiving the actual humidity, which is the humidity of the actual environment where the user exists and performs the content, from the humidity control device (8000).

The above first humidity control unit (7100) can perform the first reality humidity control step (S1030) based on the comparison result between the first content humidity and the measured reality humidity value.

When the actual humidity is higher than the first content humidity, the first humidity control unit (7100) can control the humidity control device (8000) to perform dehumidification so that the actual humidity reaches the first content humidity.

When the actual humidity is lower than the first content humidity, the first humidity control unit (7100) can control the humidity control device (8000) to perform humidification so that the actual humidity reaches the first content humidity.

According to an embodiment of the present invention, the first humidity control unit (7100) controls the actual humidity by deriving an average value of multiple content humidity as the first content humidity, thereby reducing the error of each content humidity derived from a machine-learned inference model.

Figure 21 schematically illustrates steps performed by a second humidity control unit (7200) according to one embodiment of the present invention.

As illustrated in FIG. 21, the humidity control unit (7000) includes a second humidity control unit (7200), and the second humidity control unit (7200) can perform a second content image receiving step (S1100) of receiving the content image from an external metaverse platform (5000); a second content humidity deriving step (S1110) of inputting the content image into the machine-learned inference model to derive a plurality of content humidity, and deriving the highest humidity among the plurality of content humidity as the second content humidity and the lowest humidity as the third content humidity; a second reality humidity receiving step (S1120) of receiving the reality humidity measured in the real environment in which the user exists from the humidity control device (8000); and a second reality humidity control step (S1130) of controlling the humidity control device (8000) so that the reality humidity is maintained between the second content humidity and the third content humidity.

Specifically, in the second content image reception step (S1100), the content image for the content being performed by the user can be received, which is provided by the external metaverse platform (5000), and the second humidity control unit (7200) can input the content image into the machine-learned inference model included in the humidity control unit to derive a plurality of content humidity by the method described above in FIG. 19, and perform the second content humidity derivation step (S1110) of deriving the highest humidity from the plurality of content humidity as the second content humidity and deriving the lowest humidity as the third content humidity.

According to an embodiment of the present invention, multiple content humidities can be derived for frames of the content video for a preset period of time.

Thereafter, the second humidity control unit (7200) can perform a step (S1120) of receiving the actual humidity, which is the humidity of the actual environment where the user exists and performs the content, from the humidity control device (8000).

The above second humidity control unit (7200) can perform the second reality humidity control step (S1130) based on the comparison result between the second content humidity and the third content humidity and the measured reality humidity value.

When the actual humidity is higher than the second content humidity, the second humidity control unit (7200) can control the humidity control device (8000) to perform dehumidification so that the actual humidity is maintained between the second content humidity and the third content humidity.

When the actual humidity is lower than the third content humidity, the second humidity control unit (7200) can control the humidity control device (8000) to perform humidification so that the actual humidity is maintained between the second content humidity and the third content humidity.

When the actual humidity is between the second content humidity and the third content humidity, the second humidity control unit (7200) can control the humidity control device (8000) to perform dehumidification or humidification so that the state is maintained.

According to an embodiment of the present invention, the second humidity control unit (7200) controls the actual humidity by deriving the highest humidity among the plurality of content humidity as the second content humidity and the lowest humidity as the third content humidity, thereby increasing the energy efficiency of the humidity control device (8000).

FIG. 22 schematically illustrates a humidity control method when moving a location within content according to one embodiment of the present invention.

As illustrated in FIG. 22, in the case where content moving from a specific location to another location is performed, the first reality humidity control step can stop the operation of the humidity control device (8000) for reaching the first content humidity of the specific location when the content progress rate in the specific location reaches a preset progress rate, and in the case where the second reality humidity control step, in the case where content moving from a specific location to another location is performed, the operation of the humidity control device (8000) for maintaining the humidity of the specific location between the second content humidity and the third content humidity can stop when the content progress rate in the specific location reaches a preset progress rate.

The above first reality humidity control step is performed in the first humidity control unit (7100), and the above second reality humidity control step can be performed in the second humidity control unit (7200), and in some cases, the operation of the humidity control device (8000) can be stopped to achieve the effect of efficiently consuming energy.

Specifically, when the content provided by the external metaverse platform (5000) is set to move from a specific location to another location, the operation of the humidity control device (8000) can be controlled to stop depending on the progress of the content in the specific location. More specifically, when the progress of the content in the specific location reaches a preset progress (preferably a progress close to 100 percent), the operation of the humidity control device (8000) can be stopped in consideration of moving to another location on the content after the progress of the content reaches 100 percent and controlling the actual humidity based on the content humidity of the other location.

In Fig. 22, since the content environment of content A is a rainy city and the content environment of content B is a desert, the difference between the content humidity of content A and the content humidity of content B may be large, and relatively, the content humidity of content A may be higher than the content humidity of content B. Accordingly, when the content progress rate of content A reaches a preset 80 percent at time T1, the operation of the humidity control device (8000) that controls the real humidity based on the content humidity of content A may be stopped from time T1. Thereafter, the operation of the humidity control device (8000) may be started from time T2 when the avatar used by the user moves to the desert, which is content B.

In the case of the first humidity control unit (7100), the content humidity of the A content may be the first content humidity, which is an average value of multiple content humidity derived from a machine-learned inference model for the A content video, and the operation of the humidity control device (8000) that controls the reality humidity to reach the first content humidity may be stopped from time T1, and the operation of the humidity control device (8000) may be started from time T2 when the avatar used by the user moves to the desert, which is the B content. In addition, the humidity control device (8000) may be controlled to control the reality humidity based on the first content humidity of the B content, and the operation of the humidity control device (8000) may be stopped from time T3 when the progress of the B content reaches a preset 80 percent.

In the case of the second humidity control unit (7200), the content humidity of the A content can derive the highest humidity as the second content humidity and the lowest humidity as the third content humidity from among the multiple content humidity derived from the machine-learned inference model for the A content video, and the operation of the humidity control device (8000) that controls the reality humidity to be maintained between the second content humidity and the third content humidity can be stopped from time T1, and the operation of the humidity control device (8000) can be started from time T2 when the avatar used by the user moves to the desert, which is the B content. In addition, the humidity control device (8000) can be controlled to control the reality humidity based on the second content humidity and the third content humidity of the B content, and the operation of the humidity control device (8000) can be stopped from time T3 when the progress of the B content reaches a preset 80 percent.

FIG. 23 exemplarily illustrates the internal configuration of a computing device according to one embodiment of the present invention.

As illustrated in FIG. 23, the computing device (11000) may include at least one processor (11100), a memory (11200), a peripheral interface (11300), an input/output subsystem (I/O subsystem) (11400), a power circuit (11500), and a communication circuit (11600). At this time, the internal configuration of the computing device (11000) may include the computing system illustrated in FIG. 1, or the computing device (11000) may correspond to an embodiment of the computing system illustrated in FIG. 1, and may be connected to the VR service unit (2000) and the additional content unit (4000) by the input/output subsystem (11400). The processor (11100) may be configured in plurality, and may include the first processor module and the second processor module described above.

The memory (11200) may include, for example, a high-speed random access memory, a magnetic disk, an SRAM, a DRAM, a ROM, a flash memory, or a nonvolatile memory. The memory (11200) may include a software module, a set of instructions, or other various data necessary for the operation of the computing device (11000).

At this time, access to the memory (11200) from other components such as the processor (11100) or peripheral interface (11300) may be controlled by the processor (11100).

The peripheral interface (11300) can couple input and/or output peripherals of the computing device (11000) to the processor (11100) and memory (11200). The processor (11100) can execute software modules or instruction sets stored in the memory (11200) to perform various functions for the computing device (11000) and process data.

The input/output subsystem can couple various input/output peripherals to the peripheral interface (11300). For example, the input/output subsystem can include a controller for coupling peripherals such as a monitor, a keyboard, a mouse, a printer, or, as needed, a touchscreen or a sensor to the peripheral interface (11300). In another aspect, the input/output peripherals can be coupled to the peripheral interface (11300) without going through the input/output subsystem.

The power circuit (11500) may provide power to all or part of the components of the terminal. For example, the power circuit (11500) may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for generating, managing, or distributing power.

The communication circuit (11600) may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit (11600) may enable communication with another computing device by transmitting and receiving RF signals, also known as electromagnetic signals, including RF circuits, as needed.

This embodiment of FIG. 23 is only an example of a computing device (11000), and the computing device (11000) may have some of the components illustrated in FIG. 23 omitted, may further include additional components not illustrated in FIG. 23, or may have a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touchscreen or a sensor, in addition to the components illustrated in FIG. 23, and may include a circuit for RF communication of various communication methods (WiFi, 3G, LTE, Bluetooth, NFC, Zigbee, etc.) in the communication circuit (11600). Components that can be included in the computing device (11000) may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

The methods according to the embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the embodiments may be configured as a PC-based program or an application exclusively for mobile terminals. The application to which the present invention is applied may be installed on a computing device (11000) through a file provided by a file distribution system. For example, the file distribution system may include a file transfer unit (not shown) that transfers the file according to a request from the computing device (11000).

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, the processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal waves, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over network-connected computing devices, and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known to and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, flash memories, etc. Examples of the program commands include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, appropriate results can be achieved even if the described techniques are performed in a different order from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or are replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (10)

A system for a realistic effect that controls the humidity of the content environment performed by an avatar that creates the humidity of the space where the user exists.
A VR device including: an HMD that is mounted on a user and provides images and sounds of virtual reality and detects the user's voice; a camera device that photographs the user's appearance to derive image information of the user; an electroencephalograph that is attached to the user's head and can measure the user's brain waves; a pulse meter that is attached to the user's wrist and can measure the user's pulse; and a motion sensing device that measures the user's movements;
A humidity control device that measures and controls the humidity in the real environment where the user exists;
A VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, and creates an avatar for the user to provide multiple virtual reality contents to the user;
An avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about interactions that a user has had with the avatar on the external metaverse platform, and performs an update on the avatar; and
A humidity control unit including a machine-learned inference model, and deriving content humidity of a content environment in the content image based on the machine-learned inference model and the content image provided by the external metaverse platform, and controlling the humidity control device based on the content humidity to control the real humidity of the real environment in which the user exists;
The above VR service department,
A virtual reality implementation unit that implements a virtual reality that is available only to the above users and provides the above multiple virtual reality contents;
A VR communication unit that communicates with the VR device and exchanges information;
A user information input unit that receives user information including the user's biometric information received from the VR device; and activity information and medical information received from the user's smartphone;
An avatar generation unit that generates an avatar of the user based on the user information within the virtual reality implemented by the virtual reality implementation unit; and
Includes a VR content provider that provides the plurality of contents to an avatar created in the virtual reality and updates the appearance of the avatar each time it performs the same;
The above humidity control unit includes a first humidity control unit,
The above first humidity control unit,
A first content video receiving step for receiving the content video from an external metaverse platform;
A first content humidity derivation step of inputting the above content image into the machine-learned inference model to derive multiple content humidity, and deriving the average value of the multiple content humidity as the first content humidity;
A first real humidity receiving step for receiving the real humidity measured in the real environment in which the user exists from the humidity control device; and
A first reality humidity control step is performed to control the humidity control device so that the above reality humidity reaches the first content humidity;
The above first reality humidity control step is,
When performing content that moves from a specific location to another location, when the content progress at the specific location reaches a preset progress, the operation of the humidity control device to reach the first content humidity of the specific location is stopped.
The above avatar creation part,
Determine the basic appearance of the avatar based on multiple image information,
Determine the avatar's aging information based on the above biometric information, and add a first graphic effect corresponding to the aging information to the above basic appearance,
Determine the avatar's disease information based on medical information including one or more medical conditions entered by the user,
The above VR content provider is,
A first content provider that provides content that enables the user to perform cognitive ability and concentration enhancement training, and whose difficulty is determined based on the aging information of the user;
A second content provider that determines the level of difficulty based on the activity information of the user and provides content that enables the user to perform physical activity strengthening training;
A third-party content provider that determines the type of content based on the above disease information of the user and provides content that can perform training on body parts or cognitive abilities requiring rehabilitation training;
An aging information control unit that calculates one or more scores based on the performance results according to the performance of the content provided by the first content provider, the second content provider, and the third content provider, the brainwave information measured by the brainwave measuring device during the performance period of the content, and the pulse information measured by the pulse measuring device, and controls the aging information according to the sum of the one or more scores; and
Including an avatar appearance update unit that updates the appearance of the avatar of the corresponding user by adjusting the first graphic effect according to the adjusted aging information by the above aging information adjustment unit;
The above biometric information is,
A system for realistic effect, comprising: image information input through the camera device; brainwave information of the user measured by the brainwave measuring device; and pulse information of the user measured by the pulse measuring device.
In claim 1,
The above inference model is,
A system for realism effect, which extracts multiple frames from the input content video according to preset rules and derives the content humidity of each frame from each of the multiple frames.
delete delete In claim 1,
The above humidity control unit includes a second humidity control unit,
The above second humidity control unit is,
A second content video reception step for receiving the content video from an external metaverse platform;
A second content humidity derivation step of inputting the above content video into the machine-learned inference model to derive multiple content humidity, and deriving the highest humidity among the multiple content humidity as the second content humidity and the lowest humidity as the third content humidity;
A second reality humidity receiving step for receiving the reality humidity measured in the real environment in which the user exists from the humidity control device; and
A system for realism effect, which performs a second reality humidity control step of controlling the humidity control device so that the above-mentioned reality humidity is maintained between the second content humidity and the third content humidity.
In claim 5,
The above second reality humidity control step is,
A system for realism effect, in which, when performing content that moves from a specific location to another location, the operation of the humidity control device for maintaining the humidity of the specific location between the second content humidity and the third content humidity is stopped when the content progress at the specific location reaches a preset progress.
In claim 1,
The above avatar update part is,
Avatar information provision step that provides information about the avatar to an external metaverse platform;
An interaction information receiving step for receiving interaction information corresponding to the avatar from the external metaverse platform; and
A system for realism effect, which performs an avatar update step of extracting information related to an avatar from the above interaction information and updating the avatar based on the information related to the avatar.
In claim 1,
The above avatar creation part,
A basic appearance determination unit that derives appearance feature information from a plurality of pieces of image information, derives character determination parameters in the form of a one-dimensional vector by averaging the derived appearance feature information, and determines the basic appearance of the avatar based on the character determination parameters;
An aging information determination unit that inputs the above biometric information into one or more artificial neural network models to determine aging information of the avatar and adds a first graphic effect corresponding to the aging information to the basic appearance;
A movement speed determination unit that determines the movement speed of the avatar by increasing the movement speed as the movement distance or number of steps increases based on the movement distance or number of steps for a preset time derived from the user's smartphone; and
A system for a realistic effect, comprising: a disease information determination unit for determining disease information of an avatar based on medical information including one or more medical conditions entered by the user, and adding a second graphic effect corresponding to the disease information to the basic appearance to which the first graphic effect has been added.
delete A method for a realistic effect implemented in a system for a realistic effect that controls the humidity of a content environment performed by an avatar that creates the humidity of a space in which a user exists,
The above system,
A VR device including: an HMD that is mounted on a user and provides images and sounds of virtual reality and detects the user's voice; a camera device that photographs the user's appearance to derive image information of the user; an electroencephalograph that is attached to the user's head and can measure the user's brain waves; a pulse meter that is attached to the user's wrist and can measure the user's pulse; and a motion sensing device that measures the user's movements;
A humidity control device that measures and controls the humidity in the real environment where the user exists;
A VR service unit that communicates with a VR device worn by the user, provides information about virtual reality to the VR device, and creates an avatar for the user to provide multiple virtual reality contents to the user;
An avatar update unit that provides information about the avatar to an external metaverse platform, receives interaction information about a user interacting with the avatar on the external metaverse platform, and performs an update on the avatar; and
A humidity control unit including a machine-learned inference model, and deriving content humidity of a content environment in the content image based on the machine-learned inference model and the content image provided by the external metaverse platform, and controlling the real humidity of the real environment in which the user exists based on the content humidity;
The above method,
A virtual reality implementation step that implements a virtual reality in which only the user can use it and multiple virtual reality contents are provided by the VR service department;
A VR communication step for exchanging information by performing communication with the VR device through the VR service unit;
A user information input step for receiving user information including the user's biometric information received from the VR device by the VR service department; and activity information and medical information received from the user's smartphone;
An avatar creation step for creating an avatar of the user based on the user information within the virtual reality implemented by the virtual reality implementation step by the VR service department; and
A VR content providing step capable of providing the plurality of contents to an avatar created in the virtual reality by the VR service unit and updating the appearance of the avatar each time the same is performed;
The above humidity control unit includes a first humidity control unit,
The above first humidity control unit,
A first content video receiving step for receiving the content video from an external metaverse platform;
A first content humidity derivation step of inputting the above content video into the machine-learned inference model to derive multiple content humidity, and deriving the average value of the multiple content humidity as the first content humidity;
A first real humidity receiving step for receiving the real humidity measured in the real environment in which the user exists from the humidity control device; and
A first reality humidity control step is performed to control the humidity control device so that the above reality humidity reaches the first content humidity;
The above first reality humidity control step is,
When performing content that moves from a specific location to another location, when the content progress at the specific location reaches a preset progress, the operation of the humidity control device to reach the first content humidity of the specific location is stopped.
The above avatar creation step is,
Determine the basic appearance of the avatar based on multiple image information,
Determine the avatar's aging information based on the above biometric information, and add a first graphic effect corresponding to the aging information to the above basic appearance,
Determine the avatar's disease information based on medical information including one or more medical conditions entered by the user,
The above VR content provision stage is:
A first content provision step in which the difficulty level is determined based on the aging information of the user and content is provided that enables the user to perform cognitive ability and concentration enhancement training;
A second content provision step in which the difficulty level is determined based on the activity information of the user and content is provided that enables the user to perform physical activity strengthening training;
A third content provision step in which the type of content is determined based on the above disease information of the user and content is provided that can perform training on a body part or cognitive ability requiring rehabilitation training;
An aging information control step for calculating one or more scores based on the performance results according to the performance of the content provided in the first content providing step, the second content providing step, and the third content providing step, the brainwave information measured by the brainwave measuring device during the performance period of the content, and the pulse information measured by the pulse measuring device, and controlling the aging information according to the sum of the one or more scores; and
Including an avatar appearance update step for updating the appearance of the avatar of the corresponding user by adjusting the first graphic effect according to the adjusted aging information through the above aging information adjustment step;
The above biometric information is,
A method for providing a realistic effect, comprising: image information input through the camera device; brain wave information of the user measured by the brain wave measuring device; and pulse information of the user measured by the pulse measuring device.