KR102216517B1 - VR Content Providing System According to Bio-signal - Google Patents

Abstract

It relates to VR content, and specifically, to a system that provides appropriate VR content according to a bio-signal. The VR content providing system according to the embodiment includes a head mount for displaying VR to a user, an EEG measuring device included in the head mount to measure EEG, an electrocardiogram measuring device included in the head mount to measure an electrocardiogram, and an upper body of the user. In a VR content providing system according to a bio-signal comprising a respiration meter attached to measure respiration, and a camera sensor for photographing motions of the user using VR content, the first step of measuring emotional change according to the electrocardiogram , The second step of measuring the emotional change according to the respiration meter, the third step of measuring the beta wave of the brain wave according to the EEG meter, the fourth step of measuring the emotional change according to the camera sensor, the first step to A fifth step of determining a user's emotional change by using the measurement result of at least one of the fourth steps, and transmitting a difficulty drop signal when the emotion change is greater than or equal to a first set value, and first setting the difficulty drop signal A sixth step of lowering the difficulty level of the VR content when receiving more than the frequency; a VR content providing system according to a bio-signal including.

Description

VR Content Providing System According to Bio-signal}

It relates to VR content, and specifically, to a system that provides appropriate VR content according to a bio-signal.

Unless otherwise indicated herein, the content described in this section is not prior art to the claims of this application, and inclusion in this section is not admitted to be prior art.

With the advent of the 5G era, expectations for VR technology are getting stronger. VR technology is classified as a promising technology in the future, but due to a lack of VR contents, the development of VR technology is not being promoted as expected. In this situation, provision of appropriate VR contents has become an essential factor to increase the number of users of VR equipment. Accordingly, it is an object of the present invention to properly provide content provided by such VR technology in a more user-oriented manner.

The invention disclosed in Korean Patent Registration No. 10-1397287 (announced on 14.05.21) relates to an emotion inducing system standardized in each intensity level and an emotion inducing method using the same. In the emotion inducing system for inducing emotion from a user, A bio-signal detector for detecting a bio-signal; A database unit for outputting a signal of a corresponding emotion inducing protocol according to an increase or decrease pattern of the biometric parameter extracted from the biometric signal; Stimulation module unit for stimulating the body by the signal of the emotion inducing protocol; A control unit which compares and analyzes the increase/decrease pattern of the biometric parameter extracted from the biometric signal, formalizes the intensity of emotion by step, and extracts it as data; And a data storage unit for each level of emotional intensity, which stores data for each level of the intensity of emotion, which is standardized by the control unit, and is made to standardize and stage emotions by intensity through experimentation, and optimal content according to each intensity of emotion. , By objectifying the biometric parameters and the emotion inducing protocol, the present invention discloses an invention having the effect of deriving an objective result for the degree of subjective emotion of a user. Since the disclosed invention includes a change in a person's bio-signal for a specific content, it can be utilized to provide VR content according to a bio-signal.

In addition, the invention disclosed in Korean Patent Registration No. 10-1798777 (announcement on 17.11.16) relates to neuro-cinematics, which uses the steady state visual development committee to measure viewers' emotional excitement for video content and evaluate video content. How to do it is disclosed. The method for measuring the emotional excitement of a viewer with respect to the disclosed video content includes the steps of receiving a result of measuring an EEG of a viewer who watches the video content including a steady state visual development potential-induced stimulation; Extracting a frequency component corresponding to the triggered stimulus from the EEG measurement result; And measuring a change in emotional excitement of the viewer by using the magnitude of the frequency component, wherein the triggered stimulus is a stimulus output at a preset frequency during reproduction of the video content. In other words, it is a technology to identify video content by measuring the emotional excitement of the user according to the content.

In the present invention, a method of identifying a bio-signal based on such background technology and providing appropriate content using the bio-signal is disclosed herein.

Korean Patent Registration No. 10-1397287 (14.05.21 announcement) Korean Patent Registration No. 10-1798777 (17.11.16 announcement)

In order to prevent users from being over-immersed in VR contents and causing side effects or preventing users from being able to enjoy VR contents by providing extreme fear and surprise to users, users can provide appropriate VR contents by sensing biometric signals. Its purpose is to allow you to enjoy VR contents as much as possible.

A head mount for displaying VR to a user according to an embodiment, an EEG measuring device included in the head mount to measure EEG, an ECG measuring device included in the head mount to measure an electrocardiogram, and attached to the upper body of the user to measure respiration In the VR content providing system according to the bio-signal comprising a respiration meter, and a camera sensor for photographing a motion of the user using the VR content, the first step of measuring the emotional change according to the ECG meter, the respiration meter According to the second step of measuring the emotional change, the third step of measuring the beta wave of the brain wave according to the EEG meter, the fourth step of measuring the emotional change according to the camera sensor, any of the first to fourth steps A fifth step of determining a user's emotional change using one or more measurement results, and transmitting a difficulty drop signal if the emotion change is greater than or equal to a first set value, and receiving the difficulty drop signal more than a first set frequency, VR And a VR content providing system according to a bio-signal including a sixth step of decreasing the difficulty of the content.

A VR content providing system that senses and analyzes a user's bio-signals and provides them with an appropriate sense of immersion and enjoyment is disclosed. Appropriate VR contents are provided according to the user's own emotions and conditions through biometric signals, so that VR contents can be enjoyed more fun and immersively.

The effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a schematic flowchart of a VR content providing system according to a bio signal according to one or another embodiment
2 is a flow chart for a method of processing an EEG signal according to one or another embodiment
3 is a flowchart of a method of processing an operation signal according to an embodiment or another embodiment;
4 is a flowchart of a VR content providing system according to a bio signal according to an embodiment
5 is a flowchart of a system for providing VR contents according to a bio signal according to another embodiment
6 is a flowchart of a system for providing VR contents according to a bio signal according to another embodiment
7 is a flowchart of a system for decreasing the difficulty level according to one or another embodiment

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to the possessor, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

In describing embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout this specification.

1 is a schematic flowchart of a VR content providing system according to a bio signal according to an embodiment or another embodiment.

As shown in FIG. 1, a head mount for displaying VR to a user, an electroencephalogram measuring device included in the head mount to measure EEG, an electrocardiogram measuring device included in the head mount to measure an electrocardiogram, and attached to the user's upper body to measure respiration. In a system for providing VR contents according to a bio-signal including a respiration measuring device to measure and a camera sensor for photographing motions of a user using VR contents, the first step (S100) of measuring emotional changes according to an electrocardiogram measuring device, The second step (S200) of measuring the emotional change according to the EEG, the third step (S300) of measuring the beta wave of the EEG according to the EEG, the fourth step (S400), the first step of measuring the emotional change according to the camera sensor A fifth step (S500) of transmitting a difficulty level drop signal when the user's emotion change level is determined by using the measurement result of at least one of the fourth steps, and if the emotion change level is higher than the first set value, and a difficulty level drop signal are generated. A system for providing VR contents according to a bio-signal including a sixth step (S600) of lowering the difficulty of VR contents when receiving more than one set frequency is disclosed.

First, as an example of a bio signal, an electrocardiogram, respiration, brain waves, and a user's motion are sensed. Analyze emotional change with the above measured results. If the result of the emotional change analysis turns out to be fear or surprise, a signal for decreasing the difficulty level is transmitted, and when the number of transmissions of the drop signal exceeds a set value, the level of difficulty is decreased to provide content.

If the result of the emotional change analysis is not fear or surprise, the emotional change is observed by continuously sensing the electrocardiogram, respiration, brain waves, and motion.

Here, the electrocardiogram is measured by a head mount for displaying the VR to a user, and the head mount may include an EEG measuring device for measuring an EEG. The respiration meter is attached to the user's upper body to measure respiration, and the motion signal is measured by a camera sensor that photographs the motion of the user using VR content.

In S100, the electrocardiograph measures the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component of the heart rate (LF), and the high frequency component of the heart rate (HF). An emotion is judged based on the measured result. Since the determination of emotion change according to the amount of change according to the measured result is disclosed in Korean Patent Registration No. 10-1397287, a detailed description will be omitted.

In S200, the respiration meter measures the rate of change of the respiration rate (Resp_R), the respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp). An emotion is judged based on the measured result, and the determination of emotion change according to the amount of change according to the measured result is disclosed in Patent No. 10-1397287. Therefore, detailed description is omitted.

2 is a flowchart illustrating a method of processing an EEG signal according to one or another embodiment.

As shown in Fig. 2, the frequency analysis result of the 3-1 step (S310) and 3-1 of the frequency analysis of the EEG signal measured by the EEG in the 3rd step (S300) is the frequency domain of the beta wave. Step 3-2 (S320) of extracting only, step 3-3 (S330) of extracting a change amount of beta wave according to the analysis result in step 3-2, and step 3-4 of normalizing the change amount of beta wave It includes a VR content providing system according to the bio-signal characterized in that it further comprises (S340).

The EEG is measured by the EEG and is analyzed through frequency analysis (S310). The frequency analysis uses a conventional Fast-Fourier Transform (FFT). After that, the band-pass filter is used to extract a specific frequency region for extracting the beta wave part from the brain wave, and the beta wave section passes through 13 to 30 Hz (S320). When the beta wave is awakening or conscious action It usually appears when you are in an emotional state of tension or anxiety. This is a known technology, so detailed descriptions are omitted. After extracting the amount of change with the extracted beta wave (S330), the extracted amount of change is normalized to a value between 0 and 1 (S340). Here, the normalization follows a conventional normalization method, and a detailed description thereof will be omitted.

3 is a flowchart illustrating a method of processing an operation signal according to an exemplary embodiment.

As shown in Fig. 3, step 4-1 (S410) of recognizing the motion by the camera sensor, step 4-2 (S420) of determining whether the motion matches the emotional motion converted into DB, coinciding with the emotional motion It includes a VR content providing system according to a bio-signal, characterized in that it further comprises a 4-3 step (S430) of calculating the emotion accuracy according to the number of operations.

The camera sensor is installed in a space where the user enjoys the VR content, photographs the user who uses the VR content, and determines the emotional change by analyzing the captured image.

As a known technique, many studies are being conducted to judge emotion through posture analysis, which is a person's behavior. As a result of the study, when a person feels a sense of surprise, they bend back, cover their cheeks with both hands, or cover their mouths. When a person feels an emotion of fear, he or she curls up, bends backwards, or wraps his arms around his body and shoulders. In addition, it is known to cover the face with both hands or the mouth with both hands, such as overlapping hands or clenching fists. Hereinafter, since it is a known technology, detailed description will be omitted.

In S410, the user's motion is recognized through the camera sensor, and in S420, the motion corresponding to the above-described surprise or fear feeling is converted into a DB, and then, in S430, the emotion accuracy is calculated according to the number of motions. In other words, when you feel the emotion of surprise or fear, if you calculate the overlapping motion as one, there are a total of 7 motions.If the motions overlap, more accurately normalize the feeling of surprise or fear to correspond to all seven motions. 1. In the case of 0 motions, the emotion accuracy is calculated by normalizing to 0.

4 is a flowchart illustrating a system for providing VR contents according to a bio signal according to an exemplary embodiment.

As shown in FIG. 4, when it is determined that the user feels fear or surprise in the first step (S110) and the step 1-2 (S120), the second step (S210) and the second step 2-2 The step of determining whether the user feels fear or surprise by calling step S220 (S510), the amount of change of the beta wave normalized in step 3-4 (S340) and the emotion accuracy in step 4-3 (S430), respectively Providing VR content according to a bio-signal, characterized in that it comprises the step of multiplying the set weight to obtain an average value (S520), and when the average value is greater than or equal to the set value, transmitting a difficulty drop signal in the fifth step (S500) (S530). Including the system.

Specifically, the heart rate is measured first. Heart rate measurement here means measuring heart rate (HR), standard deviation of RR interval (SD_RRI), low frequency component of heart rate (LF), and high frequency component of heart rate (HF). With the above measurement results, it is judged to feel the emotion of surprise or fear through the amount of change in each parameter, and if so, measure the breath again. Measuring respiration means measuring respiration rate (Resp_R), respiration amplitude (Resp_A), standard deviation of respiration rate (Resp_Rsd), and standard deviation of respiration amplitude (Resp_Asp). With the above measurement result, if it is determined that the emotion of surprise or fear is felt through the amount of change in each parameter (S510), the process proceeds to step S520. In step S520, the average value is obtained by multiplying the change amount of the beta wave of the normalized EEG signal and the normalized emotion accuracy by each weight. In general, since the accuracy of the EEG is higher than the accuracy of motion recognition through the camera sensor, it is desirable that the weight of the change amount of the beta wave of the EEG signal is higher. Thereafter, if the average value becomes larger than the set value, the difficulty level decrease signal is transmitted (S530). After that, the difficulty level is adjusted according to the number of times per minute of the difficulty level decrease signal, which will be described later.

5 is a flowchart illustrating a system for providing VR contents according to a bio signal according to another embodiment.

As shown in FIG. 5, the heart rate (HR) measured in the first step (S100), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate are normalized, respectively. The respiration rate (Resp_R), respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp) measured in the step (S540), the second step (S200) are normalized. In the case where one or more emotional motions for surprise or fear are sensed in the step of obtaining the average respiration value (S550) and the fourth step (S400), the heart rate average value is higher than or equal to the set value (OR) At the same time, when the change amount of the beta wave normalized in step 3-4 (S340) is greater than or equal to the set value, the VR according to the bio-signal comprising the step (S560) of transmitting a difficulty drop signal in the fifth step (S500) It includes a content providing system.

In S540, the amount of change in the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate are normalized, and the average value of the normalized values is obtained. In S550, the change in respiration rate (Resp_R), respiration amplitude (Resp_A), standard deviation of respiration rate (Resp_Rsd), and standard deviation of respiration amplitude (Resp_Asp) are each normalized to obtain the average value. Thereafter, in S560, when the emotion of surprise or fear is determined by motion sensing, if any one of the average values obtained in S540 and S550 is greater than or equal to the set value (AND) and the normalized change in the beta value of the EEG is greater than the set value, the difficulty level decreases. It will transmit a signal.

6 is a flowchart illustrating a system for providing VR contents according to a bio signal according to another embodiment.

6, the heart rate (HR) measured in the first step (S100), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate are normalized, respectively. The respiration rate (Resp_R), respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp) measured in the step (S570), the second step (S200) are normalized. When one or more emotional motions for surprise or fear are sensed in the step (S580) and the fourth step (S400) of obtaining the average respiration value, the heart rate average value is greater than or equal to the set value (AND) and the average respiration value is greater than or equal to the set value. Including a VR content providing system according to a bio-signal comprising the step of transmitting a difficulty drop signal in a fifth step (S500) when the change amount of the beta wave normalized in step 3-4 (S340) is more than a set value do.

In S570, the amount of change in the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate are normalized, and the average value of the normalized values is obtained. In S580, the change in respiration rate (Resp_R), respiration amplitude (Resp_A), standard deviation of respiration rate (Resp_Rsd), and standard deviation of respiration amplitude (Resp_Asp) are each normalized to obtain the average value. Subsequently, in S590, when the emotion of surprise or fear is determined by motion sensing, if both the average values obtained in S570 and S580 are equal to or greater than the set value (AND), and the normalized change in the beta value of the EEG is equal to or greater than the set value, a difficulty drop signal is generated. Will be transmitted.

7 is a flowchart illustrating a system for decreasing the difficulty level according to one or another embodiment.

As shown in FIG. 7, if the number of times of receiving the difficulty drop signal in step 6 (S600) is greater than the first set frequency and less than the second set frequency, step 6-1 of reducing the difficulty level to medium (S610) and the sixth step (S600), if the number of times the number of times of receiving the level of difficulty decrease signal is greater than the second set frequency, the living body further comprises a step 6-2 (S620) of lowering the difficulty level to the lower (lower) It includes a VR content providing system according to signals.

That is, when the difficulty level decrease signal is transmitted according to the analysis result of the fifth step (S500), content for an appropriate level of difficulty is provided according to the frequency. For example, if the difficulty drop signal per minute is more than once and less than three times, the middle of the difficulty level is lowered, and if the level of difficulty drop per minute is more than three times, the level of difficulty is lowered to provide appropriate content to the user.

The disclosed contents are only examples, and various changes can be made by those of ordinary skill in the art without departing from the gist of the claims claimed in the claims, so the scope of protection of the disclosed contents is It is not limited to the examples.

Claims (10)

delete delete delete delete delete A head mount for displaying VR to a user, an electroencephalogram measuring device included in the head mount to measure an EEG, an electrocardiogram measuring device included in the head mount to measure an electrocardiogram, a respiration meter attached to the user's upper body to measure respiration, And in the VR content providing system according to the bio-signal comprising a camera sensor for photographing the motion of the user using the VR content,
A first step of measuring an emotional change according to the electrocardiogram;
A second step of measuring emotional change according to the respiration meter;
A third step of measuring a beta wave of an EEG according to the EEG meter;
A fourth step of measuring emotional change according to the camera sensor;
A fifth step of determining a user's emotional change degree using the measurement result of at least one of the first to fourth steps, and transmitting a difficulty drop signal if the emotional change degree is greater than or equal to a first set value;
Including; a sixth step of decreasing the difficulty level of the VR content when the difficulty drop signal is received by a first preset frequency or more,

In the first step, the electrocardiogram is a device that measures a heart rate (HR), a standard deviation of an RR interval (SD_RRI), a low frequency component (LF) of the heart rate, and a high frequency component (HF) of the heart rate,
When the rate of change of the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate rises, rises, falls, or rises, respectively, the user feels fear. Step 1-1 of determining that it is;
When the rate of change of the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate rises, rises, and is constant, the user is surprised. The second step of determining that it is; further includes,

In the second step, the respiration meter is a device that measures the respiration rate (Resp_R), the respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp),
If the rate of change of the respiration rate (Resp_R), respiration amplitude (Resp_A), standard deviation of respiration rate (Resp_Rsd), and standard deviation of respiration amplitude (Resp_Asp) decreases, rises, schedules, and rises, respectively, it is determined that the user feels fear. Step 2-1;
If the rate of change of the respiration rate (Resp_R), respiration amplitude (Resp_A), standard deviation of respiration rate (Resp_Rsd), and standard deviation of respiration amplitude (Resp_Asp) decreases, rises, rises, or rises, respectively, it is determined that the user feels a surprise. The second step; further includes,

A 3-1 step of frequency-analyzing the EEG signal measured by the EEG in the third step;
A third step of extracting only the frequency domain of the beta wave from the frequency analysis result in step 3-1;
Step 3-3 of extracting the change amount of the beta wave according to the analysis result in step 3-2; And
It further includes a 3-4 step of normalizing the change amount of the beta wave;

A 4-1 step of recognizing motion by a camera sensor;
A 4-2 step of determining whether the motion matches the DB emotional motion;
Step 4-3 of calculating the emotion accuracy according to the number of movements matched with the emotion movement; further comprising,

In step 6 above
Step 6-1 of lowering the difficulty level to medium if the number of times the difficulty drop signal is received is greater than the first preset frequency and less than the second preset frequency; And
And a 6-2 step of lowering the difficulty level to a lower level when the number of times of receiving the level of difficulty drop signal in the sixth step is greater than a second preset frequency.
The method of claim 6,
Determining whether the user feels fear or surprise by calling the steps 2-1 and 2-2 when it is determined in the first steps 1-1 and 1-2 that the user feels fear or surprise;
Calculating an average value by multiplying the change amount of the beta wave normalized in step 3-4 by a weight set for each of the emotion accuracy in step 4-3;
And transmitting a difficulty level drop signal in the fifth step when the average value is greater than or equal to a set value.
The method of claim 7,
The weight of the VR content providing system according to the bio-signal, characterized in that the weight for the change amount of the beta wave is greater than the weight of the emotion accuracy.
The method of claim 6,
Normalizing the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate measured in the first step to obtain a heart rate average value;
Obtaining a respiratory average value by normalizing the respiration rate (Resp_R), the respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp) measured in the second step, respectively;
When one or more emotional motions for surprise or fear are sensed in the fourth step, the heart rate average value is greater than or equal to a set value or (OR) the respiratory average value is greater than or equal to the set value, and the beta normalized in step 3-4. And transmitting a difficulty drop signal in the fifth step when the wave change amount is greater than or equal to a set value.
The method of claim 6,
Normalizing the heart rate (HR), the standard deviation of the RR interval (SD_RRI), the low frequency component (LF) of the heart rate, and the high frequency component (HF) of the heart rate measured in the first step to obtain a heart rate average value;
Obtaining a respiratory average value by normalizing the respiration rate (Resp_R), the respiration amplitude (Resp_A), the standard deviation of the respiration rate (Resp_Rsd), and the standard deviation of the respiration amplitude (Resp_Asp) measured in the second step, respectively;
When one or more emotional motions for surprise or fear are sensed in the fourth step, the heart rate average value is greater than or equal to the set value (AND) and the respiration average value is greater than or equal to the set value, and the beta wave normalized in step 3-4. And transmitting a difficulty drop signal in the fifth step when the amount of change is greater than or equal to the set value.

Images