KR20190004088A - Virtual Reality Education System and Method based on Bio Sensors - Google Patents

Abstract

The present invention relates to a bio-signal interlocked virtual reality education system and a method. More specifically, the present invention relates to a system and an apparatus which can increase education efficiency by displaying a virtual reality education content on a smart glass device, and interlocking bio-signal sensors to analyze cumulative concentrativeness and real-time concentrativeness of a user. According to an embodiment of the present invention, the bio-signal interlocked virtual reality education system comprises: a head mounted display which comprises an optical module transmitting an image light emitted from at least one display to an eyeball of the user, and displays a first content associated with education of the user; a brain wave sensor which senses at least one brain wave of the user; a control unit which evaluates concentrativeness of the user for the first content by using the sensed at least one brain wave; and a memory which accumulates and stores the concentrativeness of the user, wherein the control unit can control the head mounted display so that the concentrativeness of the user is displayed in real time through the head mounted display.

Description

Technical Field [0001] The present invention relates to a virtual reality education system and method,

 BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a bio-signal interworking virtual reality education system and method. And more particularly, to a system and an apparatus capable of increasing educational efficiency by displaying virtual reality education contents in a smart glass device and analyzing cumulative concentration and real time concentration degree of a user by interlocking a bio-signal sensor.

Virtual reality is a process of creating a 3D environment in which a specific environment or situation is created through a computer and creating a human-computer interaction that makes the person using the 3D content appear as if they are interacting with the actual surroundings and environment. And the like.

Generally, the three-dimensional sensation perceived by a person depends on the degree of thickness change of the lens according to the position of the observed object, the angle difference between the eyes and the object, the position and shape difference of the object on the left and right eyes, , And various other psychological and memory effects.

Among them, binocular disparity is the most important factor for a person to feel the stereoscopic effect, as the eyes of a person are about 6.5 cm apart in the horizontal direction. In other words, the binocular parallax causes the angle of the object to be viewed with the difference. Due to this difference, the images coming into each eye have different phases. When these two images are transmitted to the brain through the retina, The 3D stereoscopic image of the original can be felt.

These stereoscopic 3D contents have already been widely used in various media fields and have been well received by consumers. For example, 3D movies, 3D games, and experience displays are examples.

As described above, in addition to the universalization of virtual reality technology 3D contents, there is a need to develop a technology capable of providing a more immersive virtual reality service.

2. Description of the Related Art Generally, an image display device forms a focal point so that a virtual large-sized screen can be formed at a long distance by using a precision optical device, which is generated in a position very close to an eye, so that a user can view an enlarged virtual image An image display device.

In addition, the image display device is configured to include a see-close type in which only the image light emitted from the display device can not be seen in the surrounding environment, and a see-close mode in which the image light emitted from the display device (See-through) that can be seen.

Head-mounted display (HMD) refers to various digital devices that can be worn on the head like glasses to receive multimedia contents. Various wearable computers (Wearable Computers) have been developed in accordance with the trend of weight reduction and miniaturization of digital devices, and HMDs are also widely used. The HMD can be combined with augmented reality technology, N screen technology, etc., beyond the simple display function, to provide various convenience to the user.

For example, when a microphone and a speaker are mounted on the HMD, the user can perform a telephone conversation while wearing the HMD. Further, for example, when a camera is mounted on the HMD, the user can capture an image of a desired direction in a state in which the HMD is worn.

At present, there is an increasing need to utilize the above-described virtual reality technology for a sensory-type educational program

Currently, various educational methods using a head-mounted display (HMD) or the like have been proposed. However, there is no system or method for efficiently educating users by evaluating the concentration of users using contents in real time or cumulatively, Therefore, there is a need for a solution to this problem.

Therefore, we propose a system which can carry out educational program without any admission to the place with a simple system that can be worn by individuals with technology that combines bio sensor technology such as EEG, electromyography, ECG sensor, smart glass technology and virtual reality contents technology There is a need.

Korean Patent Publication No. 10-2014-0001167 Korean Patent Publication No. 10-2013-0137692 Korean Patent Registration No. 10-1619468

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a system and a method for training a virtual reality system linked with a bio-signal.

Specifically, the present invention provides a system and an apparatus for increasing the educational efficiency by displaying the virtual reality education contents on a smart glass device and analyzing cumulative concentration and real time concentration of the user by interlocking the living body signal sensor.

The educational device proposed by the present invention is a structure capable of displaying virtual reality contents in a smart glass worn like a pair of glasses or a helmet, and can be used without any admission of a place, and can be attached to an electroencephalogram sensor or a user's body It is possible to perform real-time and quantitative analysis of the user's concentration, cognitive function, and exercise function using the possible EMG sensor and to provide effective education and training by feedback to the user. Also, The aim is to be able to train separately.

The training apparatus proposed by the present invention can provide user-motion and position-based contents by using a motion sensor, a camera, and an external interlocking Kinect sensor built in a smart glass, so that a bodily-sensible education and training system, And it is intended to provide body analysis data such as a stress index using an electrocardiograph sensor that can be attached to a human body.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.

The bio-signal interworking virtual reality education system according to an embodiment of the present invention for realizing the above-mentioned problems comprises an optical module for delivering video light emitted from at least one display to a user's eyeball, 1 head mounted display for displaying contents; A brain wave sensor for sensing at least one brain wave of the user; A controller for evaluating the user's concentration on the first content using at least one sensed EEG; And a memory for cumulatively storing the concentration of the user, wherein the controller can control the concentration of the user to be displayed in real time through the head mount display.

The control unit may stop displaying the first content on the head mount display when the concentration of the user evaluated in real time is less than a predetermined value.

In addition, the memory may store at least one second content previously designated by the user, and when the display of the first content is interrupted, the control unit causes the second content stored in the memory to be displayed on the head- Can be controlled.

Also, the controller may extract at least one first time period in which the concentration of the user is equal to or greater than a predetermined value, using the concentration information of the user accumulated in the memory.

In addition, the controller may stop displaying the first content on the head-mounted display when the time of displaying the first content is not within the first time period.

In addition, the controller may be configured such that the memory stores at least one second content previously designated by the user, and when the first content display is stopped, the control unit causes the second content stored in the memory to be displayed on the head- As shown in FIG.

The bio-signal interworking virtual reality education system includes a plurality of bio-signal interworking virtual reality education systems, and the plurality of bio-signal interworking virtual reality education systems are connected to a network through respective wireless communication units. And share the concentration of the plurality of users.

The control unit may evaluate the ranking of the concentration of the plurality of users and control each of the wireless communication units to share the ranking of the evaluated concentration in real time.

The control unit may extract at least one first time period in which the concentration of the plurality of users is equal to or greater than a predetermined value by using the concentration information of the plurality of users cumulatively stored in each of the plurality of memories.

Also, the head mount display may include a first interface unit disposed at an eyeball front of the user and fixing the user's head horizontally based on the position of the head mount display; And a second interface unit fixing the head of the user vertically with respect to the position of the head mount display.

In addition, the head mount display may be in the form of a goggle or a helmet.

The at least one EEG may include alpha waves, beta waves, theta waves, gamma waves, gamma waves, delta waves, SMR waves and Mid- beta waves .

According to another aspect of the present invention, there is provided a bio-signal interlocking virtual reality teaching method for realizing the above-mentioned problem, wherein a head-mounted display comprising an optical module for transmitting video light emitted from at least one display to a user's eye, A first step of displaying first contents related to the education of the user; A second step in which an EEG sensor senses at least one EEG of the user; A third step of the controller evaluating the user's concentration on the first content using at least one sensed EEG; A memory accumulating the concentration of the user; And a fifth step in which the concentration of the user is displayed in real time via the head mount display.

The method may further include a sixth step of stopping the first content display on the head mount display if the concentration of the user evaluated in real time after the fifth step is less than a predetermined value.

The method may further include a fifth step of storing the at least one second content previously designated by the user before the first step, and when the first content display is stopped in the sixth step, And displaying the second content stored in the memory on the head mount display instead of the second content.

The method may further include a sixth step of extracting at least one first time period in which the concentration of the user is equal to or greater than a predetermined value using the concentration information of the user accumulated in the memory after the fifth step .

The method may further include a seventh step of stopping the display of the first content on the head mounted display when the time of displaying the first content is not within the first time period after the sixth step .

The method according to claim 1, further comprising, before the first step, the memory storing at least one second content previously designated by the user, wherein, in the seventh step, And displaying the second content stored in the memory on the head mount display instead of the second content.

And a sixth step of communicating the concentration of the user to the outside by using the wireless communication unit after the fifth step, wherein the biological signal interlocking virtual reality education system includes a plurality of biological signal interlocking virtual And a seventh step of the reality education system forming a network through each wireless communication unit and sharing the concentration of the plurality of users.

After the seventh step, an eighth step of evaluating the ranking of the concentration of the plurality of users is performed. And a ninth step in which each of the wireless communication units shares the ranking of the degree of concentration evaluated in real time.

In the seventh and eighth steps, at least one first time period in which concentration of a plurality of users is equal to or greater than a predetermined value is extracted using cumulative information of a plurality of users cumulatively stored in each of the plurality of memories, Step;

Disclosure of Invention Technical Problem [8] The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a system and a method for real-time interactive virtual reality (VRT) education.

Specifically, the present invention can provide a system and an apparatus capable of increasing educational efficiency by displaying virtual reality education contents on a smart glass device and analyzing cumulative concentration and real time concentration of a user by interlocking with a bio-signal sensor .

The educational device proposed by the present invention is a structure capable of displaying virtual reality contents in a smart glass worn like a pair of glasses or a helmet, and can be used without any admission of a place, and can be attached to an electroencephalogram sensor or a user's body By using the available EMG sensor, it is possible to analyze the user's concentration, cognitive function and movement function in real time and quantitatively, and it is possible to perform training by level according to the degree of concentration and cognitive function.

Further, the educational apparatus proposed by the present invention can provide user-motion and position-based contents using a motion sensor, a camera, an external interlocking Kinect sensor, and an electronic pen incorporated in a smart glass, It can also be used as an exercise rehabilitation system. The body electrocardiogram sensor can be used to provide physical analysis data such as exercise index and stress index.

It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and, together with the description, serve to further the understanding of the technical idea of the invention, It should not be construed as limited.
1 shows an example of a block diagram of a bio-signal interlocking virtual reality (EDL) system proposed by the present invention.
Fig. 2 shows an example of a block diagram of a smart glass applied to the present invention.
FIG. 3A shows a specific example of a bio-signal interlocking virtual reality teaching device, and FIG. 3B shows an example of an electromyography sensor used in a bio-signal interlocking virtual reality teaching device.
FIG. 4A is a detailed view of a user using the virtual reality education apparatus illustrated in FIG. 3A and FIG. 3B, and FIG. 4B is a specific view used for prevention and treatment of ADHD.
FIG. 5 is a detailed view of a user using the virtual reality educational apparatus illustrated in FIG. 3A as a personalized video learning apparatus.
FIG. 6 is a flowchart illustrating a method for enhancing a training efficiency by analyzing cumulative concentration and real-time concentration of a user by interfacing a bio-signal sensor according to the present invention.
FIG. 7 is a flowchart illustrating a method by which a user can rest and recover a concentration of content by playing predetermined content at a point in time or at a time when the user's concentration becomes low, with reference to FIG.
FIG. 8 is a flowchart illustrating a method for forming a network between displays used by a plurality of users, and for providing information on real-time concentration ranking and concentration time zones of a plurality of users to users, with reference to FIG.

VIRTUAL REAL-TIME TECHNOLOGY In addition to the generalization of 3D contents, it is required to develop a technology capable of providing a more immersive virtual reality service.

Therefore, the present invention proposes a bio-signal interlocking virtual reality education device.

Specifically, the present invention is a technology which is a fusion technology of a biosensor technology such as an electroencephalogram sensor, an electromyogram sensor, an electrocardiograph sensor, a smart glass technology and a virtual reality contents technology, and is a simple system that can be worn by an individual. And a system for receiving education.

Prior to a specific description of the present invention, the basic structure of the present invention will be described with reference to the drawings.

1 shows an example of a block diagram of a bio-signal interlocking virtual reality (EDL) system proposed by the present invention.

Referring to FIG. 1, a bio-signal interworking virtual reality education system 100 includes a wireless communication unit 110, an audio / video (A / V) input unit 120, a user input unit 130, a sensing unit 140, A controller 160, a memory 160, an interface 170, a controller 180, and a power supply 190.

However, the components shown in FIG. 1 are not essential, so that a bio-signal interlocking virtual reality training system having more or fewer components than those shown in FIG. 1 may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between a bio-signal interworking virtual reality education system and a wireless communication system or between a device and a network in which the device is located.

For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may be a server for generating and transmitting broadcast signals and / or broadcast-related information, or a server for receiving broadcast signals and / or broadcast-related information generated in advance and transmitting the broadcast signals and / or broadcast- related information to a bio- have. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only And a Digital Broadcasting System (ISDB-T) (Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a radio signal to at least one of a base station, an external device, and a server on a mobile communication network.

And various types of data according to transmission / reception of text / multimedia messages.

The wireless Internet module 113 is a module for wireless Internet access, and may be embedded or externally installed in a bio-signal interworking virtual reality education system. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. As a short range communication technology, there are Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (UDA), Ultra Wideband (UWB), ZigBee, Wireless Fidelity (Wi-Fi) A Wi-Di (Wireless Display) or the like may be used.

The position information module 115 is a module for acquiring the position of the bio-signal linked virtual reality education system, and a representative example thereof is a GPS (Global Position System) module.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal by a microphone in a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the bio-signal interfacing virtual reality education system. The user input unit 130 may receive a sensor signal generated from a sensor of the sensing unit 140 through a signal processing process. Also, it can be composed of a key pad dome switch, a touch pad (static pressure / static electricity), an electronic pen, a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the opening / closing state of a bio-signal interlocking virtual reality education system, the position of a virtual reality interlocking virtual reality education system, the presence or absence of user contact, the orientation of a virtual reality interlocking system, / Deceleration and the like, and generates a sensing signal for controlling the operation of the bio-signal interlocking virtual reality education system by detecting the current state of the bio-signal interlocking virtual reality education system.

The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating an output relating to visual, auditory or tactile sense and includes a display unit 151, an acoustic output module 152, an alarm unit 153, a haptic module 154, 155, and the like.

The display unit 151 displays (outputs) information processed in the bio-signal linked virtual reality education system.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) liquid crystal on silicon, a microdisplay, a flexible display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may be a light transmission type structure. According to this structure, the user can view objects located behind the body of the bio-signal interlocking virtual reality education system body through the area occupied by the display unit 151 of the bio-signal interlocking virtual reality education system body.

There may be two or more display units 151 according to the implementation form of the bio-signal interlocking virtual reality education system. For example, in a bio-signal interlocking virtual reality education system, a plurality of display units may be spaced apart from one another, disposed integrally, or disposed on different surfaces.

(Hereinafter, referred to as a 'touch screen') in which a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') form a mutual layer structure, It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in an internal region of the bio-signal interlocking virtual reality education system wrapped by the touch screen or in the vicinity of the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as " proximity touch & The act of actually touching the pointer on the screen is called " contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

In addition, the system 100 according to the present invention may include an EEG sensor 142.

Electro-encephalography (EEG) is a measurement of the signal generated by the brain on an electrode. It is obtained by measuring the minute brain surface signals that are synthesized by electrical signals generated from numerous neurons in the brain.

EEG signals change in time and space according to brain activity, measurement state, and brain function. Thus, EEG signals have frequency characteristics according to frequency, time characteristics, and spatial characteristics related to brain function.

EEG waves are divided into delta, theta, alpha, beta, and gamma waves depending on the frequency and voltage range. Table 1 below shows the wavelengths according to the frequency band and physical condition of the brain waves.

Table 1 shows the wavelengths according to the frequency band and physical condition of EEG.

The brain wave sensor 141 according to the present invention provides a function of detecting at least one of the plurality of brain waves.

In addition, the system 100 according to the present invention may include a motion sensor 143.

The motion sensor 143 according to the present invention refers to a technique of moving an object or a person's movement to a digital using a gyroscope sensor.

The motion sensor 143 may be classified into a mechanical type, a magnetic type, an optical type, and the like according to the mode of motion capture.

In addition, the system 100 according to the present invention may include an electromyogram sensor 144.

The electromyogram is the electrical activity of skeletal muscle detected by surface electrode or needle electrode.

 EMG generally has an excellent effect in understanding the diseases of the peripheral nervous system.

Myasthenia gravis, progressive muscular stroma, multiple myositis, etc. have characteristic EMG patterns.

The electromyogram sensor 144 according to the present invention senses a change in the electromyogram to provide a signal for a content link control function such as left / right, forward / backward manipulation, selection, and the like.

In addition, the system 100 according to the present invention may include an electrocardiogram sensor 145.

Electrocardiogram (ECG) is the detection of the cardiac electrical activity as a surface electrode.

From the electrocardiogram signal measured from the electrocardiogram sensor, heart rate measurement during exercise, angina pectoris such as angina pectoris, myocardial infarction, arrhythmia, electrolyte abnormality diagnosis, and the like are possible.

The electrocardiogram sensor according to the present invention can sense a change in the electrocardiogram and provide a signal for a movement index and a stress index calculation.

Meanwhile, the audio output module 152 can output audio data received from the wireless communication unit 110 or stored in the memory 160 in a recording mode, a voice recognition mode, a broadcast receiving mode, and the like. The sound output module 152 also outputs sound signals related to the functions performed in the bio-signal interworking virtual reality education system. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the bio-signal interlocking virtual reality education system.

The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration.

The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that they may be classified as a part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled.

For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the bio-signal interlocking virtual reality education system.

The projector module 155 is a component for performing an image project function by using a bio-signal interlocking virtual reality education system and includes a display unit 151, To display an image on the external screen or on the wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided on the side, the front side, or the back side in the longitudinal direction of the bio-signal interlocking virtual reality education system. It goes without saying that the projector module 155 may be provided at any position in the bio-signal interlocking virtual reality education system as needed.

In addition, the present invention may include a smart glass 156.

The smart glass 156 proposed by the present invention may be a head mount display (HMD).

Fig. 2 shows an example of a block diagram of a smart glass applied to the present invention.

2, an HMD 156 according to an embodiment of the present invention includes a display unit 1110, a communication unit 1120, a sensing unit 1130, a camera unit 1140, and a processor 1150 .

The HMD 156 may include a first body and a second body. The first body corresponds to the body of the HMD 156 and may include a display unit 1110, a first communication unit, a first motion sensing unit, a processor 1150, and the like. The second body may be a body detachable from the body of the HMD 156, and may include a second motion sensing unit, a camera unit 1140, a second communication unit, and the like. The above-described first body and second body are, as one embodiment, capable of changing some of the constituent units or adding new constituent units to the first body and the second body, as required by those skilled in the art.

Display unit 1110 may display visual information. Here, the visual information may include contents, applications, images, moving pictures, and the like. Display unit 1110 may also output visual information to the screen based on control instructions of processor 1150. [ In this specification, the display unit 1110 may be included in the first body of the HMD 156.

Meanwhile, in this specification, the HMD 156 can output an image on a display screen in various ways. As an example, the HMD 156 may output an image in a see-through manner.

Here, the see-through method is a method in which the display screen is transparent and the user can use the contents while recognizing the surrounding environment while wearing the HMD 156. As another example, the HMD 156 may output an image in a front-light manner. Here, the forward scanning method represents a method of displaying a reflected image through a reflector such as a mirror without projecting the light directly onto the eye.

Also, as another example, the HMD 156 may output an image in a see-closed fashion. Here, the dark type method can not view the external environment through the display screen, and shows the manner of using the contents through the display screen. In the present specification, the HMD 156 is assumed to display an image in a perspective or a film-like manner.

The communication unit 1120 can communicate with external devices and transmit / receive data using various protocols. In addition, the communication unit 1120 can connect to a network by wire or wireless and send / receive digital data such as contents. For example, the communication unit may be a WLAN (Wireless LAN), Wi-Fi, Wi-Di, Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA Access) communication standard can be used.

In this specification, the communication unit 1120 may include a first communication unit and a second communication unit. As described above, the first communication unit may be provided in the first body, and the second communication unit may be provided in the second body. For example, the first communication unit and the second communication unit transmit / receive signals from / to the processor 1150 included in the first body, and transmit / receive signals from the camera unit 1140 included in the second body .

The sensing unit 1130 may sense the ambient environment of the HMD 156 using at least one sensor mounted on the HMD 156 and deliver it to the processor 1150 in the form of a signal.

The sensing unit 1130 may include at least one sensing means. In one embodiment, the at least one sensing means includes at least one sensor selected from the group consisting of a gravity sensor, a geomagnetism sensor, a motion sensor, a gyroscope sensor, an acceleration sensor, an infrared sensor, an inclination sensor, a brightness sensor, A sensor, a depth sensor, a pressure sensor, a banding sensor, an audio sensor, a video sensor, a global positioning system (GPS) sensor, a touch sensor, and a grip sensor.

The sensing unit 1130 collectively refers to the various sensing means described above and senses the various inputs of the user and the environment of the HMD 156 and transmits the sensing result so that the processor 1150 can perform the operation accordingly . The sensors described above may be included in the HMD 156 as separate elements or integrated into at least one element.

In this specification, the sensing unit 1130 may include an angle sensing unit 131 and an input sensing unit 132. In addition, the angle sensing unit 131 and the input sensing unit 132 may be provided in the first body and the second body, respectively. For example, the angle sensing unit of the first body may sense the rotation angle of the first body, and the input sensing unit of the first body may detect the input signal of the user to the first body have. In addition, for example, the angle sensing unit provided on the second body may sense the rotation angle of the second body, and the input sensing unit provided on the second body may detect a user's input signal with respect to the second body, can do. Also, a separation signal and a mounting signal between the first body and the second body can be detected through various sensing means provided on the first body and the second body. Also, the image capturing signal can be detected through the input sensing unit provided in the first body or the second body.

The camera unit 1140 can take an image. More specifically, the camera unit 1140 can take an image in the forward direction. Here, the front direction may correspond to the direction in which the camera unit 1140 faces. In addition, the camera unit 1140 may sense the image within the field of view area and provide it to the processor 1150. Here, the angle of view field represents the range of horizontal and vertical viewing angles that can be included in a certain screen when sensing an image.

In this specification, the camera unit 1140 may be included in the second body. Meanwhile, in this specification, the camera unit 1140 may be provided not only to the second body but also to the first body. The camera unit 1140 included in the second body may be mounted on the first body to sense an image. In addition, the camera unit 1140 included in the second body may be separated from the first body to sense the image.

Processor 1150 can process data, control each unit of HMD 156 described above, and control data transmission / reception between units. In this specification, processor 1150 may be included in the first body. For example, the processor 1150 may include a display unit 1110, a first angle sensing unit, a first communication unit, as well as a camera unit 1140 included in the second body, a second communication unit The sensor unit 1130, and the second angle sensing unit. The processor 1150 may be provided separately from the first body as well as the second body.

In the present specification, when the second body is separated from the first body, the processor 1150 can obtain the first rotation angle and the second rotation angle. In addition, the processor 1150 may display a separate image preview interface that displays an image obtained by performing a first rotation compensation on the sensed image based on the second rotation angle. Processor 1150 may also detect the first capturing signal. The processor 1150 may also capture the sensed image and store the captured image a second rotationally compensated image based on the first rotational angle and the second rotational angle.

In one embodiment of the present disclosure, the operations performed by the HMD 156 may be controlled by the processor 1150. [ For convenience, the drawings and the following description are collectively referred to as performing / controlling the HMD 156.

2, the HMD 156 may include a power unit, a storage unit, an audio unit, and the like.

The power unit can supply power to the HMD 156 as a power source connected to a battery or an external power source inside the HMD 156. [ In addition, the storage unit can store various digital data such as audio, pictures, movies, and applications. The storage unit may represent various digital data storage spaces such as a flash memory, a random access memory (RAM), and a solid state drive (SSD). Further, the audio unit can receive or output audio data through a microphone and a speaker.

In addition, the HMD 156 proposed by the present invention can be manufactured in a goggle type.

The memory unit 160 may store a program for processing and controlling the control unit 180 and temporarily store input / output data (e.g., message, audio, still image, For example. The frequency of use of each of the data may be stored in the memory unit 160 as well. In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The bio-signal interworking virtual reality education system may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a pathway to all external devices connected to the bio-signal interworking virtual reality education system. The interface unit 170 receives data from an external device or supplies power to each component in the bio-signal interlocking virtual reality education system, or transmits data to the external devices in the biology signal interlocking virtual reality education system do. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the usage right of the bio-signal linked virtual reality education system. The identification module includes a user identification module (UIM), a subscriber identity module (SIM) A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Therefore, the identification device can be connected to the bio-signal interlocking virtual reality education system through the port.

When the bio-signal interlocking virtual reality education system is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the bio-signal interworking virtual reality education system, Signal may be the path through which the mobile device is communicated. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile device is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the bio-signal interworking virtual reality education system.

The control unit 180 may transmit a control signal for operating the hardware of the output unit or may store the control signal in the memory by using the signal input through the wireless communication unit, the A / V input unit, the user input unit, the sensing unit, and the interface unit. The control unit may include a processor module for analyzing the sensor signal to calculate an analysis index such as a cognitive function index and a motor function index. The control unit can receive a signal from the memory and control the function of the output unit.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, controllers, micro-controllers, microprocessors, and other electronic units for performing other functions. In some cases, The embodiments described may be implemented by the control unit 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

FIG. 3A shows a specific example of a bio-signal interlocking virtual reality teaching device, and FIG. 3B shows an example of an electromyography sensor used in a bio-signal interlocking virtual reality teaching device.

Referring to FIG. 3A, the bio-signal interworking virtual reality apparatus 100 proposed by the present invention may include a band-shaped interface unit 170, a brain wave sensor 142, and a smart glass 156.

Also, as shown in FIG. 3B, the bio-signal interlocking virtual reality teaching apparatus 100 proposed by the present invention may further comprise an electromyogram sensor 144.

Referring to FIG. 3A, the band-shaped interface unit 170 includes a first portion 170a coupled to the user's head vertically with respect to the forehead and a second portion 170b coupled to the user's head horizontally do.

In addition, the brain-wave sensor 142 is disposed near the forehead or the temple of the user to provide a function of measuring a user's brain wave.

As described above, the brain-wave sensor 142 can detect delta, theta, alpha, beta, gamma wave, and the like, which are classified according to the frequency and voltage range.

The smart glass 156 may be a head mounted display (HMD), and may be provided in the form of a goggle.

The user can be provided with the virtual reality content through the smart glass 156.

3B, the electromyogram sensor 144 may include a plurality of sensors, and may have a bracelet shape that is filled in at least a part of the user's arm.

As described above, it is possible to sense the electrical activity of the skeletal muscle of the user through the electromyogram sensor 144. [

FIG. 4A is a detailed view of a user using the virtual reality education apparatus illustrated in FIGS. 3A and 3B.

4A, a user has a band-like interface (not shown) including a first portion 170a coupled to the user's head vertically with respect to the forehead and a second portion 170b coupled to the user's head horizontally 170 may be connected to the bio-signal interworking virtual reality teaching apparatus 100. [

In addition, a goggle-type smart glass 156 is disposed on the front of the user's eye, an EEG sensor 142 is disposed near the forehead or the temple of the user, and the EMG sensor 144 is mounted on the user's arm in the form of a bracelet do.

The bio-signal interworking virtual reality education apparatus 100 proposed by the present invention can be used for prevention and treatment of cognitive dysfunction such as attention deficit hyperactivity disorder (ADHD) and dementia.

That is, for the treatment of hyperactivity disorder (ADHD), the contents of the virtual reality are displayed through the smart glass 156 in the form of goggles and solved the problem of presenting the contents through the control based on the signal of the EMG sensor 144. The concentration of the user can be checked through the brain wave sensor 142 and the content of the virtual reality can be changed and displayed only when the degree of concentration is high.

In addition, in order to prevent and treat cognitive dysfunction such as dementia, the contents of the virtual reality are displayed through the goggle type smart glass 156, and the problem and the action given to the user are solved through the electromyogram sensor 144 , The content of the virtual reality is changed and displayed only when it is solved, so that it is possible to prevent the decline of the cognitive function together with the strength movement.

In addition, the smart glass 156 according to the present invention is a structure that can be easily worn by a user in the form of a spectacles, a headband, or a helmet to enjoy virtual reality contents.

In addition, the virtual reality contents according to the present invention are contents for enhancing and treating cognitive functions, and can be configured to control the degree of difficulty according to cognitive functions (dementia, ADHD, etc.) Status analysis and reporting is possible.

The smart glass 156 according to the present invention enlarges and displays cognitive rehabilitative contents on the user's eyes.

The brain wave sensor 142, the motion sensor 143, the camera 121, the audio 150 and the wired / wireless communication module 110 are built in and utilize a bio-signal, a position-based signal, Content composition is possible.

Particularly, it is possible to perform neuro feedback training and cognitive function analysis using EEG signals, and to upload and upgrade contents using the wired / wireless transmission / reception module 110.

In addition, the electromyogram sensor 144 according to the present invention is a wrist band type multi-channel electromyography measuring system.

The EMG sensor 144 can transmit and receive an EMG signal by incorporating a wireless transmission / reception module, and can incorporate a motion sensor 143 and a position sensor 115. The EMG sensor and the muscle index can be calculated using the EMG sensor 144.

The bio-signal interworking virtual reality teaching apparatus 100 according to the present invention can be used for home use.

FIG. 4B is a view showing a concrete use of the bio-signal interlocking virtual reality education system proposed by the present invention in the prevention and treatment of attention deficit hyperactivity disorder (ADHD).

Referring to FIG. 4B, the bio-signal interlocking virtual reality teaching apparatus 100 according to the present invention can be used as an ADHD, a dementia cognitive function enhancing training machine.

That is, it is possible to induce interest by using game contents according to degree of difficulty, to provide various contents, to upgrade the contents of a new version, and to perform cognitive function analysis To monitor treatment and prevention effects through self-diagnosis.

On the other hand, various educational methods using a head-mounted display (HMD) or the like have been proposed. However, there is no system or method for evaluating the concentration of users using contents in real time or cumulatively, Therefore, there is a need for a solution to this problem.

Therefore, we propose a system which can carry out educational program without any admission to the place with a simple system that can be worn by individuals with technology that combines bio sensor technology such as EEG, electromyography, ECG sensor, smart glass technology and virtual reality contents technology There is a need.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a system and method for training a virtual reality in a bio-signal interlocking system.

Specifically, the present invention provides a system and an apparatus for increasing the educational efficiency by displaying the virtual reality education contents on a smart glass device and analyzing cumulative concentration and real time concentration of the user by interlocking the living body signal sensor.

The bio-signal interworking virtual reality teaching apparatus 100 proposed by the present invention can be used as a personal wearable learning machine which can be typically worn by an individual and can learn moving pictures.

FIG. 5 is a detailed view of a user using the virtual reality educational apparatus illustrated in FIG. 3A as a personalized video learning apparatus.

Referring to FIG. 5, it is possible to learn by using various text or moving picture contents worn by individuals at home, to pay for upgrading contents of a new version, to provide a result of concentration analysis through bio-signals, Which can be used as a system capable of learning in an effective learning environment.

Specific methods of the present invention will be described with reference to the drawings.

First Embodiment

In the first embodiment of the present invention, the cumulative concentration and the real-time concentration of the user are analyzed by interlocking the bio-signal sensor.

FIG. 6 is a flowchart illustrating a method for enhancing a training efficiency by analyzing cumulative concentration and real-time concentration of a user by interfacing a bio-signal sensor according to the present invention.

Referring to FIG. 6, first, the content related to education is played through the HMD 156 (S100).

Here, the education-related contents may include text, image, video, and the like.

In addition, the education-related contents to be played may be received from the outside through the wireless communication unit 110 and played or stored in the memory 160 in advance and then played after loading.

When the education-related contents are played, a step S200 of sensing at least one brain wave of the user using the brain-wave sensor 142 is performed.

As described above, at least one EEG includes at least one of alpha-beta, beta-beta, theta, gamma wave, delta wave, SMR wave and Mid- beta wave .

Then, the control unit 180 evaluates the real-time concentration degree of the user using at least one sensed EEG and performs a step S300 of feeding back to the user.

That is, a change of at least one of alpha-wave (alpha), beta wave (beta), theta wave (?), Gamma wave (?), Delta wave (?), SMR wave and Mid- The concentration can be fed back to the user in real time.

In the present embodiment, evaluation of the user's concentration using the EEG is described as a representative example. However, it is also possible to accurately determine the concentration of the user by further applying the user's gaze tracking function.

In addition, the sensing information through the electrocardiographic sensor described above may be additionally utilized.

Feedback of the degree of concentration according to the control unit 180 may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

Then, in accordance with the control of the control unit 180, the memory 160 stores the evaluated real time concentration degree (S400).

Thereafter, when the play of the educational contents is terminated or a predetermined period of time elapses, the controller 180 analyzes the concentration degree stored in the memory 160 for a predetermined period of time, evaluates a time period during which the user can concentrate well, (S500).

For example, as a result of collecting and analyzing the user's concentration for a week, the user can feedback the evaluation information that the concentration from 13:00 to 17:00 is the highest and the concentration from 06:00 to 08:00 is the lowest.

Evaluation feedback for a predetermined period may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

In the present invention, an electronic pen automatically interlocked with the HMD 156 may be additionally used.

That is, the user may control the operation of the HMD 156 through the operation of the user's electronic pen, or the user may perform handwriting while the education-related contents are being played, thereby allowing the memory to store it.

Second Embodiment

Meanwhile, in the second embodiment of the present invention, the cumulative concentration and the real-time concentration of the user are interlocked by interlocking the bio-signal sensor, and the user can play the pre-set contents at the time or when the concentration of the user drops, I can recover.

FIG. 7 is a flowchart illustrating a method by which a user can rest and recover a concentration of content by playing predetermined content at a point in time or at a time when the user's concentration becomes low, with reference to FIG.

Referring to FIG. 7, first, the content related to education is played through the HMD 156 (S100).

Here, the education-related contents may include text, image, video, and the like.

In addition, the education-related contents to be played may be received from the outside through the wireless communication unit 110 and played or stored in the memory 160 in advance and then played after loading.

When the education-related contents are played, a step S200 of sensing at least one brain wave of the user using the brain-wave sensor 142 is performed.

As described above, at least one EEG includes at least one of alpha-beta, beta-beta, theta, gamma wave, delta wave, SMR wave and Mid- beta wave .

Then, the control unit 180 evaluates the real-time concentration degree of the user using at least one sensed EEG and performs a step S300 of feeding back to the user.

That is, a change of at least one of alpha-wave (alpha), beta wave (beta), theta wave (?), Gamma wave (?), Delta wave (?), SMR wave and Mid- The concentration can be fed back to the user in real time.

Feedback of the degree of concentration according to the control unit 180 may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

In this case, in the second embodiment, when the concentration of the user who has been evaluated by the controller 180 is equal to or less than a predetermined degree of concentration, the process proceeds to step S310 of stopping the play of the education contents.

That is, since the effect of playing the educational contents is inevitable in a state where the concentration is low, when the concentration of the evaluated user is equal to or less than a preset concentration level, .

If the answer to the question is YES, predefined contents may be played.

In addition, the controller 180 may cause the HMD 156 to play the content previously designated by the user.

For example, a music list, a video list, and the like designated by the user may be displayed through the HMD 156 to induce the rest of the user.

If the user-specified content is played for a predetermined period of time, the control unit 180 may stop the play and cause the content related to the education to be played again.

Then, in accordance with the control of the control unit 180, the memory 160 stores the evaluated real time concentration degree (S400).

Thereafter, when the play of the educational contents is terminated or a predetermined period of time elapses, the controller 180 analyzes the concentration degree stored in the memory 160 for a predetermined period of time, evaluates a time period during which the user can concentrate well, (S500).

For example, as a result of collecting and analyzing the user's concentration for a week, the user can feedback the evaluation information that the concentration from 13:00 to 17:00 is the highest and the concentration from 06:00 to 08:00 is the lowest.

Evaluation feedback for a predetermined period may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

If the education-related content is played again after the feedback on the focusable time through S500 is delivered, the control unit 180 stops the play of the educational contents in a time period other than the time zone in which the user can concentrate (S510) Can be performed.

That is, since the effect of playing the educational contents is inevitable in a time zone in which the concentration is low, it is possible to automatically stop the play of the educational contents or check whether the execution of the education contents is executed in a time zone where the user's concentration is low.

In addition, the control unit 180 may cause the HMD 156 to play the content previously designated by the user (S520).

For example, a music list, a video list, and the like designated by the user may be displayed through the HMD 156 to induce the rest of the user.

If the user-specified content is played for a predetermined period of time, the control unit 180 may stop the play and cause the content related to the education to be played again.

Third Embodiment

In the third embodiment of the present invention, a network is formed between displays used by a plurality of users, and information on the real-time concentration ranking and the concentration time zones of a plurality of users can be provided to users.

FIG. 8 is a flowchart illustrating a method for forming a network between displays used by a plurality of users, and for providing information on real-time concentration ranking and concentration time zones of a plurality of users to users, with reference to FIG.

Referring to FIG. 8, first, a step S50 of forming a network between a plurality of HMDs 156 is performed.

Where the network may be established using local or remote communication.

The local area communication may include Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and Wi-Fi (Wireless Fidelity).

Further, the long-distance communication may be performed by using a code division multiple access (CDMA), a frequency division multiple access (FDMA), a time division multiple access (TDMA), an orthogonal frequency division multiple access (OFDMA), a single carrier frequency division multiple access . ≪ / RTI >

Thereafter, a step S100 of playing contents related to education through the HMD 156 is performed.

Here, the education-related contents may include text, image, video, and the like.

In addition, the education-related contents to be played may be received from the outside through the wireless communication unit 110 and played or stored in the memory 160 in advance and then played after loading.

When the education-related contents are played, a step S200 of sensing at least one brain wave of the user using the brain-wave sensor 142 is performed.

As described above, at least one EEG includes at least one of alpha-beta, beta-beta, theta, gamma wave, delta wave, SMR wave and Mid- beta wave .

Then, the control unit 180 evaluates the real-time concentration degree of the user using at least one sensed EEG and performs a step S300 of feeding back to the user.

That is, a change of at least one of alpha-wave (alpha), beta wave (beta), theta wave (?), Gamma wave (?), Delta wave (?), SMR wave and Mid- The concentration can be fed back to the user in real time.

Feedback of the degree of concentration according to the control unit 180 may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

Unlike the first and second embodiments, after step S300, the controller 180 evaluates the real-time concentration ranking of a plurality of users, and the concentration ranking evaluated through the wireless communication unit 110 is shared among the plurality of displays (S330).

Through the step S330, it is possible to induce competition between the users, thereby acquiring the education contents with higher concentration.

Then, in accordance with the control of the control unit 180, the memory 160 stores the evaluated real time concentration degree (S400).

Thereafter, when the play of the educational contents is terminated or a predetermined period of time elapses, the controller 180 analyzes the concentration degree stored in the memory 160 for a predetermined period of time, evaluates a time period during which the user can concentrate well, (S500).

For example, as a result of collecting and analyzing the user's concentration for a week, the user can feedback the evaluation information that the concentration from 13:00 to 17:00 is the highest and the concentration from 06:00 to 08:00 is the lowest.

Evaluation feedback for a predetermined period may be displayed on the HMD 156 or displayed to the user through the sound output module 152, the alarm unit 153, or the projector module 155, and the like.

In addition, in the third embodiment, a step S530 of adding an average of a plurality of users to the concentrated time zone and feedback to the user is added.

Therefore, the user can acquire information on the concentration time zone of the user, as well as the information on the concentration time zone, which is higher in average on the average of the plurality of users.

The present invention can provide a bio-signal interlocking virtual reality education system and method.

Specifically, the present invention can provide a system and an apparatus capable of increasing educational efficiency by displaying virtual reality education contents on a smart glass device and analyzing cumulative concentration and real time concentration of a user by interlocking with a bio-signal sensor .

The educational device proposed by the present invention is a structure capable of displaying virtual reality contents in a smart glass worn like a pair of glasses or a helmet, and can be used without any admission of a place, and can be attached to an electroencephalogram sensor or a user's body Using this possible EMG sensor, real-time and quantitative analysis of the user's concentration, cognitive function, and motor function is possible, and it is possible to perform training according to the degree of concentration and level-specific training according to the degree of cognitive function.

Further, since the education apparatus proposed by the present invention can provide user-motion and position-based contents using a motion sensor, a camera, and an external interlocking Kinect sensor built in a smart glass, it is possible to provide not only a sensible training but also cognitive rehabilitation, System, and can provide body analysis data such as stress index and exercise state using an electrocardiograph sensor that can be attached to a human body.

In addition, the present invention can be used for the prevention and treatment of attention deficit hyperactivity disorder (ADHD) and for the prevention and treatment of dementia in relation to prevention and treatment of cognitive dysfunction.

The present invention can be used for cognitive function rehabilitation and motor function rehabilitation due to brain dysfunction, cognitive function strengthening training, and attention and concentration training in relation to prevention and treatment of brain dysfunction.

Further, the present invention may be used for a concentration game using biological signals in connection with the game market.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet) .

In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers of the technical field to which the present invention belongs.

It should be noted that the above-described apparatus and method are not limited to the configurations and methods of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments are selectively combined .

Claims (21)

A head mount display comprising an optical module for transmitting video light emitted from at least one display to a user's eyeball and displaying first content related to a user's education;
A brain wave sensor for sensing at least one brain wave of the user;
A controller for evaluating the user's concentration on the first content using at least one sensed EEG; And
And a memory for cumulatively storing the concentration of the user,
Wherein,
And controls the user to display the concentration of the user in real time through the head mount display. The method according to claim 1,
Wherein,
When the concentration of the user evaluated in real time is less than a predetermined value,
And stops displaying the first content on the head mount display. 3. The method of claim 2,
Wherein the memory stores at least one second content previously designated by the user,
When the first content display is stopped,
Wherein the controller controls the second content stored in the memory to be displayed on the head-mounted display instead of the second content. The method according to claim 1,
Wherein,
Wherein at least one first time interval in which the concentration of the user is equal to or greater than a predetermined value is extracted using the concentration information of the user accumulated in the memory. 5. The method of claim 4,
Wherein,
When the time at which the first content is displayed is not within the first time period,
And stops displaying the first content on the head mount display. 6. The method of claim 5,
Wherein,
Wherein the memory stores at least one second content previously designated by the user,
When the first content display is stopped,
Wherein the controller controls the second content stored in the memory to be displayed on the head-mounted display instead of the second content. The method according to claim 1,
And a wireless communication unit for externally communicating the concentration of the user,
The bio-signal interlocking virtual reality education system is a plurality of,
Wherein the plurality of bio-signal interworking virtual reality education systems form a network through respective wireless communication units and share the concentration of the plurality of users. 8. The method of claim 7,
Wherein,
Evaluating a ranking of the concentration of the plurality of users,
Wherein each of the wireless communication units controls the ranking of the degree of concentration to be shared in real time. 8. The method of claim 7,
Wherein,
Wherein at least one first time interval in which the concentration of a plurality of users is equal to or greater than a predetermined value is extracted using the concentration information of a plurality of users cumulatively stored in each of the plurality of memories, . The method according to claim 1,
Wherein the head mount display is disposed at an eyeball front of the user,
A first interface unit for horizontally fixing the head of the user based on a position of the head mount display; And
And a second interface for fixing the user's head vertically with respect to the position of the head mount display. 3. The method of claim 2,
Wherein the head mount display is a goggle type or a helmet type. The method according to claim 1,
Wherein the at least one EEG comprises:
A virtual-reality virtual reality education system comprising an alpha wave, a beta wave, a theta wave, a gamma wave, a delta wave, an SMR wave and a Mid- . A head mount display comprising an optical module for transmitting image light emitted from at least one display to a user's eye, the first mount displaying a first content related to a user's education;
A second step in which an EEG sensor senses at least one EEG of the user;
A third step of the controller evaluating the user's concentration on the first content using at least one sensed EEG;
A memory accumulating the concentration of the user; And
And a fifth step of displaying the concentration of the user in real time through the head mount display. 14. The method of claim 13,
After the fifth step,
And stopping the first content display on the head mount display if the concentration of the user evaluated in real time is less than a preset value. 15. The method of claim 14,
Before the first step, the memory further stores at least one second content previously designated by the user,
And displaying the second contents stored in the memory instead on the head mount display when the first content display is interrupted in the sixth step Way. 14. The method of claim 13,
After the fifth step,
And a sixth step of extracting at least one first time interval in which the concentration of the user is equal to or greater than a predetermined value by using the concentration information of the user cumulatively stored in the memory, Way. 17. The method of claim 16,
After the sixth step,
And stopping the display of the first content on the head-mounted display if the time for displaying the first content is not within the first time period. Way. 18. The method of claim 17,
Prior to the first step,
Wherein the memory stores at least one second content previously designated by the user,
And displaying the second contents stored in the memory instead on the head mount display when the first contents display is interrupted in the seventh step. Way. 14. The method of claim 13,
After the fifth step,
And a sixth step of communicating the concentration of the user to the outside using a wireless communication unit,
The bio-signal interlocking virtual reality education system is a plurality of,
Further comprising a seventh step of the plurality of bio-signal interworking virtual reality education systems forming a network through respective wireless communication units and sharing the concentration of the plurality of users. . 20. The method of claim 19,
After the seventh step,
An eighth step of evaluating a ranking of the concentration of the plurality of users; And
And each of the wireless communication units shares the ranking of the degree of concentration evaluated in real time by the wireless communication unit. 20. The method of claim 19,
After the seventh step,
And extracting at least one first time period in which the concentration of the plurality of users is equal to or greater than a predetermined value by using the concentration information of the plurality of users cumulatively stored in each of the plurality of memories, Vital Signal Interlocking Virtual Reality Education Method.

Images