KR101799980B1 - Apparatus, system and method for controlling virtual reality image and simulator - Google Patents

Abstract

The present invention relates to an apparatus, a system and a method for controlling a virtual reality image and a motion simulator, and more particularly, to an apparatus and method for controlling a virtual reality image and a motion simulator by using an output device for displaying virtual reality image contents, The apparatus comprising: an input unit for receiving a user's brain wave or voice signal; A database in which brain waves or voice signals and motion data of a character in a virtual reality image are mapped and stored; And a control unit for retrieving motion data of the character from the database based on the inputted EEG or voice signal and synchronously controlling the motion simulator with the virtual reality image contents displayed on the output device based on the motion data.

Description

TECHNICAL FIELD [0001] The present invention relates to an apparatus, a system, and a method for controlling a virtual reality image and a motion simulator,

The present invention relates to an apparatus, a system and a method for controlling a virtual reality image and a motion simulator. More particularly, the present invention relates to an apparatus, system and method for controlling a virtual reality space and a motion simulator by using a brain wave or a voice signal as an input signal, To an apparatus, a system and a method for controlling an output device for displaying contents and a motion simulator on which a user can ride.

Virtual Reality (VR) is the application of multimedia technology to make an environment that does not exist in reality feel like it actually exists. Such a virtual reality is called 'virtual reality' in order to display a processing world or a remote space using various images or computer graphics to make it feel as if it is in the spot.

In recent years, virtual reality contents using HMD (Head Mounted Display), which can be worn on a human head and enjoyed virtual reality, are provided. They are used in various fields such as education, entertainment, games, and theme parks.

In particular, the market of a system combining a motion simulator that moves to match the contents of a virtual reality image displayed by the HMD is actively expanding. In the system in which the motion simulator is combined with the virtual reality image, the movement of the motion simulator determined according to the progress time of the image is provided in accordance with the order programmed in advance, and the virtual reality image is changed in real- So that the motion simulator can not have a corresponding motion. This not only reduces the effects of virtual reality, but also limits the use of it in various ways.

Korean Patent Publication No. 10-2015-0135154: Virtual Reality Motion Simulator

According to an embodiment of the present invention, there is provided an apparatus and method for moving a character in a virtual reality space according to a command desired by a user and controlling a virtual reality image content and a motion simulator .

In this case, it is aimed to control the motion of the virtual reality image contents and the motion simulator by using brain waves or voice signals.

The technical problem to be solved by the embodiments of the present invention is not limited to the technical problems mentioned above, and various technical problems can be derived from the contents to be described below within a range obvious to the ordinary artisan.

In order to solve the above-mentioned problems, a virtual reality video and motion simulator control apparatus includes an output device for displaying virtual reality video contents, a control device for controlling a motion simulator operating in synchronization with the virtual reality video, An input unit for receiving a user's brain wave or voice signal; A database in which the brain wave or voice signal and motion data of a character in the virtual reality image are mapped and stored; And a controller for retrieving motion data of the character from the database based on the inputted EEG or voice signal and for controlling the motion simulator in synchronization with the virtual reality image contents displayed on the output device based on the motion data, .

At this time, the motion data may include motions related to the moving direction, speed, rotation, and posture of the character.

Also, the displayed virtual reality image content includes a virtual reality space background viewed from the character's viewpoint, and the displayed virtual reality space background includes position coordinates of the character in the virtual reality space, Can be determined on a basis.

In addition, the inputted EEG or speech signal can be displayed in the virtual reality image content by converting the contents of the mapped motion data into text.

In addition, the control device may be mounted on an HMD (Head Mounted Device) device.

In addition, the rate at which the motion data is reflected to the virtual reality image contents and the motion simulator may vary according to the magnitude of the amplitude of the input EEG or voice signal.

The virtual reality system includes an output device for outputting a virtual reality image; A motion simulator that operates in synchronization with the most realistic image and on which a person can ride; And a control device for controlling the output device and the motion simulator, wherein the control device comprises: an input unit for receiving a user's brain wave or voice signal; A database in which the brain wave or voice signal and motion data of a character in the virtual reality image are mapped and stored; And a controller for retrieving motion data of the character from the database based on the inputted EEG or voice signal and for controlling the motion simulator in synchronization with the virtual reality image contents displayed on the output device based on the motion data, .

At this time, the motion data may include motions related to the moving direction, speed, rotation, and posture of the character.

Also, the displayed virtual reality image content includes a virtual reality space background viewed from the character's viewpoint, and the displayed virtual reality space background includes position coordinates of the character in the virtual reality space, Can be determined on a basis.

In addition, the inputted EEG or speech signal can be displayed in the virtual reality image content by converting the contents of the mapped motion data into text.

In addition, the control device may be mounted on an HMD (Head Mounted Device) device.

In addition, the rate at which the motion data is reflected to the virtual reality image contents and the motion simulator may vary according to the magnitude of the amplitude of the input EEG or voice signal.

A method of controlling a virtual reality video and a motion simulator according to a user's brain wave or voice signal, the method comprising: receiving a user's brain wave or voice signal; Retrieving motion data of the character from the database based on the input brain wave or voice signal; And synchronizing the motion simulator with the virtual reality image content displayed on the output device based on the motion data.

At this time, the motion data may include motions related to the moving direction, speed, rotation, and posture of the character.

Also, the displayed virtual reality image content includes a virtual reality space background viewed from the character's viewpoint, and the displayed virtual reality space background includes position coordinates of the character in the virtual reality space, Can be determined on a basis.

In addition, the inputted EEG or speech signal can be displayed in the virtual reality image content by converting the contents of the mapped motion data into text.

In addition, the control device may be mounted on an HMD (Head Mounted Device) device.

In addition, the rate at which the motion data is reflected to the virtual reality image contents and the motion simulator may vary according to the magnitude of the amplitude of the input EEG or voice signal.

The embodiment of the present invention can control the motion of the output device and the motion simulator for outputting the virtual reality image contents according to the real time input of the external signal.

Especially, it is possible to control a character in a virtual reality space by using an EEG or voice signal as an input signal, and to control the virtual reality image contents and the motion simulator so as to correspond to the motion of the character.

The effects of the embodiments of the present invention are not limited to the above-mentioned effects, and various effects can be included within the scope that is obvious to a person skilled in the art from the following description.

1 is a block diagram of a virtual reality system according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining how virtual reality video viewpoint movement or motion of a motion simulator is mapped to a specific instruction to form a database according to an embodiment of the present invention.
3 is a flowchart illustrating a method of controlling a virtual reality image and a motion simulator according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

The embodiments disclosed herein should not be construed or interpreted as limiting the scope of the present invention. It will be apparent to those of ordinary skill in the art that the description including the embodiments of the present specification has various applications. Accordingly, any embodiment described in the Detailed Description of the Invention is illustrative for a better understanding of the invention and is not intended to limit the scope of the invention to embodiments.

The functional blocks shown in the drawings and described below are merely examples of possible implementations. In other implementations, other functional blocks may be used without departing from the spirit and scope of the following detailed description. Also, although one or more functional blocks of the present invention are represented as discrete blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

In addition, the expression "including any element" is merely an expression of an open-ended expression, and is not to be construed as excluding the additional elements.

Further, when a component is referred to as being connected or connected to another component, it may be directly connected or connected to the other component, but it should be understood that there may be other components in between.

Also, the expressions such as 'first, second', etc. are used only to distinguish a plurality of configurations, and do not limit the order or other features between configurations.

1 is a diagram illustrating a virtual reality image system according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a virtual reality imaging system includes a control device 100, an output device 200, and a motion simulator 300.

The output device 200 displays a virtual reality image, and an image to be displayed is controlled according to a user's input signal.

At this time, the output device 200 is a goggle type worn on the user's eyeball portion, and can output a first-person image. For example, it may be in the form of an HMD (Head Mounted Display) which can be worn on a human head to cover an eyeball and enjoy a virtual reality.

Here, virtual reality (VR) refers to the application of multimedia technology to make an environment that does not exist in reality actually exist. Such a virtual reality is called 'virtual reality' in order to display a processing world or a remote space using various images or computer graphics to make it feel as if it is in the spot.

The motion simulator 300 is a device to be worn on the body of a user or a body on which a user is to ride, and is a device for enhancing the actual sensation felt by a user through a virtual reality image through movement. The motion simulator 300 is controlled by the controller 100, and can be operated in synchronization with the virtual reality image.

The control device 100 controls the output device 200 and the motion simulator 300. The control device 100 receives an input signal from a user and controls the output device 200 and the motion simulator 300. The input signal includes a motion, a brain wave, a voice signal, etc., can do.

The controller 100 may include an input unit 110, a database 120, and a controller 130 according to an exemplary embodiment of the present invention.

The input unit 110 may receive a user's brain wave or voice signal.

Electroencephalogram is an action that is caused by the binding form or activity of several billions of cranial nerve cells and is generally measured by adhering electrodes on the scalp. The electroencephalogram is a wave with a potential difference of tens of microvolts, Is a physical value reflecting the state of human consciousness.

In particular, the input unit 110 may receive a user's brain wave in a manner that an electrode is attached to a head of a user, and a method of measuring brain waves includes a known method. For example, 1999-0046503 (an apparatus and method for real-time EEG using a headband) may be used.

In this case, an EEG to be used as an input signal can use an EEG generated when a user thinks about a specific instruction or a language that refers to an action.

More specifically, for example, an EEG measurement device is placed on a head of a user and an EEG occurring when a specific language is considered is measured. At this time, an EEG received by the control unit 130 through the input unit 110 is stored in the database 120 (Stocco & Rao et al., &Quot; Playing 20 Questions with the Mind: Collaborative Problem Solving by Humans Using a Brain-to-Brain Interface ", PLOS ONE , Published: 23 Sep 2015).

Also, the input unit 110 can measure the user's brain waves by classifying the amplitude of the input brain waves different from each other even if they are similar. At this time, even if the user intends to move to the "right" in the same way, the amplitude of the EEG may be different when the approaching speed of the obstacle is imminent and when it approaches slowly. Thus, the intensity of the command input from the same user, for example, movement to the right side, is divided, and when the user issues a command, the reaction speed of the character in the virtual reality space according to the command, that is, And can be used to vary the time until.

The input unit 110 may receive a voice signal through a microphone that receives sound. At this time, the control unit 130 may analyze the voice signal and determine that the user has input a specific command or action.

Also, the input unit 110 can measure a voice signal of a user by dividing the amplitude of the input signal by the amplitude of the input signal.

Referring to FIG. 2, motion data of a character in a virtual reality according to a user command generated by analog characteristics (e.g., frequency characteristics, pattern characteristics, etc.) of brain waves or voice signals is mapped and stored in the database 120 have.

The motion data is a table in which specific behaviors of characters according to brain waves or voice signals are mapped. The specific action may be, for example, data including movement related to the movement direction, speed, rotation, and posture of the character. In addition, movement data of the motion simulator 300 to be performed according to an EEG or a voice signal may be stored.

In addition, the database 120 stores a virtual reality space for a user to experience a virtual reality, and position coordinates can be specified and stored for each position of the virtual reality space. The virtual reality space is a virtual three-dimensional space, and the user can view the virtual reality space background displayed as the first person's point of view of the character in the virtual reality through the output device 100 as the virtual reality image content.

The control unit 130 searches the database 120 for the motion data of a character mapped with the input brain wave or voice signal and synchronizes and controls the virtual reality image contents and the motion simulator displayed on the output device based on the retrieved motion data .

At this time, the virtual reality image content viewed by the user through the output device 200 includes the virtual reality space background viewed from the character's viewpoint, and the displayed virtual reality space background corresponds to the position coordinates and motion data of the character in the virtual reality space Can be determined on a basis.

For example, if an instruction to move a character is mapped according to a brain wave or a voice signal of a user, the controller 130 can move the character by changing coordinates at which the character is located. At this time, It is possible to display a field of view looking at the real space, that is, a virtual reality image content on the output device 200.

At this time, the control device 100 can sense a user's input, for example, a head movement and reflect it on the movement of the character, so that the direction of the field of view changes according to the movement of the user's head, A different virtual reality space background can be output as the virtual reality image content.

For this purpose, the controller 300 may be mounted on an HMD (Head Mounted Device) and coupled with the output device 100.

In addition, the character does not move at the position coordinates where the character can not be reached. At this time, the virtual reality image contents may also correspond to the movement of the character, so that the field of view may not change. For example, in the flat ground, the movement of the position coordinates of the character is not limited, so that the visual field may change according to the movement of the character. However, if there is a wall with limited movement of the position coordinates, the movement of the character is limited, .

Meanwhile, it is preferable that the motion simulator 300 is previously programmed so that the motion simulator 300 moves correspondingly with the movement of the character, for example, the moving direction, speed, rotation, and posture.

For example, when the controller 130 receives a command corresponding to 'right shift' from the brain wave or voice signal input from the input unit 110, the controller 120 searches for motion data mapped to the command 'right shift' from the database 120 The motion simulator 300 changes the position coordinates of the character to the right according to the searched motion data and outputs the virtual reality image contents to the output device 100 based on the position coordinates of the character, You can have it move to the right.

At this time, the controller 130 may vary the speed at which the motion data is reflected to the virtual reality video contents and the motion simulator, depending on the magnitude of the amplitude of the EEG or voice signal being input. For example, when the controller 130 receives a command corresponding to 'rightward movement' from the EEG or voice signal received from the input unit 110, the control unit 130 determines from the input time that the inputted amplitude is stronger, The time to start moving the position coordinates of the character to the right can be obtained. In addition, as the magnitude of the amplitude of the brain wave or the voice signal increases, the reaction speed at which the motion simulator 300 starts to move in accordance with the motion of the character can also be accelerated.

In addition, the control unit 130 may vary the motion of the character according to amplitude of the received EEG or voice signal, or may change the movement speed of the character. For example, the greater the amplitude of the brain waves or the voice signal, the larger the size of the character's movement or the radius of the movement, and the greater the speed of movement.

In addition, when the input signal is a voice signal, the speed of movement of the controlled character according to the motion data or the reaction speed at which the character starts to move may be changed according to the speed of the voice the user speaks. For example, the faster the speech speed in speech, the faster the response speed of the virtual reality image or the motion simulator related to the time until the character moves after inputting the voice signal, or the speed of the character's movement in the virtual reality image .

In addition, the control unit 130 may convert the contents of the motion data mapped with the user's brain waves or voice signals into text, and output text in the virtual reality image contents. Accordingly, when the brain wave or the voice signal input from the input unit 110 is a command corresponding to the 'rightward movement', the motion data mapped with the command 'right shift' is retrieved from the database 120, 'Right Move'.

In another embodiment, when the EEG or voice signal, the motion data of the virtual reality image, and the motion data of the motion simulator 300 are respectively mapped to the database 120, the controller 130 controls the output device 200 and the motion simulator 300, respectively.

Meanwhile, the database 120 stores motion data of a virtual reality image in order to reduce a time difference between the viewpoint movement of the virtual reality image and the motion of the motion simulator 300, It is preferable that the motion simulator 300 preprograms and controls the motion simulator 300 so as to have a motion corresponding to the virtual reality image.

Meanwhile, the control unit 130 included in the above-described embodiment may be implemented by an arithmetic unit including a memory including instructions programmed to perform these functions, and a microprocessor for executing the instructions.

3 is a flowchart illustrating a method of controlling a virtual reality image and a motion simulator according to an exemplary embodiment of the present invention.

The virtual reality video and motion simulator control method includes: a step (S310) of receiving a user's brain wave or voice signal in a virtual reality video and motion simulator output control according to a user's brain wave or voice signal; (S320) of retrieving motion data of the character from the database based on the inputted EEG or speech signal; And synchronizing and controlling the motion simulator with the virtual reality image content displayed on the output device based on the motion data (S330). At this time, each step is implemented by the control device 100.

At this time, the motion data may include movements related to the moving direction, speed, rotation, and posture of the character.

Also, the displayed virtual reality image contents are determined based on the position coordinates of the character in the virtual reality space and the motion data searched.

In addition, the inputted EEG or speech signal can be converted into text and the contents of the mapped motion data can be displayed in the virtual reality image contents.

In addition, the speed at which the motion data is reflected in the virtual reality image contents and the motion simulator may vary depending on the magnitude of the amplitude of the EEG or voice signal input.

The above-described embodiments of the present invention can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs) , FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

Thus, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: Control device
110: input unit
120: Database
130:
200: Output device
300: Motion simulator

Claims (20)

An apparatus for controlling a motion simulator operating in synchronization with a virtual reality image, the apparatus comprising: an output device for displaying virtual reality image contents;
An input unit for receiving a user's brain wave or voice signal;
An EEG signal generated when the user considers a predetermined command or action of the EEG or a voice signal indicating a predetermined command or action of the voice signal and a motion data corresponding to a motion of the character in the virtual reality video are mapped A stored database; And
And a motion simulator for synchronizing the movement of the motion simulator with the motion of the virtual reality image content displayed on the output device based on the searched motion data by searching for motion data of the character from the database on the basis of the input brain wave or voice signal, ;
And a virtual reality image and a motion simulator.
The method according to claim 1,
The motion data includes:
And a motion associated with the movement direction, speed, rotation, and posture of the character.
The method according to claim 1,
The virtual reality image contents, which are displayed,
Wherein the displayed virtual reality space background includes position coordinates of the character in the virtual reality space and virtual reality images and motion simulators determined based on the searched data of motions, controller.
The method according to claim 1,
The input EEG or speech signal is transmitted to a user,
Wherein the content of the mapped motion data is converted into text and displayed in the virtual reality image content.
The method according to claim 1,
The control device includes:
An apparatus for controlling a virtual reality image and a motion simulator mounted on an HMD (Head Mounted Device) apparatus.
The method according to claim 1,
At least one of a speed at which the motion data is reflected in the virtual reality image contents and the motion simulator, a width of the motion of the character, and a speed of movement of the character are determined according to the magnitude of the amplitude of the input EEG or voice signal A virtual reality video and motion simulator control device that changes.
The method according to claim 1,
Wherein the virtual reality image content and the motion simulator are arranged in a virtual reality mode in which at least one of a speed at which the motion data is reflected in the virtual reality image content and the motion simulator, a width of the motion of the character, Image and motion simulator control device.

An output device for outputting a virtual reality image;
A motion simulator operating in synchronization with the virtual reality image and capable of being carried by a person; And
A control device for controlling the output device and the motion simulator;
Lt; / RTI >
The control device includes:
An input unit for receiving a user's brain wave or voice signal;
An EEG signal generated when the user considers a predetermined command or action of the EEG or a voice signal indicating a predetermined command or action of the voice signal and a motion data corresponding to a motion of the character in the virtual reality video are mapped A stored database; And
And a motion simulator for synchronizing the movement of the motion simulator with the motion of the virtual reality image content displayed on the output device based on the searched motion data by searching for motion data of the character from the database on the basis of the input brain wave or voice signal, ;
And a virtual reality system.
9. The method of claim 8,
The motion data includes:
And a motion associated with the movement direction, speed, rotation, and posture of the character.
9. The method of claim 8,
The virtual reality image contents, which are displayed,
Wherein the virtual reality space background includes a virtual reality space background viewed from the time of the character and the displayed virtual reality space background is determined based on position coordinates of the character in the virtual reality space and the searched data.
9. The method of claim 8,
The input EEG or speech signal is transmitted to a user,
Wherein the content of the mapped motion data is converted into text and displayed within the virtual reality video content.
9. The method of claim 8,
Wherein the control device is mounted on an HMD (Head Mounted Device) device.
9. The method of claim 8,
At least one of a speed at which the motion data is reflected in the virtual reality image contents and the motion simulator, a width of the motion of the character, and a speed of movement of the character are determined according to the magnitude of the amplitude of the input EEG or voice signal A Changing Virtual Reality Video System.
9. The method of claim 8,
Wherein the virtual reality image content and the motion simulator are arranged in a virtual reality mode in which at least one of a speed at which the motion data is reflected in the virtual reality image content and the motion simulator, a width of the motion of the character, Image system.
A method for controlling movement of a virtual reality image and a motion simulator output according to a user's brain wave or voice signal,
Receiving a user's brain wave or voice signal;
A brain wave generated when the user considers a predetermined command or action among the input brain waves, or a voice in the virtual reality image stored in the database on the basis of a speech signal designating a predetermined command or action of the input voice signal, Searching for motion data corresponding to the motion of the motion data;
Synchronizing the movement of the motion simulator with the motion of the virtual reality image content displayed on an output device that outputs the virtual reality image based on the searched motion data;
And a virtual reality image and a motion simulator control method.
16. The method of claim 15,
The motion data includes:
And a motion associated with the movement direction, speed, rotation, and posture of the character.
16. The method of claim 15,
The virtual reality image contents, which are displayed,
Wherein the displayed virtual reality space background includes position coordinates of the character in the virtual reality space and virtual reality images and motion simulators determined based on the searched data of motions, Control method.
16. The method of claim 15,
And the contents of the mapped motion data are converted into texts and displayed in the virtual reality image contents.
16. The method of claim 15,
At least one of a speed at which the motion data is reflected in the virtual reality image contents and the motion simulator, a width of the motion of the character, and a speed of movement of the character are determined according to the magnitude of the amplitude of the input EEG or voice signal A Method for Controlling Different Virtual Reality Video and Motion Simulator.
16. The method of claim 15,
Wherein the virtual reality image content and the motion simulator are arranged in a virtual reality mode in which at least one of a speed at which the motion data is reflected in the virtual reality image content and the motion simulator, a width of the motion of the character, Control method of video and motion simulator.

Images