KR102386359B1 - System and method for controlling exoskeleton robot using brain waves according to motion imagination - Google Patents

Abstract

Robot control system according to an embodiment of the present invention, a display unit for displaying an image; an EEG detector for detecting the user's EEG; a processing unit for determining an operation command corresponding to the detected EEG and generating a control signal corresponding to the operation command; and a robot driven according to the control signal. The image is an image that induces a user's imagination related to a specific motion, and the brain wave is detected in a part of the brain that is activated as the user imagines the specific motion. According to an embodiment, an image related to a specific motion is induced by providing an image to the user, and the exoskeleton robot can be controlled using the brain waves detected accordingly. In this way, the method of detecting brain waves by inducing imagination of motion does not have side effects such as seizures even when used for a long period of time compared to the existing visual stimulation method, and the user can control the robot by continuing to imagine even if the provision of images is stopped. can last

Description

Robot control system and control method using brain waves according to motion imagination

The present invention relates to a robot control system and a control method, and more particularly, to a system and method for controlling a robot using brain waves detected according to a motion image after providing an image inducing an image related to a specific motion to a user is about

[Description of state-funded R&D]

This research was conducted under the supervision of the Korea Research Institute of Science and Technology for the information communication and broadcasting R&D project of the Ministry of Science and ICT : 2017-0-00432) supported.

Recently, a technology for controlling an electronic device or an exoskeleton robot using a human biosignal has been actively studied. Brain-Computer Interface (BCI) technology is a representative technology that detects a user's brain waves through electroencephalography (EEG) and generates a corresponding control signal to control a device. BCI technology can be used in various industries such as cognitive research, motor rehabilitation, exoskeleton robots for spinal paralysis patients, or electric wheelchairs.

In general, in order for a user to generate a specific EEG by his or her will to control the device, learning and training for a long time is required. The reality is that it is very low.

In order to solve this problem, a system for controlling a device using an EEG signal (such as an SSVEP signal or a P300 signal) activated through a visual stimulus of a specific frequency has been developed. For example, SSVEP (Steady State Visually Evoked Potential) is an electrical signal generated from the brain that is activated when the human retina is stimulated by a specific frequency (about 3.5 Hz to 75 Hz). Using this biomechanism of inducing a specific EEG through visual stimulation, for example, a user can generate a specific EEG more easily by looking at an LED light of a specific frequency, and can control an external device using BCI. be able to

However, if this type of visual stimulus (e.g., the behavior of looking at LED lights) is continued for a long time, the user may have seizures, making it difficult to use for a long time. It was difficult to use it simultaneously with motions such as driving and driving.

Registered Patent Publication No. 10-1549325

Accordingly, an object of the present invention is to provide a system and method for controlling a robot by detecting brain waves in a manner that induces imagination related to a specific motion to the user, rather than the conventional visual stimulation method, and generates a control signal corresponding thereto. . In addition, it aims to solve problems that may occur in a control system using brain waves according to motion imagination (malfunction due to user's intentional misrecognition, reduction in recognition rate due to adaptation to stimuli, etc.).

According to an embodiment of the present invention, a robot control system using brain waves according to motion imagination, a display unit for displaying an image; an EEG detector for detecting the user's EEG; a processing unit for determining an operation command corresponding to the detected EEG and generating a control signal corresponding to the operation command; and a robot driven according to the control signal, wherein the image is an image that induces a user to imagine related to a specific motion, and the brain wave can be detected in a part of the brain that is activated as the user imagines the specific motion. there is.

According to an embodiment of the present disclosure, a mode switching operation recognition unit for recognizing a mode switching operation of the user may be further included, and the processing unit may determine an operation command corresponding to the brain wave from among operations classified as a current mode.

According to an embodiment, the mode switching operation may include an eye blinking motion or a hand motion, and the mode switching motion recognition unit may include a camera for recognizing the eye blinking or a motion sensor for recognizing the hand motion. .

According to an embodiment, the processing unit determines an operation command from among operations related to sitting in a first mode in which the user is seated, and receives an operation command from among operations related to standing in a second mode in which the user is in a standing state. In the third mode in which the user is walking, the motion command may be determined from among motions related to walking.

According to an embodiment, the processing unit measures the time at which the EEG is detected, determines the operation command corresponding to the EEG only when the sum of the measured times is equal to or greater than a set value, and allows the user to recognize the measured time. so that it can be visually displayed through the display unit.

According to an embodiment, a vibration unit for stimulating a sensory sensor by contacting a body part of the user is further included, and the processing unit controls the vibrating unit to increase the strength of the brain wave when the intensity of the detected EEG decreases. This can stimulate the user's senses.

According to an embodiment, a voice output unit for outputting a voice inducing imagination related to the specific operation to the user may be further included.

According to an embodiment, at least one of the display unit, the brain wave detection unit, and the processing unit may be included in a head mounted display (HMD).

According to an embodiment of the present invention, a robot control method using brain waves according to a motion image includes: displaying an image inducing an image related to a specific motion to a user; detecting a user's EEG detected in a part of the brain that is activated as the user imagines the specific motion; determining an operation command corresponding to the detected brain wave; generating a control signal corresponding to the operation command; and transmitting the control signal to the robot to drive the robot.

According to an embodiment, the method may further include recognizing a user's mode switching operation, and in determining the operation command, an operation command corresponding to the brain wave may be determined from among operations classified as a current mode.

According to an embodiment, the mode switching operation may include an eye blinking operation or a hand operation, and may further include changing a mode according to the eye blinking operation or the hand operation.

According to an embodiment, measuring the time at which the EEG is detected; and visually displaying the measured time so that the user can recognize it, wherein in the step of determining the operation command, the EEG corresponds to the EEG only when the sum of the time at which the EEG is detected is equal to or greater than the set value. You can decide which action command to do.

According to an embodiment, when the intensity of the EEG is reduced, the method may further include stimulating the user's sensory organs through a vibrator in contact with a body part of the user so that the intensity of the EEG can be maintained.

According to an embodiment, the method may further include outputting a voice for inducing an imagination related to the specific operation to the user.

A computer program stored in a computer-readable recording medium for executing the EEG-based robot control method according to the embodiment may be provided.

According to the robot control system according to an embodiment of the present invention, imagination related to a specific motion is induced by providing an image to the user, and the exoskeleton robot can be controlled using the brain waves detected accordingly. In this way, the method of detecting brain waves by inducing the imagination of motion does not have side effects such as seizures even when used for a long period of time compared to the existing visual stimulation method (for example, a method to watch an LED of a specific frequency), and provides images Even when interrupted, the user can continue to control the robot by continuing to imagine.

According to another embodiment of the present invention, the user can switch the control mode by performing a specific operation, and can increase the gesture recognition rate by pre-classifying the operations that can be controlled in each mode. According to these embodiments, it is possible to solve problems that may occur in the existing BCI system, such as a malfunction due to a user's intentional misrecognition, and a reduction in recognition rate due to adaptation to a stimulus.

1 is a block diagram showing the configuration of an EEG-based robot control system according to an embodiment.
2 is a diagram illustrating a method of operating an EEG-based robot control system according to an embodiment.
3 shows an example of a control mode switched through a user's motion and operation commands that can be determined in each mode in an EEG-based robot control system according to an embodiment.
4A and 4B show that, in the EEG-based robot control system according to an embodiment, the time at which EEG of the same characteristic is detected is displayed with a gauge so that the user can recognize it.
5 is a block diagram showing the configuration of an EEG-based robot control system according to another embodiment.
6 is a diagram illustrating stimulation of a user's sensory organs by controlling a vibrator in order to maintain a constant intensity of a detected EEG in an EEG-based robot control system according to an embodiment.
7 is a flowchart illustrating steps of a method for controlling an EEG-based robot according to an embodiment.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the scope of the claims is not limited or limited by the embodiments.

The terms used in this specification have been selected as currently widely used general terms as possible while considering their functions, but may vary depending on the intention or custom of a person skilled in the art or the emergence of new technology. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the corresponding specification. Therefore, it is intended to clarify that the terms used in this specification should be interpreted based on the actual meaning of the terms and the contents of the entire specification, rather than the names of simple terms.

Further, embodiments described herein may have aspects that are entirely hardware, partially hardware and partially software, or entirely software. As used herein, "unit", "module", "device", or "system" etc. refers to hardware, a combination of hardware and software, or a computer-related entity such as software. refers to For example, a part, module, device, server or system may refer to hardware constituting part or all of a platform and/or software such as an application for driving the hardware.

Hereinafter, preferred embodiments will be described in more detail with reference to the drawings.

1 shows the configuration of an EEG-based robot control system according to an embodiment. Referring to FIG. 1 , the robot control system using brain waves according to motion imagination includes a display unit 100 for displaying an image, an brain wave detection unit 200 for detecting the brain waves of a user U, and corresponding to the detected brain waves. It is composed of a processing unit 300 for determining an operation command and generating a control signal corresponding to the operation command, and a robot 400 driven according to the control signal.

The display unit 100 is configured to provide an image for inducing an imagination related to a specific operation to the user. Unlike the LED light used to induce brain waves in the existing visual stimulus-based BCI, these images are photos, videos, drawings, etc. related to the motion that the user actually wants to implement through the robot (exoskeleton robot, remote robot, etc.). . In the conventional visual stimulus-based BCI, LED light of a specific frequency is irradiated to activate the occipital region of the user's brain, and the device is controlled using the EEG detected thereby, which may bring side effects such as seizures when used for a long time. In addition, since the user cannot activate the occipital lobe region on their own without visual stimulation, EEG is not detected when the visual stimulation is stopped, making it difficult to continuously maintain control.

Accordingly, the present invention does not provide a visual stimulus directly to activate the brain, but provides a motion-related image in order to induce a motion-related imagination to the user and generate an EEG according to the motion-related image. Since these images are composed of photos, videos, and drawings that simply indicate motion, side effects such as seizures do not occur even when viewed for a long period of time, and users can maintain activation of specific brain regions through their imagination even if image provision is stopped. . Therefore, the user does not need to keep an eye on the LED light, so the risk can be reduced when walking or driving.

The EEG detection unit 200 is configured to detect EEG generated in a part of the brain that is activated as the user imagines a specific motion while watching the image provided through the display unit 100 . EEG refers to a current signal generated by brain activity and can be data in the form of an EEG signal. The electroencephalogram (EEG) is a recording of electric potential generated in the cerebral cortex as an electrical signal, and may be measured using an electrode sensor attached to the user's head.

The EEG signal is generated when a specific part of the brain is activated under a specific situation experienced by the human body, and the desired characteristics (wavelength, frequency, intensity, etc.) It can generate electroencephalogram signals. Accordingly, the regions of the brain that are activated are different according to the scene or motion imagined by the user, and thus EEGs having different characteristics are detected. BCI converts the detected EEG into the form of an EEG signal and generates a control signal corresponding to each signal to control the device.

The processing unit 300 is configured to determine an operation command corresponding to the detected EEG and generate a control signal corresponding to the operation command. The processing unit 300 receives the EEG signal and determines a robot operation command corresponding to each EEG signal according to a command stored in the form of a program language. In an embodiment, a Filter-Bank Common Spatial Pattern (FBCSP) and a Mutual Information-based Best Individual Feature (MIBIF) technique may be used to classify user intentions according to EEG analysis. According to this, EEG is divided into several frequency bands based on frequency, and spatial features are extracted from the corresponding bands through common spatial pattern (CSP) filtering. Among the extracted spatial features, meaningful features are selected through mutual-information. As an example of such brain wave analysis and intention analysis technology, various types of intention analysis technology belonging to the BCI technology field may be utilized.

The robot motion command is a motion that the user wants to implement with the robot 400 , and refers to, for example, motions of walking or sitting or standing in a place by operating the joints of the exoskeleton robot mounted on the user's arm or leg. The processing unit 300 generates a control signal for driving each component of the robot so that the robot 400 can perform an appropriate operation (ie, an operation imagined by the user) and transmits the generated control signal to the robot 400 .

The robot 400 is driven by the control signal. In one example, the robot 400 may be an exoskeleton robot mounted on a user's limbs or an electric wheelchair that can move while the user is seated. This type of robot is manufactured to assist or replace the movement of users with physical disabilities such as arms and legs. As another example, the robot 400 may be a remote robot operating in a space far from the user. This type of robot can be used to move or perform work in a remote location.

2 is a diagram illustrating a method of operating an EEG-based robot control system according to an embodiment. For example, when a user imagines a walking motion while watching an image related to walking output to the display unit 100, the EEG detector 200 detects a specific EEG detected when a person imagines a walking motion and provides information on this. It is transmitted to the processing unit 300 . The processing unit 300 generates a control signal to enable the exoskeleton robot 400 to perform a walking motion by assisting the user's leg movement and transmits the generated control signal to the exoskeleton robot 400 . The exoskeleton robot 400 drives each component according to a control signal to perform a walking motion or assist a user in walking.

According to an embodiment, the display unit 100 , the brain wave detection unit 200 , and the processing unit 300 may be integrally included in the head mounted display (HMD). A head mounted display device provides an image to the user through a binocular display, detects brain waves through electrodes attached to the head, and uses a built-in processor or a wirelessly connected terminal (smartphone, laptop, desktop, tablet PC, etc.) A control signal corresponding to brain waves can be generated and transmitted to an external robot.

3 shows an example of a control mode switched through a user's motion and operation commands that can be determined in each mode in the EEG-based robot control system according to an embodiment. According to this embodiment, the user can switch the control mode by performing a specific operation, and by pre-classifying the operations that can be controlled in each mode, the gesture recognition rate can be increased.

Referring to FIG. 3 , when the user performs a mode change operation (blinking an eye, a hand gesture, etc.), the mode change operation recognition unit 310 recognizes such an operation and switches the control mode. For example, the control system is initially set to 'seating mode', which assumes that the user is in a seated state, and commands an operation related to sitting, such as standing up from the seat or changing the seating direction as a follow-up operation. This is the input mode. When the user performs, for example, a mode switching operation of blinking the eyes three times in a row, a small camera installed around the user's eyes (eg, installed on the side of the binocular display of the HMD) detects such blinking, and the processing unit 300 . switches the control mode to the 'standing mode' accordingly. The 'standing mode' is a mode in which the user is assumed to stand up from his/her seat, and as a follow-up motion, an operation command related to standing, such as taking a seat or starting walking, is inputted. When the user blinks his/her eyes three times in succession again in the 'standing mode', the processing unit 300 switches the control mode to the 'walking mode'. The 'walking mode' is a mode in which it is assumed that the user is walking, and as a follow-up operation, a motion command related to walking is input, such as stopping in place or changing a walking direction.

In this way, by classifying the control mode so that the operation command is inputted only from among the related operations, the possibility of misrecognizing the intention according to the imagination can be reduced. If the motion-specific characteristics of the EEG according to the motion imagination are not significantly conspicuous, for example, the user imagines a 'walking motion' but may be recognized as a 'standing motion' with similar characteristics of the EEG. Since there is a high possibility of misrecognition when trying to recognize multiple motions at once, as in the embodiment, the recognition rate and control stability can be improved by classifying the control modes into multiple and allowing only motions derived from each control mode to be input. . For example, in the above embodiment, when the current control mode is the 'standing mode', even imagining a 'standing operation', an operation command is not input and only a 'walking operation' is input, so that an unintended malfunction can be prevented.

The mode switching operation may be set to various operations stored in advance. For example, the control mode may be set to be sequentially switched whenever a 'movement of waving a hand from side to side' is performed, and for this purpose, a motion sensor for recognizing a user's hand gesture may be utilized.

When a user controls the robot by imagining a motion, even if the user's imagination is aided by providing an image related to the motion, the user may unconsciously imagine another motion and the robot may operate unintentionally. In particular, in the case of an exoskeleton robot or electric wheelchair that assists the user in walking, such an unconscious error (for example, a situation in which the user wants the robot to stop but unconsciously imagines the driving motion and causes the robot to drive unintentionally) poses a great risk. can cause

In the present invention, in order to prevent errors due to such unconscious imagination, when the EEG having the same characteristics continues without executing a corresponding motion command for temporary EEG detection (that is, when the user intentionally continues to imagine a specific action) ) to execute the operation command only. According to an embodiment, the processing unit is configured to measure the time at which the EEG is detected and determine the operation command corresponding to the EEG only when the sum of the measured times is equal to or greater than a set value. In addition, the measured time is visually displayed through the display unit so that the user can recognize it.

4A and 4B show that, in the EEG-based robot control system according to an embodiment, the time at which EEG of the same characteristic is detected is displayed with a gauge so that the user can recognize it. For example, when the user wants to command the robot to stand up in place, the user consciously (either with the help of an image or by himself) imagines related to the standing motion. According to this imagination, when a specific area of the brain is activated, EEG a with characteristic a is detected. The processing unit does not directly transmit the control signal corresponding to the EEG a to the robot, but displays the detection time as a gauge as shown in FIG. 4A until the EEG a detection time reaches a preset threshold time (eg, 5 seconds). By looking at the gauge displayed on the display, the user can know how long to imagine the motion in order to command the robot to stand up.

If the user unconsciously or consciously imagines another motion, for example, a walking motion, while imagining the standing motion, the walking motion is not immediately executed, and a new gauge for this is displayed as shown in FIG. 4B . If this imagination is done unconsciously unlike the user's intention, and if the user still wants to command the standing motion, the gauge of FIG. 4A can be filled by imagining the standing motion again, and if the user's intention is the gait motion, the Continue imagining the gait motion until you fill the gauge. In this way, by setting a buffer between the imagination of the motion and the execution of the actual motion command, it is possible to prevent malfunction of the robot due to the unconscious imagination.

5 is a block diagram showing the configuration of an EEG-based robot control system according to another embodiment. The robot control system according to the embodiment shown in FIG. 5 has an overall similar configuration to the robot control system described with reference to FIG. 1 , but additionally a voice output unit 110 for providing auditory stimulation to the user and sensory stimulation to the user It further includes a vibrating unit 120 for providing.

The voice output unit 110 outputs a voice inducing imagination related to the target motion so that the user can more easily imagine and continue the target motion, that is, the motion related to the image provided by the display unit 100 . A voice (music, message, beep, etc.) for easily recalling a certain action may be set differently for each user.

The vibrator 120 is a device for stimulating a sensory sensor by being in contact with a user's body part (eg, a finger). The vibrating unit 120 may vibrate with a specific intensity, frequency, and pattern, and stimulates the user's somatosensory to induce specific brain waves, that is, steady-state somatosensory evoked potentials (SSSEP). In an embodiment, the vibrating unit 120 may be connected to the processing unit 300 by wire or wirelessly to receive a control signal and a feedback signal, thereby controlling vibration characteristics. The vibrating unit 120 may vibrate by, for example, a built-in battery or a small motor built therein by receiving power from the outside, but is not limited to a specific structure.

In this way, in inducing EEG by stimulating the user's sensory organs through the vibrating unit, the intensity, frequency, or pattern of vibration applied to the user is changed by an external factor, or the intensity of EEG detected by the user's sensory apparatus adapting to the vibration If is weak, it becomes difficult to analyze the user's intention and determine the operation command.

FIG. 6 illustrates stimulation of the user's sensory organs by controlling the vibrating unit in order to keep the intensity of the detected EEG constant in the EEG-based robot control system according to an embodiment. Referring to FIG. 6, ① when vibration stimulation is applied to the sensory device to detect the user's brain waves ② accordingly, while controlling the robot ③ when the intensity of the brain waves detected by external factors or adaptation of the sensory sensor decreases ④ processing unit 300 Controls the vibrating unit to increase the intensity of vibration or change the vibration pattern to increase the strength of ⑤ brain waves. Through such a feedback system, the recognition rate can be improved by maintaining the strength of the EEG at a certain level.

In the above, an embodiment of a system for controlling a robot by detecting brain waves in a manner that induces a user's imagination related to a specific motion and generating a control signal corresponding thereto has been described. According to this embodiment, it is possible to solve problems that may occur in the control system, such as a malfunction due to a user's erroneous recognition of intention, a decrease in recognition rate due to adaptation to a stimulus, and the like.

Hereinafter, each step of the EEG-based robot control method according to an embodiment will be described with reference to FIG. 7 . A description overlapping with the EEG-based robot control system according to the above-described embodiments will be omitted.

Referring to FIG. 7 , a step of displaying an image inducing imagination related to a specific motion to the user is first performed ( S100 ). Here, the image includes photos, videos, drawings, etc. related to the motions that the user actually wants to implement through a robot (exoskeleton robot, remote robot, etc.), as described above. The image may be displayed on a general display (LCD, LED, OLED, etc.) or may be displayed through a head mounted display (HMD) mounted on the user's head to display different images in both eyes.

Next, a step of detecting the user's brain waves detected in the part of the brain that is activated as the user imagines the specific motion is performed (S200). EEG is detected in the form of an EEG signal through electrodes attached to the user's head. The EEG signal is generated when a specific part of the brain is activated under a specific situation experienced by the human body, and the desired characteristics (wavelength, frequency, intensity, etc.) It can generate electroencephalogram signals. Accordingly, the regions of the brain that are activated are different according to the scene or motion imagined by the user, and thus EEGs having different characteristics are detected.

Next, a step of determining an operation command corresponding to the detected EEG is performed (S300). The robot motion command may include, for example, motions of walking or sitting or standing in a place by operating the joints of the exoskeleton robot mounted on the user's arm or leg. As described above, various types of EEG detection and intention analysis technologies belonging to the BCI technology field such as Filter-Bank Common Spatial Pattern (FBCSP) and Mutual Information-based Best Individual Feature (MIBIF) may be utilized.

Next, generating a control signal corresponding to the operation command (S400) and transmitting the control signal to the robot to drive the robot are performed (S500). The robot may be, for example, an exoskeleton robot that is mounted on a user's limbs to assist the movement, an electric wheelchair that can move while the user is seated, or a remote robot configured to operate in a remote location separately from the user.

As described above, the user can switch the control mode (eg, sitting mode, standing mode, walking mode, etc.) by performing a mode switching operation (eg, blinking an eye, a hand operation, etc.), and in each control mode, a subsequent Only action commands classified as actions to be performed can be input. In this way, by classifying the control mode so that the operation command is inputted only from among the related operations, the possibility of misrecognizing the intention according to the imagination can be reduced.

In one embodiment, the step of measuring the time at which the brain wave is detected, and the step of visually displaying the measured time so that the user can recognize it may be further performed. In the step of determining the operation command, the operation command corresponding to the EEG may be determined only when the sum of the time during which the EEG is detected is equal to or greater than a preset value. According to the embodiment, for temporary EEG detection, the action command is executed only when the EEG having the same characteristic continues without executing the corresponding action command (that is, when the user intentionally continues to imagine a specific action). Mistakes based on imagination can be prevented. The EEG detection time according to the imagination of the motion may be displayed as a gauge as shown in FIGS. 4A and 4B .

According to an embodiment, the method may further include outputting a voice for inducing an imagination related to the specific operation to the user. The voice induces imagination related to the target motion so that the user can more easily imagine and continue the target motion. A voice (music, message, beep, etc.) for easily recalling a certain action may be set differently for each user.

According to an embodiment, the method may further include stimulating the sensory sensor of the user through a vibrator in contact with a body part of the user so that the intensity of the brain wave is maintained when the intensity of the detected EEG is reduced. The vibrator stimulates the user's somatosensory sense to induce a steady-state somatosensory evoked potential (SSSEP). When the strength of the detected EEG is weakened, it becomes difficult to analyze the user's intention and determine the operation command. According to the present embodiment, when the intensity of the detected EEG decreases, the processing unit operates to increase the intensity of the EEG by controlling the vibration unit to increase the intensity of vibration or change the vibration pattern. Through such a feedback system, the recognition rate can be improved by maintaining the strength of the EEG at a certain level.

The brain wave-based robot control method according to the embodiments may be implemented as an application or implemented in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

According to the robot control system described above, it is possible to induce imagination related to a specific motion by providing an image to the user, and control the exoskeleton robot using the brain waves detected accordingly. In this way, the method of detecting brain waves by inducing the imagination of motion does not have side effects such as seizures even when used for a long period of time compared to the existing visual stimulation method (for example, a method to watch an LED of a specific frequency), and provides images Even when interrupted, the user can continue to control the robot by continuing to imagine.

According to another embodiment of the present invention, the user can switch the control mode by performing a specific operation, and can increase the gesture recognition rate by pre-classifying the operations that can be controlled in each mode. According to these embodiments, it is possible to solve problems that may occur in the existing BCI system, such as a malfunction due to a user's intentional misrecognition, and a reduction in recognition rate due to adaptation to a stimulus.

Although the above has been described with reference to the embodiments, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below you will understand

Claims (15)

As a robot control system using brain waves according to motion imagination,
a display unit for displaying an image;
an EEG detector for detecting the user's EEG;
a mode switching operation recognition unit for recognizing a user's mode switching operation;
a processing unit for determining an operation command corresponding to the detected EEG and generating a control signal corresponding to the operation command; and
Including a robot driven according to the control signal,
The image is an image that induces imagination related to a specific motion to the user, and the EEG is detected in a part of the brain that is activated as the user imagines the specific motion,
The processing unit determines the operation command corresponding to the brain wave from among the operations classified as the current mode, and does not determine the corresponding operation command if the operation corresponding to the brain wave does not exist among the operations classified as the current mode,
The processing unit measures the time at which the EEG is detected, determines an operation command corresponding to the EEG only when the sum of the measured times is equal to or greater than a set value, and through the display unit so that the user can recognize the measured time. A robot control system, characterized in that it is visually displayed.
delete According to claim 1,
The mode switching operation includes an eye blinking operation or a hand operation,
The mode switching motion recognition unit, the robot control system, characterized in that it comprises a camera for recognizing the eye blink or a motion sensor for recognizing the hand motion.
4. The method of claim 3,
In the first mode in which the user is seated, the processing unit determines an operation command from among operations related to sitting, and in the second mode in which the user stands up, determines an operation command from among the operations related to standing, and allows the user to walk In the third mode, which is a state in progress, a robot control system, characterized in that it determines a motion command from among motions related to walking.
delete According to claim 1,
Further comprising a vibrator for stimulating the sensory sensor in contact with the user's body part,
When the intensity of the detected EEG decreases, the processing unit controls the vibrator to increase the EEG intensity to stimulate the user's sensory organs.
According to claim 1,
The robot control system, characterized in that it further comprises a voice output unit for outputting a voice inducing imagination related to the specific motion to the user.
According to claim 1,
At least one of the display unit, the brain wave detection unit, and the processing unit is a head mounted display (HMD), characterized in that included in the robot control system.
As a robot control method using brain waves according to motion imagination,
displaying an image that induces imagination related to a specific motion to the user;
detecting a user's EEG detected in a part of the brain that is activated as the user imagines the specific motion;
measuring a time at which the EEG is detected;
visually displaying the measured time so that a user can recognize it;
recognizing a user's mode switching operation;
determining an operation command corresponding to the detected EEG when the sum of the time during which the EEG is detected is equal to or greater than a set value;
generating a control signal corresponding to the operation command; and
Comprising the step of transmitting the control signal to the robot to drive the robot,
In the step of determining the operation command, the operation command corresponding to the brain wave is determined from among the operations classified as the current mode, and when the operation corresponding to the brain wave does not exist among the operations classified as the current mode, the corresponding operation command Characterized in that not to determine, the robot control method.
delete 10. The method of claim 9,
The mode switching operation includes an eye blinking operation or a hand operation,
The robot control method, characterized in that it further comprises the step of switching the mode according to the eye blinking motion or the hand motion.
delete 10. The method of claim 9,
When the intensity of the EEG decreases, the method further comprising: stimulating the user's sensory organs through a vibrating unit in contact with the user's body part so that the EEG intensity can be maintained.
10. The method of claim 9,
The method for controlling a robot, characterized in that it further comprises outputting a voice for inducing an imagination related to the specific motion to the user.
A computer program stored in a computer-readable recording medium for executing the robot control method according to any one of claims 9, 11, 13 and 14.

Images