WO2020042109A1

WO2020042109A1 - Brain-computer interface device for safety control and robot system

Info

Publication number: WO2020042109A1
Application number: PCT/CN2018/103330
Authority: WO
Inventors: 陈颀潇
Original assignee: 西门子（中国）有限公司
Priority date: 2018-08-30
Filing date: 2018-08-30
Publication date: 2020-03-05
Also published as: CN113039507A

Abstract

The present invention provides a robot system and a brain-computer interface device (2) for safety control. The brain-computer interface device (2) comprises: a brain wave receiving module (21) which receives a brain wave signal from at least one electrode (1); and a safety assessment module (28) which is capable of communicating with a safety control module (3), the safety control module (3) being capable of braking at least one machine or robot (4), wherein the safety assessment module (28) assesses, by analyzing the received brain wave signal, the degree of tension of the sender of the grain wave, and when determining that the degree of tension exceeds a threshold, sends a warning signal to the safety control module (3), and the safety control module (3) is capable of braking or partially braking the machine or robot according to the warning indication.

Description

Brain-computer interface device for safety control and robot system

Technical field

The invention relates to a brain-computer interface device and an automation system provided with the brain-computer interface device, especially a robot system.

Background technique

With the advent of the industrial digital age, the realization of concepts in the field of factory automation and intelligent manufacturing requires humans and machines (such as robots or other automated equipment) or collaboration between machines and machines (Collaborative Working). In some cases, for example, the accidental intrusion of some people or objects may cause the movement of the machine to collide with people or objects, or the machine moves to an unintended place, causing it to hit other machines or facilities. Therefore, safety functions are provided in automation systems, especially factory automation systems equipped with robots, in accordance with mandatory requirements. In a collaborative scenario, safety functions are particularly required; in particular, a mandatory safety function module is required for collaborative robots.

Electroencephalogram (EEG) is a method of recording brain activity using electrophysiological indicators. It can record the changes of radio waves during brain activity, which is the overall reflection of the electrophysiological activities of brain nerve cells on the cerebral cortex or scalp surface. As a tool for indirect measurement of neural activity, related research on EEG and its associated event-related potentials (ERP) is widely used in the medical field. At present, with the use of EEG and event-related potentials, a variety of components related to human brain cognitive function have been discovered. For example, the P300 component is related to the individual's endogenous attention, and the N400 component is related to semantic processing.

In order to read brain waves, Brain-Computer-Interface (BCI) is usually used. Brain-Computer Interfaces used in the medical field can usually convert the analog signals acquired by scalp electrodes into computer-usable ones. Digital signals, whose work usually includes signal acquisition, preprocessing, feature extraction, classification, etc.

Summary of the Invention

In order to better realize the safety function in the collaborative scenario, the present invention proposes to use the brain-computer interface BCI to realize the safety control of the machine or robot system.

The present invention aims to provide a brain-computer interface device for safety control of an automatic system, such as a robot system, including:

An EEG receiving module, which receives EEG signals from at least one electrode;

A safety evaluation module capable of communicating with a safety control device capable of braking at least one machine or robot, wherein the safety evaluation module:

Evaluate the nervousness of the sender of the brain wave by analyzing the received brain wave signal,

Wherein, when it is judged that the tension degree exceeds a threshold value, an alarm signal is sent to the safety control device, wherein the safety control device can brake or partially brake the machine or robot according to the alarm instruction.

By using a brain-computer interface to control the safety control equipment of a machine or robot, the emergency braking of the robot can be implemented in a timely and effective manner, especially in a man-machine collaborative scenario. This function is implemented without additional sensors and algorithms in the machine or robot, which can reduce the complexity, weight and cost of the system. The solution according to the present invention allows the operator to perceive the wrong operation of the machine, so there is no blind spot.

According to the present invention, the braking may refer to an emergency stop of the machine or robot, or a braking operation on a part of a moving part of the machine or robot so that it does not contact an operator.

Because the solution is based on a brain-computer interface, there is no need for operators to perform manual operations or feedback, so the corresponding speed is higher than the solution of setting the sensor on the robot.

According to an advantageous embodiment, the safety assessment module characterizes the degree of tension by at least one characteristic parameter of a brain wave, wherein when the characteristic parameter exceeds a characteristic parameter threshold, the brain-computer interface device reports to the security The control device sends an alarm signal. It is particularly preferred that the characteristic parameter is an amplitude of the brain wave, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold, an alarm signal is issued. Because the amplitude itself can reflect the degree of tension of the person, and the amplitude information is already easy to obtain for the processing of brain waves, and does not require a large amount of calculation, it is possible to realize the assessment of the tension in a simple manner. In addition to using the amplitude of the brain wave as a characteristic parameter, for example, frequency, frequency characteristics of specific brain wave components, rhythm characteristics, special response on a specific frequency spectrum, and the like can be used as a characteristic parameter for evaluation. Which characteristic parameter is specifically selected can be determined by selecting an appropriate algorithm for different application scenarios and design costs.

According to an advantageous embodiment, the brain-computer interface device further includes a brain wave signal preprocessing module, which can extract characteristic parameters of the brain wave and send it to the safety evaluation module for evaluation. For example, the pre-processing module can extract the amplitude of specific components of the brain wave for evaluation. The pre-processing module may be a part of the security assessment module or a separate module, which can be implemented by hardware or software.

According to an advantageous embodiment, the brain-computer interface device further comprises a signal output interface, which can be connected to at least one control device. In the scenario where the operator cooperates with multiple robots, it is possible to reduce the separate setting of sensors on each robot, and only one set of equipment of the operator can realize the emergency braking of multiple robots. The signal output interface can be a part of the safety evaluation module or a separate module, which can be implemented by means of hardware or software.

According to an advantageous embodiment, the signal output interface can be connected to the safety control device through wireless communication. For example, it can be implemented through various available wireless communication protocols, such as Bluetooth.

According to an advantageous embodiment, the brain-computer interface device of the brain wave receiving module includes a wireless communication interface capable of receiving EEG signals from the electrodes in a wireless communication manner.

According to an advantageous embodiment, the electroencephalogram electrode is integrated in a helmet or goggles, wherein the electrode is provided at a position that can be in contact with the scalp. For example, the electrodes can be integrated in the temples of the goggles, or near the ear of the helmet, so that the electrodes can contact the scalp. Of course, brainwave electrodes can also be integrated into any conceivable head-wearing device.

In another aspect of the present invention, a robot system is provided, including:

At least one brainwave electrode;

A safety control device;

A machine or robot connected to the safety control device; and

The brain-computer interface device according to any one of the above embodiments, wherein the brain-computer interface device receives the brain wave electrode and is capable of sending an alarm signal to the safety control device.

The invention also provides a method for controlling a machine or a robot based on brain wave signals, including:

S1: preprocess the received brain wave signals and extract the brain wave components to be evaluated;

S2: Compare the characteristic parameters of the brainwave components with a threshold, where

S3: When the characteristic parameter exceeds the threshold, an alarm signal is sent to the safety control device of the machine or robot, wherein the safety control device can brake or partially brake the machine or robot according to the alarm instruction. move.

According to an advantageous embodiment, the characteristic parameter is an amplitude of an electroencephalogram, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold, an alarm signal is issued.

According to an advantageous embodiment, the pre-processing comprises amplifying and filtering the brain wave signals.

The present invention also provides a computer program product tangibly stored on a computer-readable medium and including computer-executable instructions that, when executed, cause at least one processor to execute S1 To S3 and the method described in each of the above embodiments.

The present invention also provides a computer-readable medium having computer-executable instructions stored thereon, which, when executed, cause at least one processor to execute S1 to S3 and the methods described in the foregoing embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawings are only intended to schematically illustrate and explain the present invention, and do not limit the scope of the present invention. among them,

FIG. 1 exemplarily shows a robot system provided with a brain-computer interface device 2 according to an embodiment of the present invention;

2 exemplarily provides a brain-computer interface device 2 according to an embodiment of the present invention;

FIG. 3 exemplarily shows a method for safely controlling a robot according to an embodiment of the present invention.

List of reference signs :

1 scalp electrode

2 Brain-computer interface device

3 Safety controllers and safety relays

4 Machine and robot body

21 brain wave receiving module

23 brain wave signal pre-processing module

28 Safety Evaluation Module

26 signal output interface

detailed description

In order to have a clearer understanding of the technical features, objects, and effects of the present invention, specific embodiments of the present invention will now be described with reference to the drawings.

Since the discovery of electroencephalogram (EEG) signals on the scalp, it has been used in the auxiliary diagnosis of neurological diseases and in brain function research. EEG helps to realize the direct communication between the brain and the outside world. By studying the main rhythm components in the EEG signal, various induced EEG signals, and the mechanism of some specific signals and their relationships to obtain the brain's direct response to the external environment Channels of information. According to the present invention, brain waves can be applied to automated systems, such as machine systems and robotic systems, to achieve safe control of the system.

The main application scenario of the present invention is a human-machine cooperation scenario. When an operator, such as an operator on a production line, sees, hears, touches, or even smells a dangerous signal, that is, when a person perceives danger according to vision, hearing, touch, or smell, he will induce an EEG waveform. Mutation. To this end, the present invention proposes a brain-computer interface device and a robot system based on the safety control applied to a machine or a robot.

As shown in FIG. 1, the brain-computer interface-based safety system 100 includes at least one electrode 1, a brain-computer interface device 2, and a safety control device 3 for a machine or robot body.

According to the present invention, the electrode 1 may be a scalp electrode 1, which may be provided or integrated in a wearable device such as a helmet, hat, earphones, goggles, glasses, etc., to achieve contact with the scalp, or may be directly attached to the scalp so that Realize the measurement of brain waves.

The brain-computer interface device 2 can receive and process brain wave signals from the electrodes 1. When a person finds or anticipates a danger, the brain-computer interface device 2 can evaluate the operator's degree of stress based on the brain wave signals. The evaluation of the stress level can be realized based on monitoring the fluctuation of some characteristic parameters of the brain wave. For example, in some embodiments, the characteristic parameter may be the amplitude of the brain wave. When the fluctuation of the amplitude of the electroencephalogram exceeds a preset amplitude threshold, the brain-computer interface device 2 determines that an emergency situation that may cause a safety problem has occurred, and can send an alarm signal to the safety control device 3 of the robot. This alarm signal enables the safety control device 3 to trigger an emergency stop of the machine or robot 4. In addition to using the amplitude of the brain wave as a characteristic parameter, for example, frequency, frequency characteristics of specific brain wave components, rhythm characteristics, special response on a specific frequency spectrum, and the like can be used as a characteristic parameter for evaluation. Which characteristic parameter is specifically selected can be determined by selecting an appropriate algorithm for different application scenarios and design costs.

The safety control device 3 may be a robot safety controller or a safety relay or a robot controller or a part of a robot controller that can control the robot body, which can realize the braking (or stop, emergency stop operation) of the robot, As a result, the body of the machine or robot 4 stops moving. Depending on the type of machine or robot, the safety control device 3 can be implemented by hardware or software. It can be an independent safety relay or controller provided outside the machine or robot body, or it can be a module or part of the robot controller. , Or a safety module of the control software of the robot controller.

Based on the research of brain waves in the medical field, we can know that the human brain can respond to a variety of external stimuli. These external stimuli may include at least perceptible stimuli such as hearing, vision, touch, and smell. In order to realize the function of the brain-computer interface device 2 of the present invention, the brain-computer interface device 2 exemplarily shown in FIG. 2 includes a brain wave receiving module 21 and a safety evaluation module 28.

According to the present invention, the electrode 1 can receive brain wave signals at a position close to the scalp, for example, an electrode located near the ear process. This signal is transmitted to the safety evaluation module 28 through the brain wave receiving module 21 located in the brain-computer interface device 2 to evaluate the stress degree. The brain wave receiving module 21 can receive the brain wave signal from the electrode 1 through wireless transmission. In this case, a device provided with the electrode 1 such as a goggle is also provided with a wireless signal transmitting device accordingly.

In an ideal implementation, the brain-computer interface device 2 is further provided with a brain wave signal pre-processing module 23, which can pre-process the brain wave signals, such as filtering and amplifying the signals, or according to a preset The characteristic parameter type extracts specific brain wave components and the like, and then transmits the pre-processed signal to the safety evaluation module 23. For example, when an operator hears an abnormal sound and realizes that there may be danger, the brain-computer interface device 2 may receive an electroencephalogram signal from the electrode 1. The electroencephalogram signal includes components of the scalp brainwave signal generated by the brain according to hearing. For example, existing auditory experiments can use brain waves such as evoked event-related potentials ERP to assess whether a startle response occurs. The safety evaluation module 23 can evaluate, for example, the waveform of the event-related potential ERP to evaluate whether a scare due to auditory attention occurs. Therefore, the pre-processing module 23 can extract auditory-sensitive brain wave components for further evaluation. According to an embodiment, the safety evaluation module 23 may detect the amplitude of the brain wave as a characteristic parameter, and when it is detected that the amplitude peak value exceeds an amplitude threshold calculated based on experience or big data, The control device or controller 3 issues an alarm signal. The safety control device 3 stops the movement of the machine based on the alarm signal.

In one embodiment, the visually-evoked EEG signals can also be processed by the brain-machine interface device 2 according to the present invention. For example, when an operator sees a robotic arm moving towards it at a path different from a normal working track or at an abnormal speed, it may generate alertness or shock. At this time, the electrodes 1 arranged on the scalp send the collected brain waves to the brain-computer interface device 2 to evaluate the operator's stress level. The brain-computer interface device 2 receives the operator's EEG signal by using the electrode 1 and pre-processes the EEG signal, which includes, for example, using a filtering device to filter the pre-processed EEG signal, and then uses various algorithms to implement the correction The recognition of brain waves, especially the recognition of visual stimulation frequencies or components of brain waves. This pre-processing can be performed by, for example, the brain wave signal pre-processing module 23. The safety evaluation module 28 then monitors the characteristic parameters of these brain wave signals. When the characteristic parameters of the signal appear, for example, the amplitude of the brain wave exceeds a preset threshold, an alarm signal is sent to the safety control device or the safety relay 3 . The safety control device or safety relay 3 immediately stops the work of the robot or machine 4 to ensure the safety of the operator. Preferably, based on the present invention, the electroencephalogram signal pre-processing module 23 may be a part of the safety evaluation module 28 or a separate module.

In addition, the safety evaluation module 28 can also send an alarm signal to the safety control device or safety relay 3 through a signal output interface 26, wherein the signal output interface 26 can simultaneously issue alarms to multiple safety control devices 3 for different machines or robots Signal to make multiple machines or robot bodies emergency stop.

Of course, in addition to the use of hearing and vision-induced brain wave signals, the present invention can also evaluate and monitor brain waves generated based on tactile sensation and olfactory sense, to achieve an emergency stop of a machine or a robot when an operator finds a danger.

FIG. 3 exemplarily shows the method steps of the evaluation performed by the security evaluation module 28 according to the present invention.

S2: comparing the characteristic parameter of the brainwave component with a threshold value;

S3: When the characteristic parameter exceeds the threshold, an alarm signal is sent to the safety control device 3 of the machine or robot. The safety control device 3 can brake the machine or robot according to the alarm instruction.

It should be understood that although this description is described in terms of various embodiments, not every embodiment includes only an independent technical solution. This description of the description is only for clarity, and those skilled in the art should take the description as a whole. The technical solutions in the embodiments can also be appropriately combined to form other implementations that can be understood by those skilled in the art.

The above descriptions are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent change, modification, and combination made by those skilled in the art without departing from the concept and principle of the present invention shall fall within the protection scope of the present invention.

Claims

Brain-computer interface device (2) for safety control, including:

An EEG receiving module (21), which receives EEG signals from at least one electrode (1);

A safety evaluation module (28) capable of communicating with a safety control device (3) capable of braking or partially braking at least one machine or robot (4), wherein the safety evaluation module (28):

Evaluate the nervousness of the sender of the brain wave by analyzing the received brain wave signal,

Wherein, when it is judged that the tension degree exceeds a threshold value, an alarm signal is sent to the safety control device (3), wherein the safety control device (3) can brake the machine or robot according to the alarm instruction or Partial braking.
The brain-computer interface device (2) according to claim 1, characterized in that the safety evaluation module (28) characterizes the degree of tension by at least one characteristic parameter of a brain wave, wherein when the characteristic parameter exceeds one characteristic When the parameter is a threshold value, the brain-computer interface device (2) sends an alarm signal to the safety control device (3).
The brain-computer interface device (2) according to claim 2, wherein the characteristic parameter is an amplitude of an electroencephalogram, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold When the alarm signal is issued.
The brain-computer interface device (2) according to any one of claims 1 to 3, further comprising an electroencephalogram signal preprocessing module (23), which can extract characteristic parameters of the electroencephalogram signal, and It is sent to the security evaluation module (8) for evaluation.
The brain-computer interface device (2) according to any one of claims 1 to 4, wherein the brain-computer interface device (2) further comprises a signal output interface (26), which can communicate with at least one safety device. Control device (3) is connected.
The brain-computer interface device (2) according to claim 5, characterized in that the signal output interface (26) can be connected with the safety control device (3) through wireless communication.
The brain-computer interface device (2) according to any one of claims 1 to 6, characterized in that the brain-wave receiving module (21) brain-computer interface device includes a wireless communication interface, which can communicate in a wireless manner Receiving an EEG signal from the electrode (1).
Robot system (100), including:

At least one brainwave electrode (1);

A safety control device (3);

A machine or robot (4) connected to said safety control device (3); and

The brain-computer interface device (2) according to any one of claims 1 to 7, wherein the brain-computer interface device (2) receives the brain wave electrode (1), and is capable of supplying the safety control device (3) ) Send an alarm signal.
The robot system according to claim 8, characterized in that the electrode (1) is integrated in a safety helmet or goggles, and wherein the electrode (1) is provided at a position capable of abutting on the scalp.
Method for controlling machine or robot (4) based on brain wave signals, including:

Preprocess the received brain wave signals and extract the brain wave components to be evaluated;

Comparing the characteristic parameter of the brainwave component with a threshold, where

When the characteristic parameter exceeds the threshold value, an alarm signal is sent to the safety control device (3) of the machine or robot, wherein the safety control device (3) can make the machine or robot based on the alarm instruction. Brake or partial braking.
The method according to claim 9, wherein the characteristic parameter is an amplitude of the brain wave, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold, an alarm signal is issued.
The method according to claim 9 or 10, wherein the pre-processing comprises amplifying and filtering the brain wave signal.
A computer program product tangibly stored on a computer-readable medium and including computer-executable instructions which, when executed, cause at least one processor to perform any of claims 10 to 12 The method of one item.
A computer-readable medium having stored thereon computer-executable instructions that, when executed, cause at least one processor to perform a method according to any one of claims 10 to 12.