WO2020042109A1 - Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot - Google Patents

Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot Download PDF

Info

Publication number
WO2020042109A1
WO2020042109A1 PCT/CN2018/103330 CN2018103330W WO2020042109A1 WO 2020042109 A1 WO2020042109 A1 WO 2020042109A1 CN 2018103330 W CN2018103330 W CN 2018103330W WO 2020042109 A1 WO2020042109 A1 WO 2020042109A1
Authority
WO
WIPO (PCT)
Prior art keywords
brain
safety control
computer interface
interface device
robot
Prior art date
Application number
PCT/CN2018/103330
Other languages
English (en)
Chinese (zh)
Inventor
陈颀潇
Original Assignee
西门子（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子（中国）有限公司 filed Critical 西门子（中国）有限公司
Priority to CN201880097045.1A priority Critical patent/CN113039507A/zh
Priority to PCT/CN2018/103330 priority patent/WO2020042109A1/fr
Publication of WO2020042109A1 publication Critical patent/WO2020042109A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the invention relates to a brain-computer interface device and an automation system provided with the brain-computer interface device, especially a robot system.
  • Electroencephalogram is a method of recording brain activity using electrophysiological indicators. It can record the changes of radio waves during brain activity, which is the overall reflection of the electrophysiological activities of brain nerve cells on the cerebral cortex or scalp surface.
  • EEG Electroencephalogram
  • EEG event-related potentials
  • P300 component is related to the individual's endogenous attention
  • N400 component is related to semantic processing.
  • Brain-Computer-Interface In order to read brain waves, Brain-Computer-Interface (BCI) is usually used. Brain-Computer Interfaces used in the medical field can usually convert the analog signals acquired by scalp electrodes into computer-usable ones. Digital signals, whose work usually includes signal acquisition, preprocessing, feature extraction, classification, etc.
  • the present invention proposes to use the brain-computer interface BCI to realize the safety control of the machine or robot system.
  • the present invention aims to provide a brain-computer interface device for safety control of an automatic system, such as a robot system, including:
  • An EEG receiving module which receives EEG signals from at least one electrode
  • a safety evaluation module capable of communicating with a safety control device capable of braking at least one machine or robot, wherein the safety evaluation module:
  • an alarm signal is sent to the safety control device, wherein the safety control device can brake or partially brake the machine or robot according to the alarm instruction.
  • the emergency braking of the robot can be implemented in a timely and effective manner, especially in a man-machine collaborative scenario.
  • This function is implemented without additional sensors and algorithms in the machine or robot, which can reduce the complexity, weight and cost of the system.
  • the solution according to the present invention allows the operator to perceive the wrong operation of the machine, so there is no blind spot.
  • the braking may refer to an emergency stop of the machine or robot, or a braking operation on a part of a moving part of the machine or robot so that it does not contact an operator.
  • the solution is based on a brain-computer interface, there is no need for operators to perform manual operations or feedback, so the corresponding speed is higher than the solution of setting the sensor on the robot.
  • the safety assessment module characterizes the degree of tension by at least one characteristic parameter of a brain wave, wherein when the characteristic parameter exceeds a characteristic parameter threshold, the brain-computer interface device reports to the security The control device sends an alarm signal.
  • the characteristic parameter is an amplitude of the brain wave, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold, an alarm signal is issued. Because the amplitude itself can reflect the degree of tension of the person, and the amplitude information is already easy to obtain for the processing of brain waves, and does not require a large amount of calculation, it is possible to realize the assessment of the tension in a simple manner.
  • characteristic parameter for example, frequency, frequency characteristics of specific brain wave components, rhythm characteristics, special response on a specific frequency spectrum, and the like can be used as a characteristic parameter for evaluation. Which characteristic parameter is specifically selected can be determined by selecting an appropriate algorithm for different application scenarios and design costs.
  • the brain-computer interface device further includes a brain wave signal preprocessing module, which can extract characteristic parameters of the brain wave and send it to the safety evaluation module for evaluation.
  • the pre-processing module can extract the amplitude of specific components of the brain wave for evaluation.
  • the pre-processing module may be a part of the security assessment module or a separate module, which can be implemented by hardware or software.
  • the brain-computer interface device further comprises a signal output interface, which can be connected to at least one control device.
  • a signal output interface can be connected to at least one control device.
  • the signal output interface can be a part of the safety evaluation module or a separate module, which can be implemented by means of hardware or software.
  • the signal output interface can be connected to the safety control device through wireless communication.
  • wireless communication can be implemented through various available wireless communication protocols, such as Bluetooth.
  • the brain-computer interface device of the brain wave receiving module includes a wireless communication interface capable of receiving EEG signals from the electrodes in a wireless communication manner.
  • the electroencephalogram electrode is integrated in a helmet or goggles, wherein the electrode is provided at a position that can be in contact with the scalp.
  • the electrodes can be integrated in the temples of the goggles, or near the ear of the helmet, so that the electrodes can contact the scalp.
  • brainwave electrodes can also be integrated into any conceivable head-wearing device.
  • a robot system including:
  • At least one brainwave electrode At least one brainwave electrode
  • a safety control device
  • a machine or robot connected to the safety control device
  • the brain-computer interface device according to any one of the above embodiments, wherein the brain-computer interface device receives the brain wave electrode and is capable of sending an alarm signal to the safety control device.
  • the invention also provides a method for controlling a machine or a robot based on brain wave signals, including:
  • S1 preprocess the received brain wave signals and extract the brain wave components to be evaluated
  • the characteristic parameter is an amplitude of an electroencephalogram, wherein the threshold is an amplitude threshold, and when the amplitude exceeds the amplitude threshold, an alarm signal is issued.
  • the pre-processing comprises amplifying and filtering the brain wave signals.
  • the present invention also provides a computer program product tangibly stored on a computer-readable medium and including computer-executable instructions that, when executed, cause at least one processor to execute S1 To S3 and the method described in each of the above embodiments.
  • the present invention also provides a computer-readable medium having computer-executable instructions stored thereon, which, when executed, cause at least one processor to execute S1 to S3 and the methods described in the foregoing embodiments.
  • FIG. 1 exemplarily shows a robot system provided with a brain-computer interface device 2 according to an embodiment of the present invention
  • FIG. 2 exemplarily provides a brain-computer interface device 2 according to an embodiment of the present invention
  • FIG. 3 exemplarily shows a method for safely controlling a robot according to an embodiment of the present invention.
  • EEG electroencephalogram
  • the main application scenario of the present invention is a human-machine cooperation scenario.
  • an operator such as an operator on a production line, sees, hears, touches, or even smells a dangerous signal, that is, when a person perceives danger according to vision, hearing, touch, or smell, he will induce an EEG waveform. Mutation.
  • the present invention proposes a brain-computer interface device and a robot system based on the safety control applied to a machine or a robot.
  • the brain-computer interface-based safety system 100 includes at least one electrode 1, a brain-computer interface device 2, and a safety control device 3 for a machine or robot body.
  • the electrode 1 may be a scalp electrode 1, which may be provided or integrated in a wearable device such as a helmet, hat, earphones, goggles, glasses, etc., to achieve contact with the scalp, or may be directly attached to the scalp so that Realize the measurement of brain waves.
  • a wearable device such as a helmet, hat, earphones, goggles, glasses, etc.
  • the brain-computer interface device 2 can receive and process brain wave signals from the electrodes 1. When a person finds or anticipates a danger, the brain-computer interface device 2 can evaluate the operator's degree of stress based on the brain wave signals. The evaluation of the stress level can be realized based on monitoring the fluctuation of some characteristic parameters of the brain wave. For example, in some embodiments, the characteristic parameter may be the amplitude of the brain wave. When the fluctuation of the amplitude of the electroencephalogram exceeds a preset amplitude threshold, the brain-computer interface device 2 determines that an emergency situation that may cause a safety problem has occurred, and can send an alarm signal to the safety control device 3 of the robot. This alarm signal enables the safety control device 3 to trigger an emergency stop of the machine or robot 4.
  • characteristic parameter for example, frequency, frequency characteristics of specific brain wave components, rhythm characteristics, special response on a specific frequency spectrum, and the like can be used as a characteristic parameter for evaluation. Which characteristic parameter is specifically selected can be determined by selecting an appropriate algorithm for different application scenarios and design costs.
  • the safety control device 3 may be a robot safety controller or a safety relay or a robot controller or a part of a robot controller that can control the robot body, which can realize the braking (or stop, emergency stop operation) of the robot, As a result, the body of the machine or robot 4 stops moving.
  • the safety control device 3 can be implemented by hardware or software. It can be an independent safety relay or controller provided outside the machine or robot body, or it can be a module or part of the robot controller. , Or a safety module of the control software of the robot controller.
  • the brain-computer interface device 2 exemplarily shown in FIG. 2 includes a brain wave receiving module 21 and a safety evaluation module 28.
  • the electrode 1 can receive brain wave signals at a position close to the scalp, for example, an electrode located near the ear process. This signal is transmitted to the safety evaluation module 28 through the brain wave receiving module 21 located in the brain-computer interface device 2 to evaluate the stress degree.
  • the brain wave receiving module 21 can receive the brain wave signal from the electrode 1 through wireless transmission.
  • a device provided with the electrode 1 such as a goggle is also provided with a wireless signal transmitting device accordingly.
  • the brain-computer interface device 2 is further provided with a brain wave signal pre-processing module 23, which can pre-process the brain wave signals, such as filtering and amplifying the signals, or according to a preset
  • the characteristic parameter type extracts specific brain wave components and the like, and then transmits the pre-processed signal to the safety evaluation module 23.
  • the brain-computer interface device 2 may receive an electroencephalogram signal from the electrode 1.
  • the electroencephalogram signal includes components of the scalp brainwave signal generated by the brain according to hearing.
  • existing auditory experiments can use brain waves such as evoked event-related potentials ERP to assess whether a startle response occurs.
  • the safety evaluation module 23 can evaluate, for example, the waveform of the event-related potential ERP to evaluate whether a scare due to auditory attention occurs. Therefore, the pre-processing module 23 can extract auditory-sensitive brain wave components for further evaluation. According to an embodiment, the safety evaluation module 23 may detect the amplitude of the brain wave as a characteristic parameter, and when it is detected that the amplitude peak value exceeds an amplitude threshold calculated based on experience or big data, The control device or controller 3 issues an alarm signal. The safety control device 3 stops the movement of the machine based on the alarm signal.
  • the visually-evoked EEG signals can also be processed by the brain-machine interface device 2 according to the present invention.
  • the brain-machine interface device 2 receives the operator's EEG signal by using the electrode 1 and pre-processes the EEG signal, which includes, for example, using a filtering device to filter the pre-processed EEG signal, and then uses various algorithms to implement the correction
  • the recognition of brain waves especially the recognition of visual stimulation frequencies or components of brain waves.
  • This pre-processing can be performed by, for example, the brain wave signal pre-processing module 23.
  • the safety evaluation module 28 then monitors the characteristic parameters of these brain wave signals. When the characteristic parameters of the signal appear, for example, the amplitude of the brain wave exceeds a preset threshold, an alarm signal is sent to the safety control device or the safety relay 3 . The safety control device or safety relay 3 immediately stops the work of the robot or machine 4 to ensure the safety of the operator.
  • the electroencephalogram signal pre-processing module 23 may be a part of the safety evaluation module 28 or a separate module.
  • the safety evaluation module 28 can also send an alarm signal to the safety control device or safety relay 3 through a signal output interface 26, wherein the signal output interface 26 can simultaneously issue alarms to multiple safety control devices 3 for different machines or robots Signal to make multiple machines or robot bodies emergency stop.
  • the present invention can also evaluate and monitor brain waves generated based on tactile sensation and olfactory sense, to achieve an emergency stop of a machine or a robot when an operator finds a danger.
  • FIG. 3 exemplarily shows the method steps of the evaluation performed by the security evaluation module 28 according to the present invention.
  • S1 preprocess the received brain wave signals and extract the brain wave components to be evaluated

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Manipulator (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un système de robot et un dispositif d'interface cerveau-ordinateur (2) pour une commande de sécurité. Le dispositif d'interface cerveau-ordinateur (2) comprend : un module de réception d'onde cérébrale (21) qui reçoit un signal d'onde cérébrale en provenance d'au moins une électrode (1) ; et un module d'évaluation de sécurité (28) qui est apte à communiquer avec un module de commande de sécurité (3), le module de commande de sécurité (3) étant apte à freiner au moins une machine ou un robot (4), le module d'évaluation de sécurité (28) évaluant, par l'analyse du signal d'onde cérébrale reçu, le degré de tension de l'émetteur de l'onde de grain, et envoyant, lorsqu'il est déterminé que le degré de tension dépasse un seuil, un signal d'avertissement au module de commande de sécurité (3), et le module de commande de sécurité (3) étant apte à freiner ou à freiner partiellement la machine ou le robot en fonction de l'indication d'avertissement.
PCT/CN2018/103330 2018-08-30 2018-08-30 Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot WO2020042109A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880097045.1A CN113039507A (zh) 2018-08-30 2018-08-30 用于安全控制的脑机接口装置以及机器人系统
PCT/CN2018/103330 WO2020042109A1 (fr) 2018-08-30 2018-08-30 Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/103330 WO2020042109A1 (fr) 2018-08-30 2018-08-30 Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot

Publications (1)

Publication Number Publication Date
WO2020042109A1 true WO2020042109A1 (fr) 2020-03-05

Family

ID=69643359

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103330 WO2020042109A1 (fr) 2018-08-30 2018-08-30 Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot

Country Status (2)

Country Link
CN (1) CN113039507A (fr)
WO (1) WO2020042109A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112462943A (zh) * 2020-12-07 2021-03-09 北京小米松果电子有限公司 一种基于脑电波信号的控制方法、装置及介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515072A (zh) * 2021-07-29 2021-10-19 江苏中信博新能源科技股份有限公司 一种太阳能跟踪支架调试系统及方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103845137A (zh) * 2014-03-19 2014-06-11 北京工业大学 基于稳态视觉诱发脑机接口的机器人控制方法
CN104524689A (zh) * 2014-12-03 2015-04-22 上海交通大学 一种通过脑-脑接口实现异体生物控制的系统及方法
CN106529421A (zh) * 2016-10-21 2017-03-22 燕山大学 基于混合脑机接口技术的情绪及疲劳检测辅助驾驶系统
CN106901916A (zh) * 2017-03-13 2017-06-30 上海大学 一种采用脑电信号控制的可行走座椅装置及其控制系统
CN107175673A (zh) * 2017-04-13 2017-09-19 中国农业大学 车型机器人控制系统及方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102009024866A1 (de) * 2009-06-09 2010-12-16 Abb Research Ltd. Verfahren und Einrichtung zur Überwachung der Hirnaktivität eines Menschen
CN104887190A (zh) * 2015-06-30 2015-09-09 上海斐讯数据通信技术有限公司 具有辅助睡眠功能的方法、系统及集成有该系统的手机
CN105468143A (zh) * 2015-11-17 2016-04-06 天津大学 一种基于运动想象脑-机接口的反馈系统
CN206045144U (zh) * 2016-03-03 2017-03-29 太原特玛茹电子科技有限公司 一种新型智能助眠和自然唤醒的装置
CN106527683A (zh) * 2016-09-29 2017-03-22 珠海格力电器股份有限公司 一种控制媒体播放器播放的装置、方法、头戴装置及系统
CN107111372A (zh) * 2016-12-22 2017-08-29 深圳市大疆创新科技有限公司 无人机套件、无人机控制装置及控制方法
CN106983945A (zh) * 2017-05-09 2017-07-28 京东方科技集团股份有限公司 助眠和/或唤醒装置及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103845137A (zh) * 2014-03-19 2014-06-11 北京工业大学 基于稳态视觉诱发脑机接口的机器人控制方法
CN104524689A (zh) * 2014-12-03 2015-04-22 上海交通大学 一种通过脑-脑接口实现异体生物控制的系统及方法
CN106529421A (zh) * 2016-10-21 2017-03-22 燕山大学 基于混合脑机接口技术的情绪及疲劳检测辅助驾驶系统
CN106901916A (zh) * 2017-03-13 2017-06-30 上海大学 一种采用脑电信号控制的可行走座椅装置及其控制系统
CN107175673A (zh) * 2017-04-13 2017-09-19 中国农业大学 车型机器人控制系统及方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112462943A (zh) * 2020-12-07 2021-03-09 北京小米松果电子有限公司 一种基于脑电波信号的控制方法、装置及介质

Also Published As

Publication number Publication date
CN113039507A (zh) 2021-06-25

Similar Documents

Publication Publication Date Title
US6167298A (en) Devices and methods for maintaining an alert state of consciousness through brain wave monitoring
JP5373631B2 (ja) 脳波信号処理システムに基づいて精神状態を定量的に評価するための装置
US10099374B2 (en) Robotic safety using wearables
KR20190061726A (ko) 차량의 운전제어권 조정 시스템 및 방법
WO2020042109A1 (fr) Dispositif d'interface cerveau-ordinateur pour commande de sécurité et système de robot
EP3472026A1 (fr) Procédé et dispositif de sécurisation d'un déplacement automatique d'un véhicule
DE19983911T5 (de) Wachsamkeitsüberwachungssystem
KR20170036428A (ko) 웨어러블 디바이스를 이용한 운전자 모니터링 방법 및 장치
KR102103832B1 (ko) 뇌파기반 스마트 안전모 및 관리 시스템
KR102013609B1 (ko) 운전자의 생체 신호 측정 장치
US11751784B2 (en) Systems and methods for detecting drowsiness in a driver of a vehicle
JP5799092B2 (ja) ヒトに安全に刺激を与えるための装置及び方法
US9454887B1 (en) Doze alert
Lahane et al. Innovative approach to control wheelchair for disabled people using BCI
EP3522170A1 (fr) Système d'alerte d'une personne
WO2010142409A1 (fr) Méthode et dispositif de surveillance de l'activité cérébrale d'une personne
DE19545392B4 (de) Verfahren und Vorrichtung zum Schalten und/oder Steuern, insbesondere eines Rechners
Rumagit et al. Gazing time analysis for drowsiness assessment using eye gaze tracker
CN106650636A (zh) 一种基于脑机接口的驾驶警觉度实时监测装置及方法
Paszkiel Application of Brain-Computer Interface Technology on Robot Lines in Industry in the Field of Increasing Safety
JP6751525B2 (ja) 生体状態判定装置及び生体状態判定プログラム
KR102537689B1 (ko) 안전모용 생체 신호 측정 모듈
CN211181014U (zh) 脑电波触发的非接触式报警装置
JP7387252B2 (ja) 信号中継装置、および生体情報モニタリングシステム
Croock et al. Quadreplegia wheelchair control system using electroencephalography signals

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18931348

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18931348

Country of ref document: EP

Kind code of ref document: A1