WO2023002668A1 - Operation control device, operation control method, and program - Google Patents

Operation control device, operation control method, and program Download PDF

Info

Publication number
WO2023002668A1
WO2023002668A1 PCT/JP2022/009675 JP2022009675W WO2023002668A1 WO 2023002668 A1 WO2023002668 A1 WO 2023002668A1 JP 2022009675 W JP2022009675 W JP 2022009675W WO 2023002668 A1 WO2023002668 A1 WO 2023002668A1
Authority
WO
WIPO (PCT)
Prior art keywords
command
unit
brain information
user
acquired
Prior art date
Application number
PCT/JP2022/009675
Other languages
French (fr)
Japanese (ja)
Inventor
孝満 島倉
隆幸 菅原
Original Assignee
株式会社Ｊｖｃケンウッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｊｖｃケンウッド filed Critical 株式会社Ｊｖｃケンウッド
Publication of WO2023002668A1 publication Critical patent/WO2023002668A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Definitions

  • the present disclosure relates to an operation control device, an operation control method and a program.
  • the present disclosure has been made in view of the above, and aims to enable appropriate electroencephalogram commands to be executed without being affected by irrelevant stimuli.
  • the operation control device includes a brain information acquisition unit that acquires brain information of a user, and two different brain information acquired by the brain information acquisition unit. a determination unit that determines whether or not the brain information corresponds to a first command and a second command; and the determination unit acquires the brain information corresponding to the first command and the brain information corresponding to the second command. an execution unit that executes processing corresponding to the first command when it is determined that
  • An operation control method includes steps of acquiring brain information of a user, determining whether the acquired brain information is brain information corresponding to two different first commands and second commands, and executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired.
  • a program includes steps of acquiring brain information of a user, determining whether or not the acquired brain information is brain information corresponding to two different first commands and second commands, and executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired.
  • FIG. 1 is a block diagram showing a configuration example of an operation control device according to this embodiment.
  • FIG. 2 is a flow chart showing the flow of processing in the operation control device according to this embodiment.
  • FIG. 1 is a block diagram showing a configuration example of an operation control device according to this embodiment.
  • the operation control device 10 controls the operation of the operation object 50 based on the user's brain information.
  • the operation control device 10 includes an electroencephalogram acquisition unit (brain information acquisition unit) 11, an electroencephalogram decoder 12, a command buffer 13, a password buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, and a display.
  • a unit 18 and a communication unit 19 are provided.
  • the operation target 50 is, for example, a robot, but is not limited to robots, and may be any device or the like that operates by communication. Although not shown, the operation target 50 is provided with a driving device, a control device for controlling the driving device, and a communication unit capable of receiving control signals and the like. When the communication unit of the operation target 50 receives the control signal, the control device drives the driving device to operate the robot.
  • the electroencephalogram acquisition unit 11 can be worn on the user's head.
  • the electroencephalogram decoder 12, the processing unit 16, the display control unit 17, the display unit 18, and the communication unit 19 may be worn by the user, or may be placed at a predetermined position without being worn.
  • the electroencephalogram acquisition unit 11, the electroencephalogram decoder 12, the processing unit 16, the display control unit 17, the display unit 18, and the communication unit 19 may be provided integrally or separately.
  • the electroencephalogram acquisition unit 11 acquires an electroencephalogram, which is brain information of the user.
  • the electroencephalogram acquisition unit 11 has, for example, an electrical sensor (for example, electrodes) that detects electroencephalograms emitted from weak currents flowing in the neural network of the brain.
  • the electroencephalogram acquisition unit 11 detects a weak current potential (electrical signal) when the user is externally stimulated or based on the user's thoughts such as thoughts.
  • the brain information acquisition unit is not limited to the electroencephalogram acquisition unit 11 .
  • the brain information acquisition unit may acquire, for example, the blood flow volume due to brain activity, which is the brain information of the user, by, for example, near-infrared light measurement.
  • the electroencephalogram acquisition unit 11 is connected to the electroencephalogram decoder 12 and transmits electroencephalograms, which are brain information acquired from the user, to the electroencephalogram decoder 12 .
  • the electroencephalogram decoder 12 restores the user's electroencephalogram electrical signal acquired by the electroencephalogram acquisition unit 11 to the user's thinking information.
  • the relationship between a plurality of electrical signals of the user's electroencephalogram and the user's thinking information is linked in advance.
  • machine learning such as deep learning is used to link the relationship between the electrical signal of the electroencephalogram and the thinking information of the user.
  • the command buffer 13 temporarily stores the first command determined by the determination unit 21 based on the thinking information of the user.
  • the first command is a command for operating the operation target 50, and is, for example, a command such as "forward”, “backward”, “stop”, “accelerate”, “decelerate”, “turn right” or “turn left”. In this case, the relationship between the first command and the motion of the operation target 50 is linked in advance.
  • the command buffer 13 can transmit and receive various data to and from the processing unit 16 .
  • the first command is not limited to the concept indicating the action to be instructed, and may be a value or the like that complements the concept indicating the action to be instructed.
  • the electroencephalogram decoder 103 restores the user's thought information based on the electrical signal acquired by the electroencephalogram acquisition unit 11, detects the intensity of the electroencephalogram as a decimal number from 0 to 1, and converts the intensity ( number) may be added to set the first command.
  • the numerical value of the first command is the count of the maximum value of the torque output of the driving device provided on the manipulation object 50 .
  • the password buffer 14 temporarily stores the password, which is the second command determined by the determination unit 21 based on the user's thought information.
  • the password, which is the second command is a command for determining execution of the first command.
  • a password corresponding to the command is set in advance by the user and stored in the storage unit 15 .
  • the password may be set by the processing unit 16, stored in the storage unit 15, and notified to the user in advance.
  • a password is a combination of one or more character strings, images of objects, abstract concepts, and the like.
  • one common password that is common to different first commands may be set, or a plurality of different individual passwords may be set for different first commands.
  • the password buffer 14 can transmit and receive various data to and from the processing unit 16 .
  • the storage unit 15 stores passwords that are a plurality of first commands and a plurality of second commands.
  • the storage unit 15 may store learning data obtained by machine-learning the relationship between the electric signal of the user's electroencephalogram and thought information.
  • the second command is associated in advance with the first command.
  • the command buffer 13 and the password buffer 14 are recording units such as semiconductor memory elements such as RAM (Random Access Memory) and flash memory provided in the processing unit 16, for example.
  • the storage unit 15 is composed of a memory card, an SSD (Solid State Drive), an external storage device, and the like.
  • the processing unit 16 is, for example, an arithmetic processing device (control device) configured by a CPU (Central Processing Unit).
  • the processing unit 16 loads the stored program into memory and executes the instructions contained in the program.
  • the processing unit 16 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the processing unit 16 and the like.
  • the processing unit 16 has a determination unit 21 and an execution unit 22 as functions. Also, the processing unit 16 transmits the processing result to the display control unit 17 and the communication unit 19 .
  • the determination unit 21 determines whether or not the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to two different first commands and second commands (passwords). .
  • the determination unit 21 compares the user's thinking information restored by the electroencephalogram decoder 12 with the first command stored in the storage unit 15 . At this time, when the thought information of the user matches the first command, the determination unit 21 determines that the thought information of the user is the first command. At this time, the determination unit 21 stores the command determined to be the first command in the command buffer 13 . On the other hand, when the user's thought information does not match the first command, the determination unit 21 determines that the user's thought information is not the first command. At this time, the determination unit 21 does not store the command determined as not the first command in the command buffer 13 .
  • the judgment by the judging unit 21 of whether the user's thinking information restored by the electroencephalogram decoder 12 matches the first command can be made even if the user's thinking information does not completely match as long as it can be judged that the user's thinking information means the first command. Judged as a match.
  • the determination unit 21 also compares the user's thought information restored by the electroencephalogram decoder 12 with the password stored in the storage unit 15 . At this time, when the user's thought information matches the password, the determination unit 21 determines that the user's thought information is the password. The determining unit 21 stores the password in the password buffer 14 when determining that the thought information of the user is the password. On the other hand, when the user's thought information does not match the password stored in the storage unit 15, the determination unit 21 determines that the user's thought information is not the password. The determination unit 21 does not store the password in the password buffer 14 when determining that the user's thinking information is not the password.
  • the determination unit 21 determines whether the command stored in the command buffer 13 and the password stored in the password buffer 14 are associated with each other.
  • the execution unit 22 executes processing corresponding to the first command. Specifically, when the determination unit 21 determines that the user's thought information acquired first is the first command, and the determination unit 21 determines that the user's thought information acquired next is the password. , executes the process corresponding to the first command.
  • the execution unit 22 cancels the acquired first command when a preset waiting time, such as three seconds, elapses after the determination unit 21 determines acquisition of the first command. Further, when the processing unit 16 acquires a cancel command as the user's thought after the determination unit 21 determines acquisition of the first command, the execution unit 22 cancels the acquired first command.
  • the cancel command is set in advance, and the relationship between the electric signal of the user's electroencephalogram and the user's thought information (cancel command) is linked in advance and stored in the storage unit 15 .
  • the cancel command is, for example, a symbol or concept different from the first command, such as "x" or "cancel".
  • the display control unit 17 is connected to the display unit 18 .
  • the display control unit 17 transmits the processing result transmitted from the processing unit 16 to the display unit 18 for display.
  • the display control unit 17 causes the display unit 18 to display the operation state of the operation control device 10 and the thinking instructions to the user, which are the processing results of the processing unit 16 .
  • the display unit 18 displays the operating state of the operation control device 10 transmitted from the display control unit 17 and the thinking instruction to the user.
  • the display unit 18 presents necessary information to the user.
  • the display unit 18 is a display including, for example, a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) display.
  • the communication unit 19 is capable of wireless communication with the operation target 50.
  • the communication unit 19 transmits the processing information processed by the processing unit 16 to the operation target 50 .
  • the determination unit 21 determines that the electroencephalogram corresponding to the first command and the electroencephalogram corresponding to the password have been acquired
  • the execution unit 22 generates a control signal corresponding to the processing of the first command, Send to 19.
  • the communication unit 19 transmits a control signal corresponding to processing of the first command to the operation target 50 .
  • the operation target 50 operates corresponding to the processing of the first command.
  • FIG. 2 is a flow chart showing the flow of processing in the operation control device according to this embodiment.
  • step S11 the processing unit 16 controls the display control unit 17 to display an instruction to remind the user of the first command on the display unit 18.
  • FIG. The display control unit 17 displays on the display unit 18, for example, a message such as "Please recall the first command", which instructs to recall the first command, which means the content of the operation.
  • step S12 the determination unit 21 determines whether or not the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to the first command.
  • the determination unit 21 compares the user's thought information restored by the electroencephalogram decoder 12 with the first command stored in the storage unit 15, and determines that the user's thought information does not match the first command (No ), the user's thought information is determined not to be the first command, and this routine is exited.
  • the determination unit 21 determines that the user's thought information matches the first command stored in the storage unit 15 (Yes), the determination unit 21 determines that the user's thought information is the first command, and determines that the user's thought information is the first command. 1 command is stored in the command buffer 13, and the process proceeds to step S13.
  • step S13 the processing unit 16 determines whether or not a timeout has occurred due to the lapse of a preset standby time after the determination unit 21 determined acquisition of the first command.
  • the processing unit 16 cancels the acquired first command in step S18.
  • step S ⁇ b>19 the processing unit 16 controls the display control unit 17 to display to the user that the first command has been canceled on the display unit 18 .
  • the display control unit 17 displays, for example, a timed-out message such as “Timed out” on the display unit 18 .
  • step S14 determines in step S14 whether or not an electroencephalogram corresponding to the cancel command has been detected.
  • the processing unit 16 cancels the acquired first command in step S20.
  • step S ⁇ b>21 the processing unit 16 controls the display control unit 17 to display to the user that the first command has been canceled on the display unit 18 .
  • the display control unit 17 displays, on the display unit 18, a message indicating that the first command has been canceled, such as "Cancel the first command.”
  • step S15 the determination unit 21 determines whether or not the electroencephalogram corresponding to the second command (password) has been detected. judge.
  • the determination unit 21 compares the user's thinking information restored by the electroencephalogram decoder 12 with the password stored in the storage unit 15, and determines that the user's thinking information does not match the password (No).
  • the determination unit 21 determines that the user's thought information matches the password stored in the storage unit 15 (Yes), it stores the password determined as the user's thought information as a password in the password buffer 14. Further, in step S16, the determination unit 21 associates the first command determined in step S12 and stored in the command buffer 13 with the password determined in step S15 and stored in the password buffer 14 in advance. It is determined whether it is the first command and password. When determining that the first command and the password are not associated (No), the determination unit 21 returns to step S13. When determining that the first command and the password are associated (Yes), the determination unit 21 proceeds to step S17.
  • the execution unit 22 executes processing corresponding to the first command. That is, the execution unit 22 transmits a control signal corresponding to the processing of the first command to the operation object 50 through the communication unit 19 to operate the operation object 50 .
  • the processing unit 16 controls the display control unit 17 to display to the user that the first command has been executed on the display unit 18 .
  • the display control unit 17 displays, on the display unit 18, a message indicating that the first command has been executed, such as "acceleration command executed".
  • an electroencephalogram acquisition unit 11 that acquires brain information of a user, and whether the brain information acquired by the electroencephalogram acquisition unit 11 is brain information corresponding to two different first commands and second commands (passwords) is determined.
  • a determination unit 21 that determines whether the brain information corresponding to the first command and the brain information corresponding to the second command are acquired by the determination unit 21, and an execution unit that executes processing corresponding to the first command. 22.
  • the operation target 50 is operated by executing the processing corresponding to the first command. That is, even if the brain information corresponding to the first command indicating the content of the operation is acquired, if the brain information corresponding to the second command indicating the intention to execute the content of the operation is not acquired, the processing corresponding to the first command cannot be performed. Not executed. Therefore, the content and execution timing of the first command are safer and more accurate, and an appropriate electroencephalogram command can be executed without being affected by irrelevant stimuli.
  • the first command is a command indicating the operation content for operating the operation object 50
  • the second command is a command indicating the intention to execute the operation content of the operation object 50. Therefore, by acquiring the brain information corresponding to the second command, the user can confirm the user's intention to execute the operation content of the operation target 50, and can execute an appropriate electroencephalogram command.
  • the execution unit 22 cancels the acquired first command when a predetermined waiting time elapses after the determination unit 21 determines that the brain information corresponding to the first command has been acquired. do. Therefore, by setting a waiting time from when the first command is acquired until the password is remembered, the continuation of the pending state in which the first command is acquired is restricted, and erroneous operation instructions for the operation object 50 are suppressed. be able to.
  • Each component of the illustrated operation control device 10 is functionally conceptual and does not necessarily have to be physically configured as illustrated.
  • the specific form of each device is not limited to the illustrated one, and all or part of it may be functionally or physically distributed or integrated in arbitrary units according to the processing load and usage conditions of each device.
  • the configuration of the operation control device 10 is implemented, for example, as software by a program loaded into the memory.
  • functional blocks realized by cooperation of these hardware or software have been described. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
  • the operation control device, operation control method, and program of the present disclosure can be applied to, for example, a brain machine interface.
  • operation control device 11 electroencephalogram acquisition unit (brain information acquisition unit) 12 electroencephalogram decoder 13 command buffer 14 password buffer 15 storage unit 16 processing unit 17 display control unit 18 display unit 19 communication unit 50 operation target object

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The present invention comprises a brain information acquisition unit (11) that acquires brain information on a user, a determination unit (21) that determines whether or not the brain information acquired by the brain information acquisition unit (11) is brain information corresponding to a first command and a second command that are two different commands, and an execution unit (22) that executes a process corresponding to the first command when the determination unit (21) has determined that brain information corresponding to the first command and brain information corresponding to the second command are acquired.

Description

操作制御装置、操作制御方法およびプログラムOperation control device, operation control method and program
　本開示は、操作制御装置、操作制御方法およびプログラムに関する。 The present disclosure relates to an operation control device, an operation control method and a program.
　近年、脳の情報を計測する技術が発達し、脳と機械とを直接つなぐインターフェースであるブレインマシンインターフェース（ＢＭＩ：Ｂｒａｉｎ－Ｍａｃｈｉｎｅ－Ｉｎｔｅｒｆａｃｅ）が現実的になってきている。このような技術として、例えば、特許文献１に記載されたものがある。特許文献１は、ユーザが刺激を受けたとき、ユーザから複数の定常状態視覚誘発反応電位信号を得て、システムコマンド信号を供給するものである。 In recent years, the technology for measuring brain information has developed, and the Brain-Machine Interface (BMI), which is an interface that directly connects the brain and the machine, has become a reality. As such a technique, for example, there is one described in Patent Document 1. US Pat. No. 6,200,000 obtains a plurality of steady-state visual evoked response potential signals from a user when the user is stimulated to provide system command signals.
特開２０１３－１１７９５７号公報JP 2013-117957 A
　従来のＢＭＩは、脳波コマンドを実行するときに、無関係な刺激があると、その刺激が脳波に表れて実行されてしまうおそれがある。例えば、ロボットに対して「前進」と念じると、ロボットが前進する前進コマンドの場合、周囲の誰かが「前進」と言っただけで、脳が勝手に「前進」を想起し、前進コマンドが発行されてしまう。また、ロボットの動きを見ながら操作している場合、ロボットが前進している様子がモニタに映し出されていると、それを見た操作者の脳は勝手に「前進」を想起し続けるため、勝手に前進コマンドが発行されてしまう。 With conventional BMI, if there is an unrelated stimulus when executing an electroencephalogram command, there is a risk that the stimulus will appear in the electroencephalogram and be executed. For example, if you say "move forward" to a robot, in the case of a forward command for the robot to move forward, just by saying "move forward" by someone around you, your brain will automatically recall "move forward" and issue a forward command. It will be done. In addition, when operating the robot while watching the movement of the robot, if the robot moving forward is displayed on the monitor, the brain of the operator who sees it will continue to recall "moving forward" without permission. A forward command is issued without permission.
　本開示は、上記に鑑みてなされたものであって、無関係な刺激に影響されることなく、適切な脳波コマンドを実行可能とすることを目的とする。 The present disclosure has been made in view of the above, and aims to enable appropriate electroencephalogram commands to be executed without being affected by irrelevant stimuli.
　上述した課題を解決し、目的を達成するために、本開示に係る操作制御装置は、ユーザの脳情報を取得する脳情報取得部と、前記脳情報取得部が取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定する判定部と、前記判定部が前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行する実行部と、を備える。 In order to solve the above-described problems and achieve the object, the operation control device according to the present disclosure includes a brain information acquisition unit that acquires brain information of a user, and two different brain information acquired by the brain information acquisition unit. a determination unit that determines whether or not the brain information corresponds to a first command and a second command; and the determination unit acquires the brain information corresponding to the first command and the brain information corresponding to the second command. an execution unit that executes processing corresponding to the first command when it is determined that
　本開示に係る操作制御方法は、ユーザの脳情報を取得するステップと、取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定するステップと、前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行するステップと、を含む。 An operation control method according to the present disclosure includes steps of acquiring brain information of a user, determining whether the acquired brain information is brain information corresponding to two different first commands and second commands, and executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired.
　本開示に係るプログラムは、ユーザの脳情報を取得するステップと、取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定するステップと、前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行するステップと、をコンピュータに実行させる。 A program according to the present disclosure includes steps of acquiring brain information of a user, determining whether or not the acquired brain information is brain information corresponding to two different first commands and second commands, and executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired.
　本開示によれば、無関係な刺激に影響されることなく、適切な脳波コマンドを実行することができるという効果を奏する。 According to the present disclosure, there is an effect that an appropriate electroencephalogram command can be executed without being affected by irrelevant stimuli.
図１は、本実施形態に係る操作制御装置の構成例を示すブロック図である。FIG. 1 is a block diagram showing a configuration example of an operation control device according to this embodiment. 図２は、本実施形態に係る操作制御装置における処理の流れを示すフローチャートである。FIG. 2 is a flow chart showing the flow of processing in the operation control device according to this embodiment.
　以下に添付図面を参照して、本開示に係る操作制御装置、操作制御方法およびプログラムの実施形態を詳細に説明する。なお、以下の実施形態により本発明が限定されるものではない。 Embodiments of an operation control device, an operation control method, and a program according to the present disclosure will be described in detail below with reference to the accompanying drawings. In addition, the present invention is not limited by the following embodiments.
［操作制御装置］
　図１は、本実施形態に係る操作制御装置の構成例を示すブロック図である。 [Operation control device]
FIG. 1 is a block diagram showing a configuration example of an operation control device according to this embodiment.
　本実施形態において、図１に示すように、操作制御装置１０は、ユーザの脳情報に基づいて操作対象物５０を操作制御するものである。操作制御装置１０は、脳波取得部（脳情報取得部）１１と、脳波デコーダ１２と、コマンドバッファ１３と、パスワードバッファ１４と、記憶部１５と、処理部１６と、表示制御部１７と、表示部１８と、通信部１９とを備える。 In this embodiment, as shown in FIG. 1, the operation control device 10 controls the operation of the operation object 50 based on the user's brain information. The operation control device 10 includes an electroencephalogram acquisition unit (brain information acquisition unit) 11, an electroencephalogram decoder 12, a command buffer 13, a password buffer 14, a storage unit 15, a processing unit 16, a display control unit 17, and a display. A unit 18 and a communication unit 19 are provided.
　操作対象物５０は、例えば、ロボットであるが、ロボットに限定されるものではなく、通信により作動する装置等であればよい。操作対象物５０は、図示しないが、駆動装置と、駆動装置を制御する制御装置と、制御信号などを受信可能な通信部が設けられる。操作対象物５０は、通信部が制御信号を受信すると、制御装置は、駆動装置を駆動することで、ロボットを作動する。 The operation target 50 is, for example, a robot, but is not limited to robots, and may be any device or the like that operates by communication. Although not shown, the operation target 50 is provided with a driving device, a control device for controlling the driving device, and a communication unit capable of receiving control signals and the like. When the communication unit of the operation target 50 receives the control signal, the control device drives the driving device to operate the robot.
　脳波取得部１１は、ユーザの頭部に装着可能である。脳波デコーダ１２と処理部１６と表示制御部１７と表示部１８と通信部１９は、ユーザが装着していてもよいし、装着せずに所定の位置に載置されていてもよい。また、脳波取得部１１と脳波デコーダ１２と処理部１６と表示制御部１７と表示部１８と通信部１９は、一体に設けられてもよいし、それぞれ別体に設けられていてもよい。 The electroencephalogram acquisition unit 11 can be worn on the user's head. The electroencephalogram decoder 12, the processing unit 16, the display control unit 17, the display unit 18, and the communication unit 19 may be worn by the user, or may be placed at a predetermined position without being worn. The electroencephalogram acquisition unit 11, the electroencephalogram decoder 12, the processing unit 16, the display control unit 17, the display unit 18, and the communication unit 19 may be provided integrally or separately.
　脳波取得部１１は、ユーザの脳情報である脳波を取得するものである。脳波取得部１１は、例えば、脳の神経ネットワークに流れる微弱な電流から出る脳波を検出する電気センサ（例えば、電極）を有する。脳波取得部１１は、ユーザが外部から刺激を受けたときや、ユーザの思念などの思考に基づく、微弱な電流の電位（電気信号）を検出する。なお、脳情報取得部は、脳波取得部１１に限定されるものではない。脳情報取得部は、例えば、ユーザの脳情報である脳活動による血流量を、例えば、近赤外光計測などによって取得するものであってもよい。脳波取得部１１は、脳波デコーダ１２に接続され、ユーザから取得した脳情報である脳波を脳波デコーダ１２に送信する。 The electroencephalogram acquisition unit 11 acquires an electroencephalogram, which is brain information of the user. The electroencephalogram acquisition unit 11 has, for example, an electrical sensor (for example, electrodes) that detects electroencephalograms emitted from weak currents flowing in the neural network of the brain. The electroencephalogram acquisition unit 11 detects a weak current potential (electrical signal) when the user is externally stimulated or based on the user's thoughts such as thoughts. Note that the brain information acquisition unit is not limited to the electroencephalogram acquisition unit 11 . The brain information acquisition unit may acquire, for example, the blood flow volume due to brain activity, which is the brain information of the user, by, for example, near-infrared light measurement. The electroencephalogram acquisition unit 11 is connected to the electroencephalogram decoder 12 and transmits electroencephalograms, which are brain information acquired from the user, to the electroencephalogram decoder 12 .
　脳波デコーダ１２は、脳波取得部１１が取得したユーザの脳波の電気信号をユーザの思考情報に復元する。この場合、事前に、ユーザの脳波の複数の電気信号とユーザの思考情報との関係を紐付けておく。この場合、例えば、ディープラーニング（深層学習）などによる機械学習を用いて、脳波の電気信号とユーザの思考情報との関係を紐付ける。 The electroencephalogram decoder 12 restores the user's electroencephalogram electrical signal acquired by the electroencephalogram acquisition unit 11 to the user's thinking information. In this case, the relationship between a plurality of electrical signals of the user's electroencephalogram and the user's thinking information is linked in advance. In this case, for example, machine learning such as deep learning is used to link the relationship between the electrical signal of the electroencephalogram and the thinking information of the user.
　コマンドバッファ１３は、ユーザの思考情報に基づいて判定部２１が判定した第１コマンドを一時的に記憶する。第１コマンドは、操作対象物５０を作動させるためのコマンドであり、例えば、「前進」「後退」「停止」「加速」「減速」「右旋回」「左旋回」などのコマンドである。この場合、事前に、第１コマンドと操作対象物５０の動作との関係は紐付けられている。コマンドバッファ１３は、処理部１６との間で各種データの送受信が可能である。 The command buffer 13 temporarily stores the first command determined by the determination unit 21 based on the thinking information of the user. The first command is a command for operating the operation target 50, and is, for example, a command such as "forward", "backward", "stop", "accelerate", "decelerate", "turn right" or "turn left". In this case, the relationship between the first command and the motion of the operation target 50 is linked in advance. The command buffer 13 can transmit and receive various data to and from the processing unit 16 .
　なお、第１コマンドは、指示する動作を示す概念に限るものではなく、指示する動作を示す概念を補完する数値などを付加したものであってもよい。例えば、脳波デコーダ１０３は、脳波取得部１１が取得した電気信号に基づいて、ユーザの思考情報に復元すると同時に、脳波の強度を０から１までの小数で検出し、ユーザの思考情報に強度（数値）を付加して第１コマンドを設定してもよい。この場合、第１コマンドの数値は、操作対象物５０に設けられた駆動装置のトルク出力の最大値の計数である。 It should be noted that the first command is not limited to the concept indicating the action to be instructed, and may be a value or the like that complements the concept indicating the action to be instructed. For example, the electroencephalogram decoder 103 restores the user's thought information based on the electrical signal acquired by the electroencephalogram acquisition unit 11, detects the intensity of the electroencephalogram as a decimal number from 0 to 1, and converts the intensity ( number) may be added to set the first command. In this case, the numerical value of the first command is the count of the maximum value of the torque output of the driving device provided on the manipulation object 50 .
　パスワードバッファ１４は、ユーザの思考情報に基づいて判定部２１が判定した第２コマンドであるパスワードを一時的に記憶する。第２コマンドであるパスワードは、第１コマンドの実行を決定するためのコマンドである。コマンドに対応するパスワードは、予めユーザが設定し、記憶部１５に記憶される。なお、パスワードは、処理部１６が設定し、記憶部１５に記憶されると共に、予めユーザに通知しておいてもよい。パスワードは、文字列や物のイメージ、抽象的な概念などを１つ以上組み合わせたものである。パスワードは、異なる第１コマンドに対して共通である１つの共通のパスワードを設定してもよいし、異なる第１コマンドごとに異なる複数の個別のパスワードを設定してもよい。第１コマンドである「前進」「後退」「停止」「加速」「減速」「右旋回」「左旋回」ごとに個別のパスワードを設定することで、勢い余って次の第１コマンドを発行する事態を避けることができる。パスワードバッファ１４は、処理部１６との間で各種データの送受信が可能である。 The password buffer 14 temporarily stores the password, which is the second command determined by the determination unit 21 based on the user's thought information. The password, which is the second command, is a command for determining execution of the first command. A password corresponding to the command is set in advance by the user and stored in the storage unit 15 . The password may be set by the processing unit 16, stored in the storage unit 15, and notified to the user in advance. A password is a combination of one or more character strings, images of objects, abstract concepts, and the like. As for the password, one common password that is common to different first commands may be set, or a plurality of different individual passwords may be set for different first commands. By setting individual passwords for each of the first commands, "forward", "backward", "stop", "accelerate", "decelerate", "turn right", and "turn left", the next first command will be issued with momentum. situation can be avoided. The password buffer 14 can transmit and receive various data to and from the processing unit 16 .
　記憶部１５は、複数の第１コマンドおよび複数の第２コマンドであるパスワードを記憶する。また、記憶部１５は、ユーザの脳波の電気信号と思考情報との関連を機械学習させた学習データなどが格納されてもよい。第２コマンドは、予め第１コマンドと対応付けられている。 The storage unit 15 stores passwords that are a plurality of first commands and a plurality of second commands. In addition, the storage unit 15 may store learning data obtained by machine-learning the relationship between the electric signal of the user's electroencephalogram and thought information. The second command is associated in advance with the first command.
　コマンドバッファ１３およびパスワードバッファ１４は、例えば、処理部１６が備えるＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、フラッシュメモリ（Ｆｌａｓｈ　Ｍｅｍｏｒｙ）などの半導体メモリ素子などの記録部である。記憶部１５は、メモリカードやＳＳＤ（Solid　State　Drive）、外部記憶装置などで構成される。 The command buffer 13 and the password buffer 14 are recording units such as semiconductor memory elements such as RAM (Random Access Memory) and flash memory provided in the processing unit 16, for example. The storage unit 15 is composed of a memory card, an SSD (Solid State Drive), an external storage device, and the like.
　処理部１６は、例えば、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）などで構成された演算処理装置（制御装置）である。処理部１６は、記憶されているプログラムをメモリにロードして、プログラムに含まれる命令を実行する。処理部１６は、図示しない内部メモリが含まれ、内部メモリは、処理部１６におけるデータの一時記憶などに用いられる。処理部１６は、判定部２１と、実行部２２とを機能として有する。また、処理部１６は、処理結果を表示制御部１７および通信部１９に送信する。 The processing unit 16 is, for example, an arithmetic processing device (control device) configured by a CPU (Central Processing Unit). The processing unit 16 loads the stored program into memory and executes the instructions contained in the program. The processing unit 16 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the processing unit 16 and the like. The processing unit 16 has a determination unit 21 and an execution unit 22 as functions. Also, the processing unit 16 transmits the processing result to the display control unit 17 and the communication unit 19 .
　判定部２１は、脳波取得部１１が取得して脳波デコーダ１２が復元したユーザの思考情報が、異なる２つの第１コマンドおよび第２コマンド（パスワード）に対応する脳波であるか否かを判定する。 The determination unit 21 determines whether or not the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to two different first commands and second commands (passwords). .
　具体的に、判定部２１は、脳波デコーダ１２が復元したユーザの思考情報と、記憶部１５に記憶された第１コマンドとを比較する。このとき、判定部２１は、ユーザの思考情報が第１コマンドに一致したとき、ユーザの思考情報が第１コマンドであると判定する。このとき、判定部２１は、第１コマンドであると判定したコマンドをコマンドバッファ１３に格納する。一方、判定部２１は、ユーザの思考情報が第１コマンドに一致しないとき、ユーザの思考情報が第１コマンドでないと判定する。このとき、判定部２１は、第１コマンドではないと判定したコマンドをコマンドバッファ１３に格納しない。判定部２１による、脳波デコーダ１２が復元したユーザの思考情報と、第１コマンドのとの一致の判断は、ユーザの思考情報が、第１コマンドを意味することが判断できれば、完全一致ではなくとも一致と判断する。 Specifically, the determination unit 21 compares the user's thinking information restored by the electroencephalogram decoder 12 with the first command stored in the storage unit 15 . At this time, when the thought information of the user matches the first command, the determination unit 21 determines that the thought information of the user is the first command. At this time, the determination unit 21 stores the command determined to be the first command in the command buffer 13 . On the other hand, when the user's thought information does not match the first command, the determination unit 21 determines that the user's thought information is not the first command. At this time, the determination unit 21 does not store the command determined as not the first command in the command buffer 13 . The judgment by the judging unit 21 of whether the user's thinking information restored by the electroencephalogram decoder 12 matches the first command can be made even if the user's thinking information does not completely match as long as it can be judged that the user's thinking information means the first command. Judged as a match.
　また、判定部２１は、脳波デコーダ１２が復元したユーザの思考情報と、記憶部１５に記憶されたパスワードとを比較する。このとき、判定部２１は、ユーザの思考情報がパスワードに一致したとき、ユーザの思考情報がパスワードであると判定する。判定部２１は、ユーザの思考情報がパスワードであると判定したとき、パスワードをパスワードバッファ１４に格納する。一方、判定部２１は、ユーザの思考情報が記憶部１５に記憶されたパスワードに一致しないとき、ユーザの思考情報がパスワードでないと判定する。判定部２１は、ユーザの思考情報がパスワードではないと判定したとき、パスワードをパスワードバッファ１４に格納しない。 The determination unit 21 also compares the user's thought information restored by the electroencephalogram decoder 12 with the password stored in the storage unit 15 . At this time, when the user's thought information matches the password, the determination unit 21 determines that the user's thought information is the password. The determining unit 21 stores the password in the password buffer 14 when determining that the thought information of the user is the password. On the other hand, when the user's thought information does not match the password stored in the storage unit 15, the determination unit 21 determines that the user's thought information is not the password. The determination unit 21 does not store the password in the password buffer 14 when determining that the user's thinking information is not the password.
　また、判定部２１は、コマンドバッファ１３に格納されたコマンドと、パスワードバッファ１４に格納されたパスワードとが、対応付けられた関係か否かを判断する。 Also, the determination unit 21 determines whether the command stored in the command buffer 13 and the password stored in the password buffer 14 are associated with each other.
　実行部２２は、判定部２１が第１コマンドに対応する脳波およびパスワードに対応する脳波を取得したと判定したとき、第１コマンドに対応する処理を実行する。具体的に、実行部２２は、判定部２１により最初に取得したユーザの思考情報が第１コマンドであると判定され、判定部２１により次に取得したユーザの思考情報がパスワードであると判定すると、第１コマンドに対応する処理を実行する。 When the determination unit 21 determines that the electroencephalogram corresponding to the first command and the electroencephalogram corresponding to the password have been acquired, the execution unit 22 executes processing corresponding to the first command. Specifically, when the determination unit 21 determines that the user's thought information acquired first is the first command, and the determination unit 21 determines that the user's thought information acquired next is the password. , executes the process corresponding to the first command.
　但し、実行部２２は、判定部２１が第１コマンドの取得を判定してから、例えば３秒間など、予め設定された待機時間が経過すると、取得した第１コマンドをキャンセルする。また、実行部２２は、判定部２１が第１コマンドの取得を判定してから、処理部１６が、ユーザの思考としてキャンセルコマンドを取得すると、取得した第１コマンドをキャンセルする。キャンセルコマンドは、予め設定されたものであり、事前に、ユーザの脳波の電気信号とユーザの思考情報（キャンセルコマンド）との関係は紐付けられ、記憶部１５に記憶しておく。キャンセルコマンドとは、例えば、「×」「キャンセル」など、第１コマンドとは異なるシンボルや概念などである。 However, the execution unit 22 cancels the acquired first command when a preset waiting time, such as three seconds, elapses after the determination unit 21 determines acquisition of the first command. Further, when the processing unit 16 acquires a cancel command as the user's thought after the determination unit 21 determines acquisition of the first command, the execution unit 22 cancels the acquired first command. The cancel command is set in advance, and the relationship between the electric signal of the user's electroencephalogram and the user's thought information (cancel command) is linked in advance and stored in the storage unit 15 . The cancel command is, for example, a symbol or concept different from the first command, such as "x" or "cancel".
　表示制御部１７は、表示部１８が接続される。表示制御部１７は、処理部１６から送信された処理結果を表示部１８に送信して表示させる。表示制御部１７は、処理部１６の処理結果である操作制御装置１０の動作状態やユーザに対する思考指示を表示部１８に表示させる。 The display control unit 17 is connected to the display unit 18 . The display control unit 17 transmits the processing result transmitted from the processing unit 16 to the display unit 18 for display. The display control unit 17 causes the display unit 18 to display the operation state of the operation control device 10 and the thinking instructions to the user, which are the processing results of the processing unit 16 .
　表示部１８は、表示制御部１７から送信された操作制御装置１０の動作状態やユーザに対する思考指示を表示する。表示部１８は、ユーザに必要情報を提示する。表示部１８は、例えば、液晶ディスプレイ（ＬＣＤ：Ｌｉｑｕｉｄ　Ｃｒｙｓｔａｌ　Ｄｉｓｐｌａｙ）または有機ＥＬ（Ｏｒｇａｎｉｃ　Ｅｌｅｃｔｒｏ－Ｌｕｍｉｎｅｓｃｅｎｃｅ）ディスプレイなどを含むディスプレイである。 The display unit 18 displays the operating state of the operation control device 10 transmitted from the display control unit 17 and the thinking instruction to the user. The display unit 18 presents necessary information to the user. The display unit 18 is a display including, for example, a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) display.
　通信部１９は、操作対象物５０との間で無線通信が可能である。通信部１９は、処理部１６が処理した処理情報を操作対象物５０に送信する。具体的に、判定部２１が第１コマンドに対応する脳波およびパスワードに対応する脳波を取得したと判定したとき、実行部２２は、第１コマンドの処理に対応する制御信号を生成して通信部１９に送る。通信部１９は、第１コマンドの処理に対応する制御信号を操作対象物５０に送信する。操作対象物５０は、通信部１９によって送信された第１コマンドを受信することで、第１コマンドの処理に対応して作動する。 The communication unit 19 is capable of wireless communication with the operation target 50. The communication unit 19 transmits the processing information processed by the processing unit 16 to the operation target 50 . Specifically, when the determination unit 21 determines that the electroencephalogram corresponding to the first command and the electroencephalogram corresponding to the password have been acquired, the execution unit 22 generates a control signal corresponding to the processing of the first command, Send to 19. The communication unit 19 transmits a control signal corresponding to processing of the first command to the operation target 50 . By receiving the first command transmitted by the communication unit 19, the operation target 50 operates corresponding to the processing of the first command.
［操作制御方法］
　ここで、操作制御装置１０による操作制御方法について説明する。図２は、本施形態に係る操作制御装置における処理の流れを示すフローチャートである。 [Operation control method]
Here, the operation control method by the operation control device 10 will be described. FIG. 2 is a flow chart showing the flow of processing in the operation control device according to this embodiment.
　図１および図２に示すように、ステップＳ１１にて、処理部１６は、表示制御部１７を制御することで、ユーザに対して表示部１８に第１コマンドを想起させる指示を表示する。表示制御部１７は、例えば、「第１コマンドを想起してください」などの操作内容を意味する第１コマンド想起を指示するメッセージを表示部１８に表示する。 As shown in FIGS. 1 and 2, in step S11, the processing unit 16 controls the display control unit 17 to display an instruction to remind the user of the first command on the display unit 18. FIG. The display control unit 17 displays on the display unit 18, for example, a message such as "Please recall the first command", which instructs to recall the first command, which means the content of the operation.
　ステップＳ１２にて、判定部２１は、脳波取得部１１が取得して脳波デコーダ１２が復元したユーザの思考情報が第１コマンドに対応する脳波であるか否かを判定する。ここで、判定部２１は、脳波デコーダ１２が復元したユーザの思考情報と、記憶部１５に格納された第１コマンドとを比較し、ユーザの思考情報が第１コマンドに一致しないと判定（Ｎｏ）すると、ユーザの思考情報が第１コマンドでないとしてこのルーチンを抜ける。一方、判定部２１は、ユーザの思考情報が記憶部１５に格納された第１コマンドに一致したと判定（Ｙｅｓ）すると、ユーザの思考情報が第１コマンドであるとして、判定部２１は、第１コマンドをコマンドバッファ１３に格納し、ステップＳ１３に移行する。 In step S12, the determination unit 21 determines whether or not the user's thought information acquired by the electroencephalogram acquisition unit 11 and restored by the electroencephalogram decoder 12 is an electroencephalogram corresponding to the first command. Here, the determination unit 21 compares the user's thought information restored by the electroencephalogram decoder 12 with the first command stored in the storage unit 15, and determines that the user's thought information does not match the first command (No ), the user's thought information is determined not to be the first command, and this routine is exited. On the other hand, if the determination unit 21 determines that the user's thought information matches the first command stored in the storage unit 15 (Yes), the determination unit 21 determines that the user's thought information is the first command, and determines that the user's thought information is the first command. 1 command is stored in the command buffer 13, and the process proceeds to step S13.
　ステップＳ１３にて、処理部１６は、判定部２１が第１コマンドの取得を判定してから、予め設定された待機時間が経過することで、タイムアウトしたか否かを判定する。処理部１６は、タイムアウトしたと判定（Ｙｅｓ）すると、ステップＳ１８にて、取得した第１コマンドをキャンセルする。そして、ステップＳ１９にて、処理部１６は、表示制御部１７を制御することで、ユーザに対して表示部１８に第１コマンドをキャンセルしたことを表示する。表示制御部１７は、例えば、「タイムアウトです」などのタイムアウトしたメッセージを表示部１８に表示する。 In step S13, the processing unit 16 determines whether or not a timeout has occurred due to the lapse of a preset standby time after the determination unit 21 determined acquisition of the first command. When determining that the timeout has occurred (Yes), the processing unit 16 cancels the acquired first command in step S18. Then, in step S<b>19 , the processing unit 16 controls the display control unit 17 to display to the user that the first command has been canceled on the display unit 18 . The display control unit 17 displays, for example, a timed-out message such as “Timed out” on the display unit 18 .
　処理部１６は、タイムアウトしていない判定（Ｎｏ）すると、ステップＳ１４にて、キャンセルコマンドに対応する脳波を検出したか否かを判定する。処理部１６は、キャンセルコマンドに対応する脳波を検出したと判定（Ｙｅｓ）すると、ステップＳ２０にて、取得した第１コマンドをキャンセルする。そして、ステップＳ２１にて、処理部１６は、表示制御部１７を制御することで、ユーザに対して表示部１８に第１コマンドをキャンセルしたことを表示する。表示制御部１７は、例えば、「第１コマンドをキャンセルします」などの第１コマンドをキャンセルしたメッセージを表示部１８に表示する。 When determining that the timeout has not occurred (No), the processing unit 16 determines in step S14 whether or not an electroencephalogram corresponding to the cancel command has been detected. When determining that an electroencephalogram corresponding to the cancel command is detected (Yes), the processing unit 16 cancels the acquired first command in step S20. Then, in step S<b>21 , the processing unit 16 controls the display control unit 17 to display to the user that the first command has been canceled on the display unit 18 . The display control unit 17 displays, on the display unit 18, a message indicating that the first command has been canceled, such as "Cancel the first command."
　一方、処理部１６は、キャンセルコマンドに対応する脳波を検出しないと判定（Ｎｏ）すると、ステップＳ１５にて、判定部２１は、第２コマンド（パスワード）に対応する脳波を検出したか否かを判定する。ここで、判定部２１は、脳波デコーダ１２が復元したユーザの思考情報と、記憶部１５に格納されたパスワードとを比較し、ユーザの思考情報がパスワードに一致しないと判定（Ｎｏ）すると、ステップＳ１３に戻り、処理を継続する。 On the other hand, when the processing unit 16 determines (No) that the electroencephalogram corresponding to the cancel command is not detected, in step S15, the determination unit 21 determines whether or not the electroencephalogram corresponding to the second command (password) has been detected. judge. Here, the determination unit 21 compares the user's thinking information restored by the electroencephalogram decoder 12 with the password stored in the storage unit 15, and determines that the user's thinking information does not match the password (No). Returning to S13, the processing is continued.
　一方、判定部２１は、ユーザの思考情報が記憶部１５に格納されたパスワードに一致したと判定（Ｙｅｓ）すると、ユーザの思考情報がパスワードであるとして判定したパスワードをパスワードバッファ１４に格納する。さらに、判定部２１は、ステップＳ１６において、ステップＳ１２で判定されコマンドバッファ１３に格納された第１コマンドと、ステップＳ１５で判定されパスワードバッファ１４に格納されたパスワードが、予め関連性が対応付けられた第１コマンドおよびパスワードであるか判定する。判定部２１は、第１コマンドとパスワードが対応付けられていないと判定（Ｎｏ）すると、ステップＳ１３に戻る。判定部２１は、第１コマンドとパスワードが対応付けられている判定（Ｙｅｓ）すると、ステップＳ１７に移行する。 On the other hand, when the determination unit 21 determines that the user's thought information matches the password stored in the storage unit 15 (Yes), it stores the password determined as the user's thought information as a password in the password buffer 14. Further, in step S16, the determination unit 21 associates the first command determined in step S12 and stored in the command buffer 13 with the password determined in step S15 and stored in the password buffer 14 in advance. It is determined whether it is the first command and password. When determining that the first command and the password are not associated (No), the determination unit 21 returns to step S13. When determining that the first command and the password are associated (Yes), the determination unit 21 proceeds to step S17.
　ステップＳ１７にて、実行部２２は、第１コマンドに対応する処理を実行する。すなわち、実行部２２は、第１コマンドの処理に対応する制御信号を通信部１９により操作対象物５０に送信し、操作対象物５０を作動させる。そして、ステップＳ１８にて、処理部１６は、表示制御部１７を制御することで、ユーザに対して表示部１８に第１コマンドを実行したことを表示する。表示制御部１７は、例えば、「加速コマンドを実行しました」などの第１コマンドを実行したメッセージを表示部１８に表示する。 At step S17, the execution unit 22 executes processing corresponding to the first command. That is, the execution unit 22 transmits a control signal corresponding to the processing of the first command to the operation object 50 through the communication unit 19 to operate the operation object 50 . In step S<b>18 , the processing unit 16 controls the display control unit 17 to display to the user that the first command has been executed on the display unit 18 . The display control unit 17 displays, on the display unit 18, a message indicating that the first command has been executed, such as "acceleration command executed".
［効果］
　本実施形態では、ユーザの脳情報を取得する脳波取得部１１と、脳波取得部１１が取得した脳情報が異なる２つの第１コマンドおよび第２コマンド（パスワード）に対応する脳情報であるか否かを判定する判定部２１と、判定部２１が第１コマンドに対応する脳情報および第２コマンドに対応する脳情報を取得したと判定したときに第１コマンドに対応する処理を実行する実行部２２とを備える。 [effect]
In this embodiment, an electroencephalogram acquisition unit 11 that acquires brain information of a user, and whether the brain information acquired by the electroencephalogram acquisition unit 11 is brain information corresponding to two different first commands and second commands (passwords) is determined. a determination unit 21 that determines whether the brain information corresponding to the first command and the brain information corresponding to the second command are acquired by the determination unit 21, and an execution unit that executes processing corresponding to the first command. 22.
　そのため、第１コマンドに対応する脳情報および第２コマンドに対応する脳情報を取得したときに、第１コマンドに対応する処理を実行することで、操作対象物５０を作動する。すなわち、操作内容を意味する第１コマンドに対応する脳情報を取得しても、操作内容を実行する意思を表す第２コマンドに対応する脳情報を取得しないと、第１コマンドに対応する処理が実行されない。そのため、第１コマンドの内容および実行タイミングがより安全で的確に発動されることとなり、無関係な刺激に影響されることなく、適切な脳波コマンドを実行することができる。 Therefore, when the brain information corresponding to the first command and the brain information corresponding to the second command are acquired, the operation target 50 is operated by executing the processing corresponding to the first command. That is, even if the brain information corresponding to the first command indicating the content of the operation is acquired, if the brain information corresponding to the second command indicating the intention to execute the content of the operation is not acquired, the processing corresponding to the first command cannot be performed. Not executed. Therefore, the content and execution timing of the first command are safer and more accurate, and an appropriate electroencephalogram command can be executed without being affected by irrelevant stimuli.
　また、本実施形態では、第１コマンドが操作対象物５０を操作するための操作内容を意味するコマンドであり、第２コマンドが操作対象物５０の操作内容の実行の意思を意味するコマンドである。そのため、第２コマンドに対応する脳情報を取得することで、ユーザが操作対象物５０の操作内容の実行の意思を確認することができ、適切な脳波コマンドを実行することができる。 Further, in the present embodiment, the first command is a command indicating the operation content for operating the operation object 50, and the second command is a command indicating the intention to execute the operation content of the operation object 50. . Therefore, by acquiring the brain information corresponding to the second command, the user can confirm the user's intention to execute the operation content of the operation target 50, and can execute an appropriate electroencephalogram command.
　また、本実施形態では、実行部２２は、判定部２１が第１コマンドに対応する脳情報を取得したと判定してから、予め設定された待機時間が経過すると、取得した第１コマンドをキャンセルする。そのため、第１コマンドが取得されてからパスワードを想起するまでの待ち時間を設定することで、第１コマンドを取得した保留状態の継続を制限し、操作対象物５０の操作指令の間違いを抑制することができる。 In addition, in the present embodiment, the execution unit 22 cancels the acquired first command when a predetermined waiting time elapses after the determination unit 21 determines that the brain information corresponding to the first command has been acquired. do. Therefore, by setting a waiting time from when the first command is acquired until the password is remembered, the continuation of the pending state in which the first command is acquired is restricted, and erroneous operation instructions for the operation object 50 are suppressed. be able to.
　これまで本開示に係る操作制御装置１０について説明したが、上述した実施形態以外にも種々の異なる形態にて実施されてよい。 Although the operation control device 10 according to the present disclosure has been described so far, it may be implemented in various different forms other than the embodiment described above.
　図示した操作制御装置１０の各構成要素は、機能概念的なものであり、必ずしも物理的に図示の如く構成されていなくてもよい。すなわち、各装置の具体的形態は、図示のものに限られず、各装置の処理負担や使用状況などに応じて、その全部または一部を任意の単位で機能的または物理的に分散または統合してもよい。 Each component of the illustrated operation control device 10 is functionally conceptual and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to the illustrated one, and all or part of it may be functionally or physically distributed or integrated in arbitrary units according to the processing load and usage conditions of each device. may
　操作制御装置１０の構成は、例えば、ソフトウェアとして、メモリにロードされたプログラムなどによって実現される。上記実施形態では、これらのハードウェアまたはソフトウェアの連携によって実現される機能ブロックとして説明した。すなわち、これらの機能ブロックについては、ハードウェアのみ、ソフトウェアのみ、または、それらの組み合わせによって種々の形で実現できる。 The configuration of the operation control device 10 is implemented, for example, as software by a program loaded into the memory. In the above embodiments, functional blocks realized by cooperation of these hardware or software have been described. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
　上記した構成要素には、当業者が容易に想定できるもの、実質的に同一のものを含む。さらに、上記した構成は適宜組み合わせが可能である。また、本発明の要旨を逸脱しない範囲において構成の種々の省略、置換または変更が可能である。 The above components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the above configurations can be combined as appropriate. Also, various omissions, substitutions, or modifications of the configuration are possible without departing from the gist of the present invention.
　本開示の操作制御装置、操作制御方法およびプログラムは、例えば、ブレインマシンインターフェースに適用することができる。 The operation control device, operation control method, and program of the present disclosure can be applied to, for example, a brain machine interface.
　１０　操作制御装置
　１１　脳波取得部（脳情報取得部）
　１２　脳波デコーダ
　１３　コマンドバッファ
　１４　パスワードバッファ
　１５　記憶部
　１６　処理部
　１７　表示制御部
　１８　表示部
　１９　通信部
　５０　操作対象物 10 operation control device 11 electroencephalogram acquisition unit (brain information acquisition unit)
12 electroencephalogram decoder 13 command buffer 14 password buffer 15 storage unit 16 processing unit 17 display control unit 18 display unit 19 communication unit 50 operation target object

Claims (5)

  1. 　ユーザの脳情報を取得する脳情報取得部と、
    　前記脳情報取得部が取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定する判定部と、
    　前記判定部が前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行する実行部と、
    　を備える操作制御装置。     a brain information acquisition unit that acquires brain information of a user;
    a determination unit that determines whether or not the brain information acquired by the brain information acquisition unit corresponds to two different first commands and second commands;
    an execution unit that executes processing corresponding to the first command when the determination unit determines that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired;
    An operation control device comprising:
  2. 　前記第１コマンドは、操作対象物を操作するための操作内容を意味するコマンドであり、前記第２コマンドは、前記操作対象物の操作内容の実行の意思を意味するコマンドである、
    　請求項１に記載の操作制御装置。     The first command is a command that indicates the operation content for operating the operation target, and the second command is a command that indicates the intention to execute the operation content of the operation target.
    The operation control device according to claim 1.
  3. 　前記実行部は、前記判定部が前記第１コマンドに対応する脳情報を取得したと判定してから、予め設定された待機時間が経過すると、取得した前記第１コマンドをキャンセルする、
    　請求項１または請求項２に記載の操作制御装置。     The execution unit cancels the acquired first command when a preset waiting time elapses after the determination unit determines that the brain information corresponding to the first command has been acquired.
    The operation control device according to claim 1 or 2.
  4. 　ユーザの脳情報を取得するステップと、
    　取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定するステップと、
    　前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行するステップと、
    　を含む操作制御方法。     obtaining brain information of a user;
    determining whether the acquired brain information corresponds to two different first and second commands;
    executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired;
    Operation control method including;
  5. 　ユーザの脳情報を取得するステップと、
    　取得した脳情報が異なる２つの第１コマンドおよび第２コマンドに対応する脳情報であるか否かを判定するステップと、
    　前記第１コマンドに対応する脳情報および前記第２コマンドに対応する脳情報を取得したと判定したときに前記第１コマンドに対応する処理を実行するステップと、
    　をコンピュータに実行させるプログラム。     obtaining brain information of a user;
    determining whether the acquired brain information corresponds to two different first and second commands;
    executing a process corresponding to the first command when it is determined that the brain information corresponding to the first command and the brain information corresponding to the second command have been acquired;
    A program that makes a computer run
PCT/JP2022/009675 2021-07-20 2022-03-07 Operation control device, operation control method, and program WO2023002668A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-119603 2021-07-20
JP2021119603A JP2023015680A (en) 2021-07-20 2021-07-20 Operation control apparatus, operation control method, and program

Publications (1)

Publication Number Publication Date
WO2023002668A1 true WO2023002668A1 (en) 2023-01-26

Family

ID=84979098

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/009675 WO2023002668A1 (en) 2021-07-20 2022-03-07 Operation control device, operation control method, and program

Country Status (2)

Country Link
JP (1) JP2023015680A (en)
WO (1) WO2023002668A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003248541A (en) * 2002-02-22 2003-09-05 Mitsubishi Electric Corp Control device using brain wave signal and control method
WO2011105000A1 (en) * 2010-02-25 2011-09-01 パナソニック株式会社 Brain wave interface system, brain wave interface provision device, execution method of brain wave interface, and program
JP2021077280A (en) * 2019-11-13 2021-05-20 キヤノン株式会社 Device, system, and providing method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003248541A (en) * 2002-02-22 2003-09-05 Mitsubishi Electric Corp Control device using brain wave signal and control method
WO2011105000A1 (en) * 2010-02-25 2011-09-01 パナソニック株式会社 Brain wave interface system, brain wave interface provision device, execution method of brain wave interface, and program
JP2021077280A (en) * 2019-11-13 2021-05-20 キヤノン株式会社 Device, system, and providing method

Also Published As

Publication number Publication date
JP2023015680A (en) 2023-02-01

Similar Documents

Publication Publication Date Title
US11877979B2 (en) Modular components for medical devices
US20240100348A1 (en) Systems and methods of delivering therapy using an ambulatory medical device
Asaad et al. A flexible software tool for temporally-precise behavioral control in Matlab
US9860330B2 (en) Control method for communication terminal connectable to wearable terminal via network
EP3149662A1 (en) Brain-computer interface for facilitating direct selection of multiple-choice answers and the identification of state changes
EP2868274A1 (en) Systems and methods for detecting pilot over focalization
US20190004633A1 (en) Touch panel device and process method for touch panel device
WO2023002668A1 (en) Operation control device, operation control method, and program
KR20210149008A (en) Organic light emitting diode display and operation method thereof
WO2023032411A1 (en) Operation control device, operation control method, and program
JP2023042344A (en) Operation control device, operation control method and program
WO2023002680A1 (en) Operation control device, operation control method, and program
US20210259563A1 (en) System and method for heterogenous data collection and analysis in a deterministic system
WO2023042501A1 (en) Operation control device, operation control method, and program
WO2023042513A1 (en) Operation control device, operation control method, and program
JP2023042999A (en) Operation control device, operation control method and program
JP2023042407A (en) Operation control device, operation control method and program
JP2023043085A (en) Operation control device, operation control method and program
US11071503B2 (en) Physiological parameter display methods and systems
GB2550258A (en) Scripting on a telematics control unit
CN112437962A (en) Health management device, health management method, and program
KR20190078299A (en) Apparatus and system for skin diagnosis and method thereof
KR102607876B1 (en) Bedside device, method for receiveing measured biometric information of a patient to monitor the patient&#39;s condition of location, and computer program
US20160209981A1 (en) Display device
US20240185995A1 (en) Methods, apparatuses, and computer program products for monitoring environmental conditions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22845608

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE