JP2010198237A - Device for acquiring intention determination information, brain information output device, robot and method for acquiring intention determination information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for acquiring intention determination information, with improved accuracy of intention detection. <P>SOLUTION: An experiment for acquiring learning data prior to operation of a machine by a brain machine interface (BMI) is divided into the following two sessions: a training session as a first stage and a brush-up session as a second stage. In the training session, data is acquired without using the machine to be operated, and parameters of intention determination algorithm are temporarily set using the data. Thereafter, in the brush-up session, data is acquired using temporary parameters in an environment for operating the machine, and the parameters of intention determination algorithm are set, including the data. The learning data is acquired by the two sessions, thereby improving the accuracy of intention detection. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a brain information output device for outputting brain information.

  In recent years, research on brain machine interface (BMI) has been conducted with the enhancement of physiological knowledge about the brain, development of brain measurement equipment, cost reduction of high-performance computers, and advancement in the field of machine learning. It has attracted attention (Non-Patent Document 1). BMI is an interface that connects the brain and machines. BMI measures a signal of brain activity that occurs when a person thinks, or when some action is taken, and applies a signal processing technique to the measurement data, thereby extracting the user's intention.

Toward Brain-Computer Interfacing (Neural Information Processing), MIT Press, ISBN 978-0262042444

  However, it is necessary to acquire learning data to move the robot. On the other hand, the relationship between "chicken and eggs" that says that the robot needs to be moved to acquire appropriate learning data is generated, and the robot is operated. The detection accuracy of the intention was not sufficient.

  In the present invention, an object of the present invention is to provide a learning data acquisition method for improving the intention detection accuracy in the centripetal operation of a robot in BMI.

  The intention determination information acquisition device according to the first aspect of the present invention provides an intention identifier that is information for identifying a user's intention, and when the user performs a series of actions based on the intention identified by the intention identifier. First intention determination information storage capable of storing two or more pieces of first intention determination information having a pair of a first learning feature amount group that is one or more feature amounts extracted from the first learning data acquired from the brain And first learning data that acquires first learning data as learning data from the user's brain when the user performs a series of actions with one intention in an environment where the robot to be operated does not operate An acquisition unit, a first learning feature amount group acquisition unit that acquires a first learning feature amount group that is one or more feature amounts from the first learning data, and a first learning feature amount group acquisition unit that has acquired the first learning feature amount group acquisition unit A learning feature group and the first learning data A first intention determination information storage unit that stores in the first intention determination information storage unit first intention determination information that associates an intention identifier that identifies one intention when the intention is made, the intention identifier, and the intention A second learning feature quantity group that is one or more feature quantities extracted from the second learning data acquired from the user's brain when the user performs a series of actions based on the intention identified by the identifier; A second intention determination information storage unit capable of storing two or more pieces of second intention determination information having a pair, and a robot that performs an operation according to the received intention identifier. When the user performs a series of actions based on the intention of the second learning data acquisition unit that acquires second learning data that is learning data from the user's brain, and from the second learning data, one or more Second learning feature, which is a feature quantity Identifying a second learning feature value group acquisition unit that acquires a group, a second learning feature value group acquired by the second learning feature value group acquisition unit, and one intention when the second learning data is acquired The intention determination information acquisition device includes a second intention determination information storage unit that stores second intention determination information associated with an intention identifier to be stored in the second intention determination information storage unit.

  With this configuration, learning data for improving the accuracy of intention detection can be acquired by acquiring learning data in two stages.

  In addition, the intention discriminating information acquisition apparatus of the second invention stores an intention identifier corresponding to the second learning feature quantity group in the first intention discriminating information storage unit with respect to the first invention. A first intention identifier acquisition unit that acquires from two or more intention determination information; and a first intention identifier output unit that outputs the intention identifier acquired by the first intention identifier acquisition unit to a robot, The robot is an intention determination information acquisition device that receives the intention identifier and performs an operation according to the received intention identifier.

  With this configuration, learning data for improving the accuracy of intention detection can be acquired by acquiring learning data in two stages.

  In addition, the intention determination information acquisition device according to the third aspect of the present invention is the first or second aspect of the invention, wherein the first learning data acquisition unit stores the first learning data as two or more types of learning data. The first learning feature amount group acquisition unit acquires a first learning feature amount group from the first learning data for each type of two or more types of first learning data, and the first intention The discrimination information storage unit associates the first learning feature quantity group for each type of learning data with the intention identifier, and stores the first intention discrimination information for each type of learning data in the first intention discrimination information storage unit. The second learning data acquisition unit acquires second learning data that is two or more types of learning data, and the second learning feature quantity group acquisition unit includes two or more types of second learning data. For each type, a second learning feature value group is acquired from the second learning data. The second intention determination information storage unit associates the second learning feature amount group for each type of learning data with the intention identifier, and uses the second intention determination information for each type of learning data as the second intention determination information. It is an intention determination information acquisition device stored in an information storage unit.

  With this configuration, it is possible to acquire learning data for further improving the accuracy of intention detection by using two or more types of learning data.

  The intention determination information acquisition device according to the fourth aspect of the present invention relates to the intention determination information acquisition device and a brain for acquiring brain activity data, which is brain activity data, from the first to third aspects of the invention. An activity data acquisition unit; a feature quantity group acquisition unit that acquires an input feature quantity group that is one or more feature quantities from the brain activity data; and an intention identifier that corresponds to the input feature quantity group. A second intention identifier acquisition unit that acquires two or more second intention determination information stored in the information storage unit; and a second intention identifier output unit that outputs the intention identifier acquired by the second intention identifier acquisition unit. A brain information output device.

  With this configuration, it is possible to improve intention detection accuracy by using learning data acquired in two stages.

  In addition, the intention determination information acquisition device according to the fifth aspect of the present invention is the second aspect, wherein the second intention identifier acquisition unit is configured to store two or more second intention identifiers stored in the second intention determination information storage unit. The brain information output device acquires an intention identifier corresponding to the input feature amount group using intention determination information and two or more first intention determination information stored in the first intention determination information storage unit.

  With this configuration, it is possible to improve the accuracy of intention detection by using both learning data acquired in both the first stage and the second stage.

  Further, in the sixth aspect of the invention, in the intention determination information acquisition device, the brain activity data acquisition unit acquires two or more types of brain activity data, and the feature amount The group acquisition unit acquires an input feature amount group that is one or more feature amounts for each type from the two or more types of brain activity data, and the second intention identifier acquisition unit acquires each type of the brain activity data. A probability calculating means for calculating, for each intention identifier, a probability that the indicated one intention is an intention corresponding to one intention identifier among a plurality of intention identifiers, and a type calculated by the probability calculating means. And an intention determination means for acquiring an intention identifier using a probability for each and each intention identifier.

  With this configuration, it is possible to further improve the accuracy of intention detection by using two or more types of learning data.

  In addition, the intention determination information acquisition apparatus according to the seventh aspect of the present invention is the intention determination means according to the sixth aspect, wherein the intention determination means uses a different weight for the probability for each type of the brain activity data. This is a brain information output device that calculates a final probability for each identifier and obtains an intention identifier having the highest final probability for each intention identifier.

  With this configuration, it is possible to further improve the accuracy of intention detection by using two or more types of learning data.

  Further, the intention discriminating information acquiring apparatus according to the eighth aspect of the present invention is the sixth aspect of the present invention, wherein the probability calculating means only applies to a predetermined intention identifier for each type of the brain activity data. Thus, the brain information output device calculates the probability for each intention identifier.

  With this configuration, it is possible to further improve the accuracy of intention detection by using two or more types of learning data.

  According to the intention determination information acquisition apparatus according to the present invention, it is possible to improve the accuracy of intention detection.

Conceptual diagram of robot system 1 in the first embodiment Block diagram of the robot system 1 A flowchart for explaining the operation of the consent diagram determination information acquisition apparatus 111 A flowchart for explaining the operation of the brain information output apparatus 11 Flowchart explaining consent diagram identifier acquisition processing Flow chart for explaining the operation of the robot 12 Schematic of operation flow of consent diagram discrimination information acquisition device 111 Schematic of the operation of the consent diagram discrimination information acquisition device 111 The figure which shows the example of the same brain activity data (NIRS data) The figure which shows the example of the 1st learning feature-value group The figure which shows the example of the 1st intention determination information The figure which shows the example of the said 2nd intention determination information Block diagram of robot system 2 in the second embodiment Flowchart explaining an example of consent diagram identifier acquisition processing The figure which shows the measuring instrument which the same user puts on the head The figure which shows the example of the brain activity data (electroencephalogram data) The figure which shows the example of the 1st intention determination information The figure which shows the example of the said 2nd intention determination information Overview of the computer system Block diagram of the computer system Diagram for explaining the probability calculation algorithm Diagram for explaining the probability calculation algorithm

  Hereinafter, embodiments of an intention determination information acquisition apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

  (Embodiment 1)

  In the present embodiment, a learning data acquisition method for improving the accuracy of intention detection of the brain / machine interface will be described. In addition, an intention determination information acquisition device and a brain information output device that realize such a learning method will be described. In the present embodiment, a robot that receives and operates an output from the brain information output apparatus will be described.

  In the present embodiment, the following learning method is adopted. That is, an experiment for acquiring learning data performed before operating a machine (same meaning as a robot) by a brain / machine interface is divided into the following two sessions. The first stage is a training session and the second stage is a brush-up session. In the training session, data is acquired without using the machine to be operated, and parameters of the intention determination algorithm (same meaning as a later-described learning feature value group) are temporarily set using the data. After that, in the brush-up session, in the environment where the machine is operated, the data is acquired using the tentatively set parameters, and then the parameters of the intention determination algorithm including this data are fully set, so that the accuracy of the intention detection is To improve.

  FIG. 1 is a conceptual diagram of a robot system 1 in the present embodiment. The robot system 1 includes a brain information output device 11 and a robot 12. The brain information output device 11 acquires brain information by the brain activity measuring device, detects the user's intention, and sends an intention identifier for identifying the intention to the robot 12. The robot 12 receives an intention identifier from the brain information output device 11 and performs an operation according to the intention identifier. The robot 12 may be an electronic device having a so-called human shape, but may be an electronic device having another shape. The robot 12 may be a moving body such as a four-wheeled vehicle, a two-wheeled vehicle, an aircraft, or a train.

  FIG. 2 is a block diagram of the robot system 1 in the present embodiment.

  The brain information output device 11 includes an intention determination information acquisition device 111, a brain activity data acquisition unit 112, a feature amount group acquisition unit 113, a second intention identifier acquisition unit 114, and a second intention identifier output unit 115.

  The intention determination information acquisition device 111 includes a first intention determination information storage unit 1110, a first learning data acquisition unit 1111, a first learning feature value group acquisition unit 1112, a first intention determination information storage unit 1113, and a second intention determination information storage. Unit 1114, second learning data acquisition unit 1115, second learning feature value group acquisition unit 1116, second intention determination information storage unit 1117, first intention identifier acquisition unit 1118, and first intention identifier output unit 1119.

  The intention determination information acquisition device 111 is a device that acquires learning data for determining an intention.

  The robot 12 includes an operation information storage unit 121, an intention identifier reception unit 122, and an execution unit 123.

  The intention determination information acquisition device 111 is a device that acquires learning data used by the brain information output device 11 for determination of intention.

  The first intention determination information storage unit 1110 can store two or more first intention determination information. The first intention determination information includes an intention identifier and a first learning feature amount group. The intention identifier is information for identifying the user's intention. Intention is the act of moving a part of the body (for example, right hand, left hand, tongue, foot, etc.) or a specific thing (for example, moving the right hand, sticking out the tongue, running, etc.) It is reflected in the brain activity that the user thinks. In addition, the intention identifier is, for example, a part of the body that is moved by the user, and includes, for example, “right hand”, “left hand”, “tongue”, and “foot”. In addition, the intention identifier may be, for example, “right hand” and “left hand”. Although the number of intention identifiers is not limited, it is generally considered that the smaller the number of intention identifiers, the higher the possibility that the intention identifier output by the brain information output device 11 matches the user's intention. The first learning feature amount group is information of one or more feature amounts extracted from the first learning data. The first learning data is data acquired from the user's brain when the user performs a series of actions based on the intention identified by the intention identifier. The first learning data is data indicating brain activity acquired from the brain, such as data acquired by NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph). Anything is acceptable. When the first learning data is acquired, the robot 12 is in an environment where it does not operate.

  The first intention determination information storage unit 1110 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process of storing the first intention determination information in the first intention determination information storage unit 1110 does not matter. For example, the first intention determination information may be stored in the first intention determination information storage unit 1110 via the recording medium, and the first intention determination information transmitted via the communication line or the like is the first intention determination information. The determination information storage unit 1110 may store the first intention determination information input via the input device, or the first intention determination information storage unit 1110 may store the first intention determination information. .

  The first learning data acquisition unit 1111 is the first learning data that is learning data from the user's brain when the user performs a series of actions with one intention in an environment where the robot 12 to be operated does not operate. To get. Learning data is a series of brain activity data observed when a series of actions are performed. The first learning data acquisition unit 1111, the second learning data acquisition unit 1115, and the brain activity data acquisition unit 112 normally acquire the same type of data. The first learning data acquisition unit 1111 is, for example, a near infrared light measurement device. Moreover, the 1st learning data acquisition part 1111 is an electroencephalogram measurement apparatus, for example. The first learning data acquisition unit 1111 may be, for example, functional MRI (fMRI: functional magnetic resonance imaging) or PET (Positron Emission Tomography).

  The first learning feature value group acquisition unit 1112 performs signal processing on the first learning data acquired by the first learning data acquisition unit 1111 and acquires a first learning feature value group that is one or more feature values. Here, signal processing includes, for example, bias correction, baseline correction, trend removal, average value calculation, high-pass filter processing, low-pass filter processing, FFT processing, PCA processing, ICA processing, and phase locking value calculation Calculation of moving average, sensor selection, time selection, reference correction, downsampling, upsampling, power calculation, envelope calculation (Hilbert transform), processing by spatial filter, processing by inverse problem filter, and the like. The feature amount is, for example, data calculated by the above signal processing, such as an average value, a variance, a phase locking value, and the like. Note that the feature quantity constituting the first learning feature quantity group may be anything as long as it is data indicating the characteristics of the first learning data. Also, the number of signal processes performed by the first learning feature quantity group acquiring unit 1112 may be plural. Moreover, since the above-mentioned various signal processing examples are known techniques, detailed description thereof will be omitted.

  The first learning feature quantity group acquisition unit 1112 can be realized typically as an MPU, a memory, or the like. The processing procedure of the first learning feature value group acquisition unit 1112 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first intention determination information accumulating unit 1113 includes a first learning feature amount group acquired by the first learning feature amount group acquiring unit 1112 and an intention identifier for identifying one intention when the first learning data is acquired. The associated first intention determination information is accumulated in the first intention determination information storage unit 1110. Here, the input of the intention identifier may be performed by the user or may be performed by a third party, or an instruction output by the intention determination information acquisition device 111 to the user (for example, “hold the right hand or open the right hand”). The first intention determination information storage unit 1113 may acquire an intention identifier corresponding to “. The first intention determination information accumulating unit 1113 can be usually realized by an MPU, a memory, or the like. The processing procedure of the first intention determination information storage unit 1113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention determination information storage unit 1114 can store two or more second intention determination information. The second intention determination information includes an intention identifier and a second learning feature amount group. The second learning feature amount group is information of one or more feature amounts extracted from the second learning data. The second learning data is data acquired from the user's brain when the user performs a series of actions based on the intention identified by the intention identifier. The second learning data refers to brain activity acquired from the brain, such as data acquired by NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph). Any data is acceptable. When the second learning data is acquired, it is an environment in which the robot 12 operates.

  The second intention determination information storage unit 1114 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process of storing the second intention determination information in the second intention determination information storage unit 1114 is not limited. For example, the second intention determination information may be stored in the second intention determination information storage unit 1114 via a recording medium, and the second intention determination information transmitted via a communication line or the like is stored in the second intention determination information. The determination information storage unit 1114 may store the second intention determination information input via the input device, or the second intention determination information storage unit 1114 may store the second intention determination information. .

  The second learning data acquisition unit 1115 is a robot 12 that performs an operation according to the received intention identifier. When the user performs a series of actions based on one intention in an environment in which the operation target robot 12 operates. In addition, second learning data that is learning data is acquired from the brain of the user. The second learning data is a series of brain data observed when a series of actions are performed. The second learning data acquisition unit 1115 is, for example, an electroencephalogram measurement device. The second learning data acquisition unit 1115 is, for example, a near infrared light measurement device. The second learning data acquisition unit 1115 may be, for example, functional MRI (fMRI: functional magnetic resonance imaging) or PET (Positron Emission Tomography).

  The second learning feature value group acquisition unit 1116 performs signal processing on the second learning data acquired by the second learning data acquisition unit 1115 and acquires a second learning feature value group that is one or more feature values. Here, signal processing includes, for example, bias correction, baseline correction, trend removal, average value calculation, high-pass filter processing, low-pass filter processing, FFT processing, PCA processing, ICA processing, and phase locking value calculation Calculation of moving average, sensor selection, time selection, reference correction, downsampling, upsampling, power calculation, envelope calculation (Hilbert transform), processing by spatial filter, processing by inverse problem filter, and the like. The feature amount is, for example, data calculated by the above signal processing, such as an average value, a variance, a phase locking value, and the like. Note that the feature quantity constituting the second learning feature quantity group may be anything as long as it is data indicating the characteristics of the second learning data. Further, the number of signal processes performed by the second learning feature quantity group acquisition unit 1116 may be plural.

  The second learning feature quantity group acquisition unit 1116 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second learning feature value group acquisition unit 1116 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention determination information storage unit 1117 stores the second intention determination information in the second intention determination information storage unit 1114. The second intention determination information associates the second learning feature quantity group acquired by the second learning feature quantity group acquisition unit 1116 with an intention identifier for identifying one intention when the second learning data is acquired. Information. The second intention determination information accumulating unit 1117 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second intention determination information storage unit 1117 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first intention identifier acquisition unit 1118 acquires an intention identifier corresponding to the second learning feature quantity group from two or more first intention determination information stored in the first intention determination information storage unit 1110. The intention identifier corresponding to the second learning feature amount group is also referred to as the intention identifier corresponding to the second learning data acquired by the second learning data acquisition unit 1115. The first intention identifier acquisition unit 1118 calculates, for each intention identifier, the probability that the second learning feature quantity group is each intention identifier among a plurality of intention identifiers using, for example, two or more intention determination information. . Then, the first intention identifier acquisition unit 1118 acquires the intention identifier corresponding to the highest probability. The first intention identifier acquisition unit 1118 calculates, for example, the distance between the second learning feature quantity group and the first learning feature quantity group included in each of the two or more first intention determination information, and corresponds to the smallest distance. An intention identifier may be acquired. The algorithm for the first intention identifier acquisition unit 1118 to acquire the intention identifier does not matter.

More specifically, when there are two (two types) intention identifiers, for example, the first intention identifier acquisition unit 1118 calculates the probability for each intention identifier as follows. That is, the first intention identifier acquisition unit 1118 has a boundary surface for dividing the first learning feature quantity group into two classes (class 1 [first intention identifier] or class 2 [second intention identifier]). Get the information. This concept is shown in FIG.
Next, the first intention identifier acquisition unit 1118 calculates the distance of the second learning feature amount group from the boundary surface (see FIG. 21). Then, the first intention identifier acquisition unit 1118 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first intention identifier acquisition unit 1118 uses the converted value (0 to 1) as the probability of the first intention identifier. Then, the first intention identifier acquisition unit 1118 calculates the probability of the second intention identifier by “1-probability of the first intention identifier”.

  When there are four intention identifiers (four types), for example, the first intention identifier acquisition unit 1118 calculates the probability for each intention identifier as follows. That is, the process for the above two intention identifiers is expanded. First, it is considered that there are two classes of a first intention identifier and another intention identifier, and the first intention identifier acquisition unit 1118 selects two classes (class 1 [first intention] from the first learning feature quantity group. ID], or class 2 (second intention identifier, third intention identifier, or fourth intention identifier) is obtained. Next, the first intention identifier acquisition unit 1118 calculates the distance of the second learning feature amount group from the boundary surface. Then, the first intention identifier acquisition unit 1118 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first intention identifier acquisition unit 1118 uses the converted value (0 to 1) as the probability of the first intention identifier. Next, the first intention identifier acquisition unit 1118 sets the distance from the first learning feature quantity group (vector) corresponding to the second intention identifier, the third intention identifier, or the fourth intention identifier as “1-first. It is calculated by “the probability of one intention identifier”.

  Next, considering that there are two classes of the second intention identifier and another intention identifier, the first intention identifier acquisition unit 1118 selects two classes (class 1 [first Second intention identifier] or class 2 (first intention identifier or third intention identifier or fourth intention identifier) is obtained. Next, the first intention identifier acquisition unit 1118 calculates the distance of the second learning feature amount group from the boundary surface. Then, the first intention identifier acquisition unit 1118 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first intention identifier acquisition unit 1118 uses the converted value (0 to 1) as the probability of the second intention identifier. Next, the first intention identifier acquisition unit 1118 determines the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the third intention identifier, or the fourth intention identifier as “1-first. Calculated by “probability of second intention identifier”.

  Next, considering that there are two classes of a third intention identifier and another intention identifier, the first intention identifier acquisition unit 1118 selects two classes (class 1 [first Interface information to be divided into class 3 [first intention identifier or second intention identifier or fourth intention identifier]. Next, the first intention identifier acquisition unit 1118 calculates the distance of the second learning feature amount group from the boundary surface. Then, the first intention identifier acquisition unit 1118 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first intention identifier acquisition unit 1118 sets the converted value (0 to 1) as the probability of the third intention identifier. Next, the first intention identifier acquisition unit 1118 determines the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the fourth intention identifier as “1-first. The probability of the third intention identifier ”is calculated.

  Then, considering that there are two classes of a fourth intention identifier and another intention identifier, the first intention identifier acquisition unit 1118 selects two classes (class 1 [first 4 intention identifiers], or class 2 [first intention identifier or second intention identifier or third intention identifier) is obtained. Next, the first intention identifier acquisition unit 1118 calculates the distance of the second learning feature amount group from the boundary surface. Then, the first intention identifier acquisition unit 1118 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first intention identifier acquisition unit 1118 sets the converted value (0 to 1) as the probability of the fourth intention identifier. Next, the first intention identifier acquisition unit 1118 determines the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the third intention identifier as “1-first. It is calculated by “the probability of the fourth intention identifier”.

  And the 1st intention identifier acquisition part 1118 acquires the probability of a 4th intention identifier from a 1st intention identifier by merging the probability acquired above. Here, the merge means the probability of the first intention identifier (P1), the probability of the intention identifier other than the first (P234), the probability of the second intention identifier (P2), the first Probability of intention identifier other than two (P134), Probability of third intention identifier (P3), Probability of intention identifier other than third (P124), Probability of fourth intention identifier (P4) This refers to an operation to be obtained from the eight probabilities of the probability of intention identifiers other than the fourth (P123). The probability of the merged first intention identifier is P1 × P134 × P124 × P123, the probability of the merged second intention identifier is P234 × P2 × P124 × P123, and the probability of the merged third intention identifier is P234 × P134 ×. It is preferable to obtain P3 × P123 and the merged fourth intention identifier as P234 × P134 × P124 × P4. An operation for normalizing the probability may be added. Here, normalization means that the probability of adding all becomes 1 by dividing each probability by the value obtained by adding the probability of the merged first intention identifier to the probability of the fourth intention identifier. It is an operation to do.

  The first intention identifier acquisition unit 1118 uses two or more first intention determination information stored in the first intention determination information storage unit 1110 as teacher data, and uses machine learning algorithms such as SVM, SVR, and decision tree. The intention identifier corresponding to the second learning feature amount group may be acquired.

  The first intention identifier acquisition unit 1118 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first intention identifier acquisition unit 1118 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first intention identifier output unit 1119 outputs the intention identifier acquired by the first intention identifier acquisition unit 1118 to the robot 12. By the output to the robot 12, the robot 12 performs an operation according to the intention identifier. The first intention identifier output unit 1119 can be realized by, for example, a wireless or wired communication unit.

  Note that the user or the like may input a correct intention identifier into the intention determination information acquisition apparatus 111 by looking at the operation of the robot 12 according to the intention identifier output by the first intention identifier output unit 1119. In such a case, the second intention determination information storage unit 1117 configures second intention determination information in which the input intention identifier and the second learning feature amount group acquired by the second learning feature amount group acquisition unit 1116 are paired. The second intention determination information is stored in the second intention determination information storage unit 1114.

  The brain activity data acquisition unit 112 acquires brain activity data that is brain activity data from the user. The brain activity data is brain activity data acquired from the user when the user indicates one intention. The brain activity data includes data acquired by NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph), and the like. The brain activity data may be anything as long as it indicates brain activity. The brain activity data is the same type of data as the first learning data and the second learning data. The brain activity data acquisition unit 112 is, for example, a near infrared light measurement device. The brain activity data acquisition unit 112 is, for example, an electroencephalogram measurement device. Further, the brain activity data acquisition unit 112 may be, for example, functional MRI (fMRI: functional magnetic resonance imaging) or PET (Positron Emission Tomography).

  The feature amount group acquisition unit 113 acquires an input feature amount group that is one or more feature amounts from the brain activity data acquired by the brain activity data acquisition unit 112. The feature amount group acquisition unit 113 performs signal processing on the brain activity data and acquires an input feature amount group. Here, signal processing includes, for example, bias correction, baseline correction, trend removal, average value calculation, high-pass filter processing, low-pass filter processing, FFT processing, PCA processing, ICA processing, and phase locking value calculation Calculation of moving average, sensor selection, time selection, reference correction, downsampling, upsampling, power calculation, envelope calculation (Hilbert transform), processing by spatial filter, processing by inverse problem filter, and the like. The feature amount is, for example, data calculated by the above signal processing, such as an average value, a variance, a phase locking value, and the like. Note that the feature amount constituting the input feature amount group may be anything as long as it is data indicating the characteristics of the brain activity data. Further, the number of signal processes performed by the feature quantity group acquisition unit 113 may be plural.

  The feature quantity group acquisition unit 113 can be usually realized by an MPU, a memory, or the like. The processing procedure of the feature quantity group acquisition unit 113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention identifier acquisition unit 114 acquires an intention identifier corresponding to the input feature quantity group from two or more second intention determination information stored in the second intention determination information storage unit 1114. The second intention identifier acquisition unit 114 also includes two or more second intention determination information stored in the second intention determination information storage unit 1114 and two or more stored in the first intention determination information storage unit 1110. An intention identifier corresponding to the input feature amount group may be acquired using the first intention determination information. When the second intention identifier acquisition unit 114 acquires an intention identifier using both the second intention determination information and the first intention determination information, the second intention identifier acquisition unit 114 uses the weight of the first intention determination information, It is preferable to acquire the intention identifier by increasing the weight of the second intention determination information. The process of the second intention identifier acquisition unit 114 may be the same as the process of the first intention identifier acquisition unit 1118.

  That is, the second intention identifier acquisition unit 114 uses, for example, two or more second intention determination information, or two or more second intention determination information and two or more first intention determination information, to input an input feature quantity group. The probability corresponding to each intention identifier among the plurality of intention identifiers is calculated for each intention identifier. Then, the second intention identifier acquisition unit 114 acquires the intention identifier corresponding to the highest probability.

More specifically, when there are two (two types) intention identifiers, for example, the second intention identifier acquisition unit 114 calculates the probability for each intention identifier as follows. In other words, the second intention identifier acquisition unit 114 divides the second learning feature quantity group into two classes (class 1 [first intention identifier] or class 2 [second intention identifier]). Get the information. This concept is shown in FIG.
Next, the second intention identifier acquisition unit 114 calculates the distance of the input feature quantity group from the boundary surface (see FIG. 20). And the 2nd intention identifier acquisition part 114 inputs the said distance into a logistic function (sigmoid function), and converts it into the value (0 or more and 1 or less value) of 0-1. Then, the second intention identifier acquisition unit 114 sets the converted value (0 to 1) as the probability of the first intention identifier. Then, the second intention identifier acquisition unit 114 calculates the probability of the second intention identifier by “1-probability of the first intention identifier”.

  When there are four intention identifiers (four types), for example, the second intention identifier acquisition unit 114 calculates the probability for each intention identifier as follows. That is, the process for the above two intention identifiers is expanded. First, it is considered that there are two classes of a first intention identifier and another intention identifier, and the second intention identifier acquisition unit 114 determines two classes (class 1 [first intention] from the second learning feature quantity group. ID], or class 2 (second intention identifier, third intention identifier, or fourth intention identifier) is obtained. Next, the second intention identifier acquisition unit 114 calculates the distance of the input feature amount group from the boundary surface. And the 2nd intention identifier acquisition part 114 inputs the said distance into a logistic function (sigmoid function), and converts it into the value (0 or more and 1 or less value) of 0-1. Then, the second intention identifier acquisition unit 114 sets the converted value (0 to 1) as the probability of the first intention identifier. Next, the second intention identifier acquisition unit 114 calculates the distance from the second learning feature quantity group (vector) corresponding to the second intention identifier, the third intention identifier, or the fourth intention identifier as “1-first. It is calculated by “the probability of one intention identifier”.

  Next, considering that there are two classes of the second intention identifier and the other intention identifier, the second intention identifier acquisition unit 114 selects two classes (class 1 [first Second intention identifier] or class 2 (first intention identifier or third intention identifier or fourth intention identifier) is obtained. Next, the second intention identifier acquisition unit 11 calculates the distance of the input feature amount group from the boundary surface. And the 2nd intention identifier acquisition part 114 inputs the said distance into a logistic function (sigmoid function), and converts it into the value (0 or more and 1 or less value) of 0-1. Then, the second intention identifier acquisition unit 114 sets the converted value (0 to 1) as the probability of the second intention identifier. Next, the first intention identifier acquisition unit 1118 determines the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the third intention identifier, or the fourth intention identifier as “1-first. Calculated by “probability of second intention identifier”.

  Next, considering that there are two classes of a third intention identifier and another intention identifier, the second intention identifier acquisition unit 114 selects two classes (class 1 [first Interface information to be divided into class 3 [first intention identifier or second intention identifier or fourth intention identifier]. Next, the second intention identifier acquisition unit 114 calculates the distance of the input feature amount group from the boundary surface. And the 2nd intention identifier acquisition part 114 inputs the said distance into a logistic function (sigmoid function), and converts it into the value (0 or more and 1 or less value) of 0-1. Then, the second intention identifier acquisition unit 114 sets the converted value (0 to 1) as the probability of the third intention identifier. Next, the second intention identifier acquisition unit 114 sets the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the fourth intention identifier as “1-first. The probability of the third intention identifier ”is calculated.

  Next, considering that there are two classes of a fourth intention identifier and another intention identifier, the second intention identifier acquisition unit 114 determines two classes (class 1 [first 4 intention identifiers], or class 2 [first intention identifier or second intention identifier or third intention identifier) is obtained. Next, the second intention identifier acquisition unit 114 calculates the distance of the input feature amount group from the boundary surface. And the 2nd intention identifier acquisition part 114 inputs the said distance into a logistic function (sigmoid function), and converts it into the value (0 or more and 1 or less value) of 0-1. Then, the second intention identifier acquisition unit 114 sets the converted value (0 to 1) as the probability of the fourth intention identifier. Next, the second intention identifier acquisition unit 114 calculates the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the third intention identifier as “1-first. It is calculated by “the probability of the fourth intention identifier”.

  And the 2nd intention identifier acquisition part 114 acquires the probability of a 4th intention identifier from a 1st intention identifier by merging the probability acquired above. Here, the merge means the probability of the first intention identifier (P1), the probability of the intention identifier other than the first (P234), the probability of the second intention identifier (P2), the first Probability of intention identifier other than two (P134), Probability of third intention identifier (P3), Probability of intention identifier other than third (P124), Probability of fourth intention identifier (P4) This refers to an operation to be obtained from the eight probabilities of the probability of intention identifiers other than the fourth (P123). The probability of the merged first intention identifier is P1 × P134 × P124 × P123, the probability of the merged second intention identifier is P234 × P2 × P124 × P123, and the probability of the merged third intention identifier is P234 × P134 ×. It is preferable to obtain P3 × P123 and the merged fourth intention identifier as P234 × P134 × P124 × P4. An operation for normalizing the probability may be added. Here, normalization means that the probability of adding all becomes 1 by dividing each probability by the value obtained by adding the probability of the merged first intention identifier to the probability of the fourth intention identifier. It is an operation to do.

  In the above description, when calculating the distance between the input feature quantity group, the second learning feature quantity group, and the first learning feature quantity group, the second intention identifier acquisition unit 114 performs the second learning feature quantity group and the first learning feature quantity group. An average value or a weighted average value of the learning feature amount group is calculated for each feature amount, and a vector (learning feature amount group) is configured. Then, the second intention identifier acquisition unit 114 calculates the distance between the input feature quantity group and the learned feature quantity group. In addition, when weighting, it is preferable that the weight of the second learning feature amount group is increased.

  The second intention identifier acquisition unit 114 uses the second intention determination information, or the first intention determination information and the second intention determination information as teacher data, by using a machine learning algorithm such as SVM, SVR, or decision tree. You may acquire the intention identifier corresponding to a learning feature-value group. The algorithm for acquiring the intention identifier in the second intention identifier acquisition unit 114 does not matter.

  The second intention identifier acquisition unit 114 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second intention identifier acquisition unit 114 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention identifier output unit 115 outputs the intention identifier acquired by the second intention identifier acquisition unit 114. The output is preferably output to the robot 12, but may be display on a display, audio output, or the like. The second intention identifier output unit 115 can be realized by wireless or wired communication means.

  The robot 12 receives the intention identifier and performs an operation according to the received intention identifier.

  The operation information storage unit 121 can store two or more pieces of operation information. The operation information is a set of an intention identifier and an operation module that performs an operation corresponding to the intention identifier. The operation module may be realized by hardware, may be realized by software, or may be realized by hardware and software. The operation information storage unit 121 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the operation information is stored in the operation information storage unit 121 is not limited. For example, the operation information may be stored in the operation information storage unit 121 via a recording medium, and the operation information transmitted via a communication line or the like is stored in the operation information storage unit 121. Alternatively, the operation information input via the input device may be stored in the operation information storage unit 121.

  The intention identifier receiving unit 122 receives the intention identifier output from the brain information output device 11. Acceptance includes reception using communication means, delivery of information (intention identifier) by software, and the like.

  The execution unit 123 executes an operation module corresponding to the intention identifier received by the intention identifier reception unit 122. Any operation content can be realized by the operation module. The execution unit 123 can usually be realized by an MPU, a memory, or the like. The processing procedure of the execution unit 123 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the robot system 1 will be described. First, the operation of the intention determination information acquisition apparatus 111 will be described using the flowchart of FIG.

  (Step S301) The first learning data acquisition unit 1111 determines whether to acquire first learning data. If the first learning data is acquired, the process goes to step S302, and if the first learning data is not acquired, the process goes to step S307. For example, in order to acquire the first learning data, an output unit (not shown) outputs an instruction indicating that the user performs an operation corresponding to the intention (including thinking and imagination). After the instruction is output, the first learning data acquisition unit 1111 determines that the first learning data is acquired for a predetermined period (for example, 10 seconds). In addition, the instruction indicating that an action corresponding to the intention is performed is information such as “Please shake your right hand for 10 seconds” or “Please flutter your foot for 15 seconds”.

  (Step S302) The first learning data acquisition unit 1111 acquires the first learning data and accumulates at least temporarily in the recording medium.

  (Step S303) The first learning feature value group acquisition unit 1112 acquires a first learning feature value group from the first learning data acquired in step S302.

  (Step S304) The first intention determination information storage unit 1113 acquires an intention identifier. For example, it is assumed that the intention determination information acquisition apparatus 111 stores an intention identifier paired with an instruction (information indicating an instruction) output by an output unit (not shown). Then, the first intention determination information storage unit 1113 acquires an intention identifier corresponding to the output instruction, for example.

  (Step S305) The first intention determination information storage unit 1113 configures first intention determination information having the first learning feature amount group acquired in Step S303 and the intention identifier acquired in Step S304.

  (Step S306) The first intention determination information storage unit 1113 stores the first intention determination information configured in step S305 in the first intention determination information storage unit 1110. The process returns to step S301.

  (Step S307) The first learning data acquisition unit 1111 determines whether or not to end the process of acquiring the first learning data. If it ends, the process goes to step S308, and if it does not end, the process returns to step S301. Note that the first learning data acquisition unit 1111 determines to end the acquisition process of the first learning data when acquiring the first learning data corresponding to all the intention identifiers, for example.

  (Step S308) The second learning data acquisition unit 1115 determines whether or not to acquire second learning data. If the second learning data is acquired, the process goes to step S309, and if the second learning data is not acquired, the process goes to step S317. In addition, for example, in order to acquire the second learning data, an output unit (not shown) outputs an instruction to the user to perform an operation corresponding to the intention (including thinking, imagination, etc.), and then a predetermined period ( For example, the second learning data acquisition unit 1115 determines that the first learning data is acquired. Also, the instructions are information such as “Please hold and open your left hand for 10 seconds at a frequency of 1 Hz.” “Continue to move your tongue up and down or left and right in your mouth for 15 seconds.” is there.

  (Step S309) The second learning data acquisition unit 1115 acquires the second learning data and accumulates at least temporarily in the recording medium.

  (Step S310) The second learning feature value group acquisition unit 1116 acquires a second learning feature value group from the second learning data acquired in step S309.

  (Step S311) The first intention identifier acquisition unit 1118 receives the second learning feature quantity group acquired in step S310 as an input, and acquires an intention identifier corresponding to the second learning feature quantity group. Such intention identifier acquisition processing will be described with reference to the flowchart of FIG. The learning feature amount group here is a first learning feature amount group.

  (Step S312) The first intention identifier output unit 1119 outputs the intention identifier acquired in step S311 to the robot 12. The robot 12 receives the intention identifier and performs an operation corresponding to the intention identifier (see FIG. 6).

  (Step S313) The second intention determination information storage unit 1117 determines whether information related to the correctness of the intention identifier input from the user or the like has been received. If information about correctness is accepted, the process goes to step S314, and if information about correctness is not accepted, the process returns to step S313. The correct / incorrect information is information regarding whether or not the intention indicated by the intention identifier output in step S312 matches the intention indicated by the user. Further, when the intention indicated by the intention identifier output in step S312 does not match the intention indicated by the user, the information regarding correctness is, for example, an intention identifier indicating the correct intention.

  (Step S314) The second intention determination information accumulating unit 1117 determines whether or not the information about correctness received in Step S313 is information indicating “correct”. If it is information indicating “correct”, the process goes to step S316, and if it is not information indicating “correct”, the process goes to step S315.

  (Step S315) The second intention determination information storage unit 1117 acquires a correct intention identifier. The correct intention identifier is an intention identifier indicating the correct intention accepted in step S313.

  (Step S316) The second intention determination information storage unit 1117 configures second intention determination information from the intention identifier and the second learning feature amount group acquired in Step S310, and the second intention determination information is converted into the second intention determination information. Accumulate in the discrimination information storage unit 1114. The process returns to step S308.

  (Step S317) The second learning data acquisition unit 1115 determines whether or not to end the process of acquiring the second learning data. If the process ends, the process ends. If not, the process returns to step S308. For example, when the second intention determination information corresponding to all intention identifiers is accumulated in the second intention determination information storage unit 1114, the second learning data acquisition unit 1115 ends the second learning data acquisition process. Judge that.

  In the flowchart of FIG. 3, steps S301 to S307 are the training sessions described above. In the flowchart of FIG. 3, steps S308 to S317 are the brush-up session described above. In addition, the robot 12 does not operate in the training session, but the robot 12 operates in the brush-up session.

  In the flowchart of FIG. 3, in step S313, input of information regarding correctness as to whether or not the output intention identifier (operation of the robot 12) is correct is received from the user or the like, but there is no such reception process. Also good. When there is no reception process, the second intention determination information storage unit 1117 configures second intention determination information from the intention identifier acquired in step S311 and the second learning feature amount group acquired in step S310, and The second intention determination information is accumulated in the second intention determination information storage unit 1114.

  Next, the operation in which the brain information output device 11 outputs the intention identifier will be described with reference to the flowchart of FIG.

  (Step S401) The brain activity data acquisition unit 112 determines whether or not brain activity data has been acquired. If the brain activity data is acquired, the process goes to step S402. If the brain activity data is not acquired, the process returns to step S401.

  (Step S402) The feature quantity group acquisition unit 113 acquires an input feature quantity group from the brain activity data acquired in step S401.

  (Step S403) The second intention identifier acquisition unit 114 receives the input feature quantity group acquired in step S402 as an input, and acquires an intention identifier corresponding to the input feature quantity group. Such intention identifier acquisition processing will be described with reference to the flowchart of FIG. Note that the learning feature quantity group here is the second learning feature quantity group, or the second learning feature quantity group and the first learning feature quantity group.

  (Step S404) The second intention identifier output unit 115 outputs the intention identifier acquired in step S403 to the robot 12. The process returns to step S401.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Next, an example of the intention identifier acquisition process in steps S311 and S403 will be described with reference to the flowchart in FIG.

  (Step S501) The first intention identifier acquisition unit 1118 or the second intention identifier acquisition unit 114 (hereinafter referred to as the intention identifier acquisition unit) sets the counter i to 1.

  (Step S502) The intention identifier acquisition unit determines whether or not the i-th intention identifier exists. If the i-th intention identifier exists, the process goes to step S503, and if the i-th intention identifier does not exist, the process goes to step S508.

  (Step S503) The intention identifier acquisition unit acquires a learning feature amount group (vector) corresponding to the i-th intention identifier.

  (Step S504) The intention identifier acquisition unit acquires the input feature quantity group (vector).

  (Step S505) The intention identifier acquisition unit calculates the distance between the learning feature group (vector) acquired in step S503 and the feature group (vector) acquired in step S504.

  (Step S506) The intention identifier acquisition unit associates the distance acquired in step S505 with the i-th intention identifier and stores them in the recording medium at least temporarily.

  (Step S507) The intention identifier acquisition unit increments the counter i by 1. The process returns to step S502.

  (Step S508) The intention identifier acquisition unit acquires an intention identifier using the distance accumulated in step S506. For example, the intention identifier acquisition unit acquires an intention identifier paired with the smallest distance. Further, as described above, the intention identifier acquisition unit may calculate the probability for each intention identifier and acquire the intention identifier having the highest probability. Return to upper process.

  In the flowchart of FIG. 5, the operation in step S504 may be performed outside the loop and performed only once.

  Next, the operation of the robot 12 will be described with reference to the flowchart of FIG.

  (Step S601) The intention identifier receiving unit 122 determines whether the intention identifier output from the brain information output apparatus 11 has been received. If the intention identifier is accepted, the process goes to step S602. If the intention identifier is not accepted, the process returns to step S601.

  (Step S602) The execution unit 123 reads the operation module corresponding to the intention identifier received in step S601 from the operation information storage unit 121.

  (Step S603) The execution unit 123 executes the operation module read in step S602. The process returns to step S601.

  Through the above processing, the robot 12 performs an operation corresponding to an action (including thinking) intended by the user.

  In the flowchart of FIG. 6, the process ends when the power is turned off or the process ends.

  Hereinafter, the concept of the operation of the intention determination information acquisition apparatus 111 in the present embodiment will be described. A schematic diagram of the operation flow of the intention determination information acquisition apparatus 111 is shown in FIG.

  First, since there is no operation parameter (learning feature amount group) for the machine (robot 12), the machine cannot be operated.

  Therefore, in the training session, the user performs brain activity without machine operation. Then, the first learning data acquisition unit 1111 acquires data 1 (first learning data) (S701).

  Next, in a training session, the first learning feature value group acquisition unit 1112 calculates and acquires a machine operation parameter (parameter 1) from data 1 (first learning data) (S702). Parameter 1 is the first learning feature quantity group.

  Next, the user operates the machine (robot 12). That is, the second learning data acquisition unit 1115 acquires data 2 (second learning data) in the brush-up session while the user operates the machine (S703). In such a case, data indicating brain activity in a state where there is a machine operation is acquired.

  Next, in a brush-up session, the second learning feature quantity group acquisition unit 1116 acquires parameter 2 (second learning feature quantity group) from data 2 (second learning data) (S704). The parameter 2 is a parameter for operating the machine (robot 12).

  Then, the actual machine operation is performed using the parameter 2 (S705). Such step S705 is referred to as a test session.

  Further, the concept of the intention determination information acquisition apparatus 111 will be further described with reference to the conceptual diagram of FIG.

  In the training session of S801, the user performs an operation (including thinking and imagination) without operating the machine (robot 12), and learning data (first intention determination information) is acquired. In a training session, the machine is not operated, so brain activity may differ from the actual case.

  Next, in the brush-up session in S802, the user performs an operation (including thinking and imagination) with the operation of the machine (robot 12), and learning data (second intention determination information) is acquired. In the brush-up session, it is a session for learning data, but since the actual machine is operated, it is considered that the brain activity is similar to the actual case.

  Next, in the test session (actual operation) in S803, the user uses the learned data (second intention determination information, or the second intention determination information and the first intention determination information) to perform a predetermined operation. (Including thinking and imagination) and actually operate the machine.

  Hereinafter, a specific operation of the robot system 1 in the present embodiment will be described. A conceptual diagram of the robot system 1 is shown in FIG.

  The brain information output device 11 is an electronic device that functions as a so-called brain / machine interface (actual movement of the left hand / right hand). The intention identifier output from the brain information output device 11 is sent to the robot 12, and the robot 12 performs an operation corresponding to the intention identifier.

  In addition, the first learning data acquisition unit 1111, the second learning data acquisition unit 1115, and the brain activity data acquisition unit 112 are, for example, near-infrared light measurement devices. The first learning data, the second learning data, and the brain activity data are NIRS data. NIRS data is a set of hemoglobin information indicating the amount of hemoglobin in the head.

  In such a situation, the user performs a training session. In other words, for example, it is assumed that the user imagines an action of moving the “right hand”. In this case, the intention is “imagine that the right hand is moved”, and the intention identifier is “right hand”.

  Then, it is assumed that the first learning data acquisition unit 1111 of the intention determination information acquisition apparatus 111 has acquired brain activity data (NIRS data) shown in FIG. FIG. 9 shows the acquired amount of hemoglobin over time (t). Here, the hemoglobin information is acquired every 8 msec, for example.

  Next, the first learning feature quantity group acquisition unit 1112 acquires the first learning feature quantity group shown in FIG. 10 from the first brain activity data shown in FIG.

  The first intention determination information storage unit 1113 configures first intention determination information in which the intention identifier “right hand” and the first learning feature amount group illustrated in FIG. 10 are paired, and the first intention determination information storage unit 1110 stores the first intention determination information. accumulate. In FIG. 10, the first learning feature quantity group has two feature quantities, a first feature quantity and a second feature quantity. In this specific example, the feature amount group has two feature amounts for the sake of simplicity. However, it goes without saying that the feature quantity group may have three or more feature quantities.

  The above process is repeated for at least the intention identifiers “left hand”, “tongue”, and “foot”, and the first intention determination information storage unit 1110 stores at least four first intention determinations as shown in FIG. Information is stored. This completes the training session.

  The intention identifier “right hand” is information for identifying the intention obtained when the user “imagines that the right hand is moved”. The intention identifier “left hand” is information for identifying an intention obtained when the user “imagines that the left hand is moved”. The intention identifier “tongue” is information for identifying an intention obtained when the user “imagines that the tongue is moved”. The intention identifier “foot” is information for identifying the intention obtained when the user “imagines that the foot has moved”.

  Next, the user performs a brush-up session. That is, assume that the user imagines that the user has moved his / her foot in an attempt to perform an action corresponding to the intention identifier “foot” on the robot 12.

  Next, it is assumed that the second learning data acquisition unit 1115 acquires second learning data that is data of brain activity that occurs when the user imagines that the user has moved his / her foot.

  Next, the second learning feature value group acquisition unit 1116 acquires a second learning feature value group (here, two feature values of the first feature value and the second feature value) from the acquired second learning data. To do.

  Next, the first intention identifier acquisition unit 1118 acquires the intention identifier corresponding to the acquired second learning feature quantity group using the first learning feature quantity group of FIG. 11 as learning data. That is, the first intention identifier acquisition unit 1118 calculates the distance between the second learning feature quantity group and the first learning feature quantity group for each intention identifier in FIG. Then, it is assumed that the intention identifier (here “tongue”) having the smallest distance is acquired.

  Then, the first intention identifier output unit 1119 outputs the acquired intention identifier “tongue” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “tongue”. Then, the execution unit 123 reads the motion module corresponding to the accepted intention identifier “tongue” from the motion information storage unit 121. Next, the execution unit 123 executes the read operation module.

  Through the above operation, the user recognizes that the robot 12 has performed a movement different from his intention. Then, the user inputs the correct intention identifier “foot” to the intention determination information acquisition apparatus 111 using an input unit (not shown).

  Then, the second intention determination information storage unit 1117 of the intention determination information acquisition apparatus 111 receives the intention identifier “foot” input by the user.

  Next, the second intention determination information storage unit 1117 configures second intention determination information that is a pair of the intention identifier “foot” and the acquired second learning feature quantity group. It accumulates in the two intention determination information storage unit 1114.

  Next, it is assumed that the user imagines that the user has moved the right hand in an attempt to perform an action corresponding to the intention identifier “right hand” on the robot 12.

  Next, it is assumed that the second learning data acquisition unit 1115 acquires second learning data that is data of brain activity that occurs when the user imagines that the user moves his right hand.

  Next, the second learning feature value group acquisition unit 1116 acquires a second learning feature value group from the acquired second learning data.

  Next, the first intention identifier acquisition unit 1118 acquires the intention identifier corresponding to the acquired second learning feature quantity group using the first learning feature quantity group of FIG. 11 as learning data. That is, the first intention identifier acquisition unit 1118 calculates the distance between the second learning feature quantity group and the first learning feature quantity group for each intention identifier in FIG. Then, it is assumed that the intention identifier (here, “right hand”) having the smallest distance is acquired.

  Then, the first intention identifier output unit 1119 outputs the acquired intention identifier “right hand” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “right hand”. Then, the execution unit 123 reads the operation module corresponding to the accepted intention identifier “right hand” from the operation information storage unit 121. Next, the execution unit 123 executes the read operation module.

  Through the above operation, the user recognizes that the robot 12 has performed a movement according to his / her intention. Then, the user inputs information indicating correctness to the intention determination information acquisition apparatus 111 using an input unit (not shown).

  Next, the second intention determination information storage unit 1117 configures second intention determination information which is a pair of the intention identifier “right hand” and the acquired second learning feature quantity group, and the second intention determination information is stored in the second intention determination information. It accumulates in the two intention determination information storage unit 1114.

  The above processing is performed on at least the intention identifier “left hand” and the intention identifier “tongue”, and second intention determination information corresponding to all intention identifiers is accumulated in the second intention determination information storage unit 1114 ( FIG. 12).

  With the above processing, the brush-up session ends.

  Next, it is assumed that the user performs a test session for operating the actual robot 12. Here, it is assumed that the second intention determination information in FIG. 12 is used as the learning data.

  That is, for example, it is assumed that the user has an imagination corresponding to the intention identifier “left hand” “imagination that the left hand is moved”.

  Next, the brain activity data acquisition unit 112 acquires brain activity data in such a case.

  Then, the feature value group acquisition unit 113 acquires an input feature value group (here, two feature values of the first feature value and the second feature value) from the acquired brain activity data.

  Next, the second intention identifier acquisition unit 114 calculates the distance between the corresponding second feature quantity group in FIG. 12 and the input feature quantity group for each intention identifier.

  Next, the second intention identifier acquisition unit 114 acquires the intention identifier “left hand” corresponding to the closest distance. Then, the second intention identifier output unit 115 outputs the acquired intention identifier “left hand” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “left hand” output from the brain information output apparatus 11.

  Then, the execution unit 123 reads the operation module corresponding to the accepted intention identifier “left hand” from the operation information storage unit 121.

  Next, the execution unit 123 executes the read operation module. With the above processing, the robot 12 operates as intended by the user.

  As described above, according to the present embodiment, the brain information output apparatus 11 can obtain learning data through two sessions, a training session and a brush-up session. Therefore, the accuracy of intention detection in the test session can be improved.

  According to the present embodiment, the intention determination information acquisition device 111 may not be integrated with the brain information output device 11. In such a case, the brain information output device 11 includes a first intention determination information storage unit 1110, a second intention determination information storage unit 1114, a brain activity data acquisition unit 112, a feature amount group acquisition unit 113, a second intention identifier acquisition unit 114, A second intention identifier output unit 115 is provided.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that implements the intention determination information acquisition apparatus according to the present embodiment is the following program. In other words, when the user performs a series of actions based on a single intention in an environment where the robot to be operated does not operate, the program acquires first learning data as learning data from the user's brain. A first learning data acquisition unit to acquire; a first learning feature value group acquisition unit to acquire a first learning feature value group that is one or more feature values from the first learning data; and the first learning feature value group First intention determination information in which a first learning feature amount group acquired by the acquisition unit and an intention identifier for identifying one intention when the first learning data is acquired is stored in a storage medium. One intention determination information storage unit and a robot that performs an operation according to the received intention identifier, and when the user performs a series of actions under one intention in an environment in which the robot to be operated operates, From the user's brain A second learning data acquisition unit that acquires second learning data that is learning data, and a second learning feature amount group acquisition unit that acquires a second learning feature amount group that is one or more feature amounts from the second learning data And a second intention determination unit that associates the second learning feature amount group acquired by the second learning feature amount group acquisition unit with an intention identifier for identifying one intention when the second learning data is acquired. A program for causing information to function as a second intention determination information storage unit that stores information in a storage medium.

  In the above program, the computer acquires a first intention identifier for acquiring an intention identifier corresponding to the second learning feature quantity group from two or more intention determination information stored in the first intention determination information storage unit. And a program for further functioning as a first intention identifier output unit for outputting the intention identifier acquired by the first intention identifier acquisition unit to the robot.

  Moreover, the software which implement | achieves the brain information output device in this Embodiment is the following programs. That is, the program includes a computer, the above-described intention determination information acquisition device, a brain activity data acquisition unit that acquires brain activity data that is brain activity data from a user, and one or more feature quantities from the brain activity data. And a second intention identifier that acquires an intention identifier corresponding to the input feature quantity group from two or more second intention determination information stored in a storage medium. It is a program for functioning as an acquisition part and the 2nd intention identifier output part which outputs the intention identifier which the said 2nd intention identifier acquisition part acquired.

  In the above program, the second intention identifier acquisition unit acquires an intention identifier corresponding to the input feature quantity group by using two or more second intention determination information stored in the storage medium. It is preferable that the program be a function.

  (Embodiment 2)

  In this embodiment, as in the first embodiment, when acquiring learning data for detecting an intention, the experiment is divided into two, one is a training session and the other is a brush-up session. A robot system that performs the above will be described. Further, the present embodiment is different from the first embodiment in that intention detection is performed using two or more types of data such as NIRS data and brain wave data.

  Also, in the present embodiment, a brain information output device that determines the user's intention by weighting each of a plurality of brain activity data obtained from a plurality of brain activity measuring devices will be described. The weighting for the brain activity data is equivalent to the weighting for the probability obtained from the brain activity data.

  Also, in the present embodiment, a brain information output device that determines the final intention by using only the intention that each brain activity measurement device is good at will be described.

  FIG. 1 is a conceptual diagram of a robot system 1 in the present embodiment. FIG. 13 is a block diagram of the robot system 2 in the present embodiment.

  The robot system 2 includes a brain information output device 21 and a robot 12.

  The brain information output device 21 includes an intention determination information acquisition device 211, a brain activity data acquisition unit 212, a feature amount group acquisition unit 213, a second intention identifier acquisition unit 214, and a second intention identifier output unit 115.

  The intention determination information acquisition device 211 includes a first intention determination information storage unit 2110, a first learning data acquisition unit 2111, a first learning feature quantity group acquisition unit 2112, a first intention determination information storage unit 2113, and a second intention determination information storage. Unit 2114, second learning data acquisition unit 2115, second learning feature quantity group acquisition unit 2116, second intention determination information storage unit 2117, first intention identifier acquisition unit 2118, and first intention identifier output unit 1119.

  The first intention identifier acquisition unit 2118 includes first probability calculation means 21181 and first intention determination means 21182.

  The second intention identifier acquisition unit 214 includes second probability calculation means 2141 and second intention determination means 2142.

  The first intention determination information storage unit 2110 can store two or more first intention determination information created from two or more types of learning data. Two or more types of learning data are two or more types of brain activity data among NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph), etc. . The first intention determination information storage unit 2110 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The first learning data acquisition unit 2111 is first learning data that is learning data from the user's brain when the user performs a series of actions based on one intention in an environment where the robot 12 to be operated does not operate. To get. The first learning data acquisition unit 2111 acquires first learning data that is two or more types of learning data. Learning data is a series of brain activity data observed when a user performs a series of actions. The first learning data acquisition unit 2111 includes, for example, two or more of brain measurement apparatuses such as a near infrared light measurement apparatus, an electroencephalogram measurement apparatus, a functional magnetic resonance imaging (MRI), and a PET (Positron Emission Tomography). Device.

  The first learning feature value group acquisition unit 2112 acquires a first learning feature value group for each type from two or more types of first learning data. The first learning feature value group acquisition unit 2112 acquires two or more first learning feature value groups. The first learning feature value group acquisition unit 2112 can be realized typically as an MPU, a memory, or the like. The processing procedure of the first learning feature quantity group acquisition unit 2112 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first intention determination information storage unit 2113 stores the first intention determination information for each type of learning data in which the first learning feature value group for each type of learning data and the intention identifier are associated with each other. Stored in the unit 2110. The first intention determination information storage unit 2113 can be normally realized by an MPU, a memory, or the like. The processing procedure of the first intention determination information storage unit 2113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention determination information storage unit 2114 can store two or more second intention determination information created from two or more types of learning data. Two or more types of learning data are two or more types of brain activity data among NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph), etc. . The second intention determination information storage unit 2114 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium.

  The second learning data acquisition unit 2115 is the robot 12 that performs an operation according to the received intention identifier, and when the user performs a series of actions based on one intention in an environment in which the operation target robot 12 operates. In addition, second learning data that is learning data is acquired from the brain of the user. Here, the second learning data acquired by the second learning data acquisition unit 2115 is two or more types of learning data. The second learning data acquisition unit 2115 includes, for example, two or more of brain measurement devices such as a near infrared light measurement device, an electroencephalogram measurement device, a functional magnetic resonance imaging (MRI), and a PET (Positron Emission Tomography). Device.

  The second learning feature value group acquisition unit 2116 acquires a second learning feature value group for each type from two or more types of second learning data. The second learning feature quantity group acquisition unit 2116 acquires two or more second learning feature quantity groups. The second learning feature quantity group acquisition unit 2116 can be normally realized by an MPU, a memory, or the like. The processing procedure of the second learning feature value group acquisition unit 2116 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention determination information storage unit 2117 is a second intention determination information storage unit 2117 that associates a second learning feature quantity group for each type of learning data with an intention identifier for each second type of learning data. The discrimination information is accumulated in the second intention discrimination information storage unit 2114. The second intention determination information storage unit 2117 stores, for example, second intention determination information having a learning data type identifier, an intention identifier, and a second learning feature amount group in the second intention determination information storage unit 2114. The second intention determination information storage unit 2117 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second intention determination information storage unit 2117 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first intention identifier acquisition unit 2118 obtains an intention identifier corresponding to the second learning feature quantity group for two types of data (for example, NIRS data and electroencephalogram data), and two or more first intention determinations corresponding to the two types of data. Obtain from information. The first intention identifier acquisition unit 2118 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first intention identifier acquisition unit 2118 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first probability calculation unit 21181 calculates, for each intention identifier, the probability that the indicated intention is an intention corresponding to one intention identifier among a plurality of intention identifiers for each type of brain activity data. . It is preferable that the first probability calculating unit 21181 calculates the probability for each intention identifier for only a part of the predetermined intention identifier for each type of brain activity data. In such a case, the first probability calculating unit 21181 calculates the probability only for the intention identifier that the brain activity data is good at.

  First, when there are two intention identifiers (two types), for example, the first probability calculation unit 21181 calculates a probability for each intention identifier as follows. That is, the first probability calculating unit 21181 calculates the distance between the second learning feature quantity group (vector) and the first learning feature quantity group (vector) corresponding to the first intention identifier. Then, the first probability calculation unit 21181 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). The first probability calculating unit 21181 sets the converted value (0 to 1) as the probability of the first intention identifier. Then, the first probability calculation unit 21181 calculates the probability of the second intention identifier based on “1-probability of the first intention identifier”.

  When there are four intention identifiers (four types), for example, the first probability calculation unit 21181 calculates the probability for each intention identifier as follows. That is, the process for the above two intention identifiers is expanded. The first probability calculation unit 21181 calculates the distance between the second learning feature quantity group (vector) and the first learning feature quantity group (vector) corresponding to the first intention identifier. Then, the first probability calculation unit 21181 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). The first probability calculating unit 21181 sets the converted value (0 to 1) as the probability of the first intention identifier. Next, the first probability calculation unit 21181 calculates the distance from the first learning feature quantity group (vector) corresponding to the second intention identifier, the third intention identifier, or the fourth intention identifier as “1-first. Calculated by “probability of intention identifier”.

  Next, the first probability calculation unit 21181 calculates the distance between the second learning feature quantity group (vector) and the first learning feature quantity group (vector) corresponding to the second intention identifier. Then, the first probability calculation unit 21181 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). The first probability calculating unit 21181 sets the converted value (0 to 1) as the probability of the second intention identifier. Next, the first probability calculation unit 21181 calculates the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the third intention identifier, or the fourth intention identifier as “1-second. Calculated by “probability of intention identifier”.

  Next, the first probability calculation unit 21181 calculates the distance between the second learning feature quantity group (vector) and the first learning feature quantity group (vector) corresponding to the third intention identifier. Then, the first probability calculation unit 21181 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first probability calculation unit 21181 sets the converted value (0 to 1) as the probability of the third intention identifier. Next, the first probability calculation unit 21181 calculates the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the fourth intention identifier as “1-third. Calculated by “probability of intention identifier”.

  Next, the first probability calculation unit 21181 calculates the distance between the second learning feature quantity group (vector) and the first learning feature quantity group (vector) corresponding to the fourth intention identifier. Then, the first probability calculation unit 21181 inputs the distance to a logistic function (sigmoid function) and converts the distance into a value between 0 and 1 (a value between 0 and 1). Then, the first probability calculation unit 21181 sets the converted value (0 to 1) as the probability of the fourth intention identifier. Next, the first probability calculation unit 21181 calculates the distance from the first learning feature quantity group (vector) corresponding to the first intention identifier, the second intention identifier, or the third intention identifier as “1-fourth”. Calculated by “probability of intention identifier”.

  And the 1st probability calculation means 21181 acquires the probability of a 4th intention identifier from a 1st intention identifier by merging the probability acquired above. Here, the merge means the probability of the first intention identifier (P1), the probability of the intention identifier other than the first (P234), the probability of the second intention identifier (P2), the first Probability of intention identifier other than two (P134), Probability of third intention identifier (P3), Probability of intention identifier other than third (P124), Probability of fourth intention identifier (P4) This refers to an operation to be obtained from the eight probabilities of the probability of intention identifiers other than the fourth (P123). The probability of the merged first intention identifier is P1 × P134 × P124 × P123, the probability of the merged second intention identifier is P234 × P2 × P124 × P123, and the probability of the merged third intention identifier is P234 × P134 ×. It is preferable to obtain P3 × P123 and the merged fourth intention identifier as P234 × P134 × P124 × P4. An operation for normalizing the probability may be added. Here, normalization means that the probability of adding all becomes 1 by dividing each probability by the value obtained by adding the probability of the merged first intention identifier to the probability of the fourth intention identifier. It is an operation to do.

  The first intention determination unit 21182 acquires the intention identifier using the probability calculated for each type and the intention identifier calculated by the first probability calculation unit 21181. The first intention determination means 21182 calculates the final probability for each intention identifier using different weights with respect to the probability for each type of brain activity data, and the final probability for each intention identifier is the highest. It is preferable to acquire the intention identifier.

  The first intention determination means 21182 calculates a probability for each intention identifier using two or more probabilities among the probabilities calculated for each type of brain activity data. This probability is called the integration probability. The method for calculating the integration probability will be described later.

For example, there are four types of intention identifiers, left hand “Left”, right hand “Right”, tongue “Tongue”, and foot “Foot”, brain activity data is two types, and brain activity data is measured by near infrared light In the case of the time-series data of the hemoglobin amount acquired by the apparatus and the time-series data of the electroencephalogram acquired by the electroencephalogram measurement apparatus, the first intention determination unit 21182 uses the following formula 1 to calculate the integration probability. calculate.

In Formula 1, P ^NIRS _Left (in Formula 1, the superscript character string and the subscript character string are aligned at the upper limit, but the superscript character string and the subscript character are aligned vertically, Is the probability that the intention identifier is “Left” when the brain activity data of the hemoglobin amount (NIRS data) is used. Further, P ^NIRS _Right is a probability that the intention identifier is “Right”. P ^NIRS _Tongue is the probability of the intention identifier “Tonge” when the brain activity data (NIRS data) of the hemoglobin amount is used. P ^NIRS _Foot is the probability of the intention identifier “Foot” when the brain activity data (NIRS data) of the hemoglobin amount is used. P ^EEG _Left is the probability of the intention identifier “Left” when time series data (EEG data) of an electroencephalogram is used. Further, P ^EEG _Right is a probability of the intention identifier “Right” when time series data (EEG data) of an electroencephalogram is used. Further, P ^EEG _Tongue is a probability that the intention identifier is “Tonge” when time series data of EEG (EEG data) is used. Further, P ^EEG _Foot is a probability of an intention identifier “Foot” when time series data (EEG data) of an electroencephalogram is used. P ^total _text is an integration probability. P ^total _text is the final probability (integration probability) of the intention identifier TEXT (Left, Right, Tonegue, or Foot).

  In Equation 1, w is a weight parameter indicating a weight. “W = 1” indicates that the probability for each type is handled with the same weight regardless of the type of brain activity data. In the case of “w = 2”, the probability of using the time series data (EEG data) of the electroencephalogram is more emphasized than the probability of using the brain activity data (NIRS data) of the hemoglobin amount. It shows that.

In Formula 1, i can take any of “Left, Right, Tonegue, and Foot”. Further, P ^NIRS _i is a probability obtained from a near-infrared light measuring device, and P ^EEG _i is a probability obtained from an electroencephalogram.

  Note that Equation 1 is an equation for calculating the integration probability from the probabilities corresponding to two types of learning data, but the same applies even when calculating the integration probability from the probabilities corresponding to three or more types of learning data. The integration probability can be calculated by the calculation formula. In such a case, it is preferable to assign different weights (w) to the probabilities corresponding to the respective learning data.

  The first intention determination means 21182 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first intention determination means 21182 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The brain activity data acquisition unit 212 acquires two or more types of brain activity data. The brain activity data is, for example, two or more types of data among data acquired by NIRS (near infrared spectroscopy), fMRI (functional MRI), MEG (magnetoencephalograph), EEG (electroencephalograph). The brain activity data acquired by the brain activity data acquisition unit 212 may be two or more types of data indicating brain activity. The brain activity data acquisition unit 212 is, for example, a near infrared light measurement device and an electroencephalogram measurement device. The brain activity data acquisition unit 212 may be, for example, functional MRI (fMRI: functional magnetic resonance imaging) or PET (Positron Emission Tomography).

  The feature amount group acquisition unit 213 acquires an input feature amount group that is one or more feature amounts for each type from the two or more types of brain activity data acquired by the brain activity data acquisition unit 212. The feature quantity group acquisition unit 213 acquires two or more input feature quantity groups by performing signal processing on two or more types of brain activity data. The signal processing of the feature quantity group acquisition unit 213 includes, for example, bias correction, baseline correction, trend removal, average calculation, high-pass filter processing, low-pass filter processing, FFT processing, PCA processing, ICA processing, Phase Signal processing such as Locking value, moving average calculation, sensor selection, time selection, reference correction, downsampling, upsampling, power calculation, envelope calculation (Hilbert transform), spatial filter processing, inverse problem filter processing, etc. . The feature amount may be anything such as an average value or variance. The feature quantity group acquisition unit 213 can be usually realized by an MPU, a memory, or the like. The processing procedure of the feature quantity group acquisition unit 213 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention identifier acquisition unit 214 obtains an intention identifier corresponding to the input feature amount group for two types of data (for example, NIRS data and electroencephalogram data), two or more second intention determination information corresponding to the two types of data, Alternatively, it is acquired from the first intention determination information and the second intention determination information. The second intention identifier acquisition unit 214 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second intention identifier acquisition unit 214 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  For each type of brain activity data, the second probability calculation means 2141 calculates, for each intention identifier, a probability that the indicated intention is an intention corresponding to one intention identifier among a plurality of intention identifiers. . It is preferable that the second probability calculation unit 2141 calculates the probability for each intention identifier with respect to only a part of the predetermined intention identifier for each type of brain activity data. In such a case, the second probability calculation means 2141 calculates the probability only for the intention identifier that the brain activity data is good at.

  The second probability calculation means 2141 performs the same processing as the first probability calculation means 21181, although the data handled is different. That is, the first learning feature quantity group in the first probability calculation means 21181 is the second learning feature quantity group. Further, the second learning feature quantity group in the first probability calculation means 21181 is an input feature quantity group. Therefore, a specific description of the operation of the second probability calculation unit 2141 is omitted.

  The second probability calculation means 2141 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second probability calculating means 2141 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second intention determination unit 2142 acquires the intention identifier using the probability calculated for each type and the intention identifier calculated by the second probability calculation unit 2141. The second intention determination unit 2142 calculates the final probability for each intention identifier using different weights with respect to the probability for each type of brain activity data, and the final probability for each intention identifier is the highest. It is preferable to acquire the intention identifier.

  The second intention determination unit 2142 calculates a probability for each intention identifier using two or more probabilities among the probabilities calculated for each type of brain activity data. This probability is called the integration probability.

  The operation of the second intention determination unit 2142 is the same as the operation of the first intention determination unit 21182, and a description thereof will be omitted. However, the probability handled by the second intention determination unit 2142 (probability for each data type and each intention identifier) is a probability calculated by the second probability calculation unit 2141 and is different from the first intention determination unit 21182.

  The second intention determination unit 2142 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second intention determination unit 2142 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the robot system 2 will be described. First, the operation of the intention determination information acquisition apparatus 211 will be described. The operation of the intention determination information acquisition apparatus 211 is the same as the operation of the intention determination information acquisition apparatus 111 except that a plurality of types of brain activity data are used. That is, in the operation of the intention determination information acquisition device 211, the first learning data has two or more types of learning data. Then, the intention determination information acquisition apparatus 211 obtains first intention determination information and second intention determination information for each of two or more types of learning data.

  Further, the operation in which the brain information output device 21 outputs the intention identifier is different in that it uses two or more types of brain activity data, but is substantially the same as the operation in the flowchart of FIG. 4 except for the difference. An example of the intention identifier acquisition process which is the main process of the different point will be described with reference to the flowchart of FIG. This intention identifier acquisition process is an example of a process for acquiring an intention identifier using a probability.

  (Step S1401) The first intention identifier acquisition unit 2118 or the second intention identifier acquisition unit 214 (hereinafter referred to as the intention identifier acquisition unit) sets the counter i to 1.

  (Step S1402) The intention identifier acquisition unit determines whether or not the i-th intention identifier exists. If the i-th intention identifier exists, the process proceeds to step S1403. If the i-th intention identifier does not exist, the process proceeds to step S1408.

  (Step S1403) The intention identifier acquisition unit sets a counter j to 1.

  (Step S1404) The intention identifier acquisition unit determines whether or not the j-th type of learning data exists. If the j-th type of learning data exists, the process proceeds to step S1405. If the j-th type of learning data does not exist, the process proceeds to step S1410.

  (Step S1405) The intention identifier acquisition unit acquires a learning feature amount group (vector) for each type of learning data corresponding to the j-th type of learning data and the i-th intention identifier.

  (Step S1406) The intention identifier acquisition unit acquires the input feature quantity group (vector).

  (Step S1407) The intention identifier acquisition unit (the second probability calculation unit 2141 or the like) uses the learning feature amount group (vector) acquired in step S1405 and the feature amount group (vector) acquired in step S1406, The probability that the input feature quantity group corresponds to the i-th intention identifier is calculated.

  (Step S1408) The intention identifier acquisition unit associates the identifier of the j-th type of learning data, the i-th intention identifier, and the probability calculated in step S1407, and stores them in the recording medium at least temporarily.

  (Step S1409) The intention identifier acquiring unit increments the counter j by 1. The process returns to step S1404.

  (Step S1410) The intention identifier acquiring unit increments the counter i by one. The process returns to step S1402.

  (Step S1411) The intention identifier acquisition unit calculates an integration probability corresponding to each of all the intention identifiers. As a method for calculating the integration probability, for example, Equation 1 described above is used.

  (Step S1412) The intention identifier acquisition unit acquires an intention identifier corresponding to the largest integration probability. Return to upper process.

  In the flowchart of FIG. 14, the process of step S1406 is repeated in the loop, but it goes without saying that it may be performed only once outside the loop. The flowchart of FIG. 14 is an example of the intention identifier acquisition process.

  Hereinafter, a specific operation of the robot system 2 in the present embodiment will be described. A conceptual diagram of the robot system 2 is shown in FIG.

  Now, the 1st learning data acquisition part 2111 is a measuring instrument (FIG. 15) which a user puts on a head. The first learning data acquisition unit 2111 acquires, for example, a voltage when emitting near-infrared light of 780 nm, 805 nm, and 830 nm and hitting the head, and a voltage when not emitting light. Substituting into a predetermined formula (formula for calculating the amount of hemoglobin), the hemoglobin information that is the amount of hemoglobin is acquired. The first learning data acquisition unit 2111 acquires hemoglobin information from, for example, 48 channels (48 measurement points exist in FIG. 15). Note that the first learning data acquisition unit 2111 holds information of a predetermined formula. Then, the first learning data acquisition unit 2111 also acquires electroencephalogram data. Then, for example, two different types of brain activity data are acquired by the measuring instrument shown in FIG.

  The second learning data acquisition unit 2115 and the brain activity data acquisition unit 212 are also the same hardware as the first learning data acquisition unit 2111. That is, the first learning data, the second learning data, and the brain activity data are NIRS data and electroencephalogram data.

  In such a situation, the user performs a training session. In other words, for example, it is assumed that the user imagines an action of moving the “right hand”. In this case, the intention is “imagine that the right hand is moved”, and the intention identifier is “right hand”.

  Then, it is assumed that the first learning data acquisition unit 2111 of the intention determination information acquisition device 211 has acquired brain activity data (NIRS data) shown in FIG. Further, it is assumed that the first learning data acquisition unit 2111 has acquired the brain activity data (electroencephalogram data) shown in FIG. FIG. 16 shows the acquired electroencephalogram data for each elapse of time (t). In FIG. 16, it is assumed that the electroencephalogram information is acquired every 2 msec.

  Next, the first learning feature value group acquisition unit 2112 acquires a first learning feature value group corresponding to the NIRS data from the NIRS data shown in FIG. Also, the first learning feature value group acquisition unit 2112 acquires a first learning feature value group corresponding to the electroencephalogram data from the electroencephalogram data shown in FIG.

  Then, the first intention determination information storage unit 2113 associates the first learning feature amount group for each type of learning data with the intention identifier “right hand”, the first intention determination information for each type of learning data, The information is stored in one intention determination information storage unit 2110. An example of such first intention determination information is information of a record 1701 in FIG.

  The above processing is repeated for at least the intention identifiers “left hand”, “tongue”, and “foot”, and the first intention determination information storage unit 2110 stores at least four first intention determinations as shown in FIG. Information is stored. This completes the training session.

  Next, the user performs a brush-up session. That is, assume that the user imagines that the user has moved his / her foot in an attempt to perform an action corresponding to the intention identifier “foot” on the robot 12.

  Next, it is assumed that the second learning data acquisition unit 2115 acquires two types of second learning data that are data of two types of brain activities that occur when the user imagines that the user has moved his / her foot.

  Next, the second learning feature value group acquisition unit 2116 acquires two types of second learning feature value groups from the two types of acquired second learning data. The two types of second learning feature amount groups are a first feature amount and second feature amount corresponding to NIRS data, and a first feature amount and second feature amount corresponding to brain wave data.

  Next, the first intention identifier acquisition unit 2118 acquires an intention identifier corresponding to the acquired second learning feature quantity group using the first learning feature quantity group of FIG. That is, the first intention identifier acquisition unit 2118 calculates the integration probability corresponding to the first learning feature quantity group for each intention identifier of FIG. The calculation method of the integration probability is a method using Formula 1, for example. The first intention identifier acquisition unit 2118 acquires the intention identifier (here, “tongue”) having the highest integration probability.

  Then, the first intention identifier output unit 1119 outputs the acquired intention identifier “tongue” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “tongue”. Then, the execution unit 123 reads the motion module corresponding to the accepted intention identifier “tongue” from the motion information storage unit 121. Next, the execution unit 123 executes the read operation module.

  Through the above operation, the user recognizes that the robot 12 has performed a movement different from his intention. Then, the user inputs the correct intention identifier “foot” to the intention determination information acquisition apparatus 211 using an input unit (not shown).

  Then, the second intention determination information storage unit 1117 of the intention determination information acquisition apparatus 111 receives the intention identifier “foot” input by the user.

  Next, the second intention determination information storage unit 2117 configures second intention determination information that pairs the intention identifier “foot” and the second learning feature amount group for each type of acquired learning data, The second intention determination information is accumulated in the second intention determination information storage unit 1114. The data structure of the second intention determination information is the same as that in FIG.

  Next, it is assumed that the user imagines that the user has moved the right hand in an attempt to perform an action corresponding to the intention identifier “right hand” on the robot 12.

  Next, it is assumed that the second learning data acquisition unit 2115 acquires two types of second learning data generated when the user imagines that the user has moved his right hand.

  Next, the second learning feature value group acquisition unit 1116 acquires two types of second learning feature value groups from the two types of acquired second learning data.

  Next, the first intention identifier acquisition unit 2118 acquires an intention identifier corresponding to the acquired second learning feature quantity group using the first learning feature quantity group of FIG. That is, the first intention identifier acquisition unit 2118 calculates an integrated probability corresponding to the first learning feature quantity group for each second learning feature quantity group and the intention identifier of FIG. Then, it is assumed that the first intention identifier acquisition unit 2118 has acquired the intention identifier (here, “right hand”) having the highest integration probability.

  Then, the first intention identifier output unit 1119 outputs the acquired intention identifier “right hand” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “right hand”. Then, the execution unit 123 reads the operation module corresponding to the accepted intention identifier “right hand” from the operation information storage unit 121. Next, the execution unit 123 executes the read operation module.

  Through the above operation, the user recognizes that the robot 12 has performed a movement according to his / her intention. Then, the user inputs information indicating correctness to the intention determination information acquisition apparatus 211 using an input unit (not shown).

  Next, the second intention determination information storage unit 2117 configures second intention determination information which is a pair of the intention identifier “right hand” and the acquired second learning feature quantity group, and the second intention determination information is stored in the second intention determination information. It accumulates in the two intention determination information storage unit 2114.

  The above processing is also performed on at least the intention identifier “left hand” and the intention identifier “tongue”, and second intention determination information corresponding to all intention identifiers is accumulated in the second intention determination information storage unit 2114 ( FIG. 18).

  With the above processing, the brush-up session ends.

  Next, it is assumed that the user performs a test session for operating the actual robot 12. Here, it is assumed that the second intention determination information in FIG. 18 is used as the learning data.

  That is, for example, it is assumed that the user has an imagination corresponding to the intention identifier “left hand” “imagination that the left hand is moved”.

  Next, the brain activity data acquisition unit 212 acquires two types of brain activity data in such a case.

  And the feature-value group acquisition part 213 acquires an input feature-value group for every data type from two types of acquired brain activity data.

  Next, the second intention identifier acquisition unit 214 calculates an overall probability for each intention identifier. Then, the second intention identifier acquisition unit 214 acquires the intention identifier “left hand” corresponding to the closest distance. Then, the second intention identifier output unit 215 outputs the acquired intention identifier “left hand” to the robot 12.

  Next, the intention identifier receiving unit 122 of the robot 12 receives the intention identifier “left hand” output from the brain information output device 21.

  Then, the execution unit 123 reads the operation module corresponding to the accepted intention identifier “left hand” from the operation information storage unit 121.

  Next, the execution unit 123 executes the read operation module. With the above processing, the robot 12 operates as intended by the user.

  As described above, according to the present embodiment, the brain information output apparatus 11 can obtain learning data through two sessions, a training session and a brush-up session. Therefore, the accuracy of intention detection can be improved.

  In addition, according to the present embodiment, the intention can be detected with high accuracy using two or more types of brain activity data.

  In addition, according to this Embodiment, although the example using 2 types of brain activity data was mainly demonstrated, you may use 3 or more types of brain activity data.

  Furthermore, the software that implements the intention determination information acquisition apparatus according to the present embodiment is the following program. In other words, when the user performs a series of actions based on a single intention in an environment where the robot to be operated does not operate, the program acquires first learning data as learning data from the user's brain. A first learning data acquisition unit to acquire; a first learning feature value group acquisition unit to acquire a first learning feature value group that is one or more feature values from the first learning data; and the first learning feature value group First intention determination information in which a first learning feature amount group acquired by the acquisition unit and an intention identifier for identifying one intention when the first learning data is acquired is stored in a storage medium. One intention determination information storage unit and a robot that performs an operation according to the received intention identifier, and when the user performs a series of actions under one intention in an environment in which the robot to be operated operates, From the user's brain A second learning data acquisition unit that acquires second learning data that is learning data, and a second learning feature amount group acquisition unit that acquires a second learning feature amount group that is one or more feature amounts from the second learning data And a second intention determination unit that associates the second learning feature amount group acquired by the second learning feature amount group acquisition unit with an intention identifier for identifying one intention when the second learning data is acquired. A program for causing information to function as a second intention determination information storage unit that stores information in a storage medium.

  In the program, the first learning data acquisition unit acquires first learning data that is two or more types of learning data, and the first learning feature amount group acquisition unit includes two or more types of first learning data. For each of the first learning data, the first learning feature quantity group is acquired from the first learning data, and the first intention determination information storage unit stores the first learning feature quantity group and the intention identifier for each type of learning data. Is stored in the storage medium, and the second learning data acquisition unit acquires second learning data that is two or more types of learning data. The second learning feature quantity group acquisition unit acquires a second learning feature quantity group from the second learning data for each type of two or more types of second learning data, and the second intention determination information The accumulator stores the second learning for each type of learning data. Correlating the symptom amount group with the intention identifier, the second intention determination information for each type of training data, it is preferable to accumulate in the storage medium.

  Moreover, the software which implement | achieves the brain information output device in this Embodiment is the following programs. That is, the program includes a computer, the above-described intention determination information acquisition device, a brain activity data acquisition unit that acquires brain activity data that is brain activity data from a user, and one or more feature quantities from the brain activity data. And a second intention identifier that acquires an intention identifier corresponding to the input feature quantity group from two or more second intention determination information stored in a storage medium. It is a program for functioning as an acquisition part and the 2nd intention identifier output part which outputs the intention identifier which the said 2nd intention identifier acquisition part acquired.

  In the above program, the brain activity data acquisition unit acquires two or more types of brain activity data, and the feature amount group acquisition unit acquires one or more types of the brain activity data from the two or more types of brain activity data. An input feature value group that is a feature value is acquired, and the second intention identifier acquisition unit is configured to determine whether the indicated one intention is one intention identifier of a plurality of intention identifiers for each type of the brain activity data. A probability calculating means for calculating the probability of the intention corresponding to each intention identifier, and an intention determining means for acquiring the intention identifier using the type calculated by the probability calculating means and the probability for each intention identifier. It is preferable that the program is a program for causing a computer to function.

  Further, in the program, the intention determination unit calculates a final probability for each intention identifier using different weights with respect to the probability for each type of brain activity data, and determines a final probability for each intention identifier. It is preferable that the program is a program for causing a computer to function as the one that acquires the intention identifier having the highest probability.

  Further, in the above program, the probability calculating means calculates a probability for each intention identifier for only a part of predetermined intention identifiers for each type of brain activity data. It is preferable that the program is for causing the program to occur.

  FIG. 19 shows the external appearance of a computer that executes the program described in this specification to realize the intention determination information acquisition apparatus and the like of the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 19 is an overview diagram of the computer system 340, and FIG. 20 is a block diagram of the computer system 340.

  In FIG. 19, the computer system 340 includes a computer 341 including an FD drive and a CD-ROM drive, a keyboard 342, a mouse 343, and a monitor 344.

  20, in addition to the FD drive 3411 and the CD-ROM drive 3412, the computer 341 stores an MPU 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a program such as a bootup program. A RAM 3416 for temporarily storing application program instructions and providing a temporary storage space; and a hard disk 3417 for storing application programs, system programs, and data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute the functions of the intent determination information acquisition device and the like of the above-described embodiment is stored in the CD-ROM 3501 or FD 3502, inserted into the CD-ROM drive 3412 or FD drive 3411, and It may be transferred to the hard disk 3417. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 341 to execute functions such as the intention determination information acquisition apparatus according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  The program does not include processing performed by hardware, for example, processing performed by a modem or an interface card (processing performed only by hardware).

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the intention determination information acquisition apparatus according to the present invention has an effect that the intention detection accuracy can be improved, and is useful as a BMI or the like.

1, 2 Robot system 11, 21 Brain information output device 12 Robot 111, 211 Intention discrimination information acquisition device 112, 212 Brain activity data acquisition unit 113, 213 Feature quantity group acquisition unit 114, 214 Second intention identifier acquisition unit 115, 215 Second intention identifier output unit 121 Motion information storage unit 122 Intent identifier reception unit 123 Execution unit 1110, 2110 First intention determination information storage unit 1111, 2111 First learning data acquisition unit 1112, 2112 First learning feature quantity group acquisition unit 1113 2113 First intention determination information storage unit 1114, 2114 Second intention determination information storage unit 1115, 2115 Second learning data acquisition unit 1116, 2116 Second learning feature quantity group acquisition unit 1117, 2117 Second intention determination information storage unit 1118 2118 First intention identifier acquisition unit 1119 Intention identifier output unit 2141 second probability calculation means 2142 second intention determining means 21181 first probability calculation means 21182 first intention determining means

Claims (10)

Extracted from the intention identifier, which is information for identifying the user's intention, and the first learning data acquired from the user's brain when the user performs a series of actions based on the intention identified by the intention identifier. A first intention determination information storage unit capable of storing two or more first intention determination information having a pair of a first learning feature amount group that is one or more feature amounts;
A first learning data acquisition unit that acquires first learning data, which is learning data, from the user's brain when the user performs a series of actions with one intention in an environment in which the robot to be operated does not operate; ,
A first learning feature value group acquisition unit that acquires a first learning feature value group that is one or more feature values from the first learning data;
First intention determination information in which the first learning feature quantity group acquired by the first learning feature quantity group acquisition unit is associated with an intention identifier for identifying one intention when the first learning data is acquired. A first intention determination information storage unit for storing in the first intention determination information storage unit;
An intention identifier and one or more feature quantities extracted from the second learning data acquired from the user's brain when the user performs a series of actions based on the intention identified by the intention identifier. A second intention determination information storage unit capable of storing two or more pieces of second intention determination information having a pair of two learning feature amount groups;
It is a robot that performs an operation according to the received intention identifier, and is learning data from the user's brain when the user performs a series of actions under one intention in an environment in which the operation target robot operates. A second learning data acquisition unit for acquiring second learning data;
A second learning feature quantity group acquisition unit that acquires a second learning feature quantity group that is one or more feature quantities from the second learning data;
Second intention determination information in which the second learning feature quantity group acquired by the second learning feature quantity group acquisition unit is associated with an intention identifier for identifying one intention when the second learning data is acquired. An intention determination information acquisition apparatus comprising: a second intention determination information storage unit that stores the second intention determination information storage unit. A first intention identifier acquisition unit that acquires an intention identifier corresponding to the second learning feature quantity group from two or more intention determination information stored in the first intention determination information storage unit;
A first intention identifier output unit that outputs the intention identifier acquired by the first intention identifier acquisition unit to the robot;
The robot is
The intention determination information acquisition apparatus according to claim 1, wherein the intention identifier is received and an operation according to the received intention identifier is performed. The first learning data acquisition unit
Obtain first learning data that is two or more types of learning data,
The first learning feature quantity group acquisition unit
For each type of two or more types of first learning data, obtain a first learning feature value group from the first learning data,
The first intention determination information storage unit
The first intention determination information for each type of learning data in which the first learning feature amount group for each type of learning data is associated with the intention identifier is accumulated in the first intention determination information storage unit,
The second learning data acquisition unit
Obtain second learning data that is two or more types of learning data,
The second learning feature quantity group acquisition unit
For each type of two or more types of second learning data, a second learning feature value group is acquired from the second learning data,
The second intention determination information storage unit
The second intention determination information for each type of learning data in which the second learning feature quantity group for each type of learning data is associated with the intention identifier is stored in the second intention determination information storage unit. The intention determination information acquisition apparatus according to claim 2. The intention determination information acquisition device according to any one of claims 1 to 3,
A brain activity data acquisition unit that acquires brain activity data that is brain activity data from a user;
A feature value group acquisition unit that acquires an input feature value group that is one or more feature values from the brain activity data;
A second intention identifier acquisition unit that acquires an intention identifier corresponding to the input feature quantity group from two or more second intention determination information stored in the second intention determination information storage unit;
A brain information output device comprising: a second intention identifier output unit that outputs an intention identifier acquired by the second intention identifier acquisition unit. The second intention identifier acquisition unit
The input using two or more second intention determination information stored in the second intention determination information storage unit and two or more first intention determination information stored in the first intention determination information storage unit The brain information output device according to claim 4 which acquires the intention identifier corresponding to a feature-value group. The brain activity data acquisition unit
Get two or more types of brain activity data,
The feature quantity group acquisition unit
From the two or more types of brain activity data, an input feature amount group that is one or more feature amounts is obtained for each type,
The second intention identifier acquisition unit
Probability calculating means for calculating, for each intention identifier, a probability that the indicated intention is an intention corresponding to one intention identifier of a plurality of intention identifiers for each type of brain activity data;
The brain information output device according to claim 4 or 5, further comprising an intention determination unit that acquires an intention identifier by using the probability calculated by the probability calculation unit and the probability for each intention identifier. The intention determining means includes
A final probability for each intention identifier is calculated using different weights for the probabilities for each type of brain activity data, and an intention identifier having the highest final probability for each intention identifier is obtained. Item 6. The brain information output device according to Item 6. The probability calculating means includes
The brain information output device according to claim 6, wherein, for each type of brain activity data, a probability is calculated for each intention identifier for only a part of the predetermined intention identifier. An operation information storage unit that can store two or more pieces of operation information that is a set of an intention identifier and an operation module that performs an operation corresponding to the intention identifier;
An intention identifier receiving unit that receives an intention identifier output from the brain information output device according to claim 4;
A robot comprising: an execution unit that executes an operation module corresponding to the intention identifier received by the intention identifier reception unit. A first learning data acquisition step of acquiring first learning data as learning data from the user's brain when the user performs a series of actions under one intention in an environment where the robot to be operated does not operate; ,
A first learning feature value group acquisition step of acquiring a first learning feature value group that is one or more feature values from the first learning data;
A first intention for storing in a storage medium first intention determination information in which the first learning feature quantity group acquired in the first learning feature quantity group acquisition step is associated with an intention identifier for identifying the one intention. A discrimination information accumulation step;
It is a robot that performs an operation according to the received intention identifier, and is learning data from the user's brain when the user performs a series of actions under one intention in an environment in which the operation target robot operates. A second learning data acquisition step of acquiring second learning data;
A second learning feature quantity group acquisition step of acquiring a second learning feature quantity group that is one or more feature quantities from the second learning data;
Second intention to accumulate in the storage medium second intention determination information in which the second learning feature quantity group acquired in the second learning feature quantity group acquisition step and the intention identifier for identifying the one intention are associated with each other An intention determination information acquisition method comprising a determination information accumulation step.