KR102146973B1 - Apparatus and method for learning user's operation intention using artificial neural network - Google Patents

Abstract

A method of learning a user's motion intention using an artificial neural network includes: receiving information on a signal measuring the user's motion from a motion sensor unit; Outputting a motion intention of a user corresponding to information on a signal obtained by measuring the motion of the user using a previously learned artificial neural network; Receiving information on the EEG signal measured for the user from the EEG sensor unit; And when the user's motion intention corresponding to the information on the EEG signal does not match the outputted user's motion intention, a label for the outputted user's motion intention is displayed by the user corresponding to the information on the EEG signal. It may include the step of inputting the previously learned artificial neural network by modifying the operation intention.

Description

{Apparatus and method for learning user's operation intention using artificial neural network}

The present invention relates to a method and apparatus for learning a user's motion intention using an artificial neural network.

As the market for artificial intelligence objects (for example, autonomous vehicles) grows, it is predicted that the disabled and the elderly, who were unable to drive, can use the car. However, in the case of such a handicapped person, there is a difficulty in that the body is not free, so that even if the user wants to manipulate the internal functions of the vehicle, they cannot operate at will. In the case of the disabled or the elderly, it is expected that the noise according to the movement will be large due to physical unnaturalness.

Due to this noise, it is expected that the performance of the conventional trained model will deteriorate. The present invention proposes an apparatus that continuously trains a system for the performance of a learned model in order to prevent a user who is not free from having any difficulty in using and manipulating internal functions of a device (eg, a vehicle) at will.

The technical problem to be achieved in the present invention is to provide a method of learning a user's motion intention using an artificial neural network.

Another technical problem to be achieved in the present invention is to provide an apparatus for learning a user's motion intention using an artificial neural network.

Another technical problem to be achieved in the present invention is to provide a computer-readable recording medium in which a program for executing a method of learning a user's motion intention using an artificial neural network on a computer is recorded.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

In order to achieve the above technical problem, a method of learning a motion intention of a user using an artificial neural network includes: receiving information on a signal measuring the motion of the user from a motion sensor unit; Outputting a motion intention of a user corresponding to information on a signal obtained by measuring the motion of the user using a previously learned artificial neural network; Receiving information on the EEG signal measured for the user from the EEG sensor unit; And when the user's motion intention corresponding to the information on the EEG signal does not match the outputted user's motion intention, a label for the outputted user's motion intention is displayed by the user corresponding to the information on the EEG signal. It may include the step of inputting the previously learned artificial neural network by modifying the operation intention.

The method may further include learning to recognize the output label of the motion intention of the user as the motion intention of the user corresponding to the information on the EEG signal.

The method includes information on the EEG signal and a user's operation intention corresponding to the information on the EEG signal whether or not the user's operation intention corresponding to the information on the EEG signal matches the output user's operation intention. And determining based on a predefined mapping relationship between the user's motion intention corresponding to the information on the signal measuring the user's motion and the output user's motion intention. The predefined mapping relationship may include a relationship corresponding to each row in the following table.

Here, the first, second and third frequencies are characterized by not having a multiple relationship with each other. The measured EEG signal may correspond to a signal caused by a blinking stimulus.

The outputting of the user's motion intention is performed when the measured signal for the user's motion is determined to be greater than or equal to a predefined feature value, and the predefined feature value may include an average signal strength of the EMG signal. . The artificial neural network may include deep neural networks (DNN). The motion sensor unit and the brain wave sensor unit may be portable and worn by the user.

In order to achieve the above other technical problem, an apparatus for learning a user's motion intention using an artificial neural network includes: a communication unit receiving information about a signal measuring the user's motion from a motion sensor unit; And a processor that outputs a user's motion intention corresponding to information on a signal measured by the user's motion using a previously learned artificial neural network, wherein the communication unit includes an EEG signal measured for the user from the EEG sensor unit. When the information on the EEG signal is received, and the processor indicates that the user's motion intention corresponding to the information on the EEG signal does not match the outputted user's motion intention, the output label of the user's motion intention The user's motion intention corresponding to the information on is corrected and input to the previously learned artificial neural network.

The processor may learn to recognize the output label of the user's motion intention as the user's motion intention corresponding to the information on the EEG signal.

The processor determines whether or not the user's operation intention corresponding to the information on the EEG signal matches the output user's operation intention, the information on the EEG signal, the user's operation intention corresponding to the information on the EEG signal, and It may be determined based on a predefined mapping relationship between the user's motion intention corresponding to the information on the signal measuring the user's motion.

The measured EEG signal may correspond to a signal caused by a blinking stimulus. The processor outputs the user's motion intention corresponding to the measured signal for the user's motion when it is determined that the measured signal for the user's motion is greater than or equal to a predefined feature value. The predefined feature value includes the average signal strength of the EMG signal. The artificial neural network includes deep neural networks (DNN).

The motion sensor unit and the EEG sensor unit are portable and wearable by the user. The EEG sensor unit includes an Electroencephalography (EEG) sensor. The motion sensor unit includes an EMG sensor unit or an acceleration sensor unit. The signal measured for the user's motion includes an EMG signal or an acceleration signal of the user.

The method of learning a user's motion intention according to an embodiment of the present invention can reduce errors that occur when a device is operated using a machine learning algorithm to recognize the user's motion intention.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid in understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, the technical spirit of the present invention will be described.
1 is a diagram illustrating a 10-20 EEG system as an example of an EEG sensor system.
2 is a diagram illustrating a layer structure of an artificial neural network.
3 is a diagram illustrating a block diagram of a learning device for learning a user's motion intention using an artificial neural network according to the present invention.
4 is a flowchart briefly explaining a method of learning a user's motion intention using an artificial neural network according to the present invention.
5 is a diagram illustrating a system including a device 300 for learning a user's motion intention using an artificial neural network according to the present invention.
6 shows a graph of brain waves detected by the EEG sensor 520 when the user 500 views a stimulus image of 15 Hz.
7 is a flowchart illustrating a method of learning a user's motion intention using an artificial neural network according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description to be disclosed hereinafter together with the accompanying drawings is intended to describe exemplary embodiments of the present invention, and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details to provide a thorough understanding of the present invention. However, one of ordinary skill in the art appreciates that the invention may be practiced without these specific details.

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted, or may be shown in a block diagram form centering on core functions of each structure and device. In addition, the same components will be described with the same reference numerals throughout the present specification.

The present invention proposes a method for learning a user's motion intention using an artificial neural network and a learning device for the same. The learning device is an EEG sensor unit and a motion sensor unit worn by the user (eg, an EMG sensor unit or an acceleration sensor unit, etc., and will be described below as an EMG sensor unit for convenience of explanation). It receives signals and EMG signals and learns the user's motion intentions using an artificial neural network. Prior to describing the present invention, an EEG sensor unit and a motion sensor unit will be briefly described.

An EEG sensor unit (eg, an electroencephalography (EEG) sensor) measures the flow of electricity generated when signals are transmitted between cranial nerves in the nervous system. The neurophysiological measurement of the brain's electrical activity includes recording through electrodes attached to the scalp.

1 is a diagram illustrating a 10-20 EEG system as an example of an EEG sensor system.

In order to measure EEG, the position of the electrode on the scalp is required, and a 10-20 EEG system can be used as a system for the measurement method. The 10-20 EEG system is the same as that shown in Fig. 1 and is named descriptive. 10 and 20 of "10-20" refer to 10% and 20% of the distance between NASION (anterior) and INION (posterior).

In the channel names shown in FIG. 1, an odd number always refers to an electrode position on the left side of the head, and an even number refers to an electrode position on the right side of the head. The letter "z" is used to indicate a point between the center line or the center line between NASION (Nz) and INION (Iz). F, C, P, O, and T are the acronyms for the brain region, where F is the frontal lobe, C is the central nervous system, P is the temporal lobe, O is the occipital lobe, and T is the Represents the parietal lobe (Temporal).

In FIG. 1, NASION refers to a small notch under the forehead and above the nose, and INION refers to a small bump or bump at the base of the neck and occiput. The distance between these two points can be measured using a tape measure, etc., it is accurate to measure in centimeters, and a typical measurement is 36 centimeters.

In NASION, 10% above this distance is the midpoint between the two prefrontal lobe sites called FP1 and FP2. In addition, Fz is where 20% of the distance between NASION and INION has gone further. If NASION and INION move 20% more in Fz, it is Cz, the midpoint between NASION and INION.

As an example, brain waves are artificially delta (δ) waves (0.2 ~ 3.99 Hz), theta (θ) waves (4 ~ 7.99 Hz), and alpha (α) waves (8 ~ 12.99 Hz) depending on the range of the vibrating frequency. , Beta β wave (13 ~ 29.99 Hz), gamma g wave (30 ~ 50 Hz). Delta waves are predominantly pronounced during deep sleep in normal people or in newborns. Theta waves appear mainly in the process leading to emotional stability or sleep, and are related to various states such as memory, creativity, and concentration. Alpha waves mainly appear in a relaxed state, such as relaxation, and the amplitude increases as the state becomes more stable and comfortable. Beta waves mainly appear in the frontal region and appear during all conscious activities, such as when awake and when speaking. In particular, it appears predominantly in anxious state or tension, and in complex calculations. Gamma waves vibrate faster than beta waves, and are emotionally more nervous, or are related to advanced cognitive information processing such as reasoning and judgment.

1 illustrates an EEG measurement channel measured by an EEG sensor of a wearable device according to the present invention. Four EEG channels, Fp1, A1, A2, and Cz, can be used. Among EEG sensors, EEG1 measures drowsiness and concentration in the Fp1 channel, and EEG 2 measures drowsiness in the Cz channel (EEG2). Channel A1 is used as REF (Reference) and channel A2 is used as GND (Ground).

The following is an example that can be the characteristic values of the EEG signal.

1. Mean: indicates the average signal strength of the EEG signal.

2. Standard deviation: This is the standard deviation of the EEG signal strength.

3. Zero Crossing: Indicates the number of zero crossings when the EEG signal is differentiated by the second order.

4. Kurtosis: A value obtained by normalizing the fourth-order momentum of the signal to the square of the standard deviation, and indicates a value that determines the peakedness of the probability distribution of the signal strength.

5. Crest factor: The ratio of the peak value of the signal. It is similar to Kurtosis and is a characteristic value that describes the impedance of the signal.

6. Power Spectrum Density: It represents the power spectrum density generated through Fourier transform of the EEG signal.

7. Correlation: Shows auto-correlation of EEG signals.

8. Threshold Crossing: Shows the number of filtered raw data that exceeds a specific threshold in one Epoch.

9. Skewness: As the third-order momentum of the EEG signal, it represents the degree to which the distribution of signal strength is biased.

10. Entropy: indicates the measurement of the predictability of the signal.

11. Band Energy: It represents the energy in the frequency domain related to stress.

12. Spectral Flux: Means the rate of change over time of the power spectrum when analyzing the time frequency of a signal (PSD(t)/PSD(t-1))

Next, the EMG sensor unit will be described.

As a wearable device, the EMG sensor unit may have a shape that can be worn on a user's wrist or arm. The EMG sensor unit may extract or acquire data on the EMG signal by sensing a user's bio-signal, respectively. The EMG sensor unit may include an electromyography (EMG) sensor. In this case, preferably, in the case of EMG, a value may be received and processed at a minimum sampling rate of 400 Hz per second. The EMG sensor unit may transmit data or information on the calculated or extracted EMG signals to the learning apparatus of the present invention. In this case, the EMG sensor unit may convert data on the extracted EMG signal into digital signal data and transmit it to the learning apparatus of the present invention.

EMG The extracted value from the signal or Feature value

1. Mean: It represents the average signal strength of the EMG signal.

2. Standard deviation: This is the standard deviation of the EMG signal strength.

3. Zero Crossing: Shows the number of zero crossings when the EMG signal is differentiated by the second order.

4. Kurtosis: A value obtained by normalizing the fourth-order momentum of the signal to the square of the standard deviation, and indicates a value that determines the peakedness of the probability distribution of the signal strength.

5. Crest factor: The ratio of the peak value of the signal. It is similar to Kurtosis and is a characteristic value that describes the impedance of the signal.

6. Power Spectrum Density: It represents the power spectrum density generated through Fourier transform of the EMG signal.

7. Correlation: indicates auto-correlation of EMG signals.

8. Threshold Crossing: Shows the number of filtered raw data that exceeds a specific threshold in one Epoch.

9. Skewness: As the third-order momentum of the EMG signal, it represents the degree to which the distribution of signal intensity is biased.

10. Entropy: indicates the measurement of the predictability of the signal.

11. Band Energy: It represents the energy in the frequency domain related to concentration.

12. Spectral Flux: Means the rate of change over time of the power spectrum when analyzing the time frequency of the signal (PSD(t)/PSD(t-1))

Hereinafter, deep learning, artificial neural networks, and deep neural networks, which are prerequisites for a method of learning a user's motion intention using an artificial neural network according to the present invention, will be briefly described.

Deep learning is a kind of artificial neural network (ANN) that uses the theory of human neural networks. It is composed of a layer structure and is one between the input layer and the output layer. It is a set of machine learning models or algorithms that refer to a deep neural network (DNN) that has the above hidden layer (hereinafter referred to as the middle layer). Simply put, Deep Learning can be said to be an artificial neural network with deep layers.

It is estimated that the human brain is made up of 25 billion neurons. The brain is made up of neurons, and each neuron (Neuron) refers to one neuron that makes up a neural network. Neurons contain one cell body, one protrusion of the cell body, one axon (Axon or nurite) and usually several dendrite or protoplasmic processes. The exchange of information between these neurons is transmitted through junctions between neurons called synapses. It is very simple to look at just one nerve cell apart, but when these nerve cells gather, they can possess human intelligence. The dendrites receive signals from other neurons (Input), and the axon is a part that extends very long from the cell body and transmits signals to other neurons (Output). There is a connection called synapse that connects the axon and the dendrites that transmit signals between nerve cells. The signal is not transmitted unconditionally, but only when the signal strength exceeds a certain value (threshold). Is to do. In other words, each synapse has a different connection strength and decides whether or not to transmit a signal.

An artificial neural network (ANN), a field of artificial intelligence, is a mathematical model that mimics the brain structure (neural network) of biology (usually human). That is, the artificial neural network is implemented by mimicking the information processing and transmission process of such biological neurons. Similar to the way the human brain solves problems, a neural network has excellent parallelism because each neuron operates independently. In addition, since information is scattered across many connecting lines, even if a problem occurs in some neurons, it does not affect the whole, so it is resistant to a certain level of errors and has the ability to learn about a given environment.

Deep neural network can be seen as a descendant of artificial neural network, and it is the latest version of artificial neural network with success cases in areas where numerous artificial intelligence technologies have failed in the past by transcending existing limitations. Looking at the details of modeling an artificial neural network by mimicking a biological neural network, in terms of processing unit, biological neurons are nodes, and in connection, synapses are weights. It was modeled as shown in Table 1.

Biological neural network Artificial neural network Cell body Node Dendrites Input Axon Output Synapse Weight

2 is a diagram illustrating a layer structure of an artificial neural network.

Just as human biological neurons are connected to each other through a synapse, even in the case of artificial neural networks, individual neurons are connected to each other through synapses, so that a plurality of layers are connected to each other. It can be updated by weight. In this way, it is used in fields for learning and cognition with its multi-layered structure and connection strength.

Each node is connected by weighted links, and the whole model learns while adjusting the weights repeatedly. The weights represent the importance of each node as a basic means for long-term memory. In simple terms, the artificial neural network trains the entire model by initializing these weights and adjusting the weights by updating them with the dataset to be trained. After training is complete, when new inputs come in, appropriate outputs are inferred. The learning principle of artificial neural networks can be viewed as a process in which intelligence is formed from the generalization of experience, and is accomplished in a bottom-up manner. In FIG. 2, when there are two or more intermediate layers (i.e., 5 to 10), the layer is considered to be deep and is referred to as a deep neural network, and the learning and inference model made through such a deep neural network can be referred to as deep learning. have.

The artificial neural network can play a certain role even if it has one intermediate layer (commonly referred to as'hidden layer') excluding inputs and outputs, but if the complexity of the problem increases, the number of nodes or layers You have to increase the number. Among them, it is effective to increase the number of layers to take a multi-layered structure model, but the scope of application is limited due to the limitation that efficient learning is impossible and the amount of computation for learning the network is large.

However, by overcoming the existing limitations as above, the artificial neural network can take a deep structure. As a result, it is possible to construct a complex and expressive model, and breakthrough results have been announced in various fields such as voice recognition, face recognition, object recognition, and text recognition.

A deep neural network (DNN) is an artificial neural network (ANN) composed of several hidden layers between an input layer and an output layer. Deep neural networks, like general artificial neural networks, can model complex non-linear relationships. For example, in a deep neural network structure for an object identification model, each object can be expressed as a hierarchical structure of basic elements of an image. In this case, the additional layers may gather features of the progressively gathered lower layers. This feature of deep neural networks makes it possible to model complex data with fewer units than similarly performed artificial neural networks.

Previous deep neural networks have usually been designed as front-end neural networks, but recent studies have successfully applied deep learning structures to recurrent neural networks (RNNs). An example is the application of deep neural network structure to the field of language modeling. In the case of a convolutional neural network (CNN), not only has it been well applied in the field of computer vision, but each successful application case is well documented. More recently, convolutional neural networks have been applied to the field of acoustic modeling for Automatic Speech Recognition (ASR), and are evaluated as being applied more successfully than existing models. Deep neural networks can be trained with standard error backpropagation algorithms. In this case, the weights may be updated through stochastic gradient descent using the following equation.

3 is a diagram illustrating a block diagram of a learning device for learning a user's motion intention using an artificial neural network according to the present invention.

Referring to FIG. 3, a learning apparatus 300 for learning a user's motion intention using an artificial neural network may include a communication unit 310 and a processor 320. The communication unit 310 and the processor 320 are electrically connected in the learning device 300 for learning the user's motion intention.

The communication unit 310 receives a signal measuring the user's movement from the motion sensor unit through wireless communication or the like. The processor 320 may output (or estimate, determine) the user's motion intention corresponding to the signal measured by the user's motion using the previously learned artificial neural network. And, when the communication unit 310 receives information on the EEG signal measured for the user from the EEG sensor unit, the processor 320 determines the user's operation intention corresponding to the information on the EEG signal. It is determined whether it does not match the intention of the operation. If they do not match, the processor 320 modifies the output label (or correct answer) of the user's motion intention to the user's motion intention corresponding to the information on the EEG signal and inputs it to the previously learned artificial neural network.

The processor 320 learns to recognize and correct the label (or correct answer) of the user's motion intention output from the previously learned artificial neural network to the user's motion intention corresponding to the information on the EEG signal. The processor 320 determines whether the user's operation intention corresponding to the information on the EEG signal matches the outputted user's operation intention, the user's operation corresponding to the information on the EEG signal and the information on the EEG signal. It may be determined based on a predefined mapping relationship between the intention, the user's motion intention corresponding to the information on the signal measuring the user's motion, and the output user's motion intention. Here, the predefined mapping relationship may include a relationship corresponding to each row in Table 2 below.

Frequency of the EEG signal received from the EEG sensor unit User's motion intention estimated from EEG signals Information on a signal that measures the user's movement received from a motion sensor unit (eg, a root degree sensor unit, an acceleration sensor unit, etc.) User's motion intention output through pre-learned artificial neural network First frequency Intent to act A First motion reaction Intent to act A 2nd frequency Intent B action Second motion reaction Intent B action 3rd frequency Intent C behavior 3rd motion reaction Intent C behavior

Referring to Table 2, the predefined mapping relationship or rule is that the frequency of the EEG signal received from the EEG sensor unit is the first frequency, and the user’s motion intention corresponding to the first frequency is A. In this case, from the motion sensor unit. The information on the received signal measuring the user's motion is a first motion response, and the user's motion intention corresponding to the first motion is also motion A. For example, referring to Table 2, it is assumed that the processor 320 outputs that the user's motion intention is the B motion as intended through the previously learned artificial neural network. In this case, if the frequency of the EEG signal received from the communication unit 310 by the processor 320 is the first frequency, the user's operation intention corresponding to the first frequency is the A operation, so it is consistent with the outputted user's operation intention, the B operation. You can judge that you don't. Then, the processor 320 recognizes that there is an error in the user's motion intention output through the pre-learned artificial neural network, and assigns a label (or correct answer) of the user's motion intention to the user's corresponding EEG signal information. It is corrected to the motion intention (B motion) and input into the previously learned artificial neural network. In this way, it is proposed to generate a blinking stimulus to provide an answer to the learning system by monitoring the EEG generated in the user's brain when the user looks at the stimulus, and the learning system to incrementally learn using this answer.

Here, the measured EEG signal may correspond to a signal caused by a blinking stimulus. The processor 320 may determine the user's motion intention corresponding to the measured signal for the user's motion when it is determined that the signal measured for the user's motion is greater than or equal to a predefined feature value. The artificial neural network may include deep neural networks (DNN). In addition, the EEG sensor unit may include an Electroencephalography (EEG) sensor. The motion sensor unit may include an EMG sensor unit or an acceleration sensor unit. The signal measured for the user's movement may include the user's EMG signal or an acceleration signal. Hereinafter, for convenience of explanation, the motion sensor unit will be described assuming the EMG sensor unit.

4 is a flowchart briefly explaining a method of learning a user's motion intention using an artificial neural network according to the present invention.

Referring to FIG. 4, an apparatus 300 for learning a user's motion intention using an artificial neural network receives information about a signal measuring a user's motion from a motion sensor unit. The apparatus 300 for learning the user's motion intention using an artificial neural network outputs the user's motion intention corresponding to the information on the signal measured by the user's motion using the previously learned artificial neural network. The apparatus 300 for learning the user's motion intention using an artificial neural network receives information on the EEG signal measured for the user from the EEG sensor unit. Further, the apparatus 300 for learning the user's motion intention using an artificial neural network determines whether the user's motion intention corresponding to the information on the EEG signal matches the output user's motion intention.

If it is determined that they do not match, the output label for the user's motion intention may be modified to the user's motion intention corresponding to the information on the EEG signal and input to the previously learned artificial neural network.

In addition, the apparatus 300 for learning the user's motion intention using the artificial neural network may learn to recognize the output label of the user's motion intention as the user's motion intention corresponding to the information on the EEG signal. The apparatus 300 for learning the user's motion intention using an artificial neural network determines whether the user's motion intention corresponding to the information on the EEG signal matches the output user's motion intention, the information on the EEG signal, Determination based on a predefined mapping relationship between the user's motion intention corresponding to the information on the EEG signal, the user's motion intention corresponding to the information on the signal measuring the user's motion, and the output user's motion intention can do. Here, the predefined mapping relationship is shown in Table 2 above.

The measured EEG signal may correspond to a signal caused by a blinking stimulus. The apparatus 300 for learning the user's motion intention using an artificial neural network performs the step of outputting the user's motion intention when it is determined that the measured signal for the user's motion is greater than or equal to a predefined feature value. The predefined feature value may include an average signal strength of an EMG signal. The motion sensor unit and the EEG sensor unit are portable and wearable by the user.

5 is a diagram illustrating a system including a device 300 for learning a user's motion intention using an artificial neural network according to the present invention.

As shown in FIG. 5, it is assumed that the user 500 (for example, a person with a physical disability or an elderly person) is driving. The learning device 300 may receive a signal for an EMG response from the EMG sensor 510. The learning device 300 estimates or outputs a motion intention of the user based on an EMG classification algorithm using a previously learned artificial neural network. In this case, the learning device 300 receives information on the EEG signal (such as information on the EEG frequency measured for the user) from the EEG sensor 520 of the user. The learning device 300 determines whether the output user's motion intention matches the user's motion intention corresponding to the received brainwave frequency information. If they do not match, the learning device 300 performs incremental learning using the received data. That is, the learning device 300 modifies the output label for the user's motion intention to the user's motion intention corresponding to the information on the EEG signal and inputs it to the previously learned artificial neural network, and the outputted user's It may be learned to recognize the label of the motion intention as the motion intention of the user corresponding to the information on the EEG signal.

EEG sensor 520 is the brain of the user 500 as the user 500 views any one of a plurality of flashing stimulus displays (head-up display) 530, 531, 532, 533 provided in the vehicle. The EEG frequency measured at can be calculated. Each display is provided in a different position in the vehicle, and the frequencies of the signals blinking in each display are not multiples. In addition, it is assumed that the operation intention of the user corresponding to the display corresponding to each location is differently matched, and that the user 500 recognizes this matching relationship.

For example, suppose that the user 500 moves his arm upward in order to intend an operation A (eg, to play music) among the functions inside the vehicle. The user 500 looks at the blinking stimulus on the display at the corresponding position corresponding to the operation A. However, the user's motion intention, which is output by using an artificial neural network (or EMG classification algorithm) that has been previously learned by receiving information on the EMG response in which the user's arm is moved upward, is the A motion ( For example, if the operation B (turning on the air conditioner), which is a different operation instead of playing music), the learning device 300 processes this as an incorrect result due to noise or the like for the movement of the user 500. The learning apparatus 300 incrementally learns by inputting the user's true motion intention, A motion (eg, playing music), into a pre-learned artificial neural network based on the received brainwave frequency information.

6 shows a graph of brain waves detected by the EEG sensor 520 when the user 500 views a stimulus image of 15 Hz.

When the user 500 watches a stimulation image of 15 Hz, a frequency proportional to the frequency of the stimulation image is observed in the brain. For example, the EEG sensor 520 may detect brain waves of 15 Hz, 30 Hz, and 45 Hz, which are multiples of 15 Hz, as shown in FIG. 6. When the user 500 watches a stimulus image of 15 Hz, a frequency proportional to the frequency of the stimulus image is observed in the brain, so a display (head-up display) 530, 531, 532 for a plurality of flashing stimuli provided in a vehicle , 533) The frequencies of the blinking signals in each should not have a multiple relationship with each other. For example, the frequency of the blinking signal in each of the displays (head-up display) 530, 531, 532, and 533 may be 15Hz, 17Hz, 18Hz, and 19Hz, respectively.

7 is a flowchart illustrating a method of learning a user's motion intention using an artificial neural network according to the present invention.

Referring to FIG. 7, it is assumed that a user 500 (for example, a physically disabled person) is driving while wearing an EEG measurement band (or EEG sensor unit) for EEG measurement and an EMG sensor unit for EMG measurement. . The EEG sensor unit and the EMG sensor unit worn by the user perform monitoring and determine whether the feature value exceeds a predetermined threshold value by extracting the feature value. If the EMG sensor unit determines that the feature value for the extracted EMG response exceeds a predetermined threshold value, the EMG sensor unit transmits it to the communication unit 300 of the learning device 300 by a wireless method or the like. Here, the feature value may be the average intensity of the EMG signal.

The processor 320 of the learning device 300 outputs or estimates the user's motion intention using the EMG classification algorithm of the artificial neural network that has been previously learned about the EMG information received from the EMG sensor unit.

In addition, the communication unit 310 of the learning device 300 receives information on the EEG signal measured for generating an SSVEP stimulus for operation confirmation from the EEG sensor unit. In this case, the processor 320 of the learning device 300 may determine the user's operation intention corresponding to the information on the EEG signal received by the EEG classification algorithm or a preset rule. The processor 320 of the learning device 300 determines whether the user's motion intention output by the EMG algorithm classifier and the user's motion intention identified by the EEG classification algorithm (or a preset rule) match.

If not, the processor 320 of the learning device 300 corrects the label of the information on the corresponding EMG response. After modification, the processor 320 of the learning device 300 performs an EEG algorithm device control, stores the observed EMG and SSVEP-responsive EEG in a data set for incremental learning to learn a previously learned artificial neural network.

As described above, a learning method for reducing errors that occur when a device is manipulated by using a machine learning algorithm model to recognize a user's intention and a learning apparatus for the same are proposed.

The embodiments described above are those in which components and features of the present invention are combined in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, it is also possible to constitute an embodiment of the present invention by combining some components and/or features. The order of operations described in the embodiments of the present invention may be changed. Some configurations or features of one embodiment may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is obvious that the embodiments may be configured by combining claims that do not have an explicit citation relationship in the claims or may be included as new claims by amendment after filing.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the present invention. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (24)

Receiving, by a communication unit, information on a signal measuring the user's movement from the motion sensor unit;
Outputting, by a processor, a motion intention of a user corresponding to information on a signal obtained by measuring the motion of the user using a previously learned artificial neural network;
Receiving, by the communication unit, information on the EEG signal measured for the user from the EEG sensor unit; And
When the user's operation intention corresponding to the information on the EEG signal and the outputted user's operation intention do not match, the processor sets a label for the output user's operation intention corresponding to the information on the EEG signal. A method of learning the user's motion intention using the artificial neural network, comprising the step of modifying the motion intention of the user and inputting it into the previously learned artificial neural network. The method of claim 1,
The method of learning a user's motion intention using an artificial neural network, further comprising the step of learning, by the processor, to recognize the output label of the user's motion intention as the motion intention of the user corresponding to the information on the EEG signal . The method of claim 1,
The processor determines whether the user's operation intention corresponding to the information on the EEG signal matches the output user's operation intention, the information on the EEG signal, and the user's operation intention corresponding to the information on the EEG signal. , The user using an artificial neural network, further comprising the step of determining based on a predefined mapping relationship between the user's motion intention corresponding to the information on the measured signal of the user's motion and the output user's motion intention How to learn the intention of motion. The method of claim 3,
The predefined mapping relationship includes a relationship corresponding to each row in the following table. A method of learning a user's motion intention using an artificial neural network.

The method of claim 4,
The first, second, and third frequencies are not a multiple relationship between each other, characterized in that, using an artificial neural network to learn the user's motion intention. The method of claim 1,
The measured EEG signal corresponds to a signal caused by a blinking stimulus, and a method of learning a user's motion intention using an artificial neural network. The method of claim 1,
The step of outputting the motion intention of the user by the processor is performed when the measured signal for the motion of the user is determined to be greater than or equal to a predefined feature value. A method of learning the motion intention of the user using an artificial neural network. The method of claim 7,
The predefined feature value includes an average signal strength of an EMG signal. A method of learning a user's motion intention using an artificial neural network. The method of claim 1,
The artificial neural network includes a deep neural network (DNN), a method of learning a user's motion intention using an artificial neural network. The method of claim 1,
The motion sensor unit and the EEG sensor unit, characterized in that the user is portable and wearable, a method for learning a user's motion intention using an artificial neural network. A communication unit for receiving information on a signal measuring the user's movement from the motion sensor unit; And
Including a processor for outputting the user's motion intention corresponding to the information on the signal measured the user's motion using a previously learned artificial neural network,
The communication unit receives information on the EEG signal measured for the user from the EEG sensor unit,
When the user's operation intention corresponding to the information on the EEG signal and the outputted user's operation intention do not match, the processor sets a label of the output user's operation intention to the user corresponding to the information on the EEG signal A device for learning the user's motion intention using the artificial neural network, which is modified as the motion intention of and inputs to the previously learned artificial neural network. The method of claim 11,
The processor learns to recognize the output label of the user's motion intention as the user's motion intention corresponding to the information on the EEG signal, using an artificial neural network to learn the motion intention of the user. The method of claim 11,
The processor determines whether the user's operation intention corresponding to the information on the EEG signal matches the output user's operation intention, the information on the EEG signal, the user's operation intention corresponding to the information on the EEG signal, and An apparatus for learning a user's motion intention using an artificial neural network, which determines based on a predefined mapping relationship between a user's motion intention corresponding to information on the signal measuring the motion of the user. The method of claim 13,
The predefined mapping relationship includes a relationship corresponding to each row in the following table. An apparatus for learning motion intention of a user using an artificial neural network.

The method of claim 14,
The first, second, and third frequencies are not a multiple relationship between each other, characterized in that, using an artificial neural network to learn a user's motion intention. The method of claim 11,
The measured EEG signal corresponds to a signal caused by a blinking stimulus, and a device for learning a user's motion intention using an artificial neural network. The method of claim 11,
The processor outputs the user's motion intention corresponding to the signal measured for the user's motion, when the measured signal for the user's motion is determined to be greater than or equal to a predefined feature value, by using an artificial neural network. A device that learns motion intent. The method of claim 17,
The predefined feature value includes an average signal strength of an EMG signal. An apparatus for learning a user's motion intention using an artificial neural network. The method of claim 11,
The artificial neural network includes a deep neural network (DNN), a device for learning a user's motion intention using an artificial neural network. The method of claim 11,
The motion sensor unit and the brain wave sensor unit, characterized in that the user is portable and wearable, using an artificial neural network to learn the user's motion intention. The method of claim 11,
The EEG sensor unit includes an Electroencephalography (EEG) sensor, and uses an artificial neural network to learn a user's motion intention. The method of claim 11,
The motion sensor unit includes an EMG sensor unit or an acceleration sensor unit, and uses an artificial neural network to learn motion intention of a user. The method of claim 22,
The signal measured for the user's motion includes the user's EMG signal or an acceleration signal. A computer-readable recording medium in which a program for executing a method of learning a user's motion intention using the artificial neural network according to any one of claims 1 to 10 on a computer is recorded.

Images