KR102237390B1 - Self-error recognition and learning system based on smart machine using wireless eeg measurement system - Google Patents

Abstract

The present invention relates to a system for recognizing and learning a self-error based on a smart machine using a wireless brain wave measurement system. The system for recognizing and learning a self-error based on a smart machine using a wireless brain wave measurement system comprises: a brain wave measurement system for analyzing an event-related potential pattern by collecting brain waves of a user generated by an event performed in a smart machine; and a smart machine for recognizing a self-error related to a decision making corresponding to the event and performing artificial intelligence-based learning by considering a result of analyzing the event-related potential pattern provided from the brain wave measurement system. The smart machine can recognize a decision making error and perform relearning when P300 is included in the result of analyzing the event-related potential pattern.

Description

Self-error recognition and learning system based on a smart machine using a wireless EEG measurement system {SELF-ERROR RECOGNITION AND LEARNING SYSTEM BASED ON SMART MACHINE USING WIRELESS EEG MEASUREMENT SYSTEM}

The present application relates to a self-error recognition and learning system based on a smart machine using a wireless EEG measurement system.

In the case of a conventional smart machine, there is a problem in that the user must directly control the user's body or an exercise organ such as a hand or reset the system itself in order to correct the abnormal behavior. For example, in the case of a speech recognition system, a speech recognition system that incorrectly judges a user's specific vocabulary once tends to repeatedly judge the same vocabulary.

Current smart machines cannot recognize their own errors in an environment where only the same information is continuously given. In order to correct this, in the end, the user must transmit control and additional information about it in a way other than direct voice.

In addition, in the case of an autonomous driving system, a very dangerous situation can be caused when the autonomous driving system performs an abnormal operation. The problem is that in general, the user recognizes the problem and reacts physically while driving, and the princess time required to move is more than 1 second, and the situation quickly deteriorates before the vehicle is directly controlled, and control may become impossible.

In addition, the smart machine operates according to the user's intention without direct control of the user. This is possible because the decision-making process based on modern advanced machine learning and deep learning shows very flexible and high accuracy. However, the problem arises when the smart machine does not operate according to the user's intention. A smart machine that decides to operate based on machine learning or deep learning based on the surrounding environment and all inputted information continues to operate inconsistent with the user's intention without additional information and control. The error of the smart machine is corrected through the method.

The technology behind the present application is disclosed in Korean Patent Publication No. 10-1962276.

The present application is to solve the problems of the prior art described above, by observing the potential P300 related to the real-time event of the user, a wireless EEG measurement capable of recognizing and correcting the non-ideal operation of the smart machine without direct control and input from the user. It aims to provide a smart machine-based self-error recognition and learning system using the system.

The present application is to solve the problems of the prior art described above, the user does not transmit information through a physical method of control, through the event-related potential observed from the brain waves, the smart machine to recognize the abnormal operation and error by itself. The aim is to provide a smart machine-based self-error recognition and learning system using a wireless EEG measurement system that can be corrected.

The present application is intended to solve the problems of the prior art described above, and when a smart machine that has been operating normally and behaves abnormally at an instant, an event-related potential is induced in the brain of the user who recognizes this, which is transmitted as learning information of the smart machine. The purpose is to provide a smart machine-based self-error recognition and learning system using a wireless EEG measurement system capable of self-recognizing and correcting the smart machine itself.

However, the technical problem to be achieved by the embodiments of the present application is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, in a smart machine-based self-error recognition and learning system using a wireless EEG measurement system according to an embodiment of the present application, the user's An EEG measurement system that collects EEG and analyzes an event-related potential pattern, and self-error recognition related to decision-making corresponding to the event and AI-based learning in consideration of the event-related potential pattern analysis result provided from the EEG measurement system Including a smart machine that performs, the smart machine, when the event-related potential pattern analysis result includes P300, it is possible to recognize the decision-making error and perform re-learning.

In addition, the smart machine performs a decision and transmits event information related to the decision-making to the EEG measurement system, and the EEG measurement system, when the event information is received, is applied to the event-related potential pattern. It can be determined whether the P300 peak is included.

In addition, when P300 is included in the event-related potential pattern analysis result and the decision is determined to be an error, the smart machine receives information related to the P300 peak to the EEG measurement system, and receives information related to the P300 peak. It is possible to grasp the result of the existing decision before the preset time as the time at which is received.

In addition, the smart machine may store information related to the decision-making as artificial intelligence-based learning data when the decision-making is determined to be normal because P300 is not included in the event-related potential pattern analysis result.

In addition, the smart machine, when the selection of the existing decision making has two options, determines whether to re-decision to proceed with the opposite option rather than the option linked to the existing decision in which the event was performed, and If there are three or more options when making a decision, information on a new decision can be provided.

In addition, the smart machine may store a re-decision determination result determined in response to an option included in the information related to the existing decision-making and information related to the new decision-making result as artificial intelligence-based learning data.

In addition, the EEG measurement system includes a sensor unit for receiving EEG detected through an electrode, a wireless communication device transmitting an analysis result of an event-related potential pattern to the smart machine, and filtering the EEG signal in consideration of a preset frequency domain. It may be determined whether the P300 peak is included in the event-related potential pattern by using the filtering unit performing the operation and the EEG signal passed through the filtering unit.

The self-error recognition and learning method of a smart machine using a wireless EEG measurement system according to an embodiment of the present application includes the steps of performing a decision in the smart machine, and transmitting information on the event related to the decision making to the EEG measurement system. Analyzing an event-related potential pattern using the user's EEG generated by the event performed in the smart machine in the EEG measurement system, when P300 is included in the analysis result of the related potential pattern in the smart machine , Recognizing the decision-making error and performing relearning, and when the P300 is not included in the analysis result of the related potential pattern, the decision-making normal is recognized and the information related to the decision-making is stored as artificial intelligence-based learning data. It may include the step of.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present application. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present application, it is possible to prevent unnecessary problems through quick correction by allowing self-recognition of the abnormal behavior of the smart machine.

According to the above-described problem solving means of the present application, since direct correction of the user is not required in order to correct the abnormal behavior of the smart machine, the convenience is maximized and the smart machine is more convenient even for users who are inconvenient with classic hands and arms. Can be used.

According to the above-described problem solving means of the present application, an economical effect is expected by enhancing user-friendliness and promoting purchase by supplementing the fatal shortcomings of the smart machine.

According to the above-described problem solving means of the present application, in the case of brain waves, since the temporal resolution is very excellent, the use of the event-related potential makes it possible to immediately recognize the abnormal behavior of the smart machine in real time. This has the effect of very lowering the probability of accidents in safety-related smart machines such as autonomous vehicles.

According to the above-described problem solving means of the present application, information on the user's state, such as the P300 event-related potential, is more accurate and can help to implement a smart machine based on machine learning and deep learning like a person.

However, the effect obtainable in the present application is not limited to the above-described effects, and other effects may exist.

1 is a schematic configuration diagram of a smart machine-based self-error recognition and learning system using a wireless EEG measurement system according to an embodiment of the present application.
2 is an overall operation flowchart of a method for self-error recognition and learning of a smart machine using a wireless EEG measurement system according to an embodiment of the present application.
3 is a schematic block diagram of an EEG measurement system according to an embodiment of the present application.
4 is a schematic configuration diagram of an EEG measurement system according to an embodiment of the present application.
5 is a view for explaining EEG measurement using an electrode of the EEG measurement system according to an embodiment of the present application.
6 is a diagram showing a filtered signal and a conventional raw signal of the EEG measurement system according to an embodiment of the present application.
7 shows experimental methods and results for various situations in which a potential signal related to a P300 event is detected according to an embodiment of the present application.
8 shows the distribution of big data displayed by detecting a total of 400 P300 EEG patterns for 10 subjects according to an embodiment of the present application.
9 is a view for explaining a process of self-recognizing errors based on a smart machine and solving an error of a smart machine by itself through an event-related potential of a learning system according to an embodiment of the present application.
10 is a schematic operation flowchart of a method for self-error recognition and learning of a smart machine using a wireless EEG measurement system according to an embodiment of the present application.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present application. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted in order to clearly describe the present application, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the present specification, when a part is said to be "connected" with another part, it is not only the case that it is "directly connected", but also "electrically connected" or "indirectly connected" with another element interposed therebetween. "Including the case.

Throughout the present specification, when a member is positioned "on", "upper", "upper", "under", "lower", and "lower" of another member, this means that a member is located on another member. This includes not only the case where they are in contact but also the case where another member exists between the two members.

In the entire specification of the present application, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

Hereinafter, a smart machine-based self-error recognition and learning system using a wireless EEG measurement system according to an embodiment of the present application will be referred to as the present system 1 for convenience of description.

1 is a schematic configuration diagram of a smart machine-based self-error recognition and learning system using a wireless EEG measurement system according to an embodiment of the present application.

Referring to FIG. 1, the system 1 may include a smart machine 10 and an EEG measurement system 20. The smart machine 10 and the EEG measurement system 20 may be linked through a network.

According to an embodiment of the present application, the system 1 measures an EEG, checks an event-related potential, and then transmits it to the outside through wireless communication. Errors can be identified and corrected.

In addition, the present system 1 can check whether the smart machine 10 is right or wrong for an operation that the smart machine 10 has determined by itself according to a decision tree based on machine learning and deep learning, according to the user's reaction, that is, the potential associated with the P300 event. In addition, the system 1 can be quickly corrected in case of an incorrectly determined operation, and this process can be used again as a learning data set of the smart machine 10 to increase the decision making accuracy of the smart machine 10.

For reference, an event-related potential is a potential difference that appears as a result of a response to a certain stimulus in the brain. Event-related potentials that are closely related to the user's decision-making and stimulus judgment process are triggered immediately according to the instantaneous judgment of the brain even if the user does not consciously and deeply think and act.

Typically, the P300 potential component in the event-related potential is a potential component caused when an abnormal stimulus is intermittently intermittently presented while a natural, problem-free, normal visual, auditory, or tactile stimulus that is expected to be judged by the user is continuously presented. In addition, the occurrence of P300 is endogenously related to stimuli such as dopamine and norepinephrine and common hormones related to cognition, so it occurs in everyone.

Examples of networks for sharing information between the smart machine 10 and the EEG measurement system 20 include 3rd Generation Partnership Project (3GPP) networks, Long Term Evolution (LTE) networks, 5G networks, and World Interoperability for Microwave Access (WIMAX). Network, Wired and Wireless Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth Network, Wifi Network, NFC (Near) Field Communication) network, satellite broadcasting network, analog broadcasting network, digital multimedia broadcasting (DMB) network, and the like, but are not limited thereto.

According to an embodiment of the present application, the smart machine 10 performs self-error recognition and artificial intelligence-based learning related to decision making corresponding to the event in consideration of the result of analyzing the event-related potential pattern provided from the EEG measurement system 20. You can do it. In addition, the smart machine 10 may recognize an error by determining potential information related to a user's event according to the non-abnormal operation by itself. In addition, when P300 is included in the event-related potential pattern analysis result, the smart machine 10 may recognize a decision-making error and perform relearning. In addition, the smart machine 10 may use the P300 event-related potential as learning conditions and data for artificial intelligence (eg, machine learning or deep learning). In addition, the smart machine 10 may improve the operation of the smart machine 10 itself or respond to it based on the recognized error.

A smart machine is an intelligent system in which objects or devices collect information about the surrounding environment in real time without direct human control, and are autonomously moving devices that accurately understand the user's intentions and listen to requests. For example, the smart machine may include an autonomous vehicle, a virtual personal assistant, a voice recognition system, a smart robot prosthetic limb, an artificial intelligence system, and the like.

According to an exemplary embodiment of the present disclosure, the EEG measurement system 20 may collect a user's EEG generated due to an event performed by the smart machine 10 to analyze an event-related potential pattern. Here, the event-related potential is one of the potentials detectable in the EEG and is closely related to the user's decision-making and stimulus judgment process. Event-related potentials are endogenous potentials that occur according to individual responses and judgments to each stimulus regardless of the physical properties of the stimulus. Typically, the P300 potential component in the event-related potential is a potential component caused when an abnormal stimulus is intermittently intermittently presented while a natural, problem-free, normal visual, auditory, or tactile stimulus that is expected to be judged by the user is continuously presented. In simple terms, the P300 is generated by processing information about unexpected external stimuli or information such as surprise by the user. In P300, 300 means that the maximum peak is displayed after approximately 300 ms from the time point at which the abnormal stimulus is intervened.

That is, when P300 is observed in the user's EEG, the smart machine 10 receives information on the user's EEG from the EEG measurement system 20 through immediate wireless communication. Accordingly, the smart machine 10 may self-recognize that the operation method or the behavior determination method at that time is contrary to the user's intention, and correct the error or take a response thereto.

Hereinafter, the process in which the smart machine 10 self-checks and improves abnormal behaviors or errors through the EEG measurement system 20 that measures EEG through FIG. 2 and checks the event-related potential and transmits it to the outside through wireless communication will be described in detail. I want to explain.

2 is an overall operation flowchart of a method for self-error recognition and learning of a smart machine using a wireless EEG measurement system according to an embodiment of the present application.

For example, referring to FIG. 2, in step S110, the smart machine 10 may perform its own decision making. The smart machine 10 performs machine learning and deep learning-based decision making itself every moment. For example, when the smart machine 10 is a voice recognition system, a decision to listen to a user's voice and provide an answer may be performed. In addition, when the smart machine 10 is an autonomous vehicle, it is possible to make a decision on which road to drive on a forked road. In addition, when the smart machine 10 is a robot prosthetic, various decisions such as whether to raise the left arm or the right arm may exist according to the user's neural data.

In step S120, the smart machine 10 may make a decision on a specific event and transmit information on the specific event to the EEG measurement system 20. For example, the moment the smart machine 10 makes its own decision, the smart machine 10 may transmit information about the self-decision to the EEG measurement system 20.

The EEG measurement system 20 may measure the user's EEG when the smart machine 10 receives information on which decision-making has been completed. For example, when the smart machine 10 is an autonomous vehicle, the smart machine 10 determines that it will turn left on a forked road by itself, and when an event of driving in a left turn is performed, the smart machine 10 will receive the corresponding information. Can be delivered to the EEG measurement system 20. The EEG measurement system 20 may measure EEG that responds to the smart machine 10, that is, the autonomous vehicle making a decision and performing an event.

In step S130, when information on a specific event is received, the EEG measurement system 20 may determine whether the P300 peak is included in the event-related potential pattern. In other words, when information about a specific event is received from the smart machine 10, the EEG measurement system 20 measures the user's EEG and determines whether the P300 peak is included in the event-related potential pattern included in the EEG signal. can do.

In addition, the EEG measurement system 20 may stop EEG measurement when the P300 peak is found within a preset time range after EEG measurement. For example, the EEG measurement system 20 may check whether the P300 peak rises within a preset time range of 300 msec to 500 msec after the EEG measurement and stop the measurement.

On the other hand, when the P300 peak is included (occurred) in the event-related potential pattern, the smart machine 10's own decision may be determined to be an error. For reference, the event-related potential P300 can be induced and observed in the user's brain when the smart machine operates contrary to the user's intention or performs unexpectedly unexpected movements.

On the other hand, when the P300 peak is not included (occurred) in the event-related potential pattern, the smart machine 10's own decision may be determined to be normal (no error).

For example, when it is determined that the P300 peak is not included (occurred) in the event-related potential pattern in the EEG measurement system 20, the smart machine 10 stores information related to decision making as artificial intelligence-based learning data. I can. The information related to decision making may include information related to a specific event generated by a decision made by the smart machine 10 itself and a decision, and information about a user's EEG signal generated by the event. In other words, the information related to decision making may include information (data) generated by the smart machine 10 and the EEG measurement system 20 in steps S110 to S130.

In step S130, when the EEG measurement system 20 analyzes the event-related potential pattern and the P300 peak is not caused, the smart machine 10 determines the case for the overall decision-making process of step S190. And a process of repeatedly storing and storing the deep learning learning data, and the process may return to step S110. In other words, when the smart machine 10 receives information from the EEG measurement system 20 that the P300 potential has not been induced, it confirms that the existing decision-making was reasonable, and the information and cases for the overall decision-making are machine-learning and deep-learning. It can be remembered as learning data, learned, and then returned to the routine.

As another example, when the EEG measurement system 20 determines that the P300 peak is included (occurred) in the event-related potential pattern, the smart machine 10 may determine that the decision made in step S110 is an error.

In step S140, when the P300 peak is included (occurred) in the event-related potential pattern, the smart machine 10 may receive information related to the P300 peak from the EEG measurement system 20. In other words, the EEG measurement system 20 may analyze the P300 event-related potential pattern and transmit information on whether or not the P300 potential is triggered again to the smart machine 10.

In step S150, when receiving information related to the P300 peak from the EEG measurement system 20, the smart machine 10 makes an existing decision prior to a preset time when the information (information related to the P300 peak) is received. You can grasp the results. When the smart machine 10 receives information that the P300 potential has been induced, it may have an opportunity to check whether a decision made by itself 300 msec before the reception time is correct.

As an example, when the smart machine 10 is an autonomous vehicle and receives information related to the P300 peak from the EEG measurement system 20, the smart machine 10 receives information related to the P300 peak at a preset time (e.g. For example, it is possible to identify the case for the entire existing decision making before 300msec). The case for the overall existing decision making before the preset time may include a decision on choosing which route to drive on a forked road, a decision on choosing which route to drive at a crossroad, and making a decision related to lane change. .

On the other hand, when there is only one case for the entire existing decision-making before a preset time, the smart machine 10 may only perform a check for the corresponding decision-making. On the other hand, when there are a plurality of cases for the entire existing decision-making before a preset time, the smart machine 10 may perform all checks for the decision-making for each of the plurality of cases.

In step S160, the smart machine 10 may determine whether there were two options when selecting an existing decision. The smart machine 10 may perform step S170 or step S180 based on the number of options when selecting an existing decision. In addition, when there are a plurality of existing decision-making cases, the smart machine 10 may perform step S170 or step S180 for each of the existing decision-making situations.

In step S170, when there are two options when selecting the existing decision, the smart machine 10 determines whether to proceed with the re-decision making with the opposite option instead of the existing decision, and makes the decision. Events for can be performed. For example, the smart machine 10 is an autonomous vehicle, and when selecting an existing decision, the option may be a left turn or a straight line. In the smart machine 10, when the existing decision was selected, the option was to turn left or go straight, but after making a decision by making a left turn, an event of driving in a left turn may occur. The smart machine 10 may determine whether to proceed with the decision-making by going straight, which is an opposite option to the existing decision-making, and may generate an event of driving by a U-turn after making the decision by going straight. That is, if there are two existing decision-making options, the smart machine 10 may determine whether to proceed with its own decision-making again with the opposite option, and then perform a decision on the corresponding decision.

In step S180, when there are three or more options when selecting an existing decision, the smart machine 10 may receive information on a new decision. The smart machine 10 may provide a question to a user for a new decision when there are three or more options when making a decision. In the smart machine 10, if the existing decision-making options are very diverse and it is difficult to determine a new option even with information on the user's P300 potential, the error correction process may be performed by immediately asking a user a question.

For example, the smart machine 10 is an autonomous vehicle, and when selecting an existing decision, options may include straight ahead, left turn, and U-turn. The smart machine 10 may decide to go straight, which is one of the options when selecting an existing decision, and generate an event of driving straight. The smart machine 10 may receive information on a new decision when three or more options, such as a straight forward, a left turn, and a U-turn, exist when selecting an existing decision.

As another example, when there are multiple cases (situation) of the existing decision making, that is, when there are a plurality of existing decision makings before a preset time (eg, 300 msec) as the reception time point, the smart machine 10 Step S170 or step S180 may be performed for each of the decision making.

For example, when there are two options for case (situation) 1 of the existing decision making, the smart machine 10 may perform its own decision again with the opposite option in response to case (situation) 1 of the existing decision-making. . In addition, when there are three or more options in case (situation) 2 of the existing decision-making, the smart machine 10 can perform re-decision-making by providing a new decision in response to case (situation) 2 of the existing decision-making. have.

According to the exemplary embodiment of the present disclosure, when there are three or more options when selecting a decision, the smart machine 10 may provide information on a new decision to a user terminal (not shown). The smart machine 10 may receive input information for new decision making from a user terminal (not shown). The smart machine 10 may receive input information for a new decision-making from a user terminal (not shown), determine whether to proceed with a new decision-making, which is user input information, and make a decision.

According to an embodiment of the present application, the smart machine 10 is A decision making menu may be provided to a user terminal (not shown). For example, a user terminal (not shown) downloads and installs an application program provided by the horse smart machine 10, and a decision making menu may be provided through the installed application.

The smart machine 10 It may include all types of servers, terminals, or devices that transmit and receive data, content, and various communication signals to and from a user terminal (not shown) through a network, and have functions of storing and processing data.

A user terminal (not shown) is a device that is interlocked with the smart machine 10 through a network, for example, a smartphone, a smart pad, a tablet PC, a wearable device, and a personal communication system (PCS). , Global System for Mobile communication (GSM), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), International Mobile Telecommunication (IMT)-2000, Code Division Multiple Access (CDMA)-2000, It may be all kinds of wireless communication devices such as W-Code Division Multiple Access (W-CDMA) and Wireless Broadband Internet (Wibro) terminals, and fixed terminals such as desktop computers and smart TVs.

In step S190, the smart machine 10 may store the re-decision decision result determined in response to the option included in the decision-making information and information related to the new decision-making result as artificial intelligence-based learning data. The smart machine 10 may store overall information about the decision-making results of steps S170 and S180 as artificial intelligence-based learning data and return to step S110. In other words, after the error correction process of steps S170 and S180 is performed, the smart machine 10 stores information on such a case as machine learning & deep learning learning data and returns to the routine. Information related to decision making is stored as learning data based on artificial intelligence, and the smart machine 10 performs continuous learning based on the learning data, so that the smart machine 10 can recognize it and perform correction.

The above routine (steps S110 to S190) based on the user's brain waves is performed very quickly. The reason is that based on wireless communication, it is very instantaneous, and the EEG detection itself is performed before humans physically act and is very fast. Through a rapid error detection process, it is possible to lower the probability of an accident in various dangerous situations, and it is possible to help improve accuracy by helping the learning process of the smart machine 10. Since the user can guide the learning process of the smart machine 10 without physical activity, it is very convenient and can be easily used even when the body is uncomfortable.

3 is a schematic block diagram of an EEG measurement system according to an embodiment of the present application, FIG. 4 is a schematic configuration diagram of an EEG measurement system according to an embodiment of the present application, and FIG. 5 is a schematic block diagram of an EEG measurement system according to an exemplary embodiment of the present application. It is a diagram for explaining the EEG measurement using the electrode of the EEG measurement system.

According to the exemplary embodiment of the present application, the EEG measurement system 20 may be attached to a part of the user's body to measure EEG. For example, a user's body part may be a scalp and a skull. The user's EEG can be read by attaching the electrode 2 to the user's scalp or using the electrode 2 that directly invades the scalp and skull. For example, referring to FIG. 5, the electrode 2 can be realized by attaching the electrode 2 to the scalp. The EEG detected through the electrode 2 is read through the EEG measurement system 20 as shown in FIG. 4.

Referring to FIG. 3, the EEG measurement system 20 may include a sensor unit 21, a wireless communication device 22, a filtering unit 23, and a determination unit 24. However, the configuration of the EEG measurement system 20 is not limited thereto. For example, the EEG measurement system 20 controls and sets up an Electrophysiology Chip including an Amplifer element that amplifies an EEG, an Analog-Digital Converter element that converts EEG in the form of an analog signal into a digital signal, and a device related to software. It may include a control unit and the like.

According to the exemplary embodiment of the present disclosure, the sensor unit 21 may receive an EEG detected through the electrode 2. The sensor unit 21 may be attached to the user's scalp or skull and connected to the electrode 2 for collecting EEG to receive the EEG collected from the electrode 2.

In addition, the wireless communication device 22 may transmit the analysis result of the event-related potential pattern to the smart machine 10.

The wireless communication device 22 may perform wireless communication with an external device and a smart machine 10.

In addition, the filtering unit 23 may perform filtering of the EEG signal in consideration of a preset frequency range (eg, 1-8 Hz). For example, P300 event-related potentials generated in response to unexpected stimuli can be more easily analyzed and detected only through bandpass filtering in a Microcontroller or Electrophysiology Chip.

6 is a diagram illustrating a filtered signal and a conventional raw signal of the EEG measurement system according to an embodiment of the present application.

6A may be an existing raw signal collected through the electrode 2. In addition, (b) of FIG. 6 may be an EEG signal polished through a 1-30 Hz band pass filter. Further, (c) of FIG. 6 may be an EEG signal polished through a band pass filter at 1-8 Hz. The P300 signal shown in FIG. 6 is a form that is difficult to analyze because various noises are mixed, and it is difficult to see a very clearly distinct peak in the 300 msec region in which the P300 peak highlighted in red should be detected.

As shown in FIG. 6, in the signal polished through a band pass filter at 1-8 Hz as shown in FIG. 6(c), a very distinct P300 peak can be confirmed compared to other parts in the 300msec region.

That is, the filtering unit 23 may remove various types of noise in order to detect a potential signal related to a faster P300 event, and may perform filtering of the EEG signal in a preset frequency region.

In addition, the determination unit 24 may determine whether the P300 peak is included in the event-related potential pattern by using the EEG signal passed through the filtering unit 23. The P300 peak detected by the determination unit 24 may be determined based on big data rather than a simple threshold value.

7 shows experimental methods and results for various situations in which a potential signal related to a P300 event is detected according to an embodiment of the present application.

The P300 is an EEG potential that is largely induced when an unexpected stimulus enters or contradicts the user's intention. When a general stimulus is repeatedly entered, an EEG waveform in the form of a non-target shown in (b) of FIG. 6 is detected. Conversely, when other unexpected stimuli such as red, orange, and yellow are suddenly applied to the user in (a) of FIG. 6, an EEG form such as the target shown in (b) of FIG. 6 is detected.

8 shows the distribution of big data displayed by detecting a total of 400 P300 EEG patterns for 10 subjects according to an embodiment of the present application.

Referring to FIG. 8, the P300 pattern and the P300 peak are basically uniformly detected around 300 msec, and there is no significant deviation even under various other conditions. Latency, which is a temporal point at which the P300 peak is detected, and P300 peak amplitude at that time, are the most basic information used for P300 pattern classification.

In addition, it is possible to detect and distinguish P300 potential from the user by checking whether there are negative peaks in front and behind the P300 peak and how large they are.

That is, the EEG measurement system 20 may measure EEG using electrodes 2 attached to the user's scalp and skull, separate noise based on the measured EEG, and check the event-related potential P300. In addition, the EEG measurement system 20 may wirelessly transmit information on the measured P300 event-related potential to devices such as the smart machine 10 through a network.

In addition, since the EEG measurement system 20 is implemented in a form in which wireless communication is possible after being connected to the electrode, the user can perform various activities in a very free state. In addition, it is possible to help drive the smart machine 10 through the EEG measurement system 20 that can be linked with various smart machines 10 without a line.

9 is a view for explaining a process of self-recognizing errors based on a smart machine and solving an error of a smart machine by itself through an event-related potential of a learning system according to an embodiment of the present application.

This system (1) can self-solve errors of smart machines through event-related potentials, and can be utilized in various ways. As an example, the smart machine 10 selects an optimal operation mode within the surrounding environment and given information. If the information on the event-related potential can be given immediately as soon as the smart machine 10 operates abnormally, various results can be produced.

In the case of an autonomous vehicle, which is a type of smart machine 10, a malfunction may soon lead to a major accident or human injury. Even in the case of self-driving cars that have been developed and released so far, it is said that it is recommended to hold the steering wheel rather than blindly to the self-driving car in the manual. When using the event-related potential, the moment the user's brain recognizes the abnormal behavior of the autonomous vehicle, which is the smart machine 10, the autonomous vehicle itself immediately recognizes and responds to an error and prepares for danger or accidents faster than holding the steering wheel. It can lower the probability of occurring.

In the case of the artificial intelligence assistant based on the speech recognition system, which is another representative smart machine (10), the probability of malfunction of the speech recognition system is the lowest point of the probability that the artificial intelligence assistant handles or answers the wrong task, so the correct operation rate of the artificial intelligence assistant Is even lower than that of a speech recognition system. In the case of a speech recognition system, the current accuracy is only 80% to 90%. In particular, since the pronunciation varies widely from user to user, a word that fails to recognize once has a high probability of continuing to fail speech recognition unless the user corrects the pronunciation and speaks it.

In this case, if the artificial intelligence assistant detects errors in the speech recognition system on his own, rapid correction and machine learning are possible, but the incident-related potential can provide a solution to this. If the speech recognition system does not understand the user's speech, the user accepts it as an abnormal behavior of the smart machine 10, and an event-related potential is detected, and at the same time, the artificial intelligence assistant based on the speech recognition system recognizes and corrects the system error by itself. can do. In addition, it is expected that various values can be created by utilizing the potential of the user's event in various smart machines that have to decide and move their own movements.

As an example, as shown in FIG. 9, an object in the form of a vehicle is a small RC car capable of autonomous driving, and can be autonomously driven without external manipulation along a line drawn on the floor. The problem is that if there is no previously entered information on a forked road, it is impossible to choose a path at random or choose the correct path by yourself.

Referring to (a) of FIG. 9, a smart machine (that is, a small RC car capable of autonomous driving) can randomly select a path due to lack of information on path selection on a forked road or cannot select the correct path by itself, so that the user can You can drive on unintended roads.

In addition, referring to (b) of FIG. 9, according to the learning process of the smart machine 10, the P300 potential may be detected in the brain of the user being observed when the autonomous vehicle selects a wrong path by itself. That is, the smart machine 10's own decision-making may be determined as an error.

Referring to (c) of FIG. 9, when the autonomous vehicle selects the wrong path by itself, the P300 potential will be detected in the brain of the user being observed, and the smart machine 10 will be able to self-learn the smart machine 10 after the feedback process. You will be able to find the correct path on the next repeated forked road along the way.

Hereinafter, based on the details described above, the operation flow of the present application will be briefly described.

The self-error recognition and learning method of a smart machine using the wireless EEG measurement system shown in FIG. 10 may be performed by the smart machine-based self-error recognition and learning system 1 described above. Therefore, even if omitted below, the description of the smart machine-based self-error recognition and learning system (1) can be equally applied to the description of the self-error recognition and learning method of a smart machine using a wireless EEG measurement system. have.

10 is a schematic operation flowchart of a method for self-error recognition and learning of a smart machine using a wireless EEG measurement system according to an embodiment of the present application.

In step S901, the system 1 may make a decision in the smart machine 10.

In step S902, the system 1 may transmit information on an event related to decision making to the brain wave measurement system 20.

In step S903, the system 1 may analyze the event-related potential pattern by using the user's EEG generated by the event performed by the smart machine 10 in the EEG measurement system 20.

In step S904, when P300 is included in the result of analyzing a related potential pattern in the smart machine 10, the system 1 may recognize a decision-making error and perform relearning.

In step S905, when P300 is not included in the analysis result of the related potential pattern in the smart machine 10, the system 1 recognizes that the decision is normal (it is a decision without error) and receives information related to the decision. It can be stored as artificial intelligence-based learning data.

In the above description, steps S901 to S905 may be further divided into additional steps or may be combined into fewer steps, according to the embodiment of the present application. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

The self-error recognition and learning method of a smart machine using a wireless EEG measurement system according to an embodiment of the present application may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

In addition, the self-error recognition and learning method of a smart machine using the above-described wireless EEG measurement system may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The foregoing description of the present application is for illustrative purposes only, and those of ordinary skill in the art to which the present application pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present application is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application.

1: Self-error recognition and learning system based on smart machine
2: electrode
10: smart machine
20: EEG measurement system
21: sensor unit
22: wireless communication element
23: filtering unit
24: judgment

Claims (8)

In a smart machine-based self-error recognition and learning system using a wireless EEG measurement system,
An EEG measurement system for analyzing an event-related potential pattern by collecting EEG of a user generated by an event performed in a smart machine; And
A smart machine that performs self-error recognition and artificial intelligence-based learning related to decision making corresponding to the event in consideration of the analysis result of the event-related potential pattern provided from the EEG measurement system,
Including,
The EEG measurement system,
When determination completion information related to decision making corresponding to the event is received from the smart machine, the user's EEG is measured, and it is determined whether the P300 peak is included in the event-related potential pattern,
The smart machine,
If P300 is included in the event-related potential pattern analysis result, relearning is performed to make a new decision by recognizing a decision error and correcting the decision in which the error occurred,
When P300 is included in the event-related potential pattern analysis result and the decision making is determined to be an error, information related to the P300 peak is received from the EEG measurement system, and is preset to a time point at which information related to the P300 peak is received. By grasping the result of the existing decision before time, the smart machine performs a check on the existing decision made by itself,
The new decision is made in consideration of the number of options of the result of the existing decision before a preset time as the time when the information related to the P300 peak is received,
When there are two options when selecting the existing decision making, it is determined whether to proceed with the re-decision making with the opposite option instead of the option linked to the existing decision in which the event was performed,
When there are three or more options when selecting the existing decision, information on a new decision is provided from the user terminal,
By storing a plurality of information related to decision making as artificial intelligence-based learning data,
The self-error recognition and learning system based on a smart machine, characterized in that by performing the artificial intelligence-based learning to self-recognize abnormal motions or errors and to correct decision-making. delete delete The method of claim 1,
The smart machine,
When P300 is not included in the event-related potential pattern analysis result, and the decision making is determined to be normal, the information related to the decision making is stored as learning data based on artificial intelligence, self-error recognition based on a smart machine. And learning system. delete The method of claim 1,
The smart machine,
A smart machine-based self-error recognition and learning system that stores the re-decision decision result determined in response to the option included in the information related to the existing decision-making and the information related to the new decision-making result as artificial intelligence-based learning data . The method of claim 1,
The EEG measurement system,
A sensor unit for receiving an EEG detected through the electrode;
A wireless communication device that transmits an analysis result of an event-related potential pattern to the smart machine;
A filtering unit that performs filtering of an EEG signal in consideration of a preset frequency domain; And
A determination unit that determines whether a P300 peak is included in the event-related potential pattern by using the EEG signal passed through the filtering unit,
That includes, a smart machine-based self-error recognition and learning system. In the self-error recognition and learning method of a smart machine using a wireless EEG measurement system,
Making a decision in the smart machine;
Transmitting information on an event related to the decision making to an EEG measurement system;
Analyzing an event-related potential pattern using a user's EEG generated by an event performed by the smart machine in the EEG measurement system;
Performing relearning in which a decision error is recognized and a new decision is made by correcting a decision in which the error has occurred when P300 is included in the result of the analysis of the related potential pattern in the smart machine; And
If P300 is not included in the related potential pattern analysis result, recognizing that the decision making is normal and storing information related to the decision making as artificial intelligence-based learning data,
Including,
Analyzing the event-related potential pattern,
When the information on which the decision-making performed by the smart machine is completed is received, the user's EEG is measured, and it is determined whether the P300 peak is included in the event-related potential pattern,
The step of performing relearning to proceed with the new decision making,
When P300 is included in the event-related potential pattern analysis result and the decision making is determined to be an error, information related to the P300 peak is received from the EEG measurement system, and is preset to a time point at which information related to the P300 peak is received. By grasping the result of the existing decision before time, the smart machine performs a check on the existing decision made by itself,
The new decision is made in consideration of the number of options of the result of the existing decision before a preset time as the time when the information related to the P300 peak is received,
When there are two options when selecting the existing decision making, it is determined whether to proceed with the re-decision making with the opposite option instead of the option linked to the existing decision in which the event was performed,
When there are three or more options when selecting the existing decision, information on a new decision is provided from the user terminal,
The step of storing as the artificial intelligence-based learning data,
A smart machine-based self-error recognition and learning method that stores a plurality of information related to decision making as artificial intelligence-based learning data.

Images