TW202117743A - System for diagnosing dyslexia by combining brain wave and artificial intelligence including brain wave detection units and a central processing unit - Google Patents

Abstract

The present invention provides a system for diagnosing dyslexia by combining brain waves and artificial intelligence. The system includes a plurality of brain wave detection units attached to a head of a subject, and a central processing device electrically connected to the brain wave detection units. The central processing device receives a brain wave signal of the subject upon occurrence of an event, processes the brain wave signal and converts it into an event-related potential. The central processing device encodes the event-related potential to a brain wave event-related potential two-dimensional feature map according to setting positions of the brain wave detection units. A neural network module of the central processing device then uses the brain wave event-related potential two-dimensional feature map for training and learning to construct the system for diagnosing dyslexia in conjunction with artificial intelligence.

Description

A system that combines brain waves and artificial intelligence to diagnose dyslexia

The present invention relates to a system for diagnosing dyslexia by combining brain wave and artificial intelligence technology, especially a one-dimensional event-related potential detected by brain waves when an event stimulus is detected, which is first encoded into a two-dimensional After the two-dimensional feature map of brainwave event-related potentials in three dimensions, the two-dimensional feature map of brainwave event-related potentials is used to train artificial intelligence neural networks, so as to build a neural network that combines brainwaves and artificial intelligence. System for diagnosing dyslexia.

The traditional judgment of dyslexia usually requires a comprehensive assessment of complex scales, tests, worksheets, and school academic performance. Therefore, the process of collecting data takes more than one year. And in the process, it is easy to cause judgment errors due to the different expectations of the tester for the testee, and the difference in the description and analysis of the learning materials, which in turn affects whether the testee will be judged as a learning disability, so the traditional dyslexia Judgment not only takes a long time and manpower, but also delays the best time for learning and treatment for people with dyslexia.

Brain wave signals are the potential changes produced by the interconnection of hundreds of billions of neurons in the brain. Therefore, in recent years, not only people have begun to study how to observe the brain waves of the subject after an external stimulus through the brain wave instrument. In response to the signal, some people even combine the brain wave instrument with the artificial neural network (Artificial Neural Network) in artificial intelligence, and use the artificial neural network to analyze and classify the subject’s brain wave signal. The analysis of the wave signal assists the doctor in screening whether the subject has other brain diseases such as epilepsy, and serves as the basis for follow-up examinations.

Therefore, in view of the above-mentioned traditional dyslexia judgment problem, how to develop an artificial intelligence learning system that can combine brainwave signals and use artificial neural networks so that this artificial intelligence learning system can be relied upon after training and learning The brain wave signal detected by the examinee when an event occurs is used to determine whether the examinee is a person with dyslexia, which is the subject of improvement.

The purpose of the present invention is to solve the problem of excessively long judgment time for traditional dyslexia. The present invention uses the principle of Event-Related Potentials (ERPs) to induce and detect when the subject is taking reading test questions. Obtain the brainwave signal generated by the test subject on the test question stimulus, and process and convert the brainwave signal into event-related potentials, and then convert the event-related potentials of different brainwave detection units to adjacent time, Coding in space to generate the image feature data of the brainwave event-related potential two-dimensional feature map, and then use the brainwave event-related potential two-dimensional feature map to train the artificial intelligence neural network model, so that it first learns how to recognize dyslexia The two-dimensional feature maps of brainwave event-related potentials of people with dyslexia and those without dyslexia can more accurately determine whether the subject is dyslexic

To achieve the above objective, the present invention provides a system for diagnosing dyslexia by combining brain waves and artificial intelligence, which includes: a complex brain wave detection unit attached to the subject’s head in accordance with the international 10-20 electrode marking method , To detect the brain wave signal generated by the subject for at least one event; and a central processing device, including a signal processing unit and an arithmetic unit electrically connected to the signal processing unit, wherein the signal processing unit is electrically Connect these brain wave detection units, and process and convert the received brain wave signals of the subject into complex event-related potentials (ERPs). The computing unit includes an input module and an A central processing module, a type of neural network module and an output module, the central processing module is electrically connected to the input module, the type of neural network module and the output module, and the arithmetic unit receives the After waiting for the event-related potentials, the central processing module re-encodes and converts it into a brainwave event required by the input module according to the location of the brainwave detection units and the time point when the event-related potentials are generated. A two-dimensional feature map of related potentials. This type of neural network module performs arithmetic processing on the two-dimensional feature map of brainwave event-related potentials, and then outputs it to the output module to display the judgment result; wherein, this type of neural network module After receiving a plurality of two-dimensional feature maps of the brainwave event-related potentials and performing multiple training and learning, the system that combines brainwaves and artificial intelligence for dyslexia diagnosis can be established, thereby using the combined brainwaves and artificial intelligence to diagnose dyslexia. A smart system for diagnosing dyslexia to diagnose whether the subject is a dyslexic.

Further, this type of neural network module includes at least one convolutional layer, at least one ReLU layer, at least one pooling layer, and at least one fully connected layer.

Further, this type of neural network module includes three fully connected layers, and the fully connected layers include at least one sigmoid function, at least one softmax function, or a combination thereof.

Furthermore, the event-related potentials monitor the brainwave signals of N200, P300, N400, P600, N2P3, and P3N4 generated by each of the brainwave detection units for each of the events.

Further, the location of the brain wave detection unit includes nine locations of A1, A2, Fp1, Fp2, C3, C4, Fz, Cz, and Pz.

Further, the positions of the A1 and A2 are pasted on the subject's left and right earlobes as reference potentials, and the positions of the Fp1 and Fp2 are pasted on the side and below the subject's eyes to filter the subject's blinking Noise.

Further, the central processing module normalizes the brainwave event-related potential two-dimensional feature map into a grayscale image, and the grayscale image value ranges from 0 to 255.

Further, the central processing module normalizes the two-dimensional feature map of brainwave event-related potentials into a grayscale image, and the grayscale image value ranges from 0 to 1.

Further, the neural network module of this type will modify a calculation weight of the fully connected layers according to the correctness of the judgment result of the output module.

Therefore, the present invention has the following beneficial effects compared with the prior art:

1. The present invention first converts the brainwave signal of the subject into a two-dimensional feature map of brainwave event-related potentials, and then performs learning and identification on the neural network module of the central processing unit to make the neural network model After training and learning multiple sets of two-dimensional feature maps of brainwave event-related potentials for people with dyslexia and those without dyslexia, the testees can use the dyslexia diagnostic system to determine whether they are dyslexic. Shorten the time to diagnose and confirm the diagnosis of people with dyslexia and avoid misjudgments caused by artificial unobjectiveness.

2. The present invention recodes the event-related potential data of the traditional one-dimensional brainwave signal into a two-dimensional brainwave event-related potential two-dimensional feature map after considering the time and space correlation, and then uses A neural network module with a convolutional layer is used for training and learning, and used as a benchmark for judging whether it is dyslexia, thereby establishing a high-precision and real-time artificial intelligence dyslexia diagnosis system.

Here are a few preferred implementation aspects of the technical features and operation methods of this application, which will be described in conjunction with the illustration below for review and reference. Furthermore, the figures in the present invention are not necessarily drawn according to the actual scale for the convenience of explanation, and the proportions in the figures are not used to limit the scope of protection of the present invention.

Please refer to Figures 1 to 3, the present invention relates to a system that combines brain waves and artificial intelligence to diagnose dyslexia, by detecting the brain wave signals measured by the subject 1 during the reading test , Re-encode this brain wave signal into two-dimensional image data, and then use the two-dimensional image data to train a neural network with artificial intelligence; when the artificial intelligence neural network is completed, it will be completed A diagnostic system capable of diagnosing whether the brain wave signal of the subject 1 is a person with dyslexia, comprising: a plurality of brain wave detecting units 10 and a central processing device 20 electrically connected to the brain wave detecting units 10, wherein:

The brain wave detection units 10 are attached to the head of the subject 1 in accordance with the international 10-20 electrode marking method to detect when the subject 1 is stimulated by an event, such as an event stimulus such as a reading test. For the generated brain wave signals, the brain wave detection units 10 of this embodiment are arranged in nine positions including A1, A2, Fp1, Fp2, C3, C4, Fz, Cz, and Pz. In order to avoid these The brain wave detection unit 10 generates too many labels in the first figure to cause confusion. Therefore, in the first figure, only two of the brain wave detection units 10 are first selected and given labels as representatives. Among them, the A1 and the A2 The position of Fp1 and Fp2 are pasted on the left and right earlobes of subject 1 as the reference potential, and the positions of Fp1 and Fp2 are pasted on the side and below the eyes of subject 1 to filter out the blinking noise of subject 1. This implementation In the example, the event refers to the measurement of the brain wave signal of the subject 1 when the subject 1 is taking the reading test, but it is not limited to this.

The central processing device 20 includes a signal processing unit 21 and an arithmetic unit 22 electrically connected to the signal processing unit 21, wherein the signal processing unit 21 is electrically connected to the brain wave detecting units 10, and receives The brain wave signal to the subject 1 is processed and converted into complex event-related potentials ERPs (Event-Related Potentials, ERPs). Please refer to Figure 1 again. The computing unit 22 includes an input module 221, an A central processing module 222, a type of neural network module 223, and an output module 224. The central processing module 222 is electrically connected to the input module 221, the type of neural network module 223, and the output module, respectively 224. After the arithmetic unit 22 receives the event-related potentials ERPs, the central processing module 222 re-encodes and converts according to the setting positions of the brainwave detection units 10 and the time points when the event-related potentials ERPs are generated. The two-dimensional feature map 2211 of a brainwave event-related potential required to form the input module 221, please refer to Figure 3 again, and this type of neural network module 223 includes a convolutional layer 2231, a ReLU (Rectified Linear Unit, ReLU) layer 2232, a pooling layer 2233, and three fully connected layers 2234, whereby the brainwave event-related potential two-dimensional feature map 2211 sequentially passes through the convolutional layer 2231, the ReLU layer 2232, and the pool After the operation and processing of the fully connected layer 2233 and the fully connected layers 2234, they are then output to the output module 224 to display the judgment result. In this embodiment, one fully connected layer 2234 contains a sigmoid function, and the other fully connected layer 2234 contains a sigmoid function. The layer 2234 contains a softmax function, but is not limited to this, that is, the fully connected layers 2234 may all contain a sigmoid function, or all contain a softmax function, or, as in this embodiment, each contain a sigmoid function and a softmax function. The function is in the fully connected layer 2234, whereby a calculation weight W of each fully connected layer 2234 can be adjusted.

After this type of neural network module 223 receives a plurality of the brainwave event-related potential two-dimensional feature maps 2211 and performs multiple training and learning, the type of neural network module 223 will determine according to the output module 224 Whether the result is correct or not, the calculated weight W in the fully connected layers 2234 is modified to establish the system for dyslexia diagnosis combined with artificial intelligence, thereby using this system to diagnose whether the subject 1 is dyslexic By.

In order to further understand how the present invention combines brain waves and artificial intelligence to design a dyslexia diagnosis system, please refer to Figure 1 and Figure 2. The central processing device 20 will use the brain wave detection units 10 to detect the brain wave signal of the subject 1, and monitor that when the subject 1 is taking the event of the reading test, the brain wave detection unit 10 of the subject 1 receives the brain generated when the test question is stimulated Signal, and the potential or potential difference generated by the brainwave signal is processed by the signal processing unit 21, so that it can be converted into the event-related potential ERPs according to the generation time, wherein the event-related potential ERPs can be affected by Tester 1 is obtained by averaging the brain wave signal when there are multiple stimuli for the same test question, or the brain wave signal detected by the stimulus of different test questions, or the combination of the above This embodiment is to detect the brainwave signals of N200, P300, N400, P600, N2P3, and P3N4 of the subject 1 after receiving the stimulus, and N200 represents the negative potential at about 200 milliseconds, and P600 represents at 600 milliseconds The left and right positive potential, N400 represents the negative potential at about 400 milliseconds, N2P3 represents the potential difference between the positive potential at about 300 milliseconds and the negative potential at about 200 milliseconds, and P3N4 represents the positive potential at about 300 milliseconds and The potential difference between the negative potentials of about 400 milliseconds is recoded by the central processing module 222 of the arithmetic unit 22 according to the different setting positions of the brain wave detection units 10 and the time sequence relationship of the event-related potentials ERPs. The two-dimensional feature map 2211 of the brainwave event-related potential in the event of the subject 1 in the reading test is generated, and the central processing module 222 normalizes the two-dimensional feature map 2211 of the brainwave event-related potential into Grayscale image, and the normalized range of the grayscale image value is a value of 0~255, or the normalized range of the grayscale image value is a value of 0~1;

Please refer to Figures 1 to 3 again. Among them, the embodiment of the present invention uses a plurality of brain wave signals that are known to be a person with or without dyslexia to perform machine learning (machine learning). learning), therefore, the neural network module 223 of this type sequentially passes through the convolutional layer 2231, the ReLU layer 2232, the pooling layer 2233, and the fully connected two-dimensional feature maps 2211 of the brainwave event-related potentials. The layer 2234 performs arithmetic processing and outputs the judgment result to the output module 224. The central processing module 222 compares the judgment result with the actual result. This type of neural network module 223 then judges according to the output module 224 The correctness of the result is to modify the calculated weight W of the fully connected layers 2234. This type of neural network module 223 performs multiple training and learning, and after correcting the calculated weight W of the fully connected layers 2234, Increase the accuracy of the system’s diagnosis, so that after the neural network module 223 of this type has completed the training and learning of the interpretation results of a plurality of known results of the brainwave event-related potential two-dimensional feature map 2211, it will be established Set up the system that combines artificial intelligence to diagnose dyslexia. Subsequently, the brainwave signals of other subjects are encoded into the brainwave event-related potential two-dimensional feature map 2211, and then the results are displayed on the output module 224 through the calculation of the neural network module 223 It can be judged whether the subject 1 is a person with dyslexia.

Please also refer to Figure 4, under the same number of training times, the neural network module of the present invention has the convolutional layer 2231, the ReLU layer 2232, the pooling layer 2233, and the fully connected layers 2234. Therefore, after training with the two-dimensional feature map 222 of the brainwave event-related potential in the two-dimensional dimension, the correct rate of diagnosing whether the subject is a dyslexic person is higher than the traditional one-dimensional brainwave signal for training. For the accuracy value of the Multilayer perceptron (MLP), please refer to Figure 5, and it can be seen that the accuracy of the judgment of whether Subject 1 is a person with dyslexia is approximately close to the accuracy of the present invention at the 200th time. 1. It is equivalent to a 100% correct rate, but under the same number of training times, the traditional multi-layer cognitive neural network MLP's judgment correct rate is less than 0.9, which is equivalent to only a 90% correct rate. Therefore, The present invention uses the signal processing unit 21 to convert the brain wave signal data detected by the brain wave detection unit 10 into one-dimensional complex event-related potentials ERPs, and then the central processing module 222 re-encodes them into two-dimensional The two-dimensional feature map 2211 of the brainwave event-related potential in the dimensions, and after training and learning by this type of neural network module 223, can successfully build a combined artificial intelligence that can determine whether the subject has a dyslexia problem in real time. Intelligent system for diagnosing dyslexia.

Hereinafter, the characteristics of the present invention and the expected effects that can be achieved are stated as follows:

The system for diagnosing dyslexia by combining brain waves and artificial intelligence of the present invention first converts the brain wave signal of the subject from the one-dimensional event-related potential ERPs to the two-dimensional event-related potential. After the two-dimensional feature map 2211, through the training and learning of the neural network module 223 of the central processing device 20, the neural network module 223 of this type will modify the entire network during training and learning. The calculated weight W of the connection layer 2234 increases the accuracy of its judgment, thus shortening the diagnosis time for diagnosing whether the subject is a person with dyslexia, and avoiding misjudgment caused by artificial judgments that are not objective.

The content of the present invention has been described in detail above, but the above are only the preferred embodiments of the present invention, and should not be used to limit the scope of implementation of the present invention, that is, all equivalent changes and changes made in accordance with the scope of the patent application of the present invention Modifications should still fall within the scope of the patent of the present invention.

1: Subject 10: Brain wave detection unit 20: Central processing unit 21: signal processing unit 22: arithmetic unit 221: Input Module 2211: Two-dimensional feature map of brainwave event-related potentials 222: Central Processing Module 223: Neural Network-like Module 2231: Convolutional layer 2232: ReLU layer 2233: Pooling layer 2234: Fully connected layer 224: output module ERPs: event-related potentials W: calculate weight

Figure 1: A schematic diagram of the structure of the present invention. Figure 2: A schematic diagram of the operation process of the present invention. Figure 3: A schematic diagram of the architecture of the neural network operation of the present invention. Figure 4: Comparison of accuracy rates between the present invention and the traditional neural network multi-layer cognitive model (MLP) at 150 and 300 training times. Figure 5: It is a graph showing the change of the accuracy rate of the present invention and the traditional MLP mode at the same number of training times.

1: Subject

10: Brain wave detection unit

20: Central processing unit

21: signal processing unit

22: arithmetic unit

221: Input Module

2211: Two-dimensional feature map of brainwave event-related potentials

222: Central Processing Module

223: Neural Network-like Module

224: output module

ERPs: event-related potentials

Claims (9)

A system that combines brain waves and artificial intelligence to diagnose dyslexia, which includes: The complex brain wave detection unit is attached to the subject’s head in accordance with the international 10-20 electrode marking method to detect the subject’s brain wave signal when at least one event occurs; and A central processing device includes a signal processing unit and an arithmetic unit electrically connected to the signal processing unit, wherein the signal processing unit is electrically connected to the brain wave detection units, and receives the brain wave of the testee Wave signals are processed and converted into complex event-related potentials (ERPs), and the arithmetic unit includes an input module, a central processing module, a type of neural network module, and an output module. The central The processing module is electrically connected to the input module, the neural network module and the output module. After the arithmetic unit receives the event-related potentials, the central processing module uses the brain wave detection units The setting positions of the event-related potentials and the time points at which the event-related potentials are generated are recoded and converted into a two-dimensional feature map of brainwave event-related potentials required by the input module. This type of neural network module performs the brainwave event After the two-dimensional characteristic map of the related potential is processed, it is output to the output module to display the judgment result; Wherein, this type of neural network module receives a plurality of two-dimensional feature maps of the brainwave event-related potentials and performs multiple training and learning to establish and complete the system for dyslexia diagnosis combined with artificial intelligence. The system for dyslexia diagnosis combined with brain waves and artificial intelligence as described in item 1 of the scope of patent application, wherein this type of neural network module includes at least one convolutional layer, at least one ReLU layer, at least one pooling layer, and At least one fully connected layer. The system for dyslexia diagnosis combined with brain waves and artificial intelligence as described in item 2 of the scope of patent application, wherein this type of neural network module includes three fully connected layers, and the fully connected layers include at least one sigmoid function , At least one softmax function or a combination thereof. The system for dyslexia diagnosis combined with brainwaves and artificial intelligence as described in item 1 of the scope of patent application, wherein the event-related potentials monitor the N200 and P300 generated by each event of each brainwave detection unit , N400, P600, N2P3 and P3N4 brainwave signals. The system for diagnosing dyslexia combined with brain waves and artificial intelligence as described in item 1 of the scope of patent application, wherein the location of the brain wave detection unit includes A1, A2, Fp1, Fp2, C3, C4, Fz, Cz And the nine positions of Pz. For example, the system for diagnosing dyslexia combined with brain waves and artificial intelligence as described in item 5 of the scope of patent application, wherein the positions of the A1 and the A2 are pasted on the subject's left and right earlobes as reference potentials, and the Fp1 and The position of the Fp2 is pasted on the side and below the subject’s eyes to filter the subject’s blinking noise. The system for dyslexia diagnosis combining brain waves and artificial intelligence as described in item 1 of the scope of patent application, wherein the central processing module normalizes the two-dimensional feature map of brain wave event-related potentials into gray-scale images, which The range of the grayscale image value is a value from 0 to 255. The system for dyslexia diagnosis combining brain waves and artificial intelligence as described in item 1 of the scope of patent application, wherein the central processing module normalizes the two-dimensional feature map of brain wave event-related potentials into gray-scale images, which The range of the grayscale image value is a value from 0 to 1. For example, the system that combines brain waves and artificial intelligence for dyslexia diagnosis described in item 2 of the scope of patent application, wherein the neural network module of this type will correct the full connections according to the correctness of the output module's judgment result A calculated weight of the layer.

Images