CN115714832A - Operation method of telephone dialing system based on steady-state visual evoked potential - Google Patents
Operation method of telephone dialing system based on steady-state visual evoked potential Download PDFInfo
- Publication number
- CN115714832A CN115714832A CN202211300736.1A CN202211300736A CN115714832A CN 115714832 A CN115714832 A CN 115714832A CN 202211300736 A CN202211300736 A CN 202211300736A CN 115714832 A CN115714832 A CN 115714832A
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram
- dialing
- module
- stimulation
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 49
- 230000000763 evoking effect Effects 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000000638 stimulation Effects 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 39
- 238000004891 communication Methods 0.000 claims abstract description 24
- 230000003993 interaction Effects 0.000 claims abstract description 21
- 230000005540 biological transmission Effects 0.000 claims description 19
- 230000002452 interceptive effect Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 210000000869 occipital lobe Anatomy 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 230000000903 blocking effect Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000007639 printing Methods 0.000 claims description 2
- 230000008676 import Effects 0.000 claims 1
- 210000004556 brain Anatomy 0.000 abstract description 20
- 230000007659 motor function Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 208000014644 Brain disease Diseases 0.000 description 1
- 208000002193 Pain Diseases 0.000 description 1
- 230000018199 S phase Effects 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000004070 electrodeposition Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000011990 functional testing Methods 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000036407 pain Effects 0.000 description 1
- 210000001428 peripheral nervous system Anatomy 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000002269 spontaneous effect Effects 0.000 description 1
Images
Landscapes
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an operation method of a telephone dialing system based on steady-state visual evoked potential. The system belongs to the field of electroencephalogram decoding and human-computer interaction, and comprises a visual stimulation and interaction module, an electroencephalogram signal real-time acquisition module, an electroencephalogram signal processing module and a telephone dialing module; the invention aims at the problem that the patient without the motor function can not normally use the intelligent communication equipment to communicate, and provides a new way for the patient to communicate autonomously. The system provides a visual stimulation interface to induce a human brain to generate signals, the wireless brain electrical acquisition device is used for acquiring brain electrical signals in real time, and the intention of a subject is judged to output a corresponding instruction and finally the call dialing is finished by classifying steady visual evoked potentials in the brain electrical signals.
Description
Technical Field
The invention belongs to the field of electroencephalogram decoding and human-computer interaction, and relates to a telephone dialing system based on steady-state visual evoked potentials and an operation method thereof, which are used for assisting patients with motor function deficiency to communicate.
Background
At present, the total number of the disabled people in China reaches 8500 million, wherein the number of the disabled people in the limbs is nearly 2500 million, and the patients with the motor function deficiency and the brain function normal can also maintain the vital signs by depending on the development of the current social and medical technology, but cannot communicate and communicate with the outside, so that the patients suffer great pains in psychology.
A Brain Computer Interface (BCI) is a communication system which is independent of a peripheral nervous system and muscle tissues and establishes a direct communication channel between the human Brain and the outside, can realize idea control of external equipment by a user, and has important application value in the fields of Brain cognition, brain diseases, intelligent control, rehabilitation medicine and the like.
The brain-computer interface system can be divided into a spontaneous type and an evoked type according to the brain electrical signal generation mode. The self-generating brain-computer interface system has the advantages that no additional stimulation device is needed, but the user needs a lot of training, and the complexity and accuracy of the identification algorithm are low, such as a brain-computer interface system based on Motor Image (MI) potential. The effective brain wave components of the evoked brain-computer interface system are excited by an external stimulation device, and the brain-computer interface system has the characteristics of no or little training, simple identification algorithm, high accuracy and the like, but has the defect of needing an additional excitation source to generate effective brain wave components, such as a brain-computer interface system based on steady-state visual evoked potential and a brain-computer interface system based on P300 potential.
The steady state visual evoked potential is usually present in the occipital lobe area of the brain, is an electroencephalogram signal induced when human eyes are subjected to external stimulation with a certain frequency flicker, the range of the induced frequency is required to be between 4 and 90Hz, and is a continuous and stable event-related potential. The steady state visual evoked potential has the characteristics of high noise ratio, signal concentration, easy acquisition and the like, and in addition, the steady state visual evoked potential has obvious rhythm assimilation characteristics, so that the characteristic extraction method of the signal is greatly simplified. Based on the generation principle of the signal, the injury to a subject is small in the actual experiment process, and the experiment difficulty is small.
According to the latest data of statcounter, as 9 months in 2022, the Android accounts for 71.54%, the ios accounts for 27.81%, the sum of the accounts of the other mobile operating systems is less than 1% in the global mobile operating system market, the data display Android accounts for the most part, the Android has the characteristics of full open source and high degree of freedom, and based on the advantages, the Android is selected by the telephone dialing lower computer development system.
The invention designs an android-terminal telephone dialing system based on steady-state visual evoked potential, and provides a new way for autonomous communication for patients with limb disorders who cannot normally use intelligent equipment.
Disclosure of Invention
The invention aims to: aiming at the problem that patients with lack of motor functions cannot normally use intelligent communication equipment to carry out communication, the invention designs and develops a telephone dialing system based on steady-state visual evoked potential, and provides a new way for autonomous communication for the patients. The system provides a visual stimulation interface to induce a human brain to generate signals, the wireless brain electrical acquisition device is used for acquiring brain electrical signals in real time, and the intention of a subject to output a corresponding instruction is judged by classifying steady visual evoked potentials in the brain electrical signals and finally the call is dialed.
The technical scheme is as follows: the invention relates to a telephone dialing system based on steady-state visual evoked potential, which comprises a visual stimulation and interaction module, an electroencephalogram signal real-time acquisition module, an electroencephalogram signal processing module and a telephone dialing module;
the visual stimulation and interaction module is used for generating an electroencephalogram signal by providing visual stimulation and outputting and feeding back a real-time result;
the electroencephalogram signal real-time acquisition module is used for acquiring electroencephalogram signals induced by visual stimulation and transmitting the electroencephalogram signals to the processing module for use;
the electroencephalogram signal processing module is used for receiving a real-time signal, processing and analyzing the real-time signal and feeding a result back to a user;
and the telephone dialing module is used for receiving the final result and completing dialing.
Further, the visual stimulation and interaction module comprises an interaction output box and an SSVEP inducing interface, the background of the interaction output box is white, and an LCD display screen is selected as the display;
the interactive output frame is used for displaying the input numbers, and the color of the interactive output frame is black;
the SSVEP evoking interface adopts graphic flicker stimulation, the stimulation graphics are square blocks, the color is black, the number is 12, the number is respectively 1,2, 3, 4, 5, 6, 7, 8, 9, N, 0 and Y, the stimulation square blocks flicker according to different frequencies, the frequency band range is selected to be 6-20Hz, and the coding is carried out in the range to form stable visual stimulation;
the flicker stimulation coding mode is sinusoidal coding, and the formula is as follows:
where R is the display refresh frequency, typically 60Hz, n is the number of frames of the LCD display, f is the frequency of the stimulus target, and s ranges from 0 to 1.
Furthermore, the electroencephalogram signal real-time acquisition module is arranged on the head of the subject, acquires the electroencephalogram signal of the subject, amplifies, filters and digitizes the signal, and is connected with the electroencephalogram signal processing module to transmit the real-time electroencephalogram signal;
further, the real-time brain electrical signal collection module selects a 64-lead wireless brain electrical signal collection system NeuSen W64 developed by Borui Cor company, and the collection device comprises: the device comprises a 64-conductive electrode cap, a signal amplifier, a multi-parameter synchronizer, an intelligent synchronization center, signal acquisition software, a PC (personal computer) and an external display; the devices are connected in sequence; the electrode cap is worn on the head of a testee to collect electroencephalogram signals of the testee, a signal amplifier magnetically attached to the rear end of the electrode cap receives the signals collected by the electrodes in the electrode cap, amplifies and filters the signals and performs A/D (analog to digital) conversion on the signals, and finally the signals are transmitted to a Start NeuSen W software system through wireless communication to be displayed and recorded.
Furthermore, the electroencephalogram signal processing module comprises preprocessing, frequency identification and number transmission;
the preprocessing calls Basic FIR filter to filter EEG data at 4-30 Hz;
the frequency identification is calculated by adopting a Filter Bank Canonical Correlation Analysis (FBCCA) algorithm to obtain a target result;
the real-time data in the electroencephalogram signal real-time acquisition module is transmitted to establish connection through a TCP/IP protocol, a Client/Server model is adopted, and the Client and the Server respectively correspond to the real-time signal processing module and the electroencephalogram signal acquisition module. The NeuSen W64 acquisition system encapsulates TCP/IP protocol codes in Start NeuSen W software, so that data receiving can be realized by calling an interface function, IP addresses and port numbers are set in the modules, the number of channels is selected, and the establishment of the connection between the modules is completed.
The data transmission between the electroencephalogram signal real-time acquisition module and the electroencephalogram signal processing module is connected through a TCP/IP protocol, the signal processing module is a client, the electroencephalogram signal acquisition module is a server, experimenters set parameters such as a host, ports and channel numbers and then operate the electroencephalogram signal processing module, a data receiving part in the module establishes a data receiving structure body through a communication client interface class DataClient, the received transmitted data is set through IP addresses, port numbers, electrode channel numbers, sampling rates and the size of the received data, and then an open () method is called to establish connection. The data received in real time can be stored in a buffer, and TCPIP. Userdata () is called by a GetBufferData () method to obtain the real-time data received in a single experiment.
Further, the electroencephalogram signal processing module processes the received electroencephalogram signal and transmits a result, the processing content comprises preprocessing and frequency identification, and the result is transmitted as a number;
the preprocessing calls Basic FIR filter to filter EEG data at 4-30 Hz;
the frequency identification is calculated by adopting a Filter Bank Canonical Correlation Analysis (FBCCA) algorithm to obtain a target result;
the FBCCA algorithm comprises the following steps:
the number transmission adopts TCP/IP communication and adopts a Client/Server model; the electroencephalogram signal processing module is a client, an address and a port of a server to be connected are designated, a java.
Further, the telephone dialing module reads and dials numbers;
the number reading realizes data transmission through TCP/IP communication, and an android lower computer is a server; the server is established through the Serversocket class, a request is received for the Serversocket through an accept () method, a Socket object is returned, and the blocking is kept until the request arrives, so that the communication Socket object can be obtained through monitoring; then, respectively obtaining reference objects of input and output streams by encapsulating Socket objects in a new thread, and sending data to a client or receiving data from the client through the two objects to realize Socket communication, wherein the specific realization is that numbers transmitted by the client are read according to character streams through a readLine () method;
the number dialing is realized through an android program, the dialing authority, namely android.
Further, an operation method of the telephone dialing system based on the steady state visual evoked potential comprises the following specific working procedures:
connecting an electroencephalogram acquisition device, starting Start New W software after connection is finished, checking whether connection is successful, starting clicking, and checking whether data transmission is normal;
step (2), wearing a NeuSen W64 electroencephalogram guiding helmet on a subject, printing electroencephalogram paste on eight electrodes OZ, O1, O2, POZ, PO3, PO4, PO5 and PO6 in an occipital lobe area and Reference electrodes, displaying the function of electrode resistance reduction in real time according to the impedance in the Start Neusen W software, and ensuring that the impedance of the used electrodes is below 10 kilo-ohms;
step (3), turning on the electroencephalogram signal processing module to set data transmission, inputting a port number and an IP address, connecting the electroencephalogram acquisition module, and verifying whether real-time data transmission is normal or not;
step (4), opening the lower computer, connecting the computer through a USB (universal serial bus) line, installing the developed dialing software and starting a service, sending data to an android port through Matlab, verifying whether the connection is successful or not, and ensuring that the lower computer is successfully connected with the signal processing module;
step (5), explaining the experimental cautions for the testee, and operating the system after the explanation is finished;
and (6) the subject watches the stimulation interface according to the requirement to select a number, the system outputs an analysis result in real time, if the number is correct, the subject watches a stimulation square block corresponding to the next number, if the number is incorrect, the subject watches a backspace square block to delete the number, after the number selection is finished, the subject watches a stimulation square block corresponding to a dialing command to finish the dialing, if the dialing is required to be continued, the visual interaction module is operated again, and if the dialing is not required, the system is exited.
Has the advantages that: compared with the prior art, the invention has the characteristics that: the android-terminal telephone dialing system based on the steady-state visual evoked potential is stable in stimulation frequency flicker, convenient to carry, free of training of an identification algorithm, capable of providing a novel communication mode for patients with motion function loss, high in transportability of an upper computer part of the system, suitable for similar multi-target occasions and high in practical value.
Drawings
FIG. 1 is a block diagram of the components of the present invention;
FIG. 2 is a graphical interface layout of a visual stimulus interaction module of the present invention;
FIG. 3 is a graph of electrode position distribution in the present invention;
FIG. 4 is an interface for testing android software functionality in the present invention;
FIG. 5 is a flow chart of the system operation of the present invention.
Detailed Description
In order to more clearly illustrate the technical solution of the present invention, the following detailed description is made with reference to the accompanying drawings:
as shown in fig. 1, the telephone dialing system based on the steady-state visual evoked potential comprises four modules, namely a visual stimulation and interaction module, an electroencephalogram signal real-time acquisition module, an electroencephalogram signal processing module and a telephone dialing module;
the visual stimulation and interaction module induces an electroencephalogram signal of a user and outputs and feeds back the electroencephalogram signal in real time, the electroencephalogram signal passes through the electroencephalogram signal real-time acquisition module and the electroencephalogram signal processing module to identify a target frequency and output an instruction, and the target frequency is fed back to the interaction interface and transmitted to the telephone dialing module.
As shown in fig. 2, the visual stimulus and interaction module includes an output box and an SSVEP evoked interface.
The visual stimulation and interaction interface development platform is Matlab2019b, and the specific development tool is a Psychtoolbox3 tool box; the hardware used to present the stimulus in the actual experiment was a personal notebook LCD display screen with a screen refresh rate of 60Hz and a screen division frequency of 2560 (px) × 1600 (px).
The visual stimulation interactive interface background is white, the stimulation squares and the interactive output frame are black, the total number of the stimulation squares is 12, the numbers are respectively 1,2, 3, 4, 5, 6, 7, 8, 9, N, 0 and Y, the corresponding frequencies are respectively 6Hz, 7.5Hz, 9Hz, 10.5Hz, 11Hz, 12Hz, 13Hz, 14Hz, 15Hz, 17Hz, 16Hz and 20Hz, and the flicker stimulation is realized by adopting a sine coding mode;
the interactive output frame is arranged at the top of the interface, and has the main functions of receiving a signal classification result and displaying an output digital number; the 12 stimulation blocks correspond to the command cases: the blocks numbered 0-9 are number identification, when the subject watches the blocks, the digital results are displayed in an output frame above the interface through classification, the stimulation block numbered 'N' corresponds to a command for deleting, when the subject watches and selects the block, the latest digit in the output frame is deleted, the stimulation block numbered 'Y' corresponds to a command for dialing, and when the subject watches the block, the number in the output frame is transmitted to a lower computer for dialing.
The electroencephalogram signal real-time acquisition module is mainly used for acquiring electroencephalogram signals induced by visual stimulation of a subject and transmitting the electroencephalogram signals to the processing module for use.
Collection system chooses the wireless brain electricity collection system NeuSen W64 of 64 leads that borui kang official developed for use, and collection system includes: the device comprises a 64-conducting electrode cap, a signal amplifier, a multi-parameter synchronizer, an intelligent synchronization center, signal acquisition software, a PC (personal computer) and an external display. The experimental sampling frequency was set at 1000Hz.
As shown in fig. 3, the electrode distribution on the electrode cap is strictly set according to the international 10-20 system standard, which collects eight electrodes in the occipital lobe area: OZ, O1, O2, POZ, PO3, PO4, PO5, PO6.
The back end of an electroencephalogram cap in an actual online experiment acquisition system receives signals acquired by electrodes in the electroencephalogram cap through a signal amplifier connected in a magnetic mode, the signals are amplified, filtered and subjected to A/D (analog to digital) conversion, and finally the signals are transmitted to a Start NeuSen W software system through wireless communication to be displayed and recorded; the real-time transmission of the electroencephalogram signals to the processing module is realized by establishing connection through a TCP/IP protocol, the specific scheme is a Client/Server (Client/Server) model, and the Client and the Server respectively correspond to the real-time signal processing module and the electroencephalogram signal acquisition module. The NeuSen W64 acquisition system packages a TCP/IP protocol code in Start NeuSen W software, so that data receiving can be realized by calling an interface function, and an IP address and a port number need to be set and the number of channels needs to be selected in the module, so that connection with an electroencephalogram signal processing module is established.
The electroencephalogram signal processing module is mainly used for receiving electroencephalogram data recorded by the acquisition system in real time, processing and classifying the data and outputting instructions back to a subject; the module development software is Matlab2019b.
The data receiving is realized through a TCP/IP protocol, the electroencephalogram data collected by the collecting module is continuously received, a signal processing module at a Matlab end is a client, the electroencephalogram signal collecting module is a server, experimenters set parameters such as a host, a port and a channel number and then operate the electroencephalogram signal processing module, a data receiving part in the module establishes a data receiving structure body through a communication client interface class DataClient, the received transmitted data is set through setting of an IP address, a port number, an electrode channel number, a sampling rate and the size of the received data, and then an open () method is called to establish connection; real-time received data can be stored in a buffer, and TCPIP. Userdata () is called by a GetBufferData () method to obtain real-time data received by a single experiment.
After the real-time data are acquired, preprocessing is needed firstly, and based on the requirement of the real-time data on timeliness, the Basic FIR filter is directly called to filter the EEG data by 4-30 Hz.
After the preprocessing, classification and identification of the electroencephalogram signals are carried out, a filter bank typical correlation analysis method is adopted to analyze first harmonics and second harmonics of the SSVEP signals, the number of filter banks is set to be 2, an effective frequency band is divided into two sections of 4-16Hz and 16-30Hz, weighting parameters of a first harmonic filter are 1, weighting parameters of second harmonics are 0.05, sine and cosine reference signals are constructed for each stimulation frequency, then SSVEP subband component signals obtained after the filter bank analysis are respectively taken as original electroencephalogram signals to be brought into a standard CCA algorithm, weighted averaging is carried out on typical correlation coefficients corresponding to each subband to obtain correlation coefficients corresponding to the SSVEP signals, and finally the frequency corresponding to the maximum typical correlation coefficient is identification frequency.
After the frequency identification is completed, feedback is output according to the instruction corresponding to the frequency, and the detailed description is as follows: the 6Hz display output digit number 1, the 7.5Hz display output digit number 2, the 9Hz display output digit number 3, the 10.5Hz display output digit number 4, the 11Hz display output digit number 5, the 12Hz display output digit number 6, the 13Hz display output digit number 7, the 14Hz display output digit number 8, the 15Hz display output digit number 9, the 17Hz back check, the 16Hz display output digit number 0 and the 20Hz dial.
Finally, the number is transmitted to a lower computer, the transmission mode adopted here adopts TCP/IP communication, and the specific scheme is a Client/Server model; and the Matlab terminal brain electric signal processing module is a client, designates the address and the port of a server to be connected, introduces a java.
The telephone dialing module is mainly used for receiving and dialing a telephone number sent by PC end software.
The development software is Android Studio, the gradle version is 5.4.1, the Android SDK Platform-Tools version is 33.0.1, the SDK Platforms are Android API 32, the debugging machine is an entity machine Hua is nova2s, the carrying system is Hongmon 2.0, the lowest requirement on the Android version of the software is 5.0, and the software can normally run on more than 98% of Android mobile phones.
Firstly, the number needs to be read, data transmission is realized through TCP/IP communication, and the android lower computer is a server. The server is established through the Serversocket class, the request is received for the Serversocket through an accept () method, the Socket object is returned, the blocking is always kept until the request arrives, and the Socket object of communication can be obtained through monitoring; and then, respectively obtaining reference objects of input and output streams by encapsulating Socket objects in a new thread, and sending data to a client or receiving data from the client through the two objects to realize Socket communication, wherein the specific implementation is to read numbers transmitted by the client according to character streams through a readLine () method.
The method comprises the steps of receiving a number, then dialing, firstly adding the right of dialing, namely android.
Fig. 4 is a diagram of a functional test interface of android software of a lower computer, where (a) is a diagram of a software initial interface, where a connection is not established at this time, a test environment connection network ip address in the diagram is 10.100.25.176, (b) is a diagram of a connection interface successfully established between the lower computer and an electroencephalogram signal processing module, and (c) is a diagram of a dial-up interface successfully completed, where a test number is 10086, that is, china mobile communication.
The system work flow of the invention is shown in figure 5.
and 6, the examinee watches the stimulation interface to select the number according to the requirement, the system outputs an analysis result in real time, if the number is correct, the examinee watches the stimulation square corresponding to the next number, if the number is incorrect, the examinee watches the grid retreating square to delete the number, after the number selection is finished, the examinee watches the stimulation square corresponding to the dialing command to finish the dialing, if the dialing is required to be continued, the visual interaction module is operated again, and if the dialing is not required, the system is quitted.
The dialed number 119 is illustrated as follows:
a user wears the electroencephalogram acquisition device to enter a task, and after the task starts, the user annotates numbers flickering at different frequencies in the visual stimulation interactive interface.
Assuming that the user needs to input 10086, first, the user watches the button box of the number 1, waits for the input box to have the number 1, the first number input is completed, then the user watches the number 0, waits for the input box to have the number 0, the second number input is completed, then the user watches the number 0, waits for the input box to have the number 0, the third number input is completed, then the user watches the number 8, waits for the input box to have the number 8, the fourth number input is completed, preferably, the user watches the number 6, and waits for the input box to have the number 6, the fifth number input is completed.
After dialing is finished, the user watches the key frame representing the dialing instruction, namely the key frame marked with the letter Y, and then dialing can be finished, and the display of successfully dialing the lower computer is shown in fig. 4 (c).
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to those skilled in the art without departing from the principles of the present invention may be apparent to those skilled in the relevant art and are intended to be within the scope of the present invention.
Claims (7)
1. A telephone dialing system based on steady state visual evoked potentials, comprising: the system comprises a visual stimulation and interaction module, an electroencephalogram signal real-time acquisition module, an electroencephalogram signal processing module and a telephone dialing module;
the visual stimulation and interaction module is used for generating an electroencephalogram signal by providing visual stimulation and outputting and feeding back a real-time result;
the electroencephalogram signal real-time acquisition module is used for acquiring electroencephalogram signals induced by visual stimulation and transmitting the electroencephalogram signals to the processing module for use;
the electroencephalogram signal processing module is used for receiving a real-time signal, processing and analyzing the real-time signal and feeding a result back to a user;
and the telephone dialing module is used for receiving the final result and completing dialing.
2. A steady-state visual evoked potential-based telephone dialing system as in claim 1,
the visual stimulus and interaction module comprises an interaction output box and an SSVEP evoked interface;
the interactive output box is used for displaying the input numbers, and the color of the interactive output box is black;
the SSVEP evoking interface adopts graphic flicker stimulation, the stimulation graphics are square blocks, the color is black, the number is 12, the number is respectively 1,2, 3, 4, 5, 6, 7, 8, 9, N, 0 and Y, the stimulation square blocks flicker according to different frequencies, the frequency band range is selected to be 6-20Hz, and the coding is carried out in the range to form stable visual stimulation;
the flicker stimulation coding mode is sinusoidal coding, and the formula is as follows:
where R is the display refresh frequency, typically 60Hz, n is the number of frames of the LCD display, f is the frequency of the stimulus target, and s ranges from 0 to 1.
3. The steady-state visual evoked potential based telephone dialing system of claim 1,
the electroencephalogram signal real-time acquisition module is arranged on the head of a subject, acquires electroencephalogram signals of the subject, amplifies, filters and digitizes the signals, and is connected with the electroencephalogram signal processing module to transmit real-time electroencephalogram signals.
4. A steady-state visual evoked potential-based telephone dialing system as in claim 3,
the electroencephalogram signal real-time acquisition module selects a 64-lead wireless electroencephalogram acquisition system NeuSen W64, and comprises: the system comprises a 64-conductive electrode cap, a signal amplifier, a multi-parameter synchronizer, an intelligent synchronization center, signal acquisition software, a PC (personal computer) and an external display which are sequentially connected;
the electrode cap is worn on the head of a testee to collect electroencephalogram signals of the testee, a signal amplifier magnetically attached to the rear end of the electrode cap receives the signals collected by the electrodes in the electrode cap, amplifies, filters and performs A/D (analog to digital) conversion on the signals, and finally transmits the signals to a Start NeuSen W software system through wireless communication to be displayed and recorded.
5. A steady-state visual evoked potential-based telephone dialing system as in claim 1,
the electroencephalogram signal processing module comprises preprocessing, frequency identification and number transmission;
the preprocessing calls Basic FIR filter to filter EEG data at 4-30 Hz;
the frequency identification is calculated by adopting a filter bank typical correlation analysis method algorithm to obtain a target result;
the number transmission adopts TCP/IP communication and adopts a Client/Server model;
the electroencephalogram signal processing module is a client, specifies an address and a port of a server to be connected, imports a java.
6. A steady-state visual evoked potential-based telephone dialing system as in claim 1,
the telephone dialing module reads and dials numbers;
the number reading realizes data transmission through TCP/IP communication;
establishing a server through a Serversocket class, receiving a request for the Serversocket through an accept () method, returning a Socket object, and keeping blocking before the request arrives;
then, in a new thread, socket objects are encapsulated to respectively obtain reference objects of input and output streams, and data can be sent to or received from a client through the two objects to realize Socket communication;
the number dialing refers to adding a call dialing authority, namely android.
7. A method of operating a steady state visual evoked potential based telephone dialing system according to any of claims 1-6, including the steps of a specific workflow as follows:
connecting an electroencephalogram acquisition device, opening Start New W software after connection is completed, checking whether connection is successful, starting clicking, and checking whether data transmission is normal;
step (2), wearing a NeuSen W64 electroencephalogram guiding helmet on a subject, printing electroencephalogram paste on eight electrodes OZ, O1, O2, POZ, PO3, PO4, PO5 and PO6 in an occipital lobe area and Reference electrodes, displaying the function of electrode resistance reduction in real time according to the impedance in the Start Neusen W software, and ensuring that the impedance of the used electrodes is below 10 kilo-ohms;
step (3), turning on the electroencephalogram signal processing module to set data transmission, inputting a port number and an IP address, connecting the electroencephalogram acquisition module, and verifying whether real-time data transmission is normal or not;
step (4), opening the lower computer, connecting the computer through a USB (universal serial bus) line, installing the developed dialing software, starting a service, sending data to an android port through Matlab, verifying whether the connection is successful, and ensuring that the lower computer is successfully connected with a signal processing module;
step (5), explaining the experimental cautions for the testee, and operating the system after the explanation is finished;
and (6) the examinee watches the stimulation interface to select the number according to the requirement, the system outputs an analysis result in real time, if the number is correct, the examinee watches the stimulation square corresponding to the next number, if the number is incorrect, the examinee watches the grid retreating square to delete the number, after the number selection is finished, the examinee watches the stimulation square corresponding to the dialing command to finish the dialing, if the dialing is required to be continued, the visual interaction module is operated again, and if the dialing is not required, the system is quitted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211300736.1A CN115714832A (en) | 2022-10-24 | 2022-10-24 | Operation method of telephone dialing system based on steady-state visual evoked potential |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211300736.1A CN115714832A (en) | 2022-10-24 | 2022-10-24 | Operation method of telephone dialing system based on steady-state visual evoked potential |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115714832A true CN115714832A (en) | 2023-02-24 |
Family
ID=85231347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211300736.1A Pending CN115714832A (en) | 2022-10-24 | 2022-10-24 | Operation method of telephone dialing system based on steady-state visual evoked potential |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115714832A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102866775A (en) * | 2012-09-04 | 2013-01-09 | 同济大学 | System and method for controlling brain computer interface (BCI) based on multimode fusion |
| CN103607519A (en) * | 2013-10-17 | 2014-02-26 | 南昌大学 | Brain-computer interface-based telephone system for double-upper limb disabled people |
| CN104182048A (en) * | 2014-08-27 | 2014-12-03 | 西安科技大学 | Brain-computer interface based telephone system and call method thereof |
| US20170035317A1 (en) * | 2014-04-17 | 2017-02-09 | The Regents Of The University Of California | Portable brain activity sensing platform for assessment of visual field deficits |
| CN106406297A (en) * | 2016-08-03 | 2017-02-15 | 哈尔滨工程大学 | Wireless electroencephalogram-based control system for controlling crawler type mobile robot |
| CN108415569A (en) * | 2018-02-28 | 2018-08-17 | 天津大学 | Portable idea dial system based on brain wave analysis |
-
2022
- 2022-10-24 CN CN202211300736.1A patent/CN115714832A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102866775A (en) * | 2012-09-04 | 2013-01-09 | 同济大学 | System and method for controlling brain computer interface (BCI) based on multimode fusion |
| CN103607519A (en) * | 2013-10-17 | 2014-02-26 | 南昌大学 | Brain-computer interface-based telephone system for double-upper limb disabled people |
| US20170035317A1 (en) * | 2014-04-17 | 2017-02-09 | The Regents Of The University Of California | Portable brain activity sensing platform for assessment of visual field deficits |
| CN104182048A (en) * | 2014-08-27 | 2014-12-03 | 西安科技大学 | Brain-computer interface based telephone system and call method thereof |
| CN106406297A (en) * | 2016-08-03 | 2017-02-15 | 哈尔滨工程大学 | Wireless electroencephalogram-based control system for controlling crawler type mobile robot |
| CN108415569A (en) * | 2018-02-28 | 2018-08-17 | 天津大学 | Portable idea dial system based on brain wave analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101201696B (en) | Chinese input BCI system based on P300 brain electric potential | |
| CN101576772B (en) | Brain-computer interface system based on virtual instrument steady-state visual evoked potentials and control method thereof | |
| CN202408871U (en) | Human life indication monitoring system based on wireless mobile terminal | |
| CN106214391B (en) | Intelligent nursing bed based on brain-computer interface and control method thereof | |
| CN102184661B (en) | Childhood autism language training system and internet-of-things-based centralized training center | |
| CN103699226B (en) | A kind of three mode serial brain-computer interface methods based on Multi-information acquisition | |
| CN102551689A (en) | Human vital sign monitoring system based on wireless mobile terminal | |
| CN107037889B (en) | Natural writing character input method and system based on visual brain-computer interface | |
| CN101159086B (en) | Calling device based on brain electric information detection | |
| CN104503571B (en) | Idea harvester, telephone terminal and system based on brain-computer interface | |
| CN105678058A (en) | Chinese traditional auxiliary diagnosis and treatment device and system | |
| CN109471530A (en) | Brain control input method based on Steady State Visual Evoked Potential and Mental imagery | |
| CN107463249A (en) | Show the brain machine interface system and control method of VEP based on VR heads | |
| CN108681391A (en) | A kind of EEG signals dummy keyboard design method based on multi-mode | |
| CN107510562A (en) | Nursing bed system based on brain-computer interface | |
| CN105433910A (en) | Traditional Chinese medicine human health management method, device and system | |
| CN103607519A (en) | Brain-computer interface-based telephone system for double-upper limb disabled people | |
| CN104182048A (en) | Brain-computer interface based telephone system and call method thereof | |
| Lightbody et al. | A user centred approach for developing Brain-Computer Interfaces | |
| CN109326341A (en) | A kind of rehabilitation motion guiding method and apparatus | |
| CN102499636A (en) | Handheld medical multi-channel biological information acquisition mobile terminal system | |
| CN105242784B (en) | Steady State Visual Evoked Potential brain-machine interface method based on crossmodulation frequency | |
| CN113359991B (en) | Intelligent brain-controlled mechanical arm auxiliary feeding system and method for disabled people | |
| CN115714832A (en) | Operation method of telephone dialing system based on steady-state visual evoked potential | |
| CN109805921A (en) | The cross-platform method of sampling of electrocardiogram (ECG) data and cardioelectric monitor system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |