CN105929937A - Mobile phone music playing system based on steady-state visual evoked potential (SSVEP) - Google Patents

Abstract

The invention discloses a mobile phone music playing system based on the steady-state visual evoked potential (SSVEP). The mobile phone music playing system includes a visual stimulation module, a signal acquisition module, a signal processing module, a real-time transmission module and a music playing module. The visual stimulation module provides a user with SSVEP evoked potential visual stimulation in an LCD display manner or the like. The signal acquisition module acquires brain electrical signal data and transmits to a data processing program. The signal processing module performs baseline drift removal, filtering, CCA analysis and feature extraction of signals by using the data processing program, and determines a desired operation of the user. The real-time transmission module transmits an analysis result to a mobile phone via the Internet or Bluetooth or WiFi. The music playing module reacts accordingly. The method provided by the invention requires no body movement of the user, the user only needs to watch the LCD, CRT, HDU or other screen, and through analysis of the acquired brain electrical signals, the control of the mobile phone music player by the user can be achieved.

Description

A kind of mobile phone music Play System based on Steady State Visual Evoked Potential

Technical field

The present invention relates to input and output technology mutual between a kind of user and computer, belong to cognitive neuroscience, letter Breath technical field and the integrated use of automation field.

Background technology

Steady State Visual Evoked Potential (Steady-State Visual Evoked Potentials, SSVEP) is exactly to work as people When body is by the visual stimulus of a fixed frequency, the continuous print that brain visual cortex produces is relevant with stimulus frequency The response of (at the fundamental frequency of stimulus frequency or frequency multiplication), i.e. brain visual cortex flash stimulation to being in the optic centre A kind of biofeedback.The steady-state induced current potential of vision combines brain machine interface system and can be used to help to lose activity control power Patient realizes the operation of intelligent machine.Outside the successful Application of these systems makes increasing individuals with disabilities to break away from Week N&M pull cause cannot regain the energy mutual with external information with the obstacle of extraneous normal communication Power.This is helpful for the quality of life improving them.Additionally, brain machine interface system is at following and wearable device Combine, thus it is possible to vary existing man-machine interaction mode, make daily life more convenient.

SSVEP can be to reliably applied to brain-computer interface system BCIs (Brain Computer Interface).Phase For giving the BCIs of other signals (such as P300, Mental imagery), the BCIs of SSVEP is generally of more High rate of information transmission, system and experimental design are easier, and the frequency of training needed is the most fewer.

Now concerning Steady State Visual Evoked Potential and brain machine interface system at drive assist system (such as Publication No. The patent of CN104461007A) and the application such as sick room calling system (such as the granted patent of Publication No. CN101159086B) In obtained in-depth study.At present, the use of mobile phone and panel computer is the most universal, but existing brain machine connects Port system is also applied to mobile device such as mobile phone and panel computer, and traditional brain machine not over the Internet and wireless network Speed and the not high enough problem of precision is there is in interface system when analyzing user and being intended to.

Summary of the invention

Present invention solves the technical problem that and be: provide one to realize brain idea and control mobile terminal, and there is higher speed Degree and the brain machine interface system of precision.

The solution proposed for this present invention is: a kind of mobile phone music Play System based on Steady State Visual Evoked Potential, It includes visual stimulus module, signal acquisition module, signal processing module, real-time Transmission module and music playing module Five parts.SSVEP Evoked ptential visual stimulus is carried by visual stimulus module with LCD or CRT or HDU display mode Supply user；Signal acquisition module utilizes electroencephalogramsignal signal collection equipment to gather the EEG signals data of user, and will Real-time data transmission is to data processor；Signal processing module utilizes data processor to go baseline to float signal Shifting, filtering, CCA analyze and feature extraction, thus judge the operation that user is desired with；Real-time Transmission module For analysis result is sent to mobile phone by the Internet or bluetooth or WiFi；Music playing module receives analysis result also Corresponding reaction can be made.

As preferably, above-mentioned SSVEP Evoked ptential visual stimulus includes 4 white square according to different frequency flicker, Represent " play/suspend ", " instruction input switch ", " upper one is bent " and " next is bent " respectively；Do not play in music In the case of, user watches square corresponding to " play/suspend " attentively can send play instruction to music player, at sound In the case of happy broadcasting, user watches square corresponding to " play/suspend " attentively can send pause instruction；" instructing defeated Enter switch " close in the case of, user watches square corresponding to " instruction input switch " attentively can allow instruction input switch Opening, in the case of " instruction input switch " is opened, user watches the square that " instruction input switch " is corresponding attentively Instruction input switch can be allowed to close；Only in the case of " instruction input switch " is opened, music could be broadcast by user Put device to send instruction and make player carry out corresponding operating；Every fixing duration t₁Second LCD or CRT or HDU screen blank screen t₂Second, wherein, t1 is the acquisition time section of the eeg data that DAP will be analyzed, and t2 is used for pointing out use Person can carry out other operation, and t1+t2 is an operation cycle.

As preferably, the length of above-mentioned t1 is between 5 seconds to 10 seconds, and the length of t2 is between 0.1 second to 0.5 second.

Further, as the most above-mentioned 4 according to the frequency corresponding to white square of different frequency flicker be respectively 8Hz, 9Hz、11Hz、12Hz。

Instruction input switch open under conditions of, real-time Transmission module every an operation cycle by signal processing module Analysis result send mobile phone music player to by the Internet or bluetooth or WiFi.

In signal acquisition module, eeg signal acquisition frequency desirable 800～1200Hz, leading of choosing be PZ, PO7, PO3, POZ, PO4, PO8, O1, OZ, O2, between programming realization electroencephalogramsignal signal collection equipment and data processor The interface of eeg data transmission in real time.

Which square signal processing module is for judge that user watches attentively, successively the data gathered is gone base Line drift, Butterworth filtering, CCA analysis, simple Nonlinear Classification, Butterworth filtering is used for filtering off below 7Hz With the ripple of more than 30Hz, analyze a length of t when gathering every time₁The eeg data of second, because different squares represents different Instruction, the flicker frequency of the white square watched attentively in obtaining the user previous cycle, thus analyze user wish into The operation of row.

Above-mentioned music playing module is preferably mobile phone music player APP, and it is broadcast after receiving analysis result immediately Put, suspend, the upper one bent or operation of next song.

Beneficial effect:

The method that the present invention proposes need not user and carries out any limb action, it is only necessary to user watch attentively LCD (or CRT/HDU etc.) screen, by the EEG signals gathered is analyzed, it is possible to realize user and mobile phone music is broadcast Put the control of device.The speed that this mobile phone music player control method identification user is intended to is fast, and accuracy rate is high, for Following brain-computer interface utilization in terms of life & amusement is of great importance.

Accompanying drawing explanation

Fig. 1 is the flow chart of the present invention.

Fig. 2 is visual stimulus module map.

Fig. 3 is that initial data reads result.

Fig. 4 is baseline correction result.

Fig. 5 is Filtering Processing result.

Fig. 6 is CCA analysis result.

Detailed description of the invention

A kind of based on Steady State Visual Evoked Potential the mobile phone music with instantiation, the present invention provided below in conjunction with the accompanying drawings The method of Play System is described in detail.

The ultimate principle of the present invention is when user needs to operate mobile phone music player, it is not necessary to user Carry out any limb action, it is only necessary to user watches the LCD screen such as (or CRT/HDU) attentively, carries out with different frequency The white square of flicker represents different operational orders, when user is look at different white square, can regard at brain Feel that cortex can produce a continuous print with stimulus frequency about the response of (at the fundamental frequency of stimulus frequency or frequency multiplication), pass through Electroencephalogramsignal signal collection equipment carries out online acquisition to EEG signals, and by the EEG signals gathered being analyzed place Reason, it is possible to show that the operation of user is intended to, then analysis result is transferred to mobile phone music by implementing transport module Playing module, thus realize the user control to mobile phone music player.

Provide below a specific embodiment, be described in detail with the enforcement to the present invention.

In the present embodiment, electroencephalogramsignal signal collection equipment uses NeuroScan equipment, and data processor is on MATLAB Realize.Use BCI2000 platform to realize real-time eeg data between electroencephalogramsignal signal collection equipment and data processor to pass Defeated interface, the function of programming realization data processor on MATLAB.With NeuroScan coordinative composition of equipments Scan4.5 software by the EEG signals data of collection by BCI2000 platform real-time online send MATLAB to.

With reference to Fig. 1, this system includes visual stimulus module, signal acquisition module, signal processing module, real-time Transmission mould Block and five parts of music playing module.

Visual stimulus module is as shown in Figure 2.According to existing SSVEP Evoked ptential technology, design SSVEP induction electricity Position visual stimulus includes 4 white square flashed according to different frequency (desirable 8Hz, 9Hz, 11Hz, 12Hz), Represent the broadcasting/time-out of music respectively, without operation, upper one bent and next song；Use LCD display mode.The length of t1 Between 5 seconds to 10 seconds, the length of t2 is between 0.1 second to 0.5 second.Here t is taken₁=5.9, t₂=0.1, often Every 5.9 seconds lcd screen blank screens 0.1 second, it is used for pointing out user can carry out other operation.Wherein, user end It is sitting in the position from lcd screen 0.5m, and enables eyes to look squarely lcd screen center.

Use NeuroScan equipment Real-time Collection user eeg data, the desirable 1000Hz of eeg signal acquisition frequency, Wherein, owing to SSVEP Evoked ptential is primarily generated at the rear pillow part of brain, so, according to " 10-20 international standard is led Connection ", choose position on electrode cap and be numbered nine of PZ, PO7, PO3, POZ, PO4, PO8, O1, OZ, O2 and lead Connection, the default location on the electrode cap that NeuroScan is equipped with is chosen in reference electrode and earth polar.

Signal processing module is mainly realized by MATLAB, and after receiving EEG signals data, MATLAB is every 6 Second processes the most first 6 seconds EEG signals data gathered, owing to 0.1 second last lcd screen is in black state, Therefore need the data by this 0.1 second gathers to cast out.

Brain wave is carried out process and includes that baseline drift, Butterworth filtering (filter off below 8Hz's and more than 30Hz Ripple), CCA (Canonical Correlation Analysis) analyze, simple Nonlinear Classification, analyze collection every time The Shi Changwei eeg data of 5.9 seconds.Because different squares represents different instructions, thus analyzes user and wish The operation carried out.

Wherein, brain wave processes that step is specific as follows (takes user and watch the brain of the white square with 9Hz frequency scintillation attentively Electricity data instance):

The figure that the original eeg data of 5.9 seconds records of duration is drawn is as shown in Figure 3.

(1) baseline drift is gone

Visual stimulus occur the moment brain potential be SSVEP produce starting point, original SSVEP should using this some position as Reference signal measures.But due to DC drift, noise, various interference and the existence of spontaneous potential, this moment Current potential is unstable, and the starting point current potential difference of each event is the biggest, it is impossible to measure SSVEP as reference potential. Solution removes baseline drift exactly: takes and stimulates the meansigma methods that interior EEG signals for the previous period occurs as SSVEP Reference potential because the impact of the not irriate of the Evoked ptential in this time period, and DC drift, noise, interference and The fluctuation of spontaneous brain electricity is change at random, and the meansigma methods in the sufficiently long time is close to zero.This is as with reference to electricity The meansigma methods of position is exactly baseline.

In instances, the meansigma methods of data of front 200ms can be chosen as baseline, and carry out the removal of baseline drift. Result is as shown in Figure 4.

(2) filtering

Priori from SSVEP signal: the frequency of event related potential is mainly distributed on low frequency range: 8～30Hz, Therefore Butterworth filter can be used to carry out bandpass filtering, in MATLAB, butter function can be directly invoked With filtfilt function.Result is as shown in Figure 5.

(3) CCA analyzes

In carrying out the Steady State Visual Evoked Potential canonical correlation analysis with reference signal, we the most only consider that first is right Relevant canonical variable, the most only considers maximum correlation, because it has best descriptive power.Correlation coefficient concrete Algorithm is as follows: first, canonical correlation analysis is PZ that Steady State Visual Evoked Potential is stronger, PO7, PO3, POZ, PO4, Signal X and reference signal Y that PO8, O1, OZ, O2 nine lead find a pair vector W_XAnd W_Y, maximize phase Close variable x=X^TW_XAnd y=Y^TW_YBetween dependency.Pass through following formula:

$\underset{W_x,W_y}{\max }\mathrm{\ρ}=\frac{E\[x^{T}y\]}{\sqrt{E\[x^{T}x\]E\[y^{T}y\]}}=\frac{E\[W_X^T{\mathrm{XY}}^T{W}_Y\]}{\sqrt{E\[W_X^T{\mathrm{XX}}^T{W}_X\]E\[W_Y^T{\mathrm{YY}}^T{W}_Y\]}}$

The maximum correlation coefficient between X and Y can be obtained.In research process, reference signal Y is generally set by we For:

$Y=\left(\begin{array}{c}sin\left(2\mathrm{\π}ft\right)\\ cos\left(2\mathrm{\π}ft\right)\\ \sin \left(4\mathrm{\π}ft\right)\\ \cos \left(4\mathrm{\π}ft\right)\end{array}\right)$

As four road signals, wherein f is stimulus frequency, takes 8Hz to 13Hz, increases and is spaced apart 0.25Hz.Owing to adopting Integrate frequency as 1000Hz, so t takes 1/1000 to 5000/1000, be spaced apart 1/1000.We are it is believed that maximum phase Closing the frequecy characteristic corresponding to coefficient is exactly stimulus frequency.

Stimulus frequency from the flicker frequency of disparate modules on stimulator, stimulator can be arranged several (according to Different needs can arrange different number) flashing module, each module has the flicker frequency of oneself.Experimenter needs note Stimulating 5.9s depending on flicker, flashing module excites brain to produce Steady State Visual Evoked Potential.Electrode is placed scalp location, Collect 9 signals led.The sample frequency of data is 1000Hz, is filtered between 7～30Hz.Choose wherein Visual stimulus is reacted territory, rear occipital region PZ, PO7, PO3, POZ, PO4, PO8, O1, OZ, O2 of relative sensitive Nine are led and carry out canonical correlation analysis.

The operational order that different frequency representative is different.According to the frequency calculated, it can be deduced that user is wanted to carry out Operation, as shown in Figure 6, highest point respective frequencies is 9.25Hz to CCA analysis result.

(4) simple Nonlinear Classification

If the reference frequency that maximum correlation coefficient is corresponding is F, then have:

$f=\left\{\begin{array}{cc}8Hz& 7.5Hz\&le;F\&le;8.5Hz\\ 9Hz& 8.5Hz<F\&le;9.5Hz\\ 11Hz& 10.5Hz\&le;F\&le;11.5Hz\\ 12Hz& 11.5Hz<F\&le;12.5Hz\end{array}\right.$

In this example, due to F=9.25Hz, therefore the frequency that can obtain the flicker square that user is watched attentively is 9Hz.

In native system, signal processing module utilizes data processor that signal goes baseline drift, filtering, CCA Analyzing and feature extraction, feature extraction mainly carries out phase by CCA to Steady State Visual Evoked Potential and reference potential Pass property is analyzed, and extracts the frequecy characteristic that in signal, correlation coefficient is maximum, thus judges the behaviour that user is desired with Making, CCA analytical calculation is simple, and required eeg data is less, has good real-time.

In real-time delivery module, computer by the analysis result of signal processing module by the Internet (or bluetooth/WiFi Deng) send mobile phone music player to.

In music playing module, music player APP carries out the operation of correspondence after receiving analysis result immediately, Final realization feature extraction based on Steady State Visual Evoked Potential controls mobile phone music player.

It will be clear that in order to the example making enforcement is more detailed, above embodiment is preferred embodiment, right Other substitute mode can also be used to implement in some known technologies those skilled in the art；And accompanying drawing part be only for Embodiment is more specifically described, it is no intended to the present invention carried out concrete restriction.

The present invention is not limited to the concrete technical scheme described in above-described embodiment, the technical side that all employing equivalents are formed Case is the protection of application claims.

Claims (8)

1. a mobile phone music Play System based on Steady State Visual Evoked Potential, it is characterised in that include visual stimulus module, letter Number acquisition module, signal processing module, real-time Transmission module and five parts of music playing module, visual stimulus module is by SSVEP Evoked ptential visual stimulus is supplied to user with LCD or CRT or HDU display mode；Signal acquisition module utilizes brain telecommunications Number collecting device gathers the EEG signals data of user, and by real-time data transmission to data processor；Signal processing module Utilize data processor that signal goes baseline drift, filtering, CCA analyze and feature extraction, thus judge to use The operation that person is desired with；Analysis result is sent to mobile phone by the Internet or bluetooth or WiFi by real-time Transmission module；Music is broadcast Amplification module receives analysis result and can make corresponding reaction.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 1, it is characterised in that described SSVEP Evoked ptential visual stimulus includes 4 white square according to different frequency flicker, represent respectively " play/suspend ", " instruction input switch ", " upper one is bent " and " next is bent "；In the case of music is not play, user watch attentively " play/ Suspend " corresponding square can send play instruction to music player, in the case of music, user watch attentively " play/ Suspending " corresponding square can send pause instruction；In the case of " instruction input switch " closes, user watches " instruction attentively Input switch " corresponding square can allow instruction input switch open, in the case of " instruction input switch " is opened, user Watching square corresponding to " instruction input switch " attentively can allow instruction input switch close；Only open at " instruction input switch " In the case of user music player could be sent instruction and make player carry out corresponding operating；Every fixing duration t₁Second LCD or CRT or HDU screen blank screen t₂Second, t₁For the acquisition time section of the eeg data that DAP will be analyzed, t₂For carrying Show that user can carry out other operation, t₁+t₂It it is an operation cycle.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 2, it is characterised in that described The length of t1 is between 5 seconds to 10 seconds, and the length of t2 is between 0.1 second to 0.5 second.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 2, it is characterised in that described 4 are respectively 8Hz, 9Hz, 11Hz, 12Hz according to the frequency that the white square of different frequency flicker is corresponding.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 2, it is characterised in that referring to Under conditions of making input switch open, the analysis result of signal processing module is passed through by real-time Transmission module every an operation cycle The Internet or bluetooth or WiFi send mobile phone music player to.

The method of mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 1, its feature exists In in signal acquisition module, eeg signal acquisition frequency desirable 800～1200Hz, leading of choosing be PZ, PO7, PO3, POZ, PO4, PO8, O1, OZ, O2, real-time brain electricity between programming realization electroencephalogramsignal signal collection equipment and data processor The interface of data transmission.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 1, it is characterised in that described Which square signal processing module is for judge that user watches attentively, successively the data gathered is gone baseline drift, bar Special Butterworth filtering, CCA analysis, simple Nonlinear Classification, Butterworth filtering is used for filtering off below 7Hz's and more than 30Hz Ripple, analyzes a length of t when gathering every time₁The eeg data of second, because different squares represents different instructions, before obtaining user The flicker frequency of the white square watched attentively in one cycle, thus analyze the operation that user is desired with.

Mobile phone music Play System based on Steady State Visual Evoked Potential the most according to claim 1, it is characterised in that described Music playing module is mobile phone music player APP, it plays out immediately after receiving analysis result, suspends, on one bent Or the operation of next song.