CN114527875A - Electroencephalogram data acquisition system and method for synchronously tracking reading content - Google Patents

Electroencephalogram data acquisition system and method for synchronously tracking reading content Download PDF

Info

Publication number
CN114527875A
CN114527875A CN202210135418.8A CN202210135418A CN114527875A CN 114527875 A CN114527875 A CN 114527875A CN 202210135418 A CN202210135418 A CN 202210135418A CN 114527875 A CN114527875 A CN 114527875A
Authority
CN
China
Prior art keywords
reading
reader
eyeball
electroencephalogram
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210135418.8A
Other languages
Chinese (zh)
Inventor
何熲
高军晖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Nuanhenao Science And Technology Co ltd
Original Assignee
Shanghai Nuanhenao Science And Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Nuanhenao Science And Technology Co ltd filed Critical Shanghai Nuanhenao Science And Technology Co ltd
Priority to CN202210135418.8A priority Critical patent/CN114527875A/en
Publication of CN114527875A publication Critical patent/CN114527875A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Hospice & Palliative Care (AREA)
  • Child & Adolescent Psychology (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Epidemiology (AREA)
  • Developmental Disabilities (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides an electroencephalogram data acquisition system for synchronously tracking reading contents, which comprises an electronic reader serving as a client, an electroencephalogram recording subsystem and a server; the electroencephalogram recording subsystem collects electroencephalogram data and sends the electroencephalogram data to the electronic reader; the server side maintains and provides reading materials; the electronic reader sets a reading environment and selects reading materials before reading, displays the reading materials during reading, performs real-time reading tracking, outputs synchronous data of paragraph ID of the reading materials and electroencephalogram data, and uploads the output data to a server. The invention also provides a corresponding method. The system and the method of the invention determine the influence of the reading material on a certain person by synchronizing the paragraph ID of the reading material and the synchronous data of the electroencephalogram data, thereby providing more accurate reading therapy for each individual needing psychological treatment.

Description

Electroencephalogram data acquisition system and method for synchronously tracking reading content
Technical Field
The invention belongs to the field of electroencephalogram data acquisition, and particularly relates to an electroencephalogram data acquisition system and method.
Background
Electroencephalography (EEG) is a method of recording brain activity using electrophysiological markers, in which post-synaptic potentials generated in synchronization with a large number of neurons are summed up during brain activity. It records the electrical wave changes during brain activity, which is a general reflection of the electrophysiological activity of brain neurons on the surface of the cerebral cortex or scalp.
The brain waves originate from the postsynaptic potential of the dendrites at the apex of pyramidal cells. The formation of brain wave-synchronized rhythms is also associated with the activity of the cortical thalamic non-specific projection system. Brain waves are the fundamental theoretical research of brain science, and brain wave monitoring is widely applied to clinical practice and application of the brain waves.
The reading therapy is a method for helping readers relieve negative emotional disturbance by reading books or contacting other information materials so as to achieve the state of physical and mental balance. When people read novels, different physiological reactions such as electroencephalogram, electrocardio, respiration and the like can be generated due to different reading contents. Even in the same novel, different people respond differently.
Therefore, in order to provide more accurate reading therapy to each individual requiring psychological treatment, it is necessary to evaluate the reading therapy so as to know the influence of different types of novels (reading materials) on a person.
The prior art evaluated for reading therapy has several:
1) the first prior art is a method and a system for automatic psychological problem detection and automatic matching recommendation of a book reading party, which are disclosed in a patent document with the publication number of CN113160940A filed by Jiangsu university, wherein automatic calculation is performed according to detection results of various scales filled by a user on line, and psychological health conditions, psychological problems and causes of the user are obtained and stored in a correlation manner; and designing a reading and treating module, and automatically pushing a matched book according to a detection result. The system can automatically detect the psychological health problem and the back complex cause thereof, automatically match and recommend books to read according to the symptoms, reduce the labor cost, improve the efficiency and accuracy of recommendation and reading, realize the automatic perfection of the recommendation system of a book reading party, and well protect the privacy of users. The system can be applied to automatic detection of psychological problems of college students, primary and secondary school students, parents, criminals, drug addicts, social ordinary persons and the like and recommendation of accurate reading books, and is simultaneously suitable for automatic detection and evaluation of psychological problems of drug addicts and drug addicts as well as automatic detection and evaluation of psychological problems of drug addicts and treatment effects of criminals.
However, the above system only uses a scale to collect data, does not perform physiological signal collection, and does not track the reading process, so the reliability and accuracy of the data are further enhanced.
2) The second prior art is a book reader based on reading therapy disclosed in patent document with publication number CN108389629A at north China university of science and technology, wherein an electronic book reader body for reading patients during reading therapy is disclosed, the electronic book reader body comprises a touch display screen for reading characters and handheld portions respectively located at the left end and the right end of the touch display screen, a power module is further arranged inside the electronic book reader body, and the book reader further comprises a microprocessor, a patient identity recognition module, a physiological index acquisition unit during reading, a storage unit, an analysis comparison unit, a mental health determination unit, a mental health status output unit and a wireless communication module. The reading device can judge the curative effect of the patient in the treatment process by monitoring the physiological index condition of the patient in real time in the reading treatment process of the patient, so that the treatment scheme can be adjusted at any time according to the performance of the patient in the treatment process, a treatment effect reference conclusion can be timely provided for the patient, and the patient can be more effectively read and treated.
The second prior art has the disadvantage that the tracking reading process is too simple to determine which position the patient has specifically read, so that the physiological index signal and the material cannot be recorded synchronously.
3) The third prior art is the Shanghai deal emotional electroencephalogram data Set (SEED). SEED is an emotion calculation dataset based on 62-lead EEG signals published by Shanghai university of transportation, Lubao food professor 2015 at 10 months (see in particular websites [ https:// bcmi.sjtu.edu.cn/home/SEED ], documents [ Zheng W L, Lv B L.inquiring diagnostic bands and channels for EEG-based examination registration with deep neural networks, IEEE Transactions on autoimmune Mental device, 2015, 7 (3): 162 and 175 and documents [ Zheng WL, Liu W, Lu Y F, Lv B L, Circhoki A.Emion metals, A for learning model 175 and 1110, 2019, IEEE 2019). The SEED data set currently includes three subsets: SEED, SEED-IV, and SEED-VIG, where the SEED subset is the three types of emotion data sets that were released earliest, the SEED-IV subset is the four types of emotion data sets, and the SEED-VIG subset is an alertness estimation data set. The SEED data set induces different emotions of a testee through a movie video segment, and mainly comprises two parts, wherein one part is an acquired EEG signal of the testee in an experiment, and the EEG signal is subjected to down-sampling, filtering, artifact removal and other operations after being acquired so as to improve the quality of the EEG signal (it is worth pointing out that the sampled signal is firstly filtered and then is subjected to down-sampling treatment from the aspect of signal treatment, so that the information contained in the original signal is better kept); another part of the data is feature extraction performed on the preprocessed EEG signal, including various features such as Power Spectral Density (PSD), Differential Entropy (DE), asymmetric difference of Differential entropy (DASM), asymmetric quotient of Differential entropy (RASM), and the like, and features are smoothed by a moving average and Linear Dynamic System (LDS).
The third prior art has the disadvantage that it requires manual segmentation of the material to determine which segment the subject specifically sees, which leads to emotion, and cannot automatically track the viewing process of the movie, since the segments must be manually done, and the SEED has only 6 works; it is not automatically determined which position the subject has specifically read, so that the signal is recorded in synchronism with the material, requiring manual assistance.
In summary, the prior art has the disadvantage that the automatic tracking reading process is not or too simple and the synchronous recording of the signal and the material cannot be performed automatically.
Disclosure of Invention
The invention aims to provide an electroencephalogram data acquisition system and method for synchronously tracking reading contents, so that the problem that synchronous recording of signals and materials cannot be automatically carried out in the prior art is solved.
In order to solve the technical problem, the invention provides an electroencephalogram data acquisition system for synchronously tracking reading contents, which comprises an electronic reader serving as a client, an electroencephalogram recording subsystem and a server; the electroencephalogram recording subsystem is used for acquiring electroencephalogram data and sending the electroencephalogram data to the electronic reader; the server is set to maintain and provide reading materials, maintain reader information, and collect and export data output by the electronic reader; the electronic reader is set to set a reading environment and select reading materials before reading, displays the reading materials during reading, carries out real-time reading tracking, outputs synchronous data of the paragraph ID of the reading materials and the electroencephalogram data, and uploads the output data to the server.
The electronic reader comprises a screen, a camera and Bluetooth, wherein the screen is set to display reading materials in a server and display a menu, and the menu is provided with setting items for a reader to set parameters through the setting items; the camera is set to shoot an eyeball state image so as to determine the watching position of the eyeball and whether the eyeball is in an eye closing state or not, and then real-time reading tracking is carried out; the Bluetooth is set to receive the electroencephalogram data sent by the electroencephalogram recording subsystem.
On the other hand, the invention provides an electroencephalogram data acquisition method for synchronously tracking reading contents, which is based on the electroencephalogram data acquisition system for synchronously tracking reading contents, and comprises the following steps:
utilizing an electronic reader as a client to execute the following steps:
step S1: carrying out account login to obtain reader information according to the account and the login password of the reader; or setting an account number to acquire reader information;
step S2: acquiring reading environment settings and reading materials;
step S3: displaying reading materials and performing real-time reading tracking during reading;
step S4: outputting synchronous data of the paragraph ID of the reading material and the electroencephalogram data, and uploading the output data to a server;
and the server executes the following steps:
step S1': maintaining the reading material; wherein, the reading material maintenance means adding the reading material and automatically cutting the reading material;
step S2': maintaining reader information;
step S3': and summarizing and exporting output data.
The reader information comprises an account number, a login password, a name, an age, a sex, a distance between eyes of a reader and the reader, an eyeball state image of a closed eye state and eyeball state images of a plurality of fixation positions of the electronic reader;
and the step S2 includes: before reading, setting a reading environment and selecting reading materials from a server side for downloading, or directly reading the reading environment and the downloaded reading materials which are set before the last reading.
The electronic reader comprises a screen, wherein the screen is set to display reading materials in a server and display a menu, and the menu is provided with setting items for a reader to set parameters through the setting items;
in step S1, the account setting is performed through a menu displayed on a screen, and includes: setting the distance between the eyes of a reader and the reader through manual input;
setting name, age and gender by manual input;
setting an eyeball state image of the closed eye state by shooting the image of the closed eye state; and
and setting eyeball state images of a plurality of fixation positions of the electronic reader.
The number of the fixation positions is 9, and the fixation positions comprise upper left, middle upper right, middle left, middle center, middle right, lower left, middle lower middle and lower right; or the number of the gazing positions is 5, and the number comprises upper left, upper right, middle, lower left and lower right;
the eyeball state images of a plurality of fixation positions of the electronic reader are specifically set through the following steps: and sequentially displaying a picture at a plurality of positions on the screen by the electronic reader, sequentially watching each position by the reader, and sequentially pressing the shooting button to shoot and obtain eyeball state images of the plurality of watching positions of the electronic reader and store the eyeball state images.
In step S3, when reading, the electronic reader first displays the first page of the reading material on the screen, and each time the reader turns back or forward a page, the electronic reader refreshes the display content accordingly.
The electronic reader comprises a camera;
the step S3 includes: every interval of a first interval time, the following steps are executed:
step S31: the electronic reader shoots a current eyeball state image of the reader by using a camera of the electronic reader, judges whether the eyeball is in an eye closing state or not according to a matching result of the current eyeball state image and the eyeball state image in the eye closing state, and judges whether the eyeball blinks or not by using current electroencephalogram data;
step S32: according to the judgment result, if the eyeball is in a blinking state or an eye closing state, the current electroencephalogram data are abandoned;
otherwise, comparing the current eyeball state image with eyeball state images of a plurality of gaze positions of the electronic reader, calculating the eyeball gaze positions, and further determining paragraph IDs corresponding to the eyeball gaze positions according to the paragraph IDs corresponding to the paragraphs displayed at each position of each page by the reading material; then entering a sleep state;
and the step S3 further includes: every second interval, electroencephalogram data are collected.
The step S3 further includes: and during reading, carrying out face emotion recognition.
The output data includes the synchronization data, reading time, reader ID, article ID of the reading material, and brain wave type.
The electroencephalogram data acquisition system and the method for synchronously tracking the reading content determine the influence of different types of novels (reading materials) on a certain person through the synchronization of the paragraph ID of the reading materials and the synchronous data of the electroencephalogram data, so that more accurate reading therapy can be provided for each individual needing psychological treatment. The invention determines the chapters and paragraphs in the reading material through the camera, realizes reading tracking, synchronously records the electroencephalogram data corresponding to the chapters and paragraphs, can help people to understand individual differences and group commonalities in the reading material more deeply, and lays a foundation for the subsequent development of accurate reading therapy. In addition, the big data cloud storage of the synchronous data is realized through the server side, and the subsequent data mining is facilitated.
Drawings
FIG. 1 is a schematic structural diagram of an electroencephalogram data acquisition system for synchronously tracking reading contents.
Detailed Description
The present invention will be further described with reference to the following specific examples. It should be understood that the following examples are illustrative only and are not intended to limit the scope of the present invention.
As shown in FIG. 1, the electroencephalogram data acquisition system for synchronously tracking reading contents of the invention comprises an electronic reader 10 as a client, an electroencephalogram recording subsystem 20 and a server 30.
The electroencephalogram recording subsystem 20 can be a brain ring device such as NeuroSky, Muse, Emotive and the like, is configured to acquire electroencephalogram data, and sends the electroencephalogram data to the electronic reader 10 serving as a client through bluetooth. Neurosky, Muse and Emotive have SDK for secondary development, so that the functions are not difficult to realize.
In the present embodiment, the electroencephalogram data is EEG (electroencephalogram), but in other implementations, the electroencephalogram data is not limited to EEG, and can be generalized to various electroencephalogram acquisition methods.
The server 30 is configured to maintain and provide reading material, maintain reader information, and aggregate and export data output by the electronic reader 10.
The electronic reader 10 is configured to set a reading environment and select a reading material before reading, display the reading material and perform real-time reading tracking during reading, output synchronous data of the paragraph ID of the reading material and the electroencephalogram data, and upload the output data to the server 30. The output data includes, in addition to the paragraph ID and the electroencephalogram data of the reading material: reading time, reader ID, article ID of the reading material, and brain wave type. Wherein, the brain wave type is one of alpha, beta, gamma, delta and theta.
The electronic reader 10 comprises a screen 11, a camera 12, Bluetooth 13 and the like; the screen 11 is configured to display reading materials in the server 30; the camera 12 is configured to capture an eye state image to determine the gaze position of the eye and whether the eye is in a closed state, so as to perform real-time reading tracking. The bluetooth 13 is configured to receive the electroencephalogram data sent by the electroencephalogram recording subsystem 20. The screen 11 may also be arranged to display a menu with setting items for the reader to set parameters by setting items.
The electronic reader 10 may be a computer version, or a mobile phone APP installed on a mobile phone, etc.
Based on the electroencephalogram data acquisition system for synchronously tracking the reading content, the electroencephalogram data acquisition method for synchronously tracking the reading content comprises the following steps:
(1) the following steps are performed with the e-reader 10 as a client:
step S1: carrying out account login to obtain reader information according to the account and the login password of the reader; or setting an account number to acquire reader information;
the reader information includes an account number, a login password, a name, an age, a sex, a distance between the eyes of the reader and the reader, an eyeball state image of a closed-eye state, eyeball state images of a plurality of fixation positions of the electronic reader 10, and the like.
The account setting is performed through a menu displayed on the screen 11, and includes:
setting the distance between the eyes of the reader and the reader (through manual input);
setting name, age, gender (by manual input);
setting an eyeball state image of the closed-eye state (by taking an image of the closed-eye state); and
eyeball state images of a plurality of fixation positions of the electronic reader 10 are set. In this embodiment, the number of the gazing positions is 9, including upper left, middle upper right, upper left, middle center, middle right, lower left, middle lower middle, and lower right; in other embodiments, the number of gaze locations may also be 5, including upper left, upper right, center, lower left, and lower right, taking into account that the screen 11 may be relatively small.
The setting of the eyeball state images of the plurality of fixation positions of the electronic reader 10 is specifically performed by the following steps:
a picture is sequentially displayed at a plurality of positions on the screen 11 by the electronic reader 10 (except that a shooting button is arranged below the screen 11, and any display except the picture is not arranged in other areas), the reader sequentially looks at each position and sequentially presses the shooting button to shoot and obtain eyeball state images of the plurality of watching positions of the electronic reader 10 and store the eyeball state images.
In addition, in some embodiments, after the eyeball state images of a plurality of gazing positions of the electronic reader 10 are all captured, the electronic reader 10 calculates whether the eyeball state images have problems, and if so, re-captures the eyeball state images when gazing at some gazing positions of the electronic reader 10.
Step S2: acquiring reading environment settings and reading materials;
wherein the step S2 includes: before reading, setting a reading environment and selecting the reading material from the server (30) to download, or directly reading the reading environment set before the last reading and the downloaded reading material
Step S3: displaying reading materials and performing real-time reading tracking during reading;
after the reader selects the reading material in step S2, in step S3, when reading, the electronic reader 10 first displays the first page of the reading material on the screen 11, and each time the reader turns back or forward a page, the electronic reader 10 refreshes its display content accordingly.
It should be noted that the reading materials are automatically cut at the server 30 in advance, and the automatic cutting of the reading materials is completed at the server 30, so that the display positions of the text paragraphs seen by all readers on the screen are the same, and the electronic reader 10 knows the paragraph IDs corresponding to the paragraphs displayed at the positions of each page of the reading materials.
The step S3 is used to determine the eyeball state, obtain the content of the gaze location of the electronic reader 10 according to the eyeball state, and optionally perform face emotion recognition.
Therefore, the step S3 includes:
the following steps are performed once every first interval (the first interval may be set to 5 seconds, 1 second minimum, and 10 seconds maximum):
step S31: the electronic reader 10 shoots a current eyeball state image of a reader by using a camera 12 of the electronic reader, judges whether the eyeball is in an eye closing state according to a matching result of the current eyeball state image and the eyeball state image in the eye closing state, and judges whether the eyeball blinks by using current electroencephalogram data;
among them, eye closure is an action actively taken by the reader, generally lasting at least several seconds; blinking is the normal behavior of the eyes, only about 0.1 second is needed, and the blinking is unconsciously completed, and normal people blink once every few seconds to ten seconds.
Step S32: according to the judgment result, if the eyeball is in the blinking state or the eye closing state, the current electroencephalogram data is abandoned to directly enter the dormant state, and the next execution is waited from the step S31; the fixation area of the reader cannot be calculated when eyes are closed, so that the reader needs to directly wait for the next execution.
Otherwise, it indicates that the eyeball is not in a blinking or eye closing state, compares the current eyeball state image with the eyeball state images of the plurality of gaze positions of the electronic reader 10, calculates the gaze position of the eyeball, and determines a paragraph ID corresponding to the gaze position of the eyeball according to the paragraph ID corresponding to each of the paragraphs displayed at each position of each page of the reading material; then, the sleep state is entered to wait for the next execution from step S31.
The eyeball fixation position is obtained by comparing the current eyeball state image with eyeball state images of a plurality of fixation positions of the electronic reader 10 and deducing the eyeball state by using an interpolation method.
Thus, the system of the present invention obtains the paragraph ID of the current reading once every first interval (or enters the sleep state due to being in the blinking or eye-closing state).
The step S3 further includes: every second interval, electroencephalogram data are collected. In the present embodiment, the second interval time is 1 second.
The step S3 may further include: and during reading, carrying out face emotion recognition. The human face emotion recognition only needs a universal and popular algorithm, so that the relation between reading materials and emotion can be counted, and the consistency of electroencephalogram-based emotion calculation and human face-based emotion calculation can be compared.
The step S3 may further include: when reading, according to the reading history record, the reader is reminded that the reader has read. In addition, other non-core functions may also be included, including, for example: reminding of too long reading time, etc.
Step S4: and outputting the paragraph ID of the reading material and the synchronous data of the electroencephalogram data, and uploading the output data to the server 30.
The output data includes, in addition to synchronization data (paragraph ID of reading material and electroencephalogram data), also: reading time, reader ID, article ID of the reading material, and brain wave type. Wherein, the brain wave type is one of alpha, beta, gamma, delta and theta.
In this embodiment, since the system of the present invention obtains the currently read paragraph ID (or enters the sleep state due to being in the blink or eye-closing state) every a first interval (5 seconds), the electroencephalogram recording subsystem 20 sends the electroencephalogram signal to the electronic reader 10 1 time every 1 second, and 5 batches of data are obtained 5 seconds, so that the synchronous data of the paragraph ID of the reading material and the electroencephalogram signal includes a paragraph ID and five batches of corresponding electroencephalogram data.
In other embodiments, the synchronous data of the synchronous output is determined according to the ratio of the first interval time and the second interval time. If the second interval time is 1 second and the first interval time is 1 second, the synchronous data output synchronously is a paragraph ID and a batch of electroencephalogram data. If the second interval time is 1 second and the first interval time is 10 seconds, the synchronous data output synchronously is a paragraph ID and ten batches of electroencephalogram data.
Therefore, the electroencephalogram data can synchronously correspond to the paragraph ID of the reading material.
(2) And the server 30 executes the following steps:
step S1': maintaining the reading material;
the reading material maintenance refers to adding reading materials by a system administrator, automatically cutting the reading materials and the like.
When reading materials are added, the famous literary works (famous works) are selected as the added reading materials, such as a miniature novel (which can be read out in 1 page and 2 minutes) and a short novel (which can be read out in 10 pages and within 5000 characters and within 10 minutes).
The reading materials are automatically segmented by using a natural language processing tool, the article ID and the paragraph ID of each piece of reading materials are set as the reading material ID according to the segmentation result, the reading materials with the reading material ID are finally obtained, and the paragraph IDs corresponding to the paragraphs displayed at the positions of each page of the reading materials are determined for later synchronous tracking. Wherein the article ID consists of 7 characters and the paragraph ID consists of 3 numbers (001-.
In this embodiment, the reading material is literary work, however in other embodiments, the reading material may be generalized to all literary materials. The invention further utilizes the direction of eyeballs to synchronously track the segmented reading materials, and is more accurate.
Step S2': maintaining reader information;
the reader information includes an account number, a login password, a name, an age, a sex, a distance between the eyes of the reader and the reader, an eyeball state image of a closed-eye state, eyeball state images of a plurality of fixation positions of the electronic reader 10, and the like. Maintaining reader information refers to performing data backup or the like on the server 30.
Step S3': and summarizing and exporting output data.
The output data includes, in addition to synchronization data (paragraph ID of reading material and electroencephalogram data), also: reading time, reader ID, article ID of the reading material, and brain wave type. Wherein, the brain wave type is one of alpha, beta, gamma, delta and theta.
The server 30 is started before the client executes, the login of the client is synchronized to the server, and the server knows which client is currently logged in, (1) the server stores account setting information of the client, (2) the server stores an article available for downloading and reading, and (3) the server receives output data uploaded after the client finishes reading, wherein the output data comprises: reading time, reader ID, reading material ID (including article ID and paragraph ID), and brain wave type.
And (3) comparing the results:
as mentioned above, there are 3 prior art techniques or schemes most similar to the present invention, and the results of comparing the present invention with the three prior art techniques are shown in the table below.
TABLE 1 comparison of the four methods
Figure BDA0003504362200000111
It can be seen that the Shanghai deal emotion electroencephalogram data Set (SEED) provided by Shanghai university of transportation is the closest prior art. Compared with the SEED technology of Shanghai traffic university, the invention has the following advantages:
1) because the segmentation must be done manually, SEED has only 6 works; the invention can automatically cut the stimulating material in advance, and can provide a large amount of reading materials;
2) besides electroencephalogram, the invention also accurately tracks the content of reading materials by shooting eyeball states, and SEED does not exist;
3) the invention automatically completes the synchronous data of the electroencephalogram data and the reading material, and the SEED needs manual assistance;
4) the invention uploads the synchronous data to the server to be used as the preparation work of the subsequent big data mining, and the SEED does not have the function.
The electroencephalogram data acquisition system and the method for synchronously tracking the reading content determine the influence of different types of novels (reading materials) on a certain person through the synchronization of the paragraph ID of the reading materials and the synchronous data of the electroencephalogram data, so that more accurate reading therapy can be provided for each individual needing psychological treatment. The invention determines the chapters and paragraphs in the reading material through the camera, realizes reading tracking, synchronously records the electroencephalogram data corresponding to the chapters and paragraphs, can help people to understand individual differences and group commonalities in the reading material more deeply, and lays a foundation for the subsequent development of accurate reading therapy. In addition, the big data cloud storage of the synchronous data is realized through the server side, and the subsequent data mining is facilitated.
The above embodiments are merely preferred embodiments of the present invention, which are not intended to limit the scope of the present invention, and various changes may be made in the above embodiments of the present invention. All simple and equivalent changes and modifications made according to the claims and the content of the specification of the present application fall within the scope of the claims of the present patent application. The invention has not been described in detail in order to avoid obscuring the invention.

Claims (10)

1. An electroencephalogram data acquisition system for synchronously tracking reading contents is characterized by comprising an electronic reader (10) serving as a client, an electroencephalogram recording subsystem (20) and a server (30); the electroencephalogram recording subsystem (20) is used for collecting electroencephalogram data and sending the electroencephalogram data to the electronic reader (10); the server (30) is configured to maintain and provide reading materials, maintain reader information, and summarize and export data output by the electronic reader (10); the electronic reader (10) is set to set a reading environment and select reading materials before reading, display the reading materials and conduct real-time reading tracking during reading, output synchronous data of paragraph ID and electroencephalogram data of the reading materials, and upload the output data to the server (30).
2. The electroencephalogram data acquisition system for synchronously tracking reading contents according to claim 1, wherein the electronic reader (10) comprises a screen (11), a camera (12) and a Bluetooth (13), the screen (11) is arranged to display reading materials in the server (30) and display a menu, and the menu is provided with setting items for a reader to set parameters through the setting items; the camera (12) is used for shooting an eyeball state image so as to determine the gazing position of the eyeball and whether the eyeball is in an eye closing state or not, and then real-time reading tracking is carried out; the Bluetooth (13) is set to receive the electroencephalogram data sent by the electroencephalogram recording subsystem (20).
3. An electroencephalogram data acquisition method for synchronously tracking reading contents, which is based on the electroencephalogram data acquisition system for synchronously tracking reading contents according to claim 1 or 2, and comprises:
performing the following steps with an e-reader (10) as a client:
step S1: carrying out account login to obtain reader information according to the account and the login password of the reader; or setting an account number to acquire reader information;
step S2: acquiring reading environment settings and reading materials;
step S3: displaying reading materials and performing real-time reading tracking during reading;
step S4: outputting the paragraph ID of the reading material and the synchronous data of the electroencephalogram data, and uploading the output data to a server (30);
and the server (30) is used for executing the following steps:
step S1': maintaining the reading material; wherein, the reading material maintenance means adding the reading material and automatically cutting the reading material;
step S2': maintaining reader information;
step S3': and summarizing and exporting output data.
4. The electroencephalogram data acquisition method for synchronously tracking reading contents according to claim 3, wherein the reader information comprises an account number, a login password, a name, an age, a sex, a distance between eyes of a reader and the reader, an eyeball state image of a closed eye state and eyeball state images of a plurality of fixation positions of an electronic reader (10);
and the step S2 includes: before reading, a reading environment is set and reading materials from the server (30) are selected to be downloaded, or the reading environment set before the last reading and the downloaded reading materials are directly read.
5. The electroencephalogram data acquisition method for synchronously tracking reading contents according to claim 4, wherein the electronic reader (10) comprises a screen (11), the screen (11) is set to display reading materials in the server (30) and display a menu, and the menu is provided with setting items for a reader to set parameters through the setting items;
in the step S1, the account setting is performed through a menu displayed on a screen (11), including: setting the distance between the eyes of a reader and the reader through manual input;
setting name, age and gender by manual input;
setting an eyeball state image of the closed eye state by shooting the image of the closed eye state; and
eyeball state images of a plurality of fixation positions of an electronic reader (10) are set.
6. The method for acquiring electroencephalogram data of synchronously tracking reading contents according to claim 5, wherein the number of the gazing positions is 9, including upper left, middle upper, upper right, middle left, middle center, middle right, lower left, middle lower and lower right; or the number of the gazing positions is 5, and the number comprises upper left, upper right, middle, lower left and lower right;
the eyeball state images of a plurality of fixation positions of the electronic reader (10) are set through the following steps: a picture is sequentially displayed at a plurality of positions on a screen (11) by an electronic reader (10), a reader sequentially watches each position and sequentially presses a shooting button so as to shoot eyeball state images of the plurality of watching positions of the electronic reader (10) and store the eyeball state images.
7. The EEG data collection method for synchronously tracking reading contents according to claim 5, characterized in that, in step S3, when reading, the electronic reader (10) firstly displays the first page of the reading material on the screen (11), and every time the reader turns back or forwards a page, the electronic reader (10) refreshes the display content thereof accordingly.
8. The method for acquiring electroencephalogram data for synchronously tracking reading content according to claim 3, wherein the electronic reader (10) comprises a camera (12);
the step S3 includes: every interval of a first interval time, the following steps are executed:
step S31: the electronic reader (10) shoots a current eyeball state image of a reader by using a camera (12), judges whether the eyeball is in an eye closing state according to a matching result of the current eyeball state image and the eyeball state image in the eye closing state, and judges whether the eyeball blinks by using current electroencephalogram data;
step S32: according to the judgment result, if the eyeball is in a blinking state or an eye closing state, the current electroencephalogram data are abandoned;
otherwise, comparing the current eyeball state image with eyeball state images of a plurality of fixation positions of the electronic reader (10), calculating the eyeball fixation positions, and further determining paragraph IDs corresponding to the eyeball fixation positions according to the paragraph IDs corresponding to the paragraphs displayed at each position of each page by the reading material; then entering a sleep state;
and the step S3 further includes: every second interval, electroencephalogram data are collected.
9. The method for acquiring electroencephalogram data of synchronously tracking reading content according to claim 3, wherein the step S3 further comprises: and during reading, carrying out face emotion recognition.
10. The electroencephalogram data acquisition method for synchronously tracking reading contents according to claim 3, wherein the output data includes the synchronization data, reading time, reader ID, article ID of reading materials, and electroencephalogram type.
CN202210135418.8A 2022-02-14 2022-02-14 Electroencephalogram data acquisition system and method for synchronously tracking reading content Pending CN114527875A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210135418.8A CN114527875A (en) 2022-02-14 2022-02-14 Electroencephalogram data acquisition system and method for synchronously tracking reading content

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210135418.8A CN114527875A (en) 2022-02-14 2022-02-14 Electroencephalogram data acquisition system and method for synchronously tracking reading content

Publications (1)

Publication Number Publication Date
CN114527875A true CN114527875A (en) 2022-05-24

Family

ID=81621909

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210135418.8A Pending CN114527875A (en) 2022-02-14 2022-02-14 Electroencephalogram data acquisition system and method for synchronously tracking reading content

Country Status (1)

Country Link
CN (1) CN114527875A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116072297A (en) * 2023-03-09 2023-05-05 深圳市人马互动科技有限公司 Method and related device for determining mental health data based on novel interaction

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116072297A (en) * 2023-03-09 2023-05-05 深圳市人马互动科技有限公司 Method and related device for determining mental health data based on novel interaction
CN116072297B (en) * 2023-03-09 2023-06-06 深圳市人马互动科技有限公司 Method and related device for determining mental health data based on novel interaction

Similar Documents

Publication Publication Date Title
US20210000374A1 (en) System and method for instructing a behavior change in a user
Ruiz-Blondet et al. CEREBRE: A novel method for very high accuracy event-related potential biometric identification
KR102277820B1 (en) The psychological counseling system and the method thereof using the feeling information and response information
Bulling et al. Recognition of visual memory recall processes using eye movement analysis
Rizzo et al. The brain in the wild: tracking human behavior in naturalistic settings
US20180279935A1 (en) Method and system for detecting frequency domain cardiac information by using pupillary response
Ekiz et al. Can a smartband be used for continuous implicit authentication in real life
CN114527875A (en) Electroencephalogram data acquisition system and method for synchronously tracking reading content
Johnson et al. The ECHOS platform to enhance communication for nonverbal children with autism: A case study
US20200402641A1 (en) Systems and methods for capturing and presenting life moment information for subjects with cognitive impairment
Bhatt et al. An IoMT-Based Approach for Real-Time Monitoring Using Wearable Neuro-Sensors
CN109620265A (en) Recognition methods and relevant apparatus
US20180235464A1 (en) Method and system for detecting time domain cardiac parameters by using pupillary response
Rincon et al. Intelligent wristbands for the automatic detection of emotional states for the elderly
Domínguez-Jiménez et al. Emotion detection through biomedical signals: a pilot study
Niforatos et al. PulseCam: biophysically driven life logging
Hassib Mental task classification using single-electrode brain computer interfaces
Duarte et al. Extraction of notable points from ECG data: A description of a dataset related to 30-s seated and 30-s stand up
Grzeszczyk et al. CogWear: Can we detect cognitive effort with consumer-grade wearables?
Borhani et al. EEG-based visual attentional state decoding using convolutional neural network
Fangmeng et al. Emotional changes detection for dementia people with spectrograms from physiological signals
Moyeenudin et al. Analysis of Electroencephalographic Signals to Study the Behavior of Brain Frequencies for the Study of Academic Stress
Rahman et al. Classification of motor imagery using a time-localised approach
Placidi et al. Self-induced emotions as alternative paradigm for driving brain–computer interfaces
Abreu et al. PreEpiSeizures: Description and outcomes of physiological data acquisition using wearable devices during video-EEG monitoring in people with epilepsy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination