CN115844336A - Automatic real-time monitoring system and device for epileptic seizure - Google Patents

Automatic real-time monitoring system and device for epileptic seizure Download PDF

Info

Publication number
CN115844336A
CN115844336A CN202310071070.5A CN202310071070A CN115844336A CN 115844336 A CN115844336 A CN 115844336A CN 202310071070 A CN202310071070 A CN 202310071070A CN 115844336 A CN115844336 A CN 115844336A
Authority
CN
China
Prior art keywords
electroencephalogram
epileptic
patient
module
finger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310071070.5A
Other languages
Chinese (zh)
Inventor
李劲松
王沛
周天舒
田雨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to CN202310071070.5A priority Critical patent/CN115844336A/en
Publication of CN115844336A publication Critical patent/CN115844336A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention discloses an automatic real-time monitoring system and a device for epileptic seizure, comprising: the system comprises an electroencephalogram acquisition module, an electroencephalogram data preprocessing module, an electroencephalogram data feature extraction module, a seizure electroencephalogram detection module, a command sending module, a video acquisition module, a command response detection module and an epileptic seizure detection module. The epileptic seizure detection method provided by the invention simultaneously analyzes the electroencephalogram information and the video information, and can more accurately detect whether the patient has epileptic seizure; firstly, electroencephalogram signals are analyzed, and when the electroencephalogram signals detect epileptic electroencephalogram, video data are analyzed, so that the calculated amount can be greatly reduced on the premise of not reducing the detection precision; whether the consciousness loss condition exists in the patient is judged by sending an instruction and monitoring whether the patient correctly responds to the instruction.

Description

Automatic real-time monitoring system and device for epileptic seizure
Technical Field
The invention relates to the technical field of medical information, in particular to an automatic real-time epileptic seizure monitoring system and device.
Background
Epilepsy (Epilepsy) is a common disease of the nervous system, a chronic disease that results in transient cerebral dysfunction due to sudden abnormal firing of cerebral neurons. Epidemiological data show that the total prevalence rate of epilepsy in China is 7.0 per mill. Because of the difference of the brain area and the propagation path of the epileptic seizure, the epileptic seizure has multiple clinical manifestations, such as sudden consciousness loss, syncope, four-limb convulsion and the like, and produces disturbance in cognition and spirit, thus seriously damaging the physical and mental health of patients and even endangering life.
The video electroencephalogram is the most common diagnostic means for detecting epilepsy at present, and in the process of detecting a patient, on one hand, accompanying personnel are required to judge and mark whether the patient has epilepsy or not through observation, communication and other modes; on the other hand, medical staff is required to diagnose whether the patient has epilepsy or not by combining the real-time electroencephalogram signals of the patient and clinical manifestations in videos. For the patient suffering from absence epilepsy, it is difficult to confirm whether the patient suffers from loss of consciousness or not under the condition of no communication and only depending on observation, and the requirement of continuous communication with the patient for accompanying can bring great burden to accompanying.
At present, a traditional epilepsy automatic monitoring system only analyzes electroencephalogram signals and does not combine clinical manifestations of patients, and although electroencephalogram epileptic-like discharge occurs when a patient seizes epilepsy, on one hand, 0.3% -3% of people in normal people do not have epilepsy, but electroencephalogram epileptic-like discharge occurs, on the other hand, clinical manifestations of consciousness loss needed for diagnosing the absence type epilepsy are difficult to distinguish from situations such as stupor and the like only depending on videos under the condition that accompanying persons do not communicate with the patients.
Therefore, we propose an automatic real-time monitoring system and device for epileptic seizure to solve the above technical problems.
Disclosure of Invention
The invention aims to provide an automatic real-time epileptic seizure monitoring system and device, which solve the problems that in the prior art, the electroencephalogram and video can be combined to comprehensively detect whether a patient seizes epilepsia, and on the other hand, the intelligent real-time epileptic seizure monitoring system has an intelligent interaction function with the patient, automatically interacts with the patient when the patient has epileptic electroencephalogram, and assists in judging whether the patient loses consciousness or not.
The technical scheme adopted by the invention is as follows:
an automatic real-time seizure monitoring system comprising:
the electroencephalogram acquisition module: the electroencephalogram data preprocessing module is used for acquiring electroencephalogram signals and inputting the electroencephalogram signals to the electroencephalogram data preprocessing module;
the electroencephalogram data preprocessing module: the electroencephalogram data extraction module is used for preprocessing the electroencephalogram signals to obtain the frequency domain characteristics of the electroencephalogram signals per second, and inputting the frequency domain characteristics of the electroencephalogram signals per second to the electroencephalogram data characteristic extraction module;
the electroencephalogram data feature extraction module: the electroencephalogram detection module is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through an electroencephalogram characteristic extraction neural network, acquiring corresponding electroencephalogram characteristics per second and probability that the electroencephalogram signals per second are epileptic-like, inputting the epileptic-like probability to the epileptic-like electroencephalogram detection module, and inputting the electroencephalogram characteristics per second to the epileptic seizure detection module;
the epileptic electroencephalogram detection module comprises: the electroencephalogram signal processing module is used for judging the probability of epileptic appearance of the electroencephalogram signal per second in a continuous time range, judging whether a patient generates epileptic appearance electroencephalogram or not and inputting the result of the epileptic appearance electroencephalogram of the patient into the instruction sending module;
an instruction sending module: the device is used for automatically giving voice prompt to a patient according to the result of epileptic electroencephalogram of the patient and requiring the patient to put numbers with gestures;
the video acquisition module: the system comprises a command response detection module, a data acquisition module and a display module, wherein the command response detection module is used for acquiring a video of a sickbed area where a patient is located and inputting the video of the sickbed area to the command response detection module;
the instruction response detection module: the system comprises a sickbed area video acquisition module, an epileptic seizure detection module and a control module, wherein the sickbed area video acquisition module is used for sampling one frame per second into a picture sequence, and acquiring video characteristics of each frame of picture and hand characteristic point coordinates of each frame of picture by utilizing an OpenPose neural network from each frame of picture sequence;
a seizure detection module: and the electroencephalogram characteristic per second is input into the seizure detection neural network to obtain the probability of seizure of the patient, and whether the patient has seizure or not is judged according to the probability of seizure of the patient and the result of response instruction of the patient.
Further, the preprocessing in the electroencephalogram data preprocessing module specifically comprises:
the system is used for fixedly dividing the electroencephalogram signals according to one second, performing Fourier transform on the electroencephalogram signals per second, and calculating the power spectrum of the multichannel electroencephalogram signals in the time range per second, wherein the frequency resolution is 1Hz;
the method is used for carrying out Fourier transform on the electroencephalogram signals per second and data of each channel of three continuous seconds one second before and after the electroencephalogram signals per second, calculating a power spectrum of the multichannel electroencephalogram signals in a continuous three-second range, and then normalizing the frequency resolution to 1Hz by adopting an average method;
the method is used for splicing the multichannel brain electricity signal power spectrum in the time range of each second and the multichannel brain electricity signal power spectrum in the continuous three-second range to serve as the frequency domain characteristic corresponding to the brain electricity signal of each second.
Further, the electroencephalogram data feature extraction module specifically comprises:
the characteristic extraction part is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through the electroencephalogram characteristic extraction neural network to obtain corresponding electroencephalogram characteristics per second;
the channel for the electroencephalogram feature is used as a time axis and is input to a prediction part in the electroencephalogram feature extraction neural network, and the probability that the electroencephalogram signal is epileptic per second is obtained;
the electroencephalogram detection module is used for inputting epileptic probability to the epileptic electroencephalogram detection module and inputting electroencephalogram characteristics every second to the epileptic seizure detection module.
Further, the epileptiform electroencephalogram detection module specifically comprises: the electroencephalogram processing module is used for judging the probability that the electroencephalogram signals are epileptic within a continuous time range every second, when the epileptic probability of more than 50% of the electroencephalogram signals within the continuous time range is greater than 80%, the patients are epileptic electroencephalogram, otherwise, the patients are not epileptic electroencephalogram, and the results of the patients with epileptic electroencephalogram are input to the instruction sending module.
Further, the instruction response detection module specifically includes:
the system is used for sampling the sickbed area video in the continuous time including the corresponding front and back preset time when the epileptic electroencephalogram occurs to a patient into a picture sequence according to one frame per second;
the video feature of each frame of image of the front 10 layers of VGG-19 and the hand feature point coordinates of each frame of image are used for outputting each frame of image sequence by utilizing an OpenPose neural network;
the video characteristics of each frame of picture are used for inputting to the epileptic seizure detection module, and the coordinates of the hand characteristic points of each frame of picture are used for judging whether a response instruction occurs.
Further, the determining whether the response command occurs specifically includes:
is used for acquiring the coordinates of the characteristic points of the hand of each frame of picture, according to the fingertips of the index finger, the middle finger and the ring finger and respective far finger joints, respectively calculating the index finger natural length, the index finger vertical length, the middle finger natural length, the middle finger vertical length, the ring finger natural length and the ring finger vertical length of each frame of picture by the middle knuckle joint and metacarpophalangeal joint coordinates;
the index finger threshold value, the middle finger threshold value and the ring finger threshold value are respectively set according to the index finger natural length, the middle finger natural length and the ring finger natural length of each frame of picture;
when the vertical length of the index finger exceeds the threshold value of the index finger, the index finger is considered to be vertical, otherwise, the index finger is considered to be non-vertical;
if the vertical length of the middle finger exceeds the threshold value of the middle finger, the middle finger is considered to be vertical, otherwise, the middle finger is considered to be non-vertical;
when the vertical length of the ring finger exceeds the threshold value of the ring finger, the ring finger is considered to be vertical, otherwise, the ring finger is considered to be non-vertical;
when the index finger is vertical, the middle finger is vertical and the ring finger is not vertical, the patient is considered to respond to the instruction, otherwise, the patient is considered not to respond to the instruction.
Further, the seizure detection module specifically includes:
the electroencephalogram characteristic processing part is used for inputting the electroencephalogram characteristics in a continuous any time range to the epileptic seizure detection neural network to acquire time sequence electroencephalogram characteristics;
the video feature processing part is used for inputting the video features in continuous arbitrary time ranges into the epileptic seizure detection neural network to obtain time sequence video features;
the time sequence electroencephalogram characteristic and the time sequence video characteristic are spliced and input into a prediction part of a seizure detection neural network to obtain the probability of seizure of the patient;
the method is used for judging that when the probability of the patient suffering from the epilepsia exceeds 85% and the patient does not respond to the instruction, the patient is considered to suffer from the epilepsia, otherwise, the patient is considered to suffer from the epilepsia without the epilepsia.
Furthermore, the electroencephalogram acquisition module acquires electroencephalogram signals in real time according to a 10-20 standard lead system.
The invention also provides an automatic real-time epileptic seizure monitoring device, which comprises a memory and one or more processors, wherein the memory stores executable codes, and the one or more processors are used for realizing the automatic real-time epileptic seizure monitoring system when executing the executable codes.
The present invention also provides a computer readable storage medium, on which a program is stored, which when executed by a processor, implements an automatic real-time seizure monitoring system as described in any one of the above.
The invention has the beneficial effects that:
1. the epileptic seizure detection method provided by the invention simultaneously analyzes the electroencephalogram information and the video information, and can more accurately detect whether the patient has epileptic seizure.
2. The method analyzes the electroencephalogram signals, analyzes the video data when the electroencephalogram signals detect epileptic electroencephalogram, and can greatly reduce the calculated amount on the premise of not reducing the detection precision.
3. The invention judges whether the patient has the condition of consciousness loss by sending the instruction and monitoring whether the patient correctly responds to the instruction.
Drawings
Fig. 1 is a schematic structural diagram of an automatic real-time epileptic seizure monitoring system according to the present invention;
fig. 2 is a schematic structural diagram of an automatic real-time epileptic seizure monitoring device according to the present invention.
Detailed Description
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an automatic real-time seizure monitoring system includes:
the electroencephalogram acquisition module: the electroencephalogram data preprocessing module is used for acquiring electroencephalogram signals and inputting the electroencephalogram signals to the electroencephalogram data preprocessing module;
the electroencephalogram acquisition module acquires electroencephalogram signals in real time according to a 10-20 standard lead system.
The electroencephalogram data preprocessing module: the electroencephalogram data extraction module is used for preprocessing the electroencephalogram signals to obtain the frequency domain characteristics of the electroencephalogram signals per second, and inputting the frequency domain characteristics of the electroencephalogram signals per second to the electroencephalogram data characteristic extraction module;
the preprocessing in the electroencephalogram data preprocessing module specifically comprises the following steps:
the system is used for fixedly dividing the electroencephalogram signals according to one second, performing Fourier transform on the electroencephalogram signals per second, and calculating the power spectrum of the multichannel electroencephalogram signals in the time range per second, wherein the frequency resolution is 1Hz;
the method is used for carrying out Fourier transform on the electroencephalogram signals per second and data of each channel of three continuous seconds one second before and after the electroencephalogram signals per second, calculating a power spectrum of the multichannel electroencephalogram signals in a continuous three-second range, and then normalizing the frequency resolution to 1Hz by adopting an average method;
the method is used for splicing the multichannel brain electricity signal power spectrum in the time range of each second and the multichannel brain electricity signal power spectrum in the continuous three-second range to serve as the frequency domain characteristic corresponding to the brain electricity signal of each second.
The electroencephalogram data feature extraction module: the electroencephalogram detection module is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through an electroencephalogram characteristic extraction neural network, acquiring corresponding electroencephalogram characteristics per second and probability that the electroencephalogram signals per second are epileptic-like, inputting the epileptic-like probability to the epileptic-like electroencephalogram detection module, and inputting the electroencephalogram characteristics per second to the epileptic seizure detection module;
the electroencephalogram data feature extraction module specifically comprises:
the characteristic extraction part is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through the electroencephalogram characteristic extraction neural network to obtain corresponding electroencephalogram characteristics per second;
the channel for the electroencephalogram feature is used as a time axis and is input to a prediction part in the electroencephalogram feature extraction neural network, and the probability that the electroencephalogram signal is epileptic per second is obtained;
the electroencephalogram detection module is used for inputting epileptic probability to the epileptic electroencephalogram detection module and inputting electroencephalogram characteristics every second to the epileptic seizure detection module.
The epileptic electroencephalogram detection module comprises: the electroencephalogram signal processing module is used for judging the probability of epileptic appearance of the electroencephalogram signal per second in a continuous time range, judging whether a patient generates epileptic appearance electroencephalogram or not and inputting the result of the epileptic appearance electroencephalogram of the patient into the instruction sending module;
the epilepsy-like electroencephalogram detection module specifically comprises: the electroencephalogram processing module is used for judging the probability that the electroencephalogram signals are epilepsy-like every second in the continuous time range, when the epilepsy-like probability of more than 50% of the electroencephalogram signals in the continuous time range is larger than 80%, the patient generates epilepsy-like electroencephalogram, otherwise, the patient does not generate epilepsy-like electroencephalogram, and the result of the occurrence of epilepsy-like electroencephalogram of the patient is input to the instruction sending module.
An instruction sending module: the device is used for automatically giving voice prompt to a patient according to the epileptic electroencephalogram result of the patient and requiring the patient to put numbers with gestures;
the video acquisition module: the system comprises a command response detection module, a data acquisition module and a display module, wherein the command response detection module is used for acquiring a video of a sickbed area where a patient is located and inputting the video of the sickbed area to the command response detection module;
the instruction response detection module: the system comprises a sickbed area video acquisition module, an epileptic seizure detection module and a control module, wherein the sickbed area video acquisition module is used for sampling one frame per second into a picture sequence, and acquiring video characteristics of each frame of picture and hand characteristic point coordinates of each frame of picture by utilizing an OpenPose neural network from each frame of picture sequence;
the instruction response detection module specifically comprises:
the system is used for sampling the sickbed area video in the continuous time including the corresponding front and back preset time when the epileptic electroencephalogram occurs to a patient into a picture sequence according to one frame per second;
the video feature of each frame of image of the front 10 layers of VGG-19 and the hand feature point coordinates of each frame of image are used for outputting each frame of image sequence by utilizing an OpenPose neural network;
the video characteristics of each frame of picture are used for inputting to the epileptic seizure detection module, and the coordinates of the hand characteristic points of each frame of picture are used for judging whether a response instruction occurs.
The step of judging whether the response instruction occurs specifically includes:
the system comprises a plurality of sets of images, a plurality of finger joints, a middle knuckle joint and a metacarpophalangeal joint, wherein the sets of images are used for acquiring hand characteristic point coordinates of each frame of image, and respectively calculating index finger natural length, index finger vertical length, middle finger natural length, middle finger vertical length, ring finger natural length and ring finger vertical length of each frame of image according to the finger tips of index finger, middle finger and ring finger and respective far finger joints;
the index finger threshold value, the middle finger threshold value and the ring finger threshold value are respectively set according to the natural length of the index finger, the natural length of the middle finger and the natural length of the ring finger of each frame of picture;
when the vertical length of the index finger exceeds the threshold value of the index finger, the index finger is considered to be vertical, otherwise, the index finger is considered to be non-vertical;
if the vertical length of the middle finger exceeds the threshold value of the middle finger, the middle finger is considered to be vertical, otherwise, the middle finger is considered to be non-vertical;
when the vertical length of the ring finger exceeds the threshold value of the ring finger, the ring finger is considered to be vertical, otherwise, the ring finger is considered to be non-vertical;
when the index finger is vertical, the middle finger is vertical and the ring finger is not vertical, the patient is considered to respond to the instruction, otherwise, the patient is considered not to respond to the instruction.
An epileptic seizure detection module: the system is used for inputting the video characteristics of each frame of picture and the electroencephalogram characteristics per second into the epileptic seizure detection neural network to obtain the probability of epileptic seizure of a patient, and judging whether the patient has epileptic seizure or not according to the probability of epileptic seizure of the patient and the result of response instructions of the patient.
The epileptic seizure detection module specifically comprises:
the electroencephalogram characteristic processing part is used for inputting the electroencephalogram characteristics in a continuous any time range to the epileptic seizure detection neural network to acquire time sequence electroencephalogram characteristics;
the video feature processing part is used for inputting the video features in continuous arbitrary time ranges into the epileptic seizure detection neural network to obtain time sequence video features;
the time sequence electroencephalogram characteristic and the time sequence video characteristic are spliced and input into a prediction part of a seizure detection neural network to obtain the probability of seizure of the patient;
the method is used for judging that when the probability of the patient suffering from the epilepsia exceeds 85% and the patient does not respond to the instruction, the patient is considered to suffer from the epilepsia, otherwise, the patient is considered to suffer from the epilepsia without the epilepsia.
Example (b): an automatic real-time seizure monitoring system comprising:
the electroencephalogram acquisition module: the electroencephalogram data preprocessing module is used for acquiring electroencephalogram signals and inputting the electroencephalogram signals to the electroencephalogram data preprocessing module;
the electroencephalogram acquisition module acquires electroencephalogram signals in real time according to a 10-20 standard lead system.
The electroencephalogram data preprocessing module: the electroencephalogram data extraction module is used for preprocessing the electroencephalogram signals to obtain the frequency domain characteristics of the electroencephalogram signals per second, and inputting the frequency domain characteristics of the electroencephalogram signals per second to the electroencephalogram data characteristic extraction module;
the preprocessing in the electroencephalogram data preprocessing module specifically comprises the following steps:
the system is used for fixedly dividing the electroencephalogram signals according to one second, performing Fourier transform on the electroencephalogram signals per second, and calculating the power spectrum of the multichannel electroencephalogram signals in the time range per second, wherein the frequency resolution is 1Hz;
the method is used for carrying out Fourier transform on the electroencephalogram signals per second and data of each channel of three continuous seconds one second before and after the electroencephalogram signals per second, calculating a power spectrum of the multichannel electroencephalogram signals in a continuous three-second range, and then normalizing the frequency resolution to 1Hz by adopting an average method;
the method is used for splicing the multichannel brain electricity signal power spectrum in the time range of each second and the multichannel brain electricity signal power spectrum in the continuous three-second range to serve as the frequency domain Feature _ Freq corresponding to the brain electricity signal of each second.
The electroencephalogram data feature extraction module: the electroencephalogram detection module is used for extracting the frequency domain Feature _ Freq of the electroencephalogram signal per second through an electroencephalogram Feature extraction neural network, acquiring the electroencephalogram Feature H _ EEG corresponding to the electroencephalogram Feature per second and the probability that the electroencephalogram signal per second is epileptic, inputting the probability of epileptic to the epileptic electroencephalogram detection module, and inputting the electroencephalogram Feature H _ EEG per second to the epileptic seizure detection module;
the electroencephalogram data feature extraction module is composed of an electroencephalogram feature extraction neural network, and the electroencephalogram feature extraction neural network is composed of a feature extraction part and a prediction part;
the structure of the feature extraction section includes: a convolution layer c1, a batch normalization layer bn1, an active layer a1; a convolution layer c2, a batch normalization layer bn2, an active layer a2; a convolution layer c3, a batch normalization layer bn3, an active layer a3; a convolution layer c4, a batch normalization layer bn4, an active layer a4; a convolution layer c5, a batch normalization layer bn5, an active layer a5; a convolution layer c6, a batch normalization layer bn6, an active layer a6; a convolution layer c7, a batch normalization layer bn7, an active layer a7; a convolution layer c8, a batch normalization layer bn8, an active layer a8; a convolution layer c9, a batch normalization layer bn9, an active layer a9; a convolution layer c10, a batch normalization layer bn10, an active layer a10;
the prediction part structure comprises: long and short term memory network layer LSTM; a full link layer L1; a full link layer L2; a full interconnect layer L3; the unfolding layer, flaten1; a full connection layer L4; a full connection layer L5; a full interconnect layer L6; and outputting the layer Out. Wherein the activation layer adopts Relu activation function;
the electroencephalogram data feature extraction module specifically comprises:
the Feature extraction part is used for extracting the frequency domain Feature _ Freq of the electroencephalogram signal per second through an electroencephalogram Feature extraction neural network to obtain the corresponding electroencephalogram Feature H _ EEG per second;
the channel of the electroencephalogram characteristic H _ EEG is used as a time axis and is input to a prediction part in the electroencephalogram signal characteristic extraction neural network, and the probability that the electroencephalogram signal is epileptic-like per second is obtained;
inputting the long-short term memory network layer LSTM by taking the channel of the second electroencephalogram characteristic H _ EEG as a time axis, and then sequentially inputting the long-short term memory network layer LSTM and the full connection layer L1; a full link layer L2; a full interconnect layer L3; the unfolding layer, flaten1; a full link layer L4; a full interconnect layer L5; a full interconnect layer L6; the output layer Out is used for acquiring the probability that the second electroencephalogram signal is epileptic;
the electroencephalogram detection module is used for inputting epileptic probability to the epileptic electroencephalogram detection module and inputting the electroencephalogram characteristics H _ EEG per second to the epileptic seizure detection module.
The epileptic electroencephalogram detection module comprises: the electroencephalogram signal processing module is used for judging the probability of epileptic appearance of the electroencephalogram signal per second in a continuous time range, judging whether a patient generates epileptic appearance electroencephalogram or not and inputting the result of the epileptic appearance electroencephalogram of the patient into the instruction sending module;
the epilepsy electroencephalogram detection module specifically comprises: the electroencephalogram processing module is used for judging the probability that the electroencephalogram signals are epileptic within a continuous time range every second, when the epileptic probability of more than 50% of the electroencephalogram signals within the continuous time range is greater than 80%, the patients are epileptic electroencephalogram, otherwise, the patients are not epileptic electroencephalogram, and the results of the patients with epileptic electroencephalogram are input to the instruction sending module.
An instruction sending module: the device is used for automatically giving voice prompt to a patient according to the epileptic electroencephalogram result of the patient, and requiring the patient to swing a figure of 2 by using a gesture;
the video acquisition module: the system comprises a command response detection module, a data acquisition module and a display module, wherein the command response detection module is used for acquiring a video of a sickbed area where a patient is located and inputting the video of the sickbed area to the command response detection module;
the instruction response detection module: the system comprises a sickbed area video acquisition module, an epileptic seizure detection module and a control module, wherein the sickbed area video acquisition module is used for sampling one frame per second into a picture sequence, and acquiring video characteristics of each frame of picture and hand characteristic point coordinates of each frame of picture by utilizing an OpenPose neural network from each frame of picture sequence;
the instruction response detection module specifically comprises:
the system is used for sampling the sickbed area video in the continuous time including the corresponding front and back preset time when the epileptic electroencephalogram occurs to a patient into a picture sequence according to one frame per second;
the video characteristic H _ IMG of each frame of image of the front 10 layers of VGG-19 and the hand characteristic point coordinates of each frame of image are used for outputting each frame of image sequence by utilizing an OpenPose neural network;
the video characteristics H _ IMG of each frame of picture are used for inputting to the epileptic seizure detection module, and the coordinates of the hand characteristic points of each frame of picture are used for judging whether a response instruction occurs or not.
The step of judging whether the response instruction occurs specifically includes:
the system comprises a plurality of sets of images, a plurality of finger joints, a middle knuckle joint and a metacarpophalangeal joint, wherein the sets of images are used for acquiring hand characteristic point coordinates of each frame of image, and respectively calculating index finger natural length, index finger vertical length, middle finger natural length, middle finger vertical length, ring finger natural length and ring finger vertical length of each frame of image according to the finger tips of index finger, middle finger and ring finger and respective far finger joints;
and acquiring coordinates of hand feature points output by each frame of picture OpenPose every second within a preset time range after the instruction is sent, wherein the feature points comprise fingertips of an index finger, a middle finger and a ring finger, respective far finger joints, middle knuckle joints and metacarpophalangeal joints.
The index finger threshold value, the middle finger threshold value and the ring finger threshold value are respectively set according to the index finger natural length, the middle finger natural length and the ring finger natural length of each frame of picture;
when the vertical length of the index finger exceeds the threshold value of the index finger, the index finger is considered to be vertical, otherwise, the index finger is considered to be non-vertical;
for each frame of picture, coordinates (x 0, y 0), (x 1, y 1), (x 2, y 2), (x 3, y 3) of the index finger tip, the far finger joint, the middle finger joint and the metacarpophalangeal joint are obtained; calculating the natural length of the index finger:
Figure SMS_1
calculating the vertical length of the index finger
Figure SMS_2
If is greater or greater>
Figure SMS_3
If the index finger is vertical, the index finger is not vertical.
If the vertical length of the middle finger exceeds the threshold value of the middle finger, the middle finger is considered to be vertical, otherwise, the middle finger is considered to be non-vertical;
obtaining coordinates (x 0', y 0'), coordinates (x 1', y 1'), coordinates (x 2', y 2') and coordinates (x 3', y 3') of a middle finger tip, a far finger joint, a middle finger joint and a metacarpophalangeal joint; natural length in calculation:
Figure SMS_4
calculating the vertical length of the middle finger
Figure SMS_5
If is>
Figure SMS_6
If the middle finger is vertical, otherwise, the middle finger is not vertical.
When the vertical length of the ring finger exceeds the threshold value of the ring finger, the ring finger is considered to be vertical, otherwise, the ring finger is considered to be non-vertical;
obtaining coordinates (x 0', y 0'), (x 1', y 1'), (x 2', y 2'), and (x 3', y 3') of the ring finger tip, the distal finger joint, the middle finger joint, and the metacarpophalangeal joint; calculating the natural length of the ring finger:
Figure SMS_7
calculating vertical length of ring finger
Figure SMS_8
If is greater or greater>
Figure SMS_9
If the ring-free finger is vertical, otherwise, the ring-free finger is not vertical.
When the index finger is vertical, the middle finger is vertical and the ring finger is not vertical, the patient is considered to respond to the instruction, otherwise, the patient is considered not to respond to the instruction.
In the preset time range, if the index finger and the middle finger of one frame of picture are vertical and the ring finger is not vertical, the patient is considered to respond to the instruction, otherwise, the patient is considered not to respond to the instruction.
A seizure detection module: and the electroencephalogram characteristic per second is input into the seizure detection neural network to obtain the probability of seizure of the patient, and whether the patient has seizure or not is judged according to the probability of seizure of the patient and the result of response instruction of the patient.
The epileptic seizure detection module is composed of an epileptic seizure detection neural network, and the epileptic seizure detection neural network is composed of an electroencephalogram characteristic processing part, a video characteristic processing part and a prediction part; the structure of the electroencephalogram feature processing part comprises: a long-short term memory network layer LSTM1, a full connection layer L1 and an activation layer a1; the structure of the video feature processing section includes: a long-short term memory network layer LSTM2, a full connection layer L2 and an activation layer a2; the structure of the prediction part includes: a convolutional layer c1, a convolutional layer c2, an active layer a3; a full-link layer L3, a full-link layer L4, an active layer a4; a full connection layer L5, an output layer Out. Wherein the activation layer adopts Relu activation function;
the epileptic seizure detection module specifically comprises:
the electroencephalogram characteristic processing part is used for inputting the electroencephalogram characteristic H _ EEG in any continuous time range to the epileptic seizure detection neural network to obtain a time sequence electroencephalogram characteristic F _ EEG;
a video feature processing part for inputting the video features H _ IMG in continuous arbitrary time range to the epileptic seizure detection neural network to obtain a time sequence video feature F _ IMG;
the time sequence electroencephalogram characteristic F _ EEG and the time sequence video characteristic F _ IMG are spliced and input into a prediction part of an epileptic seizure detection neural network to obtain the probability of epileptic seizure of a patient;
the method is used for judging that when the probability of the patient suffering from the epilepsia exceeds 85% and the patient does not respond to the instruction, the patient is considered to suffer from the epilepsia, otherwise, the patient is considered to suffer from the epilepsia without the epilepsia.
Corresponding to the embodiment of the automatic real-time epileptic seizure monitoring system, the invention also provides an embodiment of an automatic real-time epileptic seizure monitoring device.
Referring to fig. 2, an automatic real-time seizure monitoring apparatus provided in an embodiment of the present invention includes a memory and one or more processors, where the memory stores executable code, and the one or more processors, when executing the executable code, are configured to implement an automatic real-time seizure monitoring system in the foregoing embodiments.
The embodiment of the automatic real-time epileptic seizure monitoring device can be applied to any equipment with data processing capability, such as computers and other equipment or devices. The apparatus embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for running through the processor of any device with data processing capability. From a hardware aspect, as shown in fig. 2, the present invention is a hardware structure diagram of any device with data processing capability where an automatic real-time monitoring device for epileptic seizure is located, except for the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 2, in an embodiment, any device with data processing capability where the device is located may also include other hardware according to the actual function of the any device with data processing capability, which is not described again.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. One of ordinary skill in the art can understand and implement without inventive effort.
An embodiment of the present invention further provides a computer-readable storage medium, on which a program is stored, where the program, when executed by a processor, implements an automatic real-time seizure monitoring system in the above embodiments.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any data processing device described in any previous embodiment. The computer readable storage medium may also be any external storage device of a device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), etc. provided on the device. Further, the computer readable storage medium may include both an internal storage unit and an external storage device of any data processing capable device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing-capable device, and may also be used for temporarily storing data that has been output or is to be output.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. An automatic real-time seizure monitoring system, comprising:
the electroencephalogram acquisition module: the electroencephalogram data preprocessing module is used for acquiring electroencephalogram signals and inputting the electroencephalogram signals to the electroencephalogram data preprocessing module;
the electroencephalogram data preprocessing module: the electroencephalogram data extraction module is used for preprocessing the electroencephalogram signals to obtain the frequency domain characteristics of the electroencephalogram signals per second, and inputting the frequency domain characteristics of the electroencephalogram signals per second to the electroencephalogram data characteristic extraction module;
the electroencephalogram data feature extraction module: the electroencephalogram detection module is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through an electroencephalogram characteristic extraction neural network, acquiring corresponding electroencephalogram characteristics per second and probability that the electroencephalogram signals per second are epileptic-like, inputting the epileptic-like probability to the epileptic-like electroencephalogram detection module, and inputting the electroencephalogram characteristics per second to the epileptic seizure detection module;
the epileptic electroencephalogram detection module comprises: the electroencephalogram signal processing module is used for judging the probability of epileptic appearance of the electroencephalogram signal per second in a continuous time range, judging whether a patient generates epileptic appearance electroencephalogram or not and inputting the result of the epileptic appearance electroencephalogram of the patient into the instruction sending module;
an instruction sending module: the device is used for automatically giving voice prompt to a patient according to the result of epileptic electroencephalogram of the patient and requiring the patient to put numbers with gestures;
the video acquisition module: the system comprises a command response detection module, a data acquisition module and a display module, wherein the command response detection module is used for acquiring a video of a sickbed area where a patient is located and inputting the video of the sickbed area to the command response detection module;
the instruction response detection module: the system comprises a sickbed area video acquisition module, an epileptic seizure detection module and a control module, wherein the sickbed area video acquisition module is used for sampling one frame per second into a picture sequence, and acquiring video characteristics of each frame of picture and hand characteristic point coordinates of each frame of picture by utilizing an OpenPose neural network from each frame of picture sequence;
a seizure detection module: and the electroencephalogram characteristic per second is input into the seizure detection neural network to obtain the probability of seizure of the patient, and whether the patient has seizure or not is judged according to the probability of seizure of the patient and the result of response instruction of the patient.
2. The system for automatically monitoring epileptic seizure in real time as claimed in claim 1, wherein the preprocessing in the electroencephalogram data preprocessing module specifically comprises:
the system is used for fixedly dividing the electroencephalogram signals according to one second, performing Fourier transform on the electroencephalogram signals per second, and calculating the power spectrum of the multichannel electroencephalogram signals in the time range per second, wherein the frequency resolution is 1Hz;
the power spectrum analysis device is used for performing Fourier transform on the electroencephalogram signals per second and data of each channel of three continuous seconds in the previous and next second, calculating the power spectrum of the multichannel electroencephalogram signals in the range of three continuous seconds, and then normalizing the frequency resolution to 1Hz by adopting an average method;
the method is used for splicing the multichannel brain electricity signal power spectrum in the time range of each second and the multichannel brain electricity signal power spectrum in the continuous three-second range to serve as the frequency domain characteristic corresponding to the brain electricity signal of each second.
3. The system of claim 1, wherein the electroencephalogram data feature extraction module specifically comprises:
the characteristic extraction part is used for extracting the frequency domain characteristics of the electroencephalogram signals per second through electroencephalogram characteristics to obtain corresponding electroencephalogram characteristics per second;
the channel for the electroencephalogram feature is used as a time axis and is input to a prediction part in the electroencephalogram feature extraction neural network, and the probability that the electroencephalogram signal is epileptic per second is obtained;
the electroencephalogram detection module is used for inputting epileptic probability to the epileptic electroencephalogram detection module and inputting electroencephalogram characteristics every second to the epileptic seizure detection module.
4. The system for automatically monitoring epileptic seizure in real time as claimed in claim 1, wherein the epileptic-like electroencephalogram detection module specifically comprises: the electroencephalogram processing module is used for judging the probability that the electroencephalogram signals are epileptic within a continuous time range every second, when the epileptic probability of more than 50% of the electroencephalogram signals within the continuous time range is greater than 80%, the patients are epileptic electroencephalogram, otherwise, the patients are not epileptic electroencephalogram, and the results of the patients with epileptic electroencephalogram are input to the instruction sending module.
5. The system according to claim 1, wherein the instruction response detection module specifically comprises:
the system is used for sampling the sickbed area video in the continuous time including the corresponding front and back preset time when the epileptic electroencephalogram occurs to a patient into a picture sequence according to one frame per second;
the video characteristic of each frame of picture of the front 10 layers of VGG-19 and the hand characteristic point coordinates of each frame of picture are used for outputting each frame of picture sequence by utilizing an OpenPose neural network;
the video characteristics of each frame of picture are used for inputting to the epileptic seizure detection module, and the coordinates of the hand characteristic points of each frame of picture are used for judging whether a response instruction occurs.
6. The system according to claim 5, wherein the determining whether the response command occurs specifically comprises:
the system comprises a plurality of sets of images, a plurality of finger joints, a middle knuckle joint and a metacarpophalangeal joint, wherein the sets of images are used for acquiring hand characteristic point coordinates of each frame of image, and respectively calculating index finger natural length, index finger vertical length, middle finger natural length, middle finger vertical length, ring finger natural length and ring finger vertical length of each frame of image according to the finger tips of index finger, middle finger and ring finger and respective far finger joints;
the index finger threshold value, the middle finger threshold value and the ring finger threshold value are respectively set according to the index finger natural length, the middle finger natural length and the ring finger natural length of each frame of picture;
when the vertical length of the index finger exceeds the threshold value of the index finger, the index finger is considered to be vertical, otherwise, the index finger is considered to be non-vertical;
if the vertical length of the middle finger exceeds the threshold value of the middle finger, the middle finger is considered to be vertical, otherwise, the middle finger is considered to be non-vertical;
when the vertical length of the ring finger exceeds the threshold value of the ring finger, the ring finger is considered to be vertical, otherwise, the ring finger is considered to be non-vertical;
when the index finger is vertical, the middle finger is vertical and the ring finger is not vertical, the patient is considered to respond to the instruction, otherwise, the patient is considered not to respond to the instruction.
7. The system according to claim 1, wherein the seizure detection module specifically comprises:
the electroencephalogram characteristic processing part is used for inputting the electroencephalogram characteristics in a continuous any time range to the epileptic seizure detection neural network to acquire time sequence electroencephalogram characteristics;
the video feature processing part is used for inputting the video features in continuous arbitrary time ranges into the epileptic seizure detection neural network to obtain time sequence video features;
the time sequence electroencephalogram characteristic and the time sequence video characteristic are spliced and input into a prediction part of an epileptic seizure detection neural network to obtain the probability of epileptic seizure of a patient;
the method is used for judging that when the probability of the patient suffering from the epilepsia exceeds 85% and the patient does not respond to the instruction, the patient is considered to suffer from the epilepsia, otherwise, the patient is considered to suffer from the epilepsia without the epilepsia.
8. The automatic real-time epileptic seizure monitoring system of claim 1, wherein the electroencephalogram acquisition module acquires electroencephalogram signals in real time according to a 10-20 standard lead system.
9. An automatic real-time seizure monitoring device comprising a memory having executable code stored therein and one or more processors configured to implement an automatic real-time seizure monitoring system as claimed in any one of claims 1-8 when the executable code is executed by the one or more processors.
10. A computer-readable storage medium, having stored thereon a program which, when executed by a processor, implements an automatic real-time seizure monitoring system according to any one of claims 1 to 8.
CN202310071070.5A 2023-02-07 2023-02-07 Automatic real-time monitoring system and device for epileptic seizure Pending CN115844336A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310071070.5A CN115844336A (en) 2023-02-07 2023-02-07 Automatic real-time monitoring system and device for epileptic seizure

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310071070.5A CN115844336A (en) 2023-02-07 2023-02-07 Automatic real-time monitoring system and device for epileptic seizure

Publications (1)

Publication Number Publication Date
CN115844336A true CN115844336A (en) 2023-03-28

Family

ID=85657640

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310071070.5A Pending CN115844336A (en) 2023-02-07 2023-02-07 Automatic real-time monitoring system and device for epileptic seizure

Country Status (1)

Country Link
CN (1) CN115844336A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117297546A (en) * 2023-09-25 2023-12-29 首都医科大学宣武医院 Automatic detection system for capturing seizure symptomology information of epileptic

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120083701A1 (en) * 2010-10-01 2012-04-05 Ivan Osorio Detecting, quantifying, and/or classifying seizures using multimodal data
US20120083700A1 (en) * 2010-10-01 2012-04-05 Ivan Osorio Detecting, quantifying, and/or classifying seizures using multimodal data
CN207752446U (en) * 2018-05-03 2018-08-21 林潼 A kind of gesture identification interaction systems based on Leap Motion equipment
US20190336061A1 (en) * 2018-05-01 2019-11-07 International Business Machines Corporation Epilepsy seizure detection and prediction using techniques such as deep learning methods
CN110991289A (en) * 2019-11-25 2020-04-10 达闼科技成都有限公司 Abnormal event monitoring method and device, electronic equipment and storage medium
US20210282701A1 (en) * 2020-03-16 2021-09-16 nCefalon Corporation Method of early detection of epileptic seizures through scalp eeg monitoring
CN114093501A (en) * 2021-10-19 2022-02-25 杭州电子科技大学 Intelligent auxiliary analysis method for children's motor epilepsy based on synchronous video and electroencephalogram
US20230034807A1 (en) * 2020-01-24 2023-02-02 Arizona Board Of Regents On Behalf Of Arizona State University Systems and methods including a device for personalized activity monitoring involving the hands

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120083701A1 (en) * 2010-10-01 2012-04-05 Ivan Osorio Detecting, quantifying, and/or classifying seizures using multimodal data
US20120083700A1 (en) * 2010-10-01 2012-04-05 Ivan Osorio Detecting, quantifying, and/or classifying seizures using multimodal data
US20190336061A1 (en) * 2018-05-01 2019-11-07 International Business Machines Corporation Epilepsy seizure detection and prediction using techniques such as deep learning methods
CN207752446U (en) * 2018-05-03 2018-08-21 林潼 A kind of gesture identification interaction systems based on Leap Motion equipment
CN110991289A (en) * 2019-11-25 2020-04-10 达闼科技成都有限公司 Abnormal event monitoring method and device, electronic equipment and storage medium
US20230034807A1 (en) * 2020-01-24 2023-02-02 Arizona Board Of Regents On Behalf Of Arizona State University Systems and methods including a device for personalized activity monitoring involving the hands
US20210282701A1 (en) * 2020-03-16 2021-09-16 nCefalon Corporation Method of early detection of epileptic seizures through scalp eeg monitoring
CN114093501A (en) * 2021-10-19 2022-02-25 杭州电子科技大学 Intelligent auxiliary analysis method for children's motor epilepsy based on synchronous video and electroencephalogram

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117297546A (en) * 2023-09-25 2023-12-29 首都医科大学宣武医院 Automatic detection system for capturing seizure symptomology information of epileptic

Similar Documents

Publication Publication Date Title
EP3589188B1 (en) A sensory evoked diagnostic for the assessment of cognitive brain function
CN110001652B (en) Driver state monitoring method and device and terminal equipment
US9792823B2 (en) Multi-view learning in detection of psychological states
US10213152B2 (en) System and method for real-time measurement of sleep quality
US9633168B2 (en) Biometric identity validation for use with unattended tests for medical conditions
CN115844336A (en) Automatic real-time monitoring system and device for epileptic seizure
US20220338757A1 (en) System and method for non-face-to-face health status measurement through camera-based vital sign data extraction and electronic questionnaire
CN115778330A (en) Automatic epileptic seizure detection system and device based on video electroencephalogram
CN110755091A (en) Personal mental health monitoring system and method
Rescio et al. Ambient and wearable system for workers’ stress evaluation
CN115804572B (en) Automatic epileptic seizure monitoring system and device
Senaratne et al. A multimodal dataset and evaluation for feature estimators of temporal phases of anxiety
CN117224080B (en) Human body data monitoring method and device for big data
CN111724896B (en) Drug addiction evaluation system based on multi-stimulus image or video ERP
US11253324B1 (en) Determination of appendix position using a two stage deep neural network
WO2019189971A1 (en) Artificial intelligence analysis method of iris image and retinal image to diagnose diabetes and presymptom
CN112802606A (en) Data screening model establishing method, data screening device, data screening equipment and data screening medium
CN116469557A (en) Intervention effect evaluation method, device, terminal equipment and storage medium
CN114847949A (en) Cognitive fatigue recovery method based on electroencephalogram characteristics and relaxation indexes
EP3831287A1 (en) Method and device for capacitive touch panel based biosignal measurement
CN113126762A (en) Medical data checking device and method for monitoring medical behaviors
Sarker et al. Analysis of smooth pursuit assessment in virtual reality and concussion detection using bilstm
CN116509419B (en) Electroencephalogram information processing method and system
CN112399826A (en) Providing parameters indicative of loss of consciousness of a patient under anesthesia
CN117503092A (en) ICU delirium risk real-time assessment method based on millimeter waves

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20230328

RJ01 Rejection of invention patent application after publication