WO2023273640A1

WO2023273640A1 - Epilepsy detection method and apparatus

Info

Publication number: WO2023273640A1
Application number: PCT/CN2022/092800
Authority: WO
Inventors: 邸皓轩; 李丹洪; 张晓武
Original assignee: 荣耀终端有限公司
Priority date: 2021-06-30
Filing date: 2022-05-13
Publication date: 2023-01-05
Also published as: CN115530774B; CN115530774A

Abstract

Embodiments of the present application relate to the technical field of terminals, and provide an epilepsy detection method and apparatus. The method comprises: acquiring first data; extracting first motion amplitude feature data and depth feature data from accelerometer data; extracting second motion amplitude feature data from the accelerometer data; inputting the first motion amplitude feature data and the depth feature data into a first neural network model to obtain a fall detection result; inputting the second motion amplitude feature data into a second neural network model to obtain a convulsion detection result; and when the fall detection result satisfies a first preset condition, the convulsion detection result satisfies a second preset condition, and/or a muscle stiffness detection result satisfies a third preset condition, determining that a state corresponding to the first data is an epileptic seizure state. In this way, a terminal device can recognize, on the basis of the state of a user such as fall, convulsion, or muscle stiffness, whether the user is in the epileptic seizure state, such that the accuracy and real-time performance of epilepsy detection can be achieved.

Description

Epilepsy detection method and device

This application claims the priority of the Chinese patent application with the application number 202110743810.6 and the application title "Epipsysis Detection Method and Device" submitted to the State Intellectual Property Office of China on June 30, 2021, the entire contents of which are incorporated by reference in this application .

technical field

The present application relates to the field of terminal technology, and in particular to a method and device for detecting epilepsy.

Background technique

Epilepsy is a chronic, non-communicable disease of the brain characterized by recurrent seizures. When a seizure occurs, a part of the body or the whole body can undergo brief involuntary convulsions (also known as partial seizures or generalized seizures), and sometimes seizures are also accompanied by loss of consciousness or incontinence of the user , Epilepsy currently affects approximately 50 million people worldwide. Therefore, the detection of epilepsy is of great significance.

Usually, professional detection equipment, such as EEG equipment, can be used to detect the abnormal discharge of brain neurons of epilepsy patients, and then determine whether the epilepsy patients are in a state of epileptic seizures.

However, due to the recurrent and sudden nature of epilepsy, the above epilepsy detection methods cannot realize real-time detection of the onset of epilepsy patients.

Contents of the invention

Embodiments of the present application provide a method and device for detecting epilepsy, which can realize real-time detection of epilepsy.

In the first aspect, an embodiment of the present application provides a method for epilepsy detection. The terminal device includes an acceleration sensor and an inductance sensor. The method includes: the terminal device acquires first data; the first data includes accelerometer data and electrical signal data; the accelerometer data is collected by the acceleration sensor, and the electrical signal data is collected by the inductance sensor; the terminal device extracts the first motion amplitude characteristic data and the depth characteristic data from the accelerometer data; the first motion amplitude characteristic data is data used for fall detection; the terminal device extracts the first motion amplitude characteristic data from the The second motion amplitude feature data is extracted from the accelerometer data; the second motion amplitude feature data is data used for twitch detection; the terminal device inputs the first motion amplitude feature data and depth feature data into the first neural network model to obtain fall detection Result; the terminal device inputs the second motion range characteristic data into the second neural network model to obtain the twitch detection result; the first motion range is greater than the second motion range; when the fall detection result satisfies the first preset condition and the twitch detection result satisfies the first When the second preset condition and/or the muscle stiffness detection result meets the third preset condition, the terminal device determines that the state corresponding to the first data is an epileptic seizure state; the muscle stiffness detection result is obtained based on the detection of electrical signal data; or, when When the fall detection result does not meet the first preset condition, the twitch detection result does not meet the second preset condition, and/or the muscle stiffness detection result does not meet the third preset condition, the terminal device determines that the state corresponding to the first data is non-epilepsy Seizures. In this way, the terminal device can identify whether the user is in an epileptic seizure based on the user's onset characteristics, such as falling, twitching, or muscle stiffness, and can achieve accurate and real-time detection of epilepsy.

Wherein, the first motion amplitude feature data can be the traditional feature in the fall detection method described in the embodiment of the application, the depth feature data can be the depth feature in the fall detection method described in the embodiment of the application, the first neural network model It can be the fall detection model described in the embodiment of the present application; the second motion amplitude characteristic data can be the traditional feature in the twitch detection method described in the embodiment of the present application, and the second neural network model can be the one described in the embodiment of the present application Twitch detection model; the first preset condition can be that the state corresponding to the first data in the fall detection result satisfies the fall state, the second preset condition can be that the state corresponding to the first data in the twitch detection result satisfies the twitch state, and the third The preset condition can be that when the rate of change of the electrical signal data exceeds the rate of change threshold, the muscle stiffness detection result indicates that the state corresponding to the first data is a muscle stiffness state; the first range of motion can be understood as the range of motion of the fall detection state; The second range of motion can be understood as the range of motion in the twitch detection state.

In a possible implementation manner, the depth feature data is obtained by the terminal device using the third neural network model to extract the depth feature from the accelerometer data. In this way, the deep feature extraction of the accelerometer data can use the deep feature to achieve accurate recognition of the fall state.

Wherein, the third neural network model may be the convolutional neural network model in the fall detection method described in the embodiment of the present application.

In a possible implementation, the third neural network model is obtained by the terminal device based on accelerometer sample data training, the third neural network model includes an input module, a depth convolution module, a point convolution module and an output module, and the depth The convolution module includes a convolution calculation layer with a kernel of 3*3, the first normalization layer, and the first stretch to the same latitude layer, and the point convolution module includes a convolution calculation layer with a kernel of 1*1, the first Two normalize layers and a second stretch to the same latitude layer.

In a possible implementation, the first motion amplitude characteristic data includes at least one of the following: acceleration intensity vector SMV, maximum value of SMV, minimum value of SMV, difference between maximum value and minimum value of SMV, FFT feature vector, acceleration rate of change , SMV average, acceleration variance, x-axis acceleration mean, y-axis acceleration mean or z-axis acceleration mean.

In a possible implementation, the second motion amplitude characteristic data includes at least one of the following: SMV average value, acceleration variance, average deviation, difference between the maximum accelerometer data and the minimum accelerometer data on the x-axis, and the maximum acceleration on the y-axis The difference between the accelerometer data and the minimum accelerometer data, or the z-axis maximum accelerometer data and the minimum accelerometer data.

In a possible implementation, the first neural network model is trained based on the motion amplitude feature sample data corresponding to the accelerometer sample data and the depth feature sample data corresponding to the accelerometer data, and the first neural network model has four layers Fully connected neural network model, the first neural network model includes the input layer, the first hidden layer, the second hidden layer and the output layer; the nodes in the input layer include the number of nodes and the depth corresponding to the first motion amplitude feature data The number of nodes corresponding to the feature data.

Wherein, the motion amplitude feature sample data corresponding to the accelerometer sample data can be the traditional feature sample data of the accelerometer data in the fall detection method described in the embodiment of the present application; the depth feature sample data corresponding to the accelerometer data can be this The depth feature sample data of the accelerometer data in the fall detection method described in the embodiment of the application.

In a possible implementation manner, the number of nodes in the input layer is 45, the number of nodes corresponding to the first motion amplitude feature data is 10, the number of nodes corresponding to the depth feature data is 35, and the number of nodes in the output layer is 2.

In a possible implementation manner, the terminal device extracts the second motion amplitude characteristic data from the accelerometer data, including: the terminal device uses mean filtering to filter the accelerometer data to obtain filtered data; the terminal device determines the filtered Whether the processed data satisfies the first state, the second state and/or the third state; the first state is the state in which the difference between adjacent accelerometer data in the filtered data is 0, and the second state is the filtering process The difference value of the adjacent accelerometer data in the post data satisfies the state of the first difference value range; The third state is the state that the difference value of the adjacent accelerometer data in the filtered data satisfies the second difference value range; When the terminal device determines that the filtered data does not satisfy the first state, the second state and/or the third state, the terminal device extracts the second motion amplitude feature data from the filtered data. In this way, the terminal device can avoid the impact of the static state, walking or running state and/or phase micro-movement state on the twitch detection by judging the current motion state, thereby improving the accuracy of the terminal device for twitch detection.

Wherein, the first state may be the static state described in the embodiment of the present application, the second state may be the walking or running state described in the embodiment of the present application, and the third state may be the phase micro-motion state described in the embodiment of the present application ; The first difference range may be greater than the second difference range.

In a possible implementation manner, the terminal device extracts the first motion amplitude feature data and depth feature data from the accelerometer data, including: the terminal device uses a filter to filter the accelerometer data to obtain filtered data; The terminal device performs down-sampling processing on the filtered data to obtain the down-sampled data; the terminal device extracts the first motion amplitude characteristic data and depth characteristic data from the down-sampled data. In this way, the above-mentioned processing process can remove the influence of noise in the data, and reduce the memory occupation of the above-mentioned data in subsequent models.

In a possible implementation, the filter has a window length of L ₁ and an amplitude of

filter, the filtered data Acc _L (t) satisfies the following formula:

Among them, Acc(t) is accelerometer data, and i is an integer greater than or equal to 0.

In a possible implementation, the method further includes: the terminal device displays a first interface; the first interface includes warning information; the warning information is used to indicate that the user is in a state of epileptic seizure; when the terminal device receives an operation on the warning information , the terminal device displays the second interface; the second interface is an interface corresponding to the desktop of the terminal device. In this way, the terminal device can realize real-time monitoring and recording of the user's epileptic seizure state, and reflect the current epileptic state on the interface in time, so that the user can easily notice it.

In a possible implementation manner, the method further includes: the terminal device sends the epileptic seizure state to other devices, and the other device is a device corresponding to an emergency contact during an epileptic seizure recorded by the terminal device. In this way, epileptic patients are kept in an open area and cannot call others when they have an epileptic seizure, and the terminal device can also send the current status of epileptic patients to emergency contacts, thereby helping epileptic patients to get rescue in time.

In a possible implementation manner, the electrical signal data is a surface electromyography signal sEMG.

In a possible implementation manner, the first data further includes temperature data and heart rate data. When the fall detection result satisfies the first preset condition, the twitch detection result satisfies the second preset condition, and/or the muscle stiffness detection result satisfies the third preset condition, When the preset conditions include: when the fall detection result meets the first preset condition, the twitch detection result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, and, in a possible implementation manner Among them, when the heart rate data meets the fourth preset condition and/or the temperature data meets the fifth preset condition; wherein, the terminal device also includes a temperature sensor and a proximity light sensor, the temperature data is collected by the temperature sensor, and the heart rate data is collected by the proximity light sensor Collected.

Wherein, the fourth preset condition may be that the heart rate data exceeds 30% of the average heart rate data recorded by the terminal device, and the fifth preset condition may be that the temperature data exceeds 30% of the average temperature data recorded by the terminal device.

In the second aspect, an embodiment of the present application provides an epilepsy detection device. The terminal device includes an acceleration sensor and an inductance sensor. The device includes: a processing unit for acquiring first data; the first data includes accelerometer data and electrical signal data; The meter data is collected by the accelerometer, and the electrical signal data is collected by the inductance sensor; the processing unit is also used to extract the first motion amplitude feature data and depth feature data from the accelerometer data; the first motion amplitude feature data is used for falling Detected data; the processing unit is also used to extract the second amplitude of motion characteristic data from the accelerometer data; the second amplitude of motion characteristic data is data used for convulsion detection; the processing unit is also used to extract the first amplitude of motion characteristic data and depth feature data are input to the first neural network model to obtain the fall detection result; the processing unit is also used to input the second motion range feature data to the second neural network model to obtain the twitch detection result; the first motion range is greater than the second Range of motion; when the fall detection result meets the first preset condition, the twitch detection result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, the determination unit is used to determine the state corresponding to the first data It is an epileptic seizure state; the muscle stiffness detection result is obtained based on the detection of electrical signal data; or, when the fall detection result does not meet the first preset condition, the twitch detection result does not meet the second preset condition, and/or the muscle stiffness detection When the result does not meet the third preset condition, the determining unit is further configured to determine that the state corresponding to the first data is a non-epileptic seizure state.

In a possible implementation manner, the depth feature data is obtained by the terminal device using the third neural network model to extract the depth feature from the accelerometer data.

In a possible implementation manner, the processing unit is specifically configured to filter the accelerometer data by means of mean filtering to obtain filtered data; the determining unit is specifically configured to determine whether the filtered data satisfies the first state , the second state and/or the third state; the first state is the state in which the difference between the adjacent accelerometer data in the filtered data is 0, and the second state is the adjacent accelerometer in the filtered data The state in which the data difference satisfies the first difference range; the third state is the state in which the difference between adjacent accelerometer data in the filtered data satisfies the second difference range; when the terminal device determines that the filtered data When the first state, the second state and/or the third state are not satisfied, the processing unit is further specifically configured to extract the second motion amplitude characteristic data from the filtered data.

In a possible implementation manner, the processing unit is specifically configured to: use a filter to filter the accelerometer data to obtain the filtered data; perform down-sampling processing to the filtered data to obtain the down-sampled data; extracting first motion amplitude feature data and depth feature data from the down-sampled data.

filter, the filtered data Acc _L (t) satisfies the following formula:

In a possible implementation manner, the display unit is configured to display a first interface; the first interface includes warning information; the warning information is used to indicate that the user is in a state of epileptic seizure; when the terminal device receives an operation on the warning information, The display unit is further configured to display a second interface; the second interface is an interface corresponding to the desktop of the terminal device.

In a possible implementation manner, the communication unit is configured to send the epileptic seizure state to other devices, and the other device is a device corresponding to an emergency contact during an epileptic seizure recorded by the terminal device.

In a possible implementation manner, the first data further includes temperature data and heart rate data. When the fall detection result satisfies the first preset condition, the twitch detection result satisfies the second preset condition, and/or the muscle stiffness detection result satisfies the third preset condition, Preset conditions include: when the fall detection result meets the first preset condition, the twitch detection result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, and the heart rate data satisfies the fourth preset condition When the condition and/or the temperature data meet the fifth preset condition; wherein, the terminal device further includes a temperature sensor and a proximity light sensor, the temperature data is collected by the temperature sensor, and the heart rate data is collected by the proximity light sensor.

In the third aspect, the embodiment of the present application provides an epilepsy detection device, including a processor and a memory, the memory is used to store code instructions; the processor is used to run the code instructions, so that the electronic device can perform any of the first aspect or the first aspect. An epilepsy detection method described in one implementation.

In the fourth aspect, the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the first aspect or any implementation manner of the first aspect. The epilepsy detection method described.

In a fifth aspect, a computer program product includes a computer program. When the computer program is executed, the computer executes the epilepsy detection method as described in the first aspect or any implementation manner of the first aspect.

It should be understood that the second aspect to the fifth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the advantageous effects obtained by each aspect and the corresponding feasible implementation manners are similar, so details are not repeated here.

Description of drawings

FIG. 1 is a schematic diagram of a scene provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a terminal device provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of an epilepsy detection framework provided by an embodiment of the present application;

FIG. 4 is a schematic flow chart of a fall detection method provided in an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a convolutional neural network model provided in an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a fall detection model provided in an embodiment of the present application;

FIG. 7 is a schematic flowchart of a fall detection method provided in an embodiment of the present application;

FIG. 8 is a schematic interface diagram of a terminal device provided in an embodiment of the present application;

FIG. 9 is a schematic interface diagram of another terminal device provided in the embodiment of the present application;

FIG. 10 is a schematic structural diagram of an epilepsy detection device provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of a hardware structure of a control device provided in an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a chip provided by an embodiment of the present application.

detailed description

In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. For example, the first value and the second value are only used to distinguish different values, and their sequence is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the number and execution order, and words such as "first" and "second" do not necessarily limit the difference.

It should be noted that, in this application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design described herein as "exemplary" or "for example" is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

In this application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, wherein a, b, c can be single or multiple.

Epilepsy is a chronic non-infectious disease of the brain. Epilepsy is usually accompanied by transient symptoms, such as sudden falls, convulsions, or muscle stiffness. Epilepsy seriously affects the quality of life of epilepsy patients and their families. Therefore, the detection of epilepsy is of great significance.

However, due to the recurrent and sudden nature of epilepsy, the real-time detection of epilepsy is difficult. Exemplarily, when the user is exercising or in the ward, when a convulsion suddenly occurs, due to the short onset time of epilepsy, even the shortest onset time can be less than 3 seconds (s), which makes people around epilepsy patients or doctors It is difficult to detect the onset of epileptic patients in the first place, and it is easy to cause missed detection; moreover, due to the frequent seizures of epileptic patients, it is often difficult for patients to judge whether they are in a state of epileptic seizures. For example, when an epileptic patient falls suddenly, due to The difficulty in detecting whether a fall is related to a seizure in this patient with epilepsy makes the detection of epilepsy even more difficult.

In view of this, an embodiment of the present application provides an epilepsy detection method. When a user wears a terminal device, the terminal device can detect epilepsy symptoms based on sensors, such as identifying whether the user is experiencing symptoms such as falls, convulsions, or muscle stiffness; furthermore, the terminal device Based on the recognized seizure symptoms of the user and the abnormality of the user's physical characteristics, it can be further identified whether the user is in a seizure state, so that the terminal device can not only realize the accuracy and real-time detection of seizures, but also detect seizures The status is notified in time to the emergency contacts or doctors bound to the epilepsy patient's terminal device, so as to speed up the progress of epilepsy treatment and avoid missing the best treatment time.

It can be understood that the above-mentioned terminal device may be a wearable device, such as a smart watch, a smart bracelet, a wearable virtual reality (virtual reality, VR) device, or a wearable augmented reality (augmented reality, AR) device Wait. The above-mentioned terminal device may also be a smart phone or a tablet. In the embodiment of the present application, no limitation is imposed on the specific technology and specific device form adopted by the terminal device.

In order to better understand the method of the embodiment of the present application, the application scenarios applicable to the embodiment of the present application are firstly described below.

Exemplarily, FIG. 1 is a schematic diagram of a scenario provided by an embodiment of the present application. As shown in FIG. 1 , this scenario may include 101, and the user 101 may carry a terminal device 102 capable of detecting epilepsy, for example, the terminal device may be a wearable device such as a smart watch or a smart bracelet.

In the scenario corresponding to FIG. 1 , the terminal device 102 can detect the current state of the user 101 based on built-in sensors, for example, detect whether the user 101 is in a state of stillness, movement, fall, twitching or muscle stiffness, or can also detect the user's body temperature or Human characteristics such as heart rate. For example, when the terminal device 102 detects that the user 101 is in a falling state based on the acceleration sensor, and detects that the body temperature of the user 101 exceeds 30% of the normal body temperature recorded by the terminal device 102 based on the temperature sensor, the terminal device 102 can If the body temperature is abnormal, it is determined that the user 101 is in an epileptic seizure, and then the terminal device 102 can record the time of the epileptic seizure, and send information such as the onset of the epileptic patient or the location of the epileptic patient to the emergency contact recorded by the terminal device 102 .

In order to better understand the embodiment of the present application, the structure of the terminal device in the embodiment of the present application is introduced below. Exemplarily, FIG. 2 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

The terminal device may include a processor 110, an internal memory 121, a universal serial bus (universal serial bus, USB) interface, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, and a wireless communication module 160 , an audio module 170, a speaker 170A, a receiver 170B, a sensor module 180, a button 190, an indicator 192, a camera 193, and a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, an inductance sensor 180F, a proximity light sensor 180G, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone Conductivity sensor 180M etc.

It can be understood that, the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

Processor 110 may include one or more processing units. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. A memory may also be provided in the processor 110 for storing instructions and data.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. The power management module 141 is used for connecting the charging management module 140 and the processor 110 .

The wireless communication function of the terminal device can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Antennas in end devices can be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on terminal equipment. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system ( global navigation satellite system (GNSS), frequency modulation (frequency modulation, FM) and other wireless communication solutions.

The terminal device realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal device can realize the shooting function through ISP, camera 193 , video codec, GPU, display screen 194 and application processor.

Camera 193 is used to capture still images or video. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data.

The terminal device can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. The terminal device can listen to music through the speaker 170A, or listen to hands-free calls. Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the terminal device answers a phone call or voice information, the receiver 170B can be placed close to the human ear to listen to the voice.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . The gyroscope sensor 180B can be used to determine the motion posture of the terminal device. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor.

The acceleration sensor 180E can detect the acceleration of the terminal device in various directions. In the embodiment of the present application, the acceleration sensor 180E can be a three-axis (including x-axis, y-axis and z-axis) accelerometer sensor, which is used to measure the user's accelerometer in the state of falling, non-falling, twitching and non-twitching. data (or accelerometer data or acceleration data, etc.).

The inductance sensor 180F is used to detect the electrical skin signal of the human body, and the change of the electrical skin signal of the human body can be used to represent the tension degree of the muscles in the skin. In the embodiment of the present application, the inductance sensor 180F may be used to detect surface electromyography (sEMG), etc., and the sEMG may be a biological current generated by the contraction of the surface muscles of the human body. For example, when the inductance sensor 180F detects that the sEMG signal suddenly increases within a short period of time, the terminal device may determine that the user's muscles are in a stiff state at this time.

The proximity light sensor 180G may include, for example, a light emitting diode (light emitting diode, LED) and a light detector, and the light detector may be a photodiode (photo diode, PD). In the embodiment of the present application, the proximity light sensor 180G may use photoplethysmography (photoplethysmographic, PPG) to detect the user's heart rate or other human body characteristics. Wherein, the LED in the proximity light sensor 180G can be used to emit light sources such as red light, green light or infrared light, and the PD in the proximity light sensor 180G can be used to receive the LED light signal and process the light signal into an electrical signal . For example, PD can be used to receive the light signal sent by LED and reflected back by skin tissue, and process the signal into an electrical signal, and then the terminal device can detect the user's heart rate, breathing rate or blood oxygen based on the electrical signal. feature.

The ambient light sensor 180L is used for sensing ambient light brightness.

The temperature sensor 180J is used to detect temperature. In the embodiment of the present application, when the terminal device touches the user's skin, the temperature sensor 180J can be used to measure the temperature of the skin (or it can also be understood as the user's body temperature).

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The bone conduction sensor 180M can acquire vibration signals.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The terminal device can receive key input and generate key signal input related to user settings and function control of the terminal device. The indicator 192 can be an indicator light, which can be used to indicate the charging status, the change of the battery capacity, and can also be used to indicate messages, missed calls or notifications, etc.

Exemplarily, FIG. 3 is a schematic diagram of an epilepsy detection architecture provided by an embodiment of the present application. As shown in FIG. 3 , the epilepsy detection architecture may include: a fall detection module 301 , a twitch detection module 302 , a forearm muscle electrical signal detection module 303 , a health assistance module 304 , and an epilepsy detection module 305 . Wherein, the fall detection module 301 is used to detect whether the user is in a state of falling; the twitch detection module 302 is used to detect whether the user is in a twitch state; the forearm muscle electrical signal detection module 303 is used to detect whether the user is in a muscle stiffness state; the health assistance module 304 It is used to detect the user's human body characteristics, which can be heart rate or body temperature, etc.; the epilepsy detection module 305 is used to detect whether the user is in an epileptic seizure according to the user's fall state, convulsion state, or muscle stiffness state, etc., as well as the user's human body characteristics state.

In the fall detection module 301, the terminal device can obtain the user's accelerometer data based on the three-axis accelerometer sensor, perform traditional feature extraction and deep feature extraction of the fall state based on the above accelerometer data, and input the above traditional features and depth features into Prediction is made in the trained fall detection model, and then the terminal device can output the fall detection result corresponding to the traditional feature and the deep feature, for example, the user is in a falling state or the user is in a non-falling state. Among them, the fall detection model is obtained by training the accelerometer sample data of the epileptic patient in the falling state and the accelerometer sample data of the epileptic patient in the non-falling state. The traditional feature can be understood as a feature that can be obtained based on simple calculation or statistics of the accelerometer data, and the deep feature can be a deeper and more abstract feature obtained by further mining the accelerometer data based on the neural network model.

In the twitch detection module 302, the terminal device can obtain the user's accelerometer data based on the three-axis accelerometer sensor, and judge whether the user is in a static state, walking or running state, and relatively inching state based on the above accelerometer data. For the above three states, the terminal device can extract the traditional feature of the twitch state based on the above accelerometer data, and input the above traditional feature into the trained twitch detection model for prediction, and then the terminal device can output the twitch corresponding to the traditional feature Detection results, for example, the user is in a twitching state or the user is in a non-twitching state. Wherein, the twitch detection model is obtained by training the accelerometer sample data of epileptic patients in twitch state and the accelerometer sample data of epileptic patients in non-twitch state.

In the forearm muscle electrical signal detection module 303, the terminal device can detect the sEMG signal on the user's skin surface based on the inductive sensor, and determine whether the forearm muscle is stiff by detecting whether the sEMG signal suddenly increases in a short time. It can be understood that, taking the terminal device as an example of a smart bracelet, when the user wears the smart bracelet at different parts, the smart bracelet can detect muscle stiffness in different parts. For example, when the user wears the smart bracelet in the wrist, the smart bracelet can determine whether the forearm muscles are stiff based on the detected sEMG signal; or, when the user wears the smart bracelet in the ankle, the smart bracelet can also Based on the detected sEMG signal, it is determined whether the calf muscle is stiff. In the embodiment of the present application, there is no specific limitation on the position where the user wears the terminal device and the position where the terminal device performs stiffness detection.

In the health assistance module 304, the terminal device may detect the user's body temperature based on the temperature sensor, or detect the user's heart rate and other human body characteristics based on the proximity light sensor. It can be understood that the human body features used for epilepsy detection and acquisition may include other content according to actual scenarios, which is not limited in this embodiment of the present application.

In the epilepsy detection module 305, when the terminal device determines that the user meets at least one of the state of falling, convulsions, or muscle stiffness, and the terminal device determines that the user's heart rate exceeds 30% of the heart rate in the normal state, and/or the user When the body temperature exceeds 30% of the body temperature in a normal state, the terminal device can determine that the user is in a state of epileptic seizures.

Based on this, the terminal device can not only realize the real-time detection of the user's epilepsy, but also accurately identify the specific symptoms of epilepsy, and then the doctor can accurately judge the condition of the epilepsy patient based on the detected data of the epilepsy.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be implemented independently, or may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

The epilepsy detection solution provided in the embodiments of the present application can not only detect the seizures of epilepsy patients in real time, but also identify the specific symptoms of epilepsy, for example, the symptoms of epilepsy are falls, convulsions, or muscle stiffness. Therefore, in the embodiment of the present application, the detection of the state of falling (as shown in the corresponding embodiment in Figure 4), the detection of the twitching state (as in the embodiment corresponding to Figure 7), the detection of the state of muscle stiffness, and the detection of the state based on the falling state can be realized. , twitch state, muscle stiffness state and user's human body data to realize the detection of epilepsy state.

Exemplarily, the embodiment of the present application can realize the detection of the falling state. For example, FIG. 4 is a schematic flowchart of a fall detection method provided in an embodiment of the present application. As shown in Figure 4, fall detection methods can include:

S401. The terminal device collects accelerometer data of the user.

In the embodiment of the present application, the user's accelerometer data includes: the accelerometer data of the epileptic patient when he is convulsed, and the accelerometer data of the epileptic patient when he is not convulsing.

The accelerometer data received by the terminal device from the three-axis accelerometer sensor can be:

Wherein, the Acc(t) may be a three-axis time series array synchronized by time stamp calibration. The frequency at which the three-axis accelerometer sensor collects accelerometer data may be 100 hertz (hz), which can be understood as collecting accelerometer data 100 times in 1 second, and accelerometer data is collected every 10 milliseconds (ms).

S402. The terminal device preprocesses the accelerometer data.

In the embodiment of the present application, the preprocessing process may include processing such as low-pass filtering and down-sampling. Exemplarily, in order to filter out the influence of high-frequency noise, the terminal device may input the accelerometer data acquired in S401 to a low-pass filter for filtering processing. For example, using the window length L ₁ , the amplitude

The filters of are respectively filtered in three dimensions, then the result of convolving the original data with the rectangular window W _L can be:

Furthermore, without affecting the accuracy, in order to reduce the complexity of subsequent data processing and the memory usage of subsequent model parameters, the terminal device can down-sample the data:

Acc(t)=Acc(5i) i=0,1,2,3,…

S403. The terminal device performs traditional feature extraction on the preprocessed data.

In the embodiment of the present application, the traditional features may include: acceleration intensity vector (signal magnitude vector, SMV) (or also called combined velocity), combined velocity maximum value, combined velocity minimum value, combined velocity maximum and minimum difference, fast At least 10 data such as Fourier transform (fast fourier transform, FFT) eigenvector, acceleration change rate, average value of total velocity, acceleration variance, and average value of acceleration x, y, z within 5s. Therefore, 1*10 data can be generated every second.

Among them, the combined speed can be:

Wherein, x, y, and z can be the accelerometer data on the x-axis, y-axis, and z-axis respectively. The combined speed can be the tenth combined speed among the 20 combined speeds within 1 second, which can be used to represent the instantaneous speed in the falling state in 1 second.

The maximum combined speed may be: the maximum value among multiple combined speeds within 1 second. The minimum combined speed may be: the minimum value among multiple combined speeds within 1 second.

The maximum and minimum difference of combined speed may be: the difference between the maximum value and the minimum value of multiple combined speeds within 1 second.

The FFT eigenvector may be used for converting time domain data into frequency domain data. In the embodiment of the present application, 0.5s of accelerometer data can be randomly selected every 1 second for FFT calculation.

The rate of change of the acceleration may be: the rate of change of the combined velocity. For example, the combined speed in the first second is the average value of 20 combined speeds in the first second, then the acceleration change rate in the second second can be the value of the first combined speed in the second second and the combined speed in the first second The rate of change of (or it can also be understood as deriving the average value of the acceleration in the last second). It is used to represent the acceleration change in the falling state.

The average value of the composite speed may be: the average value of multiple composite speeds within the 1 second.

The acceleration variance may be: a variance value of multiple combined velocities within 1 second.

The average value of the acceleration x, y, and z within the 5s can be: when taking the acceleration average value in the first second, x can be the average value of the 20 accelerometer data acquired by the x-axis in the first second, and y can be the The average value of 20 accelerometer data acquired by the y-axis in the first second, and z may be the average value of the 20 accelerometer data acquired by the z-axis in the first second. When taking the average acceleration value in the second second, x can be the average value of the acceleration average value of the x-axis in the first second and the average value of the 20 accelerometer data acquired by the x-axis in the second second, and y can be taken The above-mentioned acceleration average value of the y-axis in the first second and the average value of the average value of the 20 accelerometer data acquired by the y-axis in the second second, z can be the acceleration average value of the above-mentioned first second z-axis and the second The average of the averages of the 20 accelerometer data acquired by the z-axis in seconds. The average acceleration values in other seconds are similar to the above average acceleration values in the second second, and will not be repeated here.

S404. The terminal device uses a convolutional neural network model to extract deep features from the preprocessed data.

Exemplarily, FIG. 5 is a schematic structural diagram of a convolutional neural network model provided in an embodiment of the present application. As shown in Figure 5, the convolutional neural network model can include: an input module 501, a network structure module 502, and an output module 503, and the network structure module 502 can include: a deep convolution (conv depthwise) module 504, a point convolution (conv pointwise) module 505 and other convolution modules. Wherein, the deep convolution module 504 can be composed of a convolution calculation layer (conv3*3) with a kernel of 3*3, a normalization layer (batch norm), and stretching to the same latitude layer (scale). The product module 505 may be composed of a convolution calculation layer (conv1*1) with a kernel of 1*1, a normalization layer (batch norm), and stretching to the same latitude layer (scale). The output module 503 may include an activation function, such as a hyperbolic tangent function (tanh).

In the embodiment corresponding to FIG. 5 , the convolutional neural network model is trained from accelerometer sample data. For example, the terminal device inputs accelerometer data for 0.5 seconds into the convolutional neural network model, for example, the input parameter may be 10*3, and the output depth feature size, for example, the output parameter may be 1*35.

S405. The terminal device uses the fall detection model to detect a fall state corresponding to the traditional feature and the depth feature.

In the embodiment of the present application, the fall detection model is obtained by training the traditional feature sample data of the accelerometer data and the deep feature sample data of the accelerometer data.

The traditional feature in the fall detection model can also be called the first motion amplitude feature.

Exemplarily, FIG. 6 is a schematic structural diagram of a fall detection model provided in an embodiment of the present application. As shown in FIG. 6 , the fall detection model may be a four-layer fully connected neural network model, including an input layer 601 , a hidden layer 602 , a hidden layer 603 and an output layer 604 .

In this fall detection model, traditional feature data and deep feature data can be used as input data of the input layer 601, each data value can correspond to an input node, and the number of nodes in the input layer can be 45, for example, the number of nodes of traditional features can be is 10, and the number of nodes of the depth feature can be 30.

The number of nodes in the hidden layer 602 and the hidden layer 603 can be preset, for example, the number of nodes in the hidden layer 602 and the hidden layer 603 can be obtained according to the history of training the fall detection model, such as the node Both can be 15. As shown in Figure 6, w in hidden layer 602, hidden layer 603 and output layer 604 can be weights (weights), b can be bias (bias), it can be understood that when the input of hidden layer 602 When x, the output of the hidden layer 602 can be y=wx+b; the activation function in the hidden layer 602 and the hidden layer 603 can be tanh, or the activation function can also be a linear rectification function (rectified linear unit , ReLU), ReLU 6 or S-type growth function (sigmoid), etc., which are not limited in the embodiments of the present application.

The number of nodes in the output layer 604 can be 2, which is used to output whether the data corresponding to the traditional feature data and the deep feature data belongs to the falling state, and the above output layer adopts full connection.

Wherein, in the embodiment of the present application, the conjugate gradient method may be used as the training method of the fall detection model.

Exemplarily, one possible implementation of training the above fall detection model based on traditional feature sample data and deep feature sample data is: input in the neural network model to be trained, the traditional features of epilepsy patients in the falling state and the non-falling state Sample data, as well as deep feature sample data of epilepsy patients in the fall state and non-fall state, use the neural network model to be trained to output the predicted fall situation, and use the loss function to compare the gap between the predicted fall situation and the real fall situation, for example The recall rate or misrecognition rate of the predicted falls can be calculated. When the predicted falls output by the model and the real falls do not meet the loss function, adjust the model parameters and continue training; until the predicted falls output by the model are consistent with If the gap between real falls satisfies the loss function, the model training ends and a fall detection model is obtained. Furthermore, the terminal device can identify whether the user is in a falling state based on the traditional feature data and the depth feature data of the user's accelerometer data.

It can be understood that, after the terminal device obtains the user's accelerometer data in the step S401, the steps shown in S402-S405 can be implemented in the terminal device or in the server. Exemplarily, the terminal device may upload the accelerometer data obtained in the step S401 to the server, execute the steps S402-S405 in the server to obtain the traditional features and depth features of the accelerometer data, and identify the The fall state corresponding to the traditional feature and the depth feature, further, the server may send the above fall state to the terminal device.

Based on this, the terminal device can extract traditional feature data and depth feature data corresponding to the detected fall state based on the detected accelerometer data of the user's current state, and more accurately identify whether the user is in a fall state based on the fall detection model.

Exemplarily, the embodiment of the present application can realize the detection of the twitching state. For example, FIG. 7 is a schematic flowchart of a fall detection method provided in the embodiment of the present application. As shown in Figure 7, fall detection methods may include:

S701. The terminal device collects accelerometer data of the user.

In the embodiment of the present application, the user's accelerometer data may include: the accelerometer data of the epileptic patient when he is convulsed, and the accelerometer data of the epileptic patient when he is not convulsing.

S702. The terminal device performs mean filtering processing on the accelerometer data.

It can be understood that the mean filtering is used to remove the influence of noise in the accelerometer data.

S703. The terminal device uses the accelerometer data to determine whether the current user is in a static state, a walking or running state, or a phase micro-movement state.

In the embodiment of this application, when the terminal device uses the accelerometer data to judge that the current user is in a static state, a walking or running state, or a phase inching state, etc., the terminal device can perform the steps shown in S704; or, when the terminal device When judging from the accelerometer data that the current user is not in a static state, a walking or running state, or a phase inching state, the terminal device may execute the steps shown in S705.

Wherein, the static state can be understood as the difference of the combined speed in the accelerometer data approaches 0; The speed difference satisfies a certain difference range, for example, it can be 2-3, etc.; the walking or running state can be understood as the difference of the combined speed approaches the corresponding difference range of walking or running, for example, it can be 3-10 Wait. Alternatively, the aforementioned static state, walking or running state, or relative micro-movement state can also be identified based on a trained detection model.

S704. The terminal device ends the twitch detection process.

S705. The terminal device performs traditional feature extraction on the accelerometer data corresponding to the non-stationary state, the non-walking or running state, and the non-relative micro-motion state.

In the embodiment of the present application, the traditional features may include: the average value of the combined velocity, the variance of the acceleration, the average deviation, the maximum and minimum difference on the x-axis, the maximum and minimum difference on the y-axis, and the maximum and minimum difference on the z-axis. Therefore, 1*6 data can be generated every second.

Wherein, the average value of the combined velocity and the variance of the acceleration are the same as the average value of the combined velocity and the variance of the acceleration in the step shown in S403 , and will not be repeated here.

The average deviation may be: a difference from the average speed of the previous second.

The maximum and minimum difference of the x-axis may be: the difference between the maximum value of the accelerometer data and the minimum value of the accelerometer data in the x-axis within 1s.

The maximum and minimum difference of the y-axis may be: the difference between the maximum value of the accelerometer data and the minimum value of the accelerometer data in the y-axis within 1s.

The maximum and minimum difference of the z-axis may be: the difference between the maximum value of the accelerometer data and the minimum value of the accelerometer data in the z-axis within 1s.

S706. The terminal device uses the twitch detection model to detect the twitch state corresponding to the traditional feature.

In the embodiment of the present application, the fall detection model is trained by traditional feature sample data of epileptic patients in convulsive state and traditional feature sample data of epileptic patients in non-convulsive state.

The traditional feature in this twitch detection model can also be referred to as the second motion magnitude feature. The first range of motion can be understood as the range of motion during fall detection, and the second range of motion can be understood as the range of motion during detection of convulsions. Wherein, the second motion range is greater than the first motion range. It is understandable that the range of motion during fall detection is large.

It can be understood that the training method of the twitch detection model in the step S706 is similar to the training method of the fall detection model in the step S405. Exemplarily, one possible implementation of training a twitch detection model based on traditional feature sample data is as follows: input traditional feature sample data of epileptic patients in twitching state and non-twitching state in the neural network model to be trained, and use the to-be-trained The neural network model outputs the predicted twitching situation, and uses the loss function to compare the gap between the predicted twitching situation and the real twitching situation. For example, the recall rate or misrecognition rate of the predicted twitching situation can be calculated. If the loss function is not satisfied, adjust the model parameters and continue training; until the gap between the predicted twitching situation output by the model and the real twitching situation satisfies the loss function, then the model training ends and the twitch detection model is obtained. Furthermore, the terminal device can identify whether the user is in a twitching state based on the traditional feature data of the user's accelerometer data.

It can be understood that, after the terminal device obtains the user's accelerometer data in the step S701, the steps shown in S702-S706 can be implemented in the terminal device or in the server, and the specific process will not be repeated.

Based on this, the terminal device can extract the traditional feature data corresponding to the detected twitch state according to the detected accelerometer data of the user's current state, and more accurately identify whether the user is in the twitch state based on the twitch detection model.

Exemplarily, the embodiment of the present application can realize the detection of muscle stiffness. In the embodiment of the present application, the terminal device can detect the sEMG signal on the user's skin surface based on the inductance sensor, and determine whether the forearm muscle is stiff by detecting whether the sEMG signal suddenly increases in a short period of time.

Exemplarily, the terminal device can obtain the sampling points of the sEMG signal within a period of time, such as obtaining the data of 20 sampling points of the sEMG signal within 5 seconds, taking the first 5 sampling points among the 20 sampling points as an example, if the sampling points between The time difference between them is △t, when the terminal equipment determines that among the first 5 sampling points, the signal of the 5th sampling point is different from the signal of the 1st sampling point (or the signal of the 2nd sampling point, the signal of the 3rd sampling point signal or the signal of the fourth sampling point) exceeds 50%, then the terminal device can determine that the user is in a state of muscle stiffness.

It can be understood that if the signal change rate of the first 5 sampling points among the 20 sampling points does not exceed 50%, then the signal change rate of the next 5 sampling points can be judged. The specific judgment process is similar to the above. This will not be repeated here.

It can be understood that the specific sampling method and muscle stiffness judgment method provided in the embodiment of the present application may include other content according to the actual scene, which is not limited in the embodiment of the present application.

Based on this, the terminal device can more accurately identify whether the user is in a state of muscle stiffness according to the detected change of the sEMG signal of the user's current state.

Exemplarily, the terminal device may comprehensively determine whether the user is in an epileptic seizure state based on the fall detection situation, twitch detection situation, and muscle stiffness detection situation, as well as the abnormality of the user's human body feature data.

Wherein, the human body characteristic data may include data such as heart rate or body temperature. The abnormality judgment of the human body characteristic data may be that, based on the average heart rate data of the human body monitored in real time, the terminal device determines that the current heart rate data exceeds 30% of the average heart rate data of the human body, then the terminal device can determine the current abnormal heart rate; and/or , the terminal device determines that the current body temperature data exceeds 30% of the average body temperature data of the human body based on the average body temperature data monitored in real time, and the terminal device can determine that the current body temperature is abnormal.

Exemplarily, when the terminal device determines that the user meets at least one of a falling state, a twitching state, or a muscle stiffness state, and the user meets at least one of abnormal heart rate data or abnormal body temperature data, the terminal device may determine that the user is in Seizure state. For example, when the terminal device determines that the user is in a falling state, and the heart rate data exceeds 25% of the normal heart rate data recorded by the terminal device, since the user's heart rate data is not abnormal, the terminal device can determine that the user is not currently in an epileptic seizure state. It can be understood that the above-mentioned data for judging the abnormality of human body characteristics may include other content according to the actual scene, which is not limited in this embodiment of the present application.

Based on this, the terminal device can more accurately identify whether the user is in a state of epileptic seizures through epileptic seizure conditions such as falls, convulsions, and muscle stiffness, as well as human body characteristic data.

Based on the detection of the epileptic seizure state by the terminal device, in a possible implementation, the terminal device can not only record the user's epileptic seizure state in real time (as shown in the corresponding embodiment in Figure 8), but also record the disease status through The message is sent to the emergency contact stored in the terminal device (as shown in the embodiment corresponding to FIG. 9 ).

Exemplarily, FIG. 8 is a schematic interface diagram of a terminal device provided in an embodiment of the present application. In the embodiment corresponding to FIG. 8 , the terminal device is a smart watch as an example for illustration, and this example does not constitute a limitation to the embodiment of the present application.

When the smart watch receives the user's operation of opening the epilepsy record in the sports health application, the smart watch can display an interface as shown in a in Figure 8, which can display the epilepsy of the user wearing the smart watch within a period of time. The number of seizures, for example, recording the number of seizures of the user within 6.1-6.7 days.

On the interface shown in a in Figure 8, when the smart watch receives the user triggering any time point in 6.1-6.7, such as triggering the operation of the control corresponding to 6.4, the smart watch can display as shown in b in Figure 8 The interface, which can further display the specific time of 6.4 on that day, for example, one seizure around 08:00, two seizures around 12:00, one seizure around 16:00 and one seizure at 20:00. 00 around 1 attack. Alternatively, when the smart watch is bound to the user's smart phone, the smart phone can also send the data corresponding to the epileptic seizure to the smart phone, and then the user can also view the above-mentioned data corresponding to the epileptic seizure based on the records in the smart phone.

Based on this, the terminal device can realize real-time monitoring and recording of the user's epileptic seizure status, and the recorded data will be helpful for subsequent epilepsy treatment of the user.

Exemplarily, FIG. 9 is a schematic interface diagram of another terminal device provided in an embodiment of the present application. In the embodiment corresponding to FIG. 9 , the terminal device is a smart watch as an example for illustration, and this example does not constitute a limitation to the embodiment of the present application.

When the smart watch detects the symptoms of the user's epilepsy based on the epilepsy detection method provided by the embodiment of the present application, such as the symptoms of falling, the terminal device may display the interface as shown in FIG. 9 . The interface may display epileptic state warning information, and the epileptic state warning information may be that it is detected that you are currently in a falling state, and your current state has been reported to an emergency contact.

In a possible implementation manner, when the terminal device receives a user's trigger for the alarm information, the terminal device may display an interface corresponding to the desktop.

In a possible implementation, when detecting that the user is in a state of epileptic seizures, the terminal device can also obtain the location information of the user, and report the location information to the emergency contact; or, the terminal device can also emit an alarm sound, so as to obtain timely assistance.

Based on this, even if an epileptic patient is in an open area and cannot call others during an epileptic seizure, the terminal device can send the current status of the epileptic patient to the emergency contact, thereby helping the epileptic patient get help in time.

The method provided by the embodiment of the present application is described above with reference to FIG. 3-FIG. 9 , and the device for performing the above method provided by the embodiment of the present application is described below. As shown in Figure 10, Figure 10 is a schematic structural diagram of an epilepsy detection device provided in the embodiment of the present application. The epilepsy detection device may be the terminal device in the embodiment of the present application, or it may be a chip or a chip system in the terminal device .

As shown in FIG. 10 , the epilepsy detection device 100 can be used in a communication device, a circuit, a hardware component or a chip, and the epilepsy detection device includes: a display unit 1001 , a determination unit 1002 , a processing unit 1003 , and a communication unit 1004 . Among them, the display unit 1001 is used to support the steps of displaying performed by the epilepsy detection method; the determination unit 1002 is used to support the epilepsy detection device to perform the steps of determination; the processing unit 1003 is used to support the epilepsy detection device to perform the steps of information processing; the communication unit 1004 uses To support the epilepsy detection device to perform the steps of sending and receiving information.

Specifically, an embodiment of the present application provides an epilepsy detection device 100, the terminal device includes an acceleration sensor and an inductance sensor, and the device includes: a processing unit 1003, configured to acquire first data; the first data includes accelerometer data and electrical signal data; The accelerometer data is collected by the acceleration sensor, and the electrical signal data is collected by the inductance sensor; the processing unit 1003 is also used to extract the first motion amplitude feature data and depth feature data from the accelerometer data; the first motion amplitude feature data is used The data used for fall detection; the processing unit 1003 is also used to extract the second motion amplitude feature data from the accelerometer data; the second motion amplitude feature data is data used for twitch detection; the processing unit 1003 is also used to extract the first Input the feature data of the range of motion and the feature data of the depth into the first neural network model to obtain the fall detection result; the processing unit 1003 is also used to input the feature data of the second range of motion into the second neural network model to obtain the twitch detection result; the first The range of motion is greater than the second range of motion; when the fall detection result meets the first preset condition, the twitch detection result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, the determination unit 1002 is used to determine The state corresponding to the first data is an epileptic seizure state; the muscle stiffness detection result is obtained based on the detection of electrical signal data; or, when the fall detection result does not meet the first preset condition, the twitch detection result does not meet the second preset condition, And/or when the muscle stiffness detection result does not meet the third preset condition, the determining unit 1002 is further configured to determine that the state corresponding to the first data is a non-epileptic seizure state.

In a possible implementation manner, the processing unit 1003 is specifically configured to filter the accelerometer data by means of mean filtering to obtain filtered data; the determining unit 1002 is specifically configured to determine whether the filtered data satisfies the first A state, a second state and/or a third state; the first state is a state in which the difference between adjacent accelerometer data in the filtered data is 0, and the second state is the adjacent accelerometer data in the filtered data The difference of accelerometer data satisfies the state of the first difference range; the third state is the state of the difference of adjacent accelerometer data in the filtered data meets the second difference range; when the terminal device determines that after the filtering process When the data does not satisfy the first state, the second state and/or the third state, the processing unit 1003 is further specifically configured to extract the second motion amplitude feature data from the filtered data.

In a possible implementation manner, the processing unit 1003 is specifically configured to: use a filter to filter the accelerometer data to obtain the filtered data; perform down-sampling processing on the filtered data to obtain the down-sampling processing the post-processing data; extracting the first motion amplitude feature data and depth feature data from the down-sampled data.

filter, the filtered data Acc _L (t) satisfies the following formula:

In a possible implementation manner, the display unit 1001 is configured to display a first interface; the first interface includes warning information; the warning information is used to indicate that the user is in a state of epileptic seizure; when the terminal device receives an operation on the warning information , the display unit 1001 is further configured to display a second interface; the second interface is an interface corresponding to the desktop of the terminal device.

In a possible implementation manner, the communication unit 1004 is configured to send the epileptic seizure state to other devices, and the other device is a device corresponding to an emergency contact during an epileptic seizure recorded by the terminal device.

In the epilepsy detection device 100 , the display unit 1001 , the determination unit 1002 , the processing unit 1003 and the communication unit 1004 may be connected by wires. Wherein, the communication unit 1004 may be an input or output interface, a pin or a circuit, and the like. Exemplarily, the storage unit 1005 may store computer execution instructions in the terminal device, so as to enable the processing unit 1003 to execute the methods in the foregoing embodiments. The storage unit 1005 may be a register, a cache, or a RAM, etc., and the storage unit 1005 may be integrated with the processing unit 1003 . The storage unit 1005 may be a ROM or other types of static storage devices that can store static information and instructions, and the storage unit 1005 may be independent from the processing unit 1302 .

In a possible embodiment, the epilepsy detection device 100 may further include: a storage unit 1005 . The processing unit 1003 is connected to the storage unit 1005 through a line.

The storage unit 1005 may include one or more memories, and the memories may be devices used to store programs or data in one or more devices and circuits.

The storage unit 1005 can exist independently, and is connected to the processing unit 1003 of the epilepsy detection device through a communication line. The storage unit 1005 may also be integrated with the processing unit 1003 .

FIG. 11 is a schematic diagram of the hardware structure of a control device provided in the embodiment of the present application. As shown in FIG. 1103 as an example for illustration).

The processor 1101 can be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, a specific application integrated circuit (application-specific integrated circuit, ASIC), or one or more for controlling the execution of the application program program integrated circuit.

Communication lines 1104 may include circuitry that communicates information between the components described above.

The communication interface 1103 uses any device such as a transceiver for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (wireless local area networks, WLAN) and so on.

Possibly, the control device may also include a memory 1102 .

The memory 1102 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types that can store information and instructions It can also be an electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be programmed by a computer Any other medium accessed, but not limited to. The memory may exist independently and be connected to the processor through the communication line 1104 . Memory can also be integrated with the processor.

Wherein, the memory 1102 is used to store computer-executed instructions for implementing the solutions of the present application, and the execution is controlled by the processor 1101 . The processor 1101 is configured to execute computer-executed instructions stored in the memory 1102, so as to implement the method provided in the embodiment of the present application.

Possibly, the computer-executed instructions in the embodiment of the present application may also be referred to as application program code, which is not specifically limited in the embodiment of the present application.

In a specific implementation, as an embodiment, the processor 1101 may include one or more CPUs, for example, CPU0 and CPU1 in FIG. 11 .

In a specific implementation, as an embodiment, the control device may include multiple processors, for example, processor 1101 and processor 1105 in FIG. 11 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

Exemplarily, FIG. 12 is a schematic structural diagram of a chip provided by an embodiment of the present application. The chip 120 includes one or more than two (including two) processors 1220 and a communication interface 1230 .

In some implementations, the memory 1240 stores the following elements: executable modules or data structures, or subsets thereof, or extensions thereof.

In this embodiment of the present application, the memory 1240 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1220 . A part of the memory 1240 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM).

In the embodiment of the present application, the memory 1240 , the communication interface 1230 and the processor 1220 are coupled together through the bus system 1210 . Wherein, the bus system 1210 may include not only a data bus, but also a power bus, a control bus, and a status signal bus. For ease of description, the various buses are labeled bus system 1210 in FIG. 12 .

The methods described in the foregoing embodiments of the present application may be applied to the processor 1220 or implemented by the processor 1220 . The processor 1220 may be an integrated circuit chip and has signal processing capability. In the implementation process, each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 1220 or instructions in the form of software. The above-mentioned processor 1220 may be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), an off-the-shelf programmable gate Array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components, the processor 1220 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present invention .

The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. Wherein, the software module may be located in a mature storage medium in the field such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 1240, and the processor 1220 reads the information in the memory 1240, and completes the steps of the above method in combination with its hardware.

In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of computer program products. Wherein, the computer program product may be written in the memory in advance, or may be downloaded and installed in the memory in the form of software.

A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center. Computer readable storage medium can be Any available media capable of being stored by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. For example, available media may include magnetic media (e.g., floppy disks, hard disks, or tapes), optical media (e.g., A digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (for example, a solid state disk (solid state disk, SSD)), etc.

The embodiment of the present application also provides a computer-readable storage medium. The methods described in the foregoing embodiments may be fully or partially implemented by software, hardware, firmware or any combination thereof. Computer-readable media may include computer storage media and communication media, and may include any medium that can transfer a computer program from one place to another. A storage media may be any target media that can be accessed by a computer.

As a possible design, the computer-readable medium may include compact disc read-only memory (compact disc read-only memory, CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer-readable medium may include a magnetic disk memory or other disk storage devices. Also, any connected cord is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, compact disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Reproduce data.

Combinations of the above should also be included within the scope of computer-readable media. The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present invention, and should cover all Within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

A method for epilepsy detection, characterized in that the terminal device includes an acceleration sensor and an inductance sensor, and the method includes:

The terminal device acquires first data; the first data includes accelerometer data and electrical signal data; the accelerometer data is collected by the acceleration sensor, and the electrical signal data is collected by the inductance sensor;

The terminal device extracts first motion amplitude feature data and depth feature data from the accelerometer data; the first motion amplitude feature data is data used for fall detection;

The terminal device extracts second motion amplitude feature data from the accelerometer data; the second motion amplitude feature data is data used for convulsion detection;

The terminal device inputs the first motion amplitude feature data and the depth feature data into a first neural network model to obtain a fall detection result;

The terminal device inputs the second motion amplitude feature data into a second neural network model to obtain a twitch detection result; the first motion range is greater than the second motion range;

When the fall detection result satisfies the first preset condition, the twitch detection result satisfies the second preset condition, and/or the muscle stiffness detection result satisfies the third preset condition, the terminal device determines that the first data corresponds to The state is an epileptic seizure state; the muscle stiffness detection result is obtained based on the detection of the electrical signal data;

Or, when the fall detection result does not meet the first preset condition, the twitch detection result does not meet the second preset condition, and/or the muscle stiffness detection result does not meet the third preset condition condition, the terminal device determines that the state corresponding to the first data is a non-epileptic seizure state.
The method according to claim 1, wherein the deep feature data is obtained by the terminal device using a third neural network model to extract deep features from the accelerometer data.
The method according to claim 2, wherein the third neural network model is obtained by training the terminal device based on accelerometer sample data, and the third neural network model includes an input module and a deep convolution module , a point convolution module and an output module, the depth convolution module includes a convolution calculation layer with a core of 3*3, the first normalization layer and the first stretching to the same latitude layer, the point convolution module Include a convolution calculation layer with a kernel of 1*1, a second normalization layer, and a second stretch to the same latitude layer.
The method according to claim 1, wherein the first motion amplitude feature data includes at least one of the following: acceleration intensity vector SMV, SMV maximum value, SMV minimum value, the SMV maximum value and the minimum Difference, FFT eigenvector, acceleration rate, SMV average, acceleration variance, x-axis acceleration mean, y-axis acceleration mean or z-axis acceleration mean.
The method according to claim 1, wherein the second motion amplitude characteristic data comprises at least one of the following: SMV average value, acceleration variance, average deviation, and the difference between the maximum accelerometer data and the minimum accelerometer data on the x-axis Value, the difference between the maximum accelerometer data and the minimum accelerometer data on the y-axis, or the difference between the maximum accelerometer data and the minimum accelerometer data on the z-axis.
The method according to claim 1, wherein the first neural network model is obtained by training based on the motion amplitude feature sample data corresponding to the accelerometer sample data and the depth feature sample data corresponding to the accelerometer data, The first neural network model is a four-layer fully connected neural network model, including an input layer, a first hidden layer, a second hidden layer and an output layer in the first neural network model; the nodes of the input layer contains the number of nodes corresponding to the first motion amplitude feature data and the number of nodes corresponding to the depth feature data.
The method according to claim 6, wherein the number of nodes in the input layer is 45, the number of nodes corresponding to the first motion amplitude feature data is 10, the number of nodes corresponding to the depth feature data is 35, The number of nodes in the output layer is 2.
The method according to claim 1, wherein the terminal device extracts the second motion amplitude characteristic data from the accelerometer data, comprising:

The terminal device performs filtering processing on the accelerometer data by means of mean filtering to obtain filtered data;

The terminal device determines whether the filtered data satisfies a first state, a second state and/or a third state; the first state is the difference between adjacent accelerometer data in the filtered data A state with a value of 0, the second state is a state in which the difference between adjacent accelerometer data in the filtered data satisfies the first difference range; the third state is the state after the filtered The difference between adjacent accelerometer data in the data satisfies the state of the second difference range;

When the terminal device determines that the filtered data does not satisfy the first state, the second state and/or the third state, the terminal device extracts from the filtered data The second motion amplitude feature data.
The method according to claim 1, wherein the terminal device extracts the first motion amplitude feature data and depth feature data from the accelerometer data, including:

The terminal device uses a filter to filter the accelerometer data to obtain filtered data;

The terminal device performs down-sampling processing on the filtered data to obtain the down-sampling data;

The terminal device extracts the first motion amplitude feature data and the depth feature data from the downsampled data.
The method according to claim 9, wherein the filter has a window length of L 1 and an amplitude of
filter, the filtered data Acc L (t) satisfies the following formula:

Wherein, the Acc(t) is the accelerometer data, and the i is an integer greater than or equal to 0.
The method according to claim 1, further comprising:

The terminal device displays a first interface; the first interface includes warning information; the warning information is used to indicate that the user is in the epileptic seizure state;

When the terminal device receives an operation on the alarm information, the terminal device displays a second interface; the second interface is an interface corresponding to the desktop of the terminal device.
The method according to claim 1, further comprising:

The terminal device sends the epileptic seizure state to other devices, and the other device is a device corresponding to an emergency contact during an epileptic seizure recorded by the terminal device.
The method according to claim 1, wherein the electrical signal data is a surface electromyography signal sEMG.
The method according to any one of claims 1-13, wherein the first data further includes temperature data and heart rate data, and when the fall detection result satisfies the first preset condition, the convulsion detection When the result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, including:

When the fall detection result satisfies the first preset condition, the twitch detection result satisfies the second preset condition, and/or the muscle stiffness detection result satisfies the third preset condition, and, When the heart rate data meets the fourth preset condition and/or the temperature data meets the fifth preset condition; wherein the terminal device further includes a temperature sensor and a proximity light sensor, the temperature data is collected by the temperature sensor, The heart rate data is collected by the proximity light sensor.
An epilepsy detection device, characterized in that the terminal device includes an acceleration sensor and an inductance sensor, and the device includes:

A processing unit, configured to acquire first data; the first data includes accelerometer data and electrical signal data; the accelerometer data is collected by the acceleration sensor, and the electrical signal data is collected by the inductance sensor;

The processing unit is further configured to extract first motion amplitude feature data and depth feature data from the accelerometer data; the first motion amplitude feature data is data used for fall detection;

The processing unit is further configured to extract second motion amplitude feature data from the accelerometer data; the second motion amplitude feature data is data used for convulsion detection;

The processing unit is further configured to input the first motion amplitude feature data and the depth feature data into a first neural network model to obtain a fall detection result;

The processing unit is further configured to input the second motion amplitude characteristic data into a second neural network model to obtain a twitch detection result; the first motion range is greater than the second motion range;

When the fall detection result satisfies a first preset condition, the twitch detection result satisfies a second preset condition, and/or the muscle stiffness detection result satisfies a third preset condition, the determining unit is configured to determine the first data The corresponding state is an epileptic seizure state; the muscle stiffness detection result is obtained based on the detection of the electrical signal data;

Or, when the fall detection result does not meet the first preset condition, the twitch detection result does not meet the second preset condition, and/or the muscle stiffness detection result does not meet the third preset condition condition, the determining unit is further configured to determine that the state corresponding to the first data is a non-epileptic state.
The device according to claim 15, wherein the depth feature data is obtained by the terminal device using a third neural network model to extract depth features from the accelerometer data.
The device according to claim 16, wherein the third neural network model is obtained by training the terminal device based on accelerometer sample data, and the third neural network model includes an input module and a deep convolution module , a point convolution module and an output module, the depth convolution module includes a convolution calculation layer with a core of 3*3, the first normalization layer and the first stretching to the same latitude layer, the point convolution module Include a convolution calculation layer with a kernel of 1*1, a second normalization layer, and a second stretch to the same latitude layer.
The device according to claim 15, wherein the first motion amplitude feature data includes at least one of the following: acceleration intensity vector SMV, SMV maximum value, SMV minimum value, the SMV maximum value and the minimum Difference, FFT eigenvector, acceleration rate, SMV average, acceleration variance, x-axis acceleration mean, y-axis acceleration mean or z-axis acceleration mean.
The device according to claim 15, wherein the second motion amplitude feature data includes at least one of the following: SMV average value, acceleration variance, average deviation, and the difference between the maximum accelerometer data and the minimum accelerometer data on the x-axis Value, the difference between the maximum accelerometer data and the minimum accelerometer data on the y-axis, or the difference between the maximum accelerometer data and the minimum accelerometer data on the z-axis.
The device according to claim 15, wherein the first neural network model is obtained by training based on the motion amplitude feature sample data corresponding to the accelerometer sample data and the depth feature sample data corresponding to the accelerometer data, The first neural network model is a four-layer fully connected neural network model, including an input layer, a first hidden layer, a second hidden layer and an output layer in the first neural network model; the nodes of the input layer contains the number of nodes corresponding to the first motion amplitude feature data and the number of nodes corresponding to the depth feature data.
The device according to claim 20, wherein the number of nodes in the input layer is 45, the number of nodes corresponding to the first motion amplitude feature data is 10, the number of nodes corresponding to the depth feature data is 35, The number of nodes in the output layer is 2.
The device according to claim 15, wherein the processing unit is specifically configured to filter the accelerometer data by mean filtering to obtain filtered data; the determining unit is specifically configured to determine Whether the filtered data satisfies the first state, the second state and/or the third state; the first state is a state in which the difference between adjacent accelerometer data in the filtered data is 0 , the second state is the state in which the difference between adjacent accelerometer data in the filtered data satisfies the first difference range; the third state is the state in which the adjacent accelerometer data in the filtered data The difference value of the accelerometer data satisfies the state of the second difference range; when the terminal device determines that the filtered data does not satisfy the first state, the second state and/or the third state , the processing unit is further specifically configured to extract the second motion amplitude feature data from the filtered data.
The device according to claim 15, wherein the processing unit is specifically configured to: use a filter to filter the accelerometer data to obtain filtered data; Perform downsampling processing to obtain downsampled data; extract the first motion amplitude feature data and the depth feature data from the downsampled data.
The device according to claim 23, wherein the filter has a window length of L 1 and an amplitude of
filter, the filtered data Acc L (t) satisfies the following formula:

Wherein, the Acc(t) is the accelerometer data, and the i is an integer greater than or equal to 0.
The device according to claim 15, wherein the display unit is configured to display a first interface; the first interface includes warning information; the warning information is used to indicate that the user is in the epileptic seizure state; when the When the terminal device receives an operation on the alarm information, the display unit is further configured to display a second interface; the second interface is an interface corresponding to the desktop of the terminal device.
The device according to claim 15, wherein the communication unit is configured to send the epileptic seizure state to other devices, and the other device is the device corresponding to the emergency contact during the epileptic seizure recorded by the terminal device .
The device according to claim 15, wherein the electrical signal data is a surface electromyography signal sEMG.
The device according to any one of claims 15-27, wherein the first data further includes temperature data and heart rate data, and when the fall detection result satisfies the first preset condition, the convulsion detection When the result meets the second preset condition and/or the muscle stiffness detection result meets the third preset condition, it includes: when the fall detection result meets the first preset condition, and the twitch detection result meets the second preset condition. Assume that the condition and/or the muscle stiffness detection result satisfies the third preset condition, and the heart rate data satisfies the fourth preset condition and/or the temperature data satisfies the fifth preset condition; wherein, the The terminal device further includes a temperature sensor and a proximity light sensor, the temperature data is collected by the temperature sensor, and the heart rate data is collected by the proximity light sensor.
An electronic device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein, when the processor executes the computer program, the electronic device Carrying out the method as described in any one of claims 1 to 14.
A computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein, when the computer program is executed by a processor, the computer executes the method according to any one of claims 1 to 14 .
A computer program product, characterized in that it includes a computer program, and when the computer program is run, causes the computer to execute the method according to any one of claims 1 to 14.