CN114081513B - Electromyographic signal-based abnormal driving behavior detection method and system - Google Patents
Electromyographic signal-based abnormal driving behavior detection method and system Download PDFInfo
- Publication number
- CN114081513B CN114081513B CN202111518134.9A CN202111518134A CN114081513B CN 114081513 B CN114081513 B CN 114081513B CN 202111518134 A CN202111518134 A CN 202111518134A CN 114081513 B CN114081513 B CN 114081513B
- Authority
- CN
- China
- Prior art keywords
- electromyographic
- abnormal driving
- data
- driving behavior
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/20—Workers
- A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention relates to an abnormal driving behavior detection method based on an electromyographic signal, which comprises the steps of obtaining the electromyographic data of the abnormal driving behavior with a label; preprocessing the electromyographic data of the abnormal driving behaviors; training a neural network model by utilizing the preprocessed characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model; acquiring real-time driving behavior electromyographic data in the driving process, and performing signal median correction, action interval extraction and feature extraction on the data; inputting the processed characteristic information of the electromyographic data of the real-time driving behaviors into a trained classification model, outputting a prediction classification result, judging whether abnormal driving behaviors occur or not, and sending an abnormal driving warning after the abnormal driving behaviors occur. The invention uses the low-cost electromyographic sensor to acquire the electromyographic data of the driving behavior of the driver, is not influenced by the environment and passengers, is convenient to use, and overcomes the problems of difficult deployment, inconvenient wearing, high cost and the like commonly existing in the prior art.
Description
Technical Field
The invention relates to the technical field of driving behavior detection, in particular to a method and a system for detecting abnormal driving behaviors based on electromyographic signals.
Background
Various solutions are currently proposed to detect abnormal driving behavior, including extracting eye information, motion information of the driver based on vision to detect fatigue driving and distracted driving; detecting a driver's action based on an acoustic method; detecting driver activity based on the smart watch; and detecting a driver driving state and the like based on the physiological information extracted by the customized device. However, these solutions have problems of being affected by the environment and passengers, being difficult to deploy, inconvenient to wear, and high in cost.
Among them, in the vision-based method, many researchers perform abnormal driving behavior detection using a camera due to the convenience of the camera. Fan and the like utilize a front camera of the smart phone to extract eye information of a driver and predict driving behaviors. Kashievnik et al propose a method for detecting abnormal driving behavior through a front camera and a built-in sensor of a smartphone. These vision-based methods rely on a fixed high-definition camera and adequate visibility. In addition, information on the eyes and mouth is not available when the driver wears sunglasses, speaks, or sings. In order to solve the problem of low visibility, some researchers have proposed a method based on near infrared spectroscopy. Dasgupta et al use infrared illumination to capture images in low visibility environments. They illuminate the face of the driver with a front facing camera while driving at night to detect the eye state of the driver and thus predict drowsy driving. However, this system requires an expensive infrared transmitter, which may lead to driver distraction because the human eye is more sensitive to this wavelength. In addition, the driver vital signs monitoring system proposed by novara et al extracts remote photoplethysmography (rPPG) features from near-infrared images. However, large movements may cause the rPPG signal to be corrupted and illumination variations in the external light may cause intensity variations in the near infrared.
In addition, in the audio-based method, xie et al analyzed the doppler profile of the acoustic signal of the fatigue driving behavior, detecting the fatigue driving behavior in real time. Xu et al uses acoustic signals for fine breathing monitoring in a driving environment. Further, the steering tracking system proposed by Xu et al tracks the movement locus of the hand using a sound signal, and then estimates the rotation angle of the steering wheel based on a geometric transformation. However, the audio-based identification system requires not only an audio transceiver supporting high frequencies but also high power transmission of audio, and the influence of the surrounding environment and passengers may cause signal attenuation and noise.
Furthermore, huang et al, in a hardware-based customization method, uses magnetic labels on the hands and heads to detect driver activity, which is inconvenient to use. Vibrations from the vehicle, road conditions, and environment may also cause the magnetic sensor readings to be noisy. Wang et al install a plurality of rfid tags in a vehicle to detect activity information of a driver, but the movement of passengers causes interference and noise to rf signals, and a plurality of passengers in the vehicle causes deterioration of driving detection performance. Saleh et al fuse the nine-axis sensor data in the smartphone to obtain the driving state of the vehicle and infer driving behavior. In fact, according to the research introduction of Chen et al, the accuracy and stability of the built-in sensor of the smart phone are different due to different and unstable road conditions of the smart phone. Seungeun et al detect driver activity based on LSTM using eight inertial measurement units worn about the limb. Xing et al utilize heart rate and steering wheel rotation characteristics collected by a smart watch to detect fatigue driving. However, they are inconvenient to use, difficult to deploy, and costly.
Therefore, the prior art method has the problems of influence by environment and passengers, difficult deployment, inconvenient wearing, high cost and the like.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the problems in the prior art, and to provide a method and a system for detecting abnormal driving behavior based on an electromyographic signal.
In order to solve the technical problem, the invention provides an abnormal driving behavior detection method based on an electromyographic signal, which comprises the following steps:
acquiring electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
filtering, down-sampling, data framing and feature extraction processing are carried out on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
training a neural network model by using the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model;
acquiring real-time driving behavior electromyographic data in a driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information into a trained classification model, and outputting a classification result of real-time detection;
and judging whether abnormal driving behaviors occur or not according to the prediction classification result, and sending an abnormal driving warning after the abnormal driving behaviors occur.
In one embodiment of the present invention, acquiring tagged abnormal driving behavior electromyographic data using an electromyographic sensor includes:
acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
and segmenting electromyographic data of abnormal driving behaviors by using a sliding window, wherein the abnormal driving behaviors comprise forward grabbing, emergency steering, stooping and picking up, turning to fetch objects and opening a skylight.
In one embodiment of the invention, the neural network model comprises:
the GRU layers are two in number, the first layer takes the feature information of the electromyographic data of the abnormal driving behaviors as input and outputs compressed abnormal driving behavior feature vectors, and the second layer takes the compressed abnormal driving behavior feature vectors as input and outputs fine-grained features;
and the full connection layer takes fine-grained characteristics as input and outputs a classification result through a softmax activation function.
In one embodiment of the invention, when a neural network model is trained by using the characteristic information of the electromyographic data of the abnormal driving behaviors, the GRU layer inputs X of the nth sample at the t time step t Mapping to a representation hidden state H t To obtain H t First, it is necessary to obtain a gate signal Z t And R t Is composed of
Where σ (-) represents a sigmod function, W z ,W r Representing a weight matrix, b z ,b r Offset vectors representing the update gate and the reset gate, respectively;
according to R t Obtaining candidate hidden state H t Is composed of
Wherein W o And b o Representing a weight matrix and a deviation vector;
based on the candidate hidden state H t And Z t Obtaining a hidden state H t Is composed of
According to the hidden state H t Obtaining a class probability vector P t ＝s(W t H t + b), where s (-) represents the softmax function, W t And b is the weight matrix and offset vector of the fully-connected layer.
In one embodiment of the invention, when the neural network model is trained by using the characteristic information of the electromyographic data of the abnormal driving behavior, a cross-like entropy error function is adopted to minimize the difference between the network prediction result and the sample basic fact.
In an embodiment of the present invention, performing signal median correction, action interval extraction, and feature extraction processing on the real-time driving behavior electromyographic data includes:
performing signal median correction processing on the real-time driving behavior electromyographic data, and adjusting signals to the same median;
extracting a time series action signal from the corrected data based on an energy threshold value to obtain an action signal in each interval;
and (4) performing feature extraction on the action signals in the interval to obtain feature information of real-time driving behavior electromyographic data.
In addition, the present invention also provides an abnormal driving behavior detection system based on an electromyographic signal, comprising:
the data acquisition module is used for acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
the preprocessing module is used for filtering, down-sampling, data framing and feature extraction processing on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
the network training module is used for training a neural network model by utilizing the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model;
the real-time detection module is used for acquiring real-time driving behavior electromyographic data in the driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information into a trained classification model, and outputting a classification result of real-time detection;
and the judging module is used for judging whether abnormal driving behaviors occur or not according to the prediction classification result and sending an abnormal driving warning after the abnormal driving behaviors occur.
In one embodiment of the present invention, the data acquisition module includes:
the data acquisition unit is used for acquiring the electromyographic data of the abnormal driving behaviors with labels by using the electromyographic sensor;
a sliding segmentation unit for segmenting electromyographic data of abnormal driving behaviors including forward grabbing, jostling a steering wheel, stooping for picking up, turning around to take a thing, and opening a skylight using a sliding window.
In one embodiment of the invention, the preprocessing module comprises:
the filtering unit is used for filtering the abnormal driving behavior electromyographic data to obtain filtered abnormal driving behavior electromyographic data;
the down-sampling unit is used for performing down-sampling processing on the filtered abnormal driving behavior electromyographic data;
the data framing unit is used for framing the abnormal driving behavior electromyographic data subjected to the down sampling;
and the characteristic extraction unit is used for extracting the characteristics of the data subjected to the framing processing to obtain the characteristic information of the electromyographic data of the abnormal driving behaviors.
In one embodiment of the present invention, the real-time detection module includes:
the signal median correction unit is used for performing signal median correction processing on the real-time driving behavior electromyographic data and adjusting signals to the same median;
an action interval extraction unit for extracting a time-series action signal from the corrected data based on an energy threshold to obtain an action signal in each interval;
the characteristic extraction unit is used for extracting the characteristics of the action signals in the interval to obtain the characteristic information of the real-time driving behavior electromyographic data;
and the prediction classification unit is used for reconstructing the characteristic information, inputting the reconstructed characteristic information into a trained classification model, and outputting a classification result of real-time detection.
Compared with the prior art, the technical scheme of the invention has the following advantages:
1. the invention uses the low-cost electromyographic sensor to collect the electromyographic data of the driving behavior of the driver, is not influenced by the environment and passengers, is convenient to use, and overcomes the problems of influence by the environment and the passengers, difficulty in deployment, inconvenience in wearing, high cost and the like commonly existing in the prior art;
2. in the off-line training stage, a series of processing such as filtering, down-sampling, data framing and feature extraction are provided, so that the data volume of an input model is greatly reduced while the electromyographic signals of different abnormal driving behaviors are effectively distinguished, and the speed of model training and real-time classification is accelerated;
3. in the real-time detection stage, the instability of real-time data is eliminated by utilizing median correction, the detection precision is increased while the calculated amount is remarkably reduced by utilizing action signal extraction, the detection speed of the method on the smart phone can reach below 45ms, and the performance requirement of real-time detection of the mobile intelligent terminal can be met.
Drawings
In order that the present disclosure may be more readily and clearly understood, reference will now be made in detail to the present disclosure, examples of which are illustrated in the accompanying drawings.
Fig. 1 is a flow chart diagram of an abnormal driving behavior detection method based on an electromyographic signal according to the present invention.
Fig. 2 is a schematic view of the wearing position of the electromyographic sensor of the present invention.
Fig. 3 is a schematic diagram of the network training of the present invention.
FIG. 4 is a graph of the overall performance evaluation of the classification network model of the present invention.
Fig. 5 is a diagram of the overall performance evaluation of various network architectures of the present invention.
FIG. 6 is a diagram of performance evaluation of various driving behavior detection under different tightness according to the present invention.
Fig. 7 is a diagram of performance evaluation of various driving behavior detections at different wearing positions according to the present invention.
FIG. 8 is a diagram of the reasoning speed of the model of the present invention on different models of handsets.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
Example one
Referring to fig. 1 to 8, the present embodiment provides a method for detecting abnormal driving behavior based on electromyographic signals, including the following steps:
s1: acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
s2: filtering, down-sampling, data framing and feature extraction processing are carried out on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
s3: training a neural network model by using the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model;
s4: acquiring real-time driving behavior electromyographic data in a driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information to a trained classification model, and outputting a classification result of real-time detection;
s5: and judging whether abnormal driving behaviors occur or not according to the prediction classification result, and sending an abnormal driving warning after the abnormal driving behaviors occur.
The invention uses the low-cost electromyographic sensor to acquire the electromyographic data of the driving behavior of the driver, is not influenced by the environment and passengers, is convenient to use, and overcomes the problems of influence by the environment and the passengers, difficulty in deployment, inconvenience in wearing, high cost and the like commonly existing in the prior art.
Wherein, in S1, the electromyographic data of the abnormal driving behavior with the label is acquired by using an electromyographic sensor, and the method comprises the following steps:
s1.1: acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
s1.2: and segmenting electromyographic data of abnormal driving behaviors by using a sliding window, wherein the abnormal driving behaviors comprise forward grabbing, emergency steering wheel hitting, stooping for picking up, turning to get things and opening a skylight.
Preferably, the electromyographic sensor can be worn at the position shown in fig. 2, real-time data of the sensor is transmitted to a computer through Bluetooth, and then electromyographic data containing abnormal driving behaviors is divided by a sliding window of 4.5 seconds and is stored in a classified mode, wherein the classified mode comprises forward grabbing (FF), jolting steering wheel (TSW), bending and Picking (PU), turning body and fetching (TB) and opening a skylight (TS).
In S2, filtering, down-sampling, data framing, and feature extraction processing are performed on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data, including:
s2.1: kalman filtering and band-pass filtering are carried out on the electromyographic data of the abnormal driving behaviors, the error variance p of the previous state estimation of Kalman filtering is set to be 0.01, the noise variance q is set to be 0.0000001, and the parameter r of a filter for adjusting the filtering effect is set to be 0.005. The band-pass range of the band-pass filter is set to be 0-500Hz;
s2.2: after filtering, down-sampling the data to reduce the data volume input by the neural network, and setting the data volume as 4 times of down-sampling according to the requirement;
s2.3: performing frame division processing on the abnormal driving behavior electromyographic data after the down sampling, setting the length of each frame to be 0.2 second, and if the length of each frame is not 0.2 second, performing zero padding, wherein each frame is overlapped by 50% in order to keep the time sequence relation between each frame of data;
s2.4: and carrying out 1024-point fast Fourier transform on the data subjected to the framing processing to obtain frequency domain information of the electromyographic data.
In S3, the neural network model adopts a gated recursive unit network (GRU), and a network structure of the neural network model includes two GRU layers and a full connection layer, where the first layer takes the feature information of the electromyographic data of the abnormal driving behavior as input and outputs a compressed abnormal driving behavior feature vector, and the second layer takes the compressed abnormal driving behavior feature vector as input and outputs fine-grained features; and the full connection layer takes fine-grained characteristics as input, and outputs a classification result through a softmax activation function. P (f), P(s), P (P), P (tb), P (ts), P (n) represent the probabilities of grabbing forward, jolting the steering wheel, bending over to pick up, turning around to get things, and opening the skylight and driving normally, respectively. In the time domain, each GRU unit accepts state information (hidden state) of the previous time step. It takes into account a number of current and previous time steps to infer whether abnormal driving behavior has occurred.
And when the neural network model is trained by utilizing the characteristic information of the electromyographic data of the abnormal driving behaviors, most of forward grabbing, steering wheel kicking, bending and picking up, turning and fetching and skylight opening can be completed within 3.3s, 2.8s, 4.0s, 4.2s and 3.5 s. Therefore, we set the sliding window to 4.5s to contain all the abnormal driving behavior. In a GRU network, each layer maps input features into a new compressed vector that is associated with unique driving behavior. At the t time step, the GRU layer can input X of the nth sample t Mapping to a representation hidden State H t The vector of (2). To obtain H t We first obtain the gating signal Z t And R t As follows:
Z t ＝σ(W z [X t ,H t-1 ]+b z ),
R t ＝σ(W r [X t ,H t-1 ]+b r ),
where σ (-) represents a sigmod function, W z ,W r Represents a weight matrix, b z ,b r Respectively representing offset vectors of an update gate and a reset gate, wherein the range of the gating signals is 0-1, and the more the gating signals are close to 1, the more the memory data are; the closer to 0, the more data is forgotten.
Then using R t Obtaining candidate hidden state H t ：
Wherein W o And b o Representing the weight matrix and the deviation vector.
Finally, with Z t And H t Obtain a hidden state H t The following:
according to the hidden state H t Obtaining a class probability vector P t ＝s(W t H t + b), where s (-) represents the softmax function, W t And b is the weight matrix and offset vector of the fully-connected layer. Given P t Class label with the highest probability of being considered as the t-th time step
To minimize the difference between the net prediction result and the sample ground truth, we use a cross-entropy-like error function E t Which is defined as:
E t ＝C t lnC t +(1-C t )ln(1-C t ),
wherein, C t Representing the output of the network, N is the number of samples in a batch. The GRU network is set to a many-to-one configuration, since only one output is required as a prediction at a time. Further, it takes a sufficiently long time to detect abnormal driving. As described above, the longest average duration of the five abnormal driving behaviors is 4.2s, and the frame length we choose is 0.2s, so the network has 21 time steps. In order to prevent the model from being over-fitted, L2 regularization needs to be added between each GRU layer to limit the weight, so that the verification accuracy of small sample learning can be improved. For initialization, the learning rate, the size of hidden units in the GRU, the regularization strength, and the discarding rate are set to 0.0001, 128, 0.01, and 0.2, respectively, and the proportion of training samples and test samples is set to 7.
In S4, performing signal median correction, action interval extraction, and feature extraction on the real-time driving behavior electromyography data, including:
s4.1: the real-time driving behavior electromyographic data is subjected to signal median correction processing, signals are adjusted to the same median value, the amplitude of a raw electromyographic signal acquired by an electromyographic sensor may fluctuate around 2000, which affects the setting of an energy threshold Q, so that the median value of the raw signal is dynamically monitored in real-time monitoring, and then the signals are adjusted to the same median value. The method specifically comprises the steps of caching myoelectric data within a period of time (for example, within 5 seconds), taking a median value of the cached data, and then subtracting the median value from the real-time data.
S4.2: and (3) extracting the action interval of the corrected data, wherein in the real-time monitoring process, if each frame is input into the model, the calculation complexity is quite high, and the data is not suitable for being deployed in mobile equipment. Therefore, a real-time action interval extraction algorithm is designed, because the power of the electromyographic signals is increased when actions occur, the action signals of a time sequence can be extracted based on an energy threshold value, whether actions exist in a long sliding window delta T or not is firstly detected, and the energy of the period is calculated as
Wherein X i Represents the ith sample point, T 0 Representing the start time. Then comparing I with a threshold Q, if I is greater than Q, an action is considered to occur, and then fine-grained detecting whether the next signal is part of the action using a small sliding window Δ t, the signal energy of the ith small window being set as I i ', if I i ′＞Q,I ( ′ i+1) ＞Q,...,I ( ′ i+k) > Q, the signal will be stored in the action buffer if successive I' s i If 'less than Q', the continuous action is considered to be finished, and finally, the action signal is extracted into the action buffer.
S4.3: and carrying out 1024-point fast Fourier transform on the action signals in the interval to obtain frequency domain characteristic information of the real-time driving behavior electromyographic data.
In the off-line training stage, the invention provides a series of processing such as filtering, down-sampling, data framing and feature extraction, and the like, thereby effectively distinguishing the electromyographic signals of different abnormal driving behaviors, greatly reducing the data volume of the input model and accelerating the speed of model training and real-time classification.
In the real-time detection stage, the instability of real-time data is eliminated by utilizing median correction, the detection precision is increased while the calculated amount is remarkably reduced by utilizing action signal extraction, the detection speed of the method on the smart phone can reach below 45ms, and the performance requirement of real-time detection of the mobile intelligent terminal can be met.
Example two
In the following, a system for detecting abnormal driving behavior based on an electromyographic signal, which is disclosed in the second embodiment of the present invention, is introduced, and a system for detecting abnormal driving behavior based on an electromyographic signal, which is described below, and a method for detecting abnormal driving behavior based on an electromyographic signal, which is described above, may be referred to in correspondence with each other.
The embodiment of the invention discloses an abnormal driving behavior detection system based on electromyographic signals, which comprises:
the data acquisition module is used for acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
the preprocessing module is used for filtering, down-sampling, data framing and feature extraction processing on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
the network training module is used for training a neural network model by utilizing the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model;
the real-time detection module is used for acquiring real-time driving behavior electromyographic data in the driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information to a trained classification model, and outputting a classification result of real-time detection;
and the judging module is used for judging whether abnormal driving behaviors occur or not according to the prediction classification result and sending an abnormal driving warning after the abnormal driving behaviors occur.
Wherein the data acquisition module comprises:
the data acquisition unit is used for acquiring the electromyographic data of the abnormal driving behaviors with labels by using the electromyographic sensor;
a sliding segmentation unit for segmenting electromyographic data of abnormal driving behaviors including forward grabbing, jostling a steering wheel, stooping for picking up, turning around to take a thing, and opening a skylight using a sliding window.
Wherein the preprocessing module comprises:
the filtering unit is used for filtering the abnormal driving behavior electromyographic data to obtain filtered abnormal driving behavior electromyographic data;
the down-sampling unit is used for carrying out down-sampling processing on the filtered abnormal driving behavior electromyographic data;
the data framing unit is used for framing the abnormal driving behavior electromyographic data subjected to the down sampling;
and the characteristic extraction unit is used for extracting the characteristics of the data subjected to the framing processing to obtain the characteristic information of the electromyographic data of the abnormal driving behaviors.
Wherein, the real-time detection module includes:
the signal median correction unit is used for performing signal median correction processing on the real-time driving behavior electromyographic data and adjusting signals to the same median;
an action interval extraction unit for extracting a time-series action signal from the corrected data based on an energy threshold to obtain an action signal in each interval;
the characteristic extraction unit is used for extracting the characteristics of the action signals in the interval to obtain the characteristic information of the real-time driving behavior electromyographic data;
and the prediction classification unit is used for inputting the reconstructed characteristic information into a trained classification model and outputting a classification result of real-time detection.
The electromyogram signal-based abnormal driving behavior detection system of the present embodiment is used to implement the aforementioned electromyogram signal-based abnormal driving behavior detection method, and therefore, a specific implementation of the system can be found in the previous embodiment section of the electromyogram signal-based abnormal driving behavior detection method, and therefore, the specific implementation of the system can refer to the description of the corresponding partial embodiment and will not be further described herein.
In addition, since the system for detecting abnormal driving behavior based on electromyographic signals of the present embodiment is used for implementing the method for detecting abnormal driving behavior based on electromyographic signals, the function corresponds to the function of the method, and the description thereof is omitted here.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications of the invention may be made without departing from the spirit or scope of the invention.
Claims (7)
1. An abnormal driving behavior detection method based on an electromyographic signal is characterized by comprising the following steps:
acquiring electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor, wherein the abnormal driving behaviors comprise forward grabbing, emergency steering, stooping and picking up, turning to fetch objects and opening a skylight;
filtering, down-sampling, data framing and feature extraction processing are carried out on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
training a neural network model by using the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model, wherein the neural network model adopts a gated recursive unit network GRU, and the network structure of the neural network GRU comprises two GRU layers and a full connection layer, the first layer takes the characteristic information of the electromyographic data of the abnormal driving behaviors as input and outputs a compressed characteristic vector of the abnormal driving behaviors, and the second layer takes the compressed characteristic vector of the abnormal driving behaviors as input and outputs fine-grained characteristics; the full connection layer takes fine-grained characteristics as input, and outputs a classification result through a softmax activation function;
acquiring real-time driving behavior electromyographic data in a driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information into a trained classification model, and outputting a classification result of real-time detection;
judging whether abnormal driving behaviors occur or not according to the classification result of the real-time detection, and sending an abnormal driving warning after the abnormal driving behaviors occur;
the real-time driving behavior myoelectricity data is subjected to signal median correction, action interval extraction and feature extraction processing, and the method comprises the following steps of:
performing signal median correction processing on the real-time driving behavior electromyographic data, and adjusting signals to the same median;
extracting action interval of the corrected data, extracting time-series action signal based on energy threshold, detecting whether there is action in long sliding window delta T, calculating the energy of the period as
Wherein x i Represents the ith sample point, T 0 Representing a start time, then comparing I with a threshold Q, assuming an action to occur if I is greater than Q, then fine-grained detecting whether the next signal is part of the action using a small sliding window Δ t, the signal energy of the g-th sub-window being set to I' g If l' g ＞Q,I′ (g+1) ＞Q,...,I′ (g+k) > Q, the signal will be stored in the action buffer if I' g ＜Q,I′ (g+1) ＜Q,...,I′ (g+k) If the value is less than Q, the continuous action is considered to be finished, and finally, the action signal is extracted into an action cache;
and carrying out 1024-point fast Fourier transform on the action signals in the interval to obtain frequency domain characteristic information of the real-time driving behavior electromyographic data.
2. The electromyographic signal based abnormal driving behavior detection method according to claim 1, wherein acquiring tagged abnormal driving behavior electromyographic data comprises:
acquiring the electromyographic data of the abnormal driving behaviors with labels by using an electromyographic sensor;
and segmenting the electromyographic data of the abnormal driving behaviors by using a sliding window.
3. The electromyographic signal based abnormal driving behavior detection method according to claim 1, characterized in that: when the characteristic information of the electromyographic data of the abnormal driving behaviors is utilized to model a neural networkWhen training the model, at the t time step, the GRU layer inputs X of the nth sample t Mapping to a representation hidden state H t To obtain H t First, it is necessary to obtain a gate signal Z t And R t Is composed of
Where σ (-) represents a sigmod function, W z ,W r Representing a weight matrix, b z ,b r Offset vectors representing the update gate and the reset gate, respectively; />
According to R t Obtaining candidate hidden states
Is->
Wherein W o And b o Representing a weight matrix and a deviation vector;
based on the candidate hidden state
And Z t Obtaining a hidden state H t Is->
According to the hidden state H t Obtaining a class probability vector P t ＝s(W t H t + b), where s (-) represents the softmax function, W t And b is the weight matrix and offset vector of the fully-connected layer.
4. The electromyographic signal based abnormal driving behavior detection method according to claim 1, characterized in that: when the characteristic information of the electromyographic data of the abnormal driving behaviors is used for training a neural network model, a cross-entropy-like error function is adopted to minimize the difference between the network prediction result and the basic fact of the sample.
5. An abnormal driving behavior detection system based on an electromyographic signal, comprising:
the data acquisition module is used for acquiring myoelectric data of the abnormal driving behaviors with labels by using a myoelectric sensor, wherein the abnormal driving behaviors comprise forward grabbing, emergency steering, stooping picking, turning to get objects and opening a skylight;
the preprocessing module is used for filtering, down-sampling, data framing and feature extraction processing on the abnormal driving behavior electromyographic data to obtain feature information of the abnormal driving behavior electromyographic data;
the network training module is used for training a neural network model by utilizing the characteristic information of the electromyographic data of the abnormal driving behaviors to obtain a classification model, the neural network model adopts a gated recursion unit network GRU, and the network structure of the neural network model comprises two GRU layers and a full connection layer, wherein the first layer takes the characteristic information of the electromyographic data of the abnormal driving behaviors as input and outputs a compressed characteristic vector of the abnormal driving behaviors, and the second layer takes the compressed characteristic vector of the abnormal driving behaviors as input and outputs fine-grained characteristics; the full connection layer takes fine-grained characteristics as input, and outputs a classification result through a softmax activation function;
the real-time detection module is used for acquiring real-time driving behavior electromyographic data in the driving process, performing signal median correction, action interval extraction and feature extraction processing on the real-time driving behavior electromyographic data to obtain feature information of the real-time driving behavior electromyographic data, reconstructing the feature information, inputting the reconstructed feature information into a trained classification model, and outputting a classification result of real-time detection;
the judgment module is used for judging whether abnormal driving behaviors occur or not according to the classification result of the real-time detection and sending an abnormal driving warning after the abnormal driving behaviors occur;
the real-time detection module comprises:
the signal median correction unit is used for performing signal median correction processing on the real-time driving behavior electromyographic data and adjusting signals to the same median;
an operation interval extraction unit for extracting an operation interval from the data after the correction processing, and first detecting whether there is an operation in the long sliding window Δ T based on an energy threshold value extraction time-series operation signal, the energy of the cycle being calculated as
Wherein x i Represents the ith sample point, T 0 Representing a start time, then comparing I to a threshold Q, assuming an action is to occur if I is greater than Q, then fine-grained detecting whether the next signal is part of the action using a small sliding window Δ t, the signal energy of the g-th sub-bin being set to I' g If l' g ＞Q,I′ (g+1) ＞Q,...,I′ (g+k) > Q, the signal will be stored in the action buffer if I' g ＜Q,I′ (g+1) ＜Q,...,I′ (g+k) If the value is less than Q, the continuous action is considered to be finished, and finally, the action signal is extracted into an action cache;
and the characteristic extraction unit is used for carrying out 1024-point fast Fourier transform on the action signals in the interval to obtain frequency domain characteristic information of the real-time driving behavior electromyographic data.
6. The system according to claim 5, wherein the data acquisition module includes:
the data acquisition unit is used for acquiring the electromyographic data of the abnormal driving behaviors with labels by using the electromyographic sensor;
a sliding segmentation unit for segmenting the abnormal driving behavior electromyography data using a sliding window.
7. The electromyographic signal based abnormal driving behavior detection system of claim 5, wherein the preprocessing module comprises:
the filtering unit is used for filtering the abnormal driving behavior electromyographic data to obtain the filtered abnormal driving behavior electromyographic data;
the down-sampling unit is used for performing down-sampling processing on the filtered abnormal driving behavior electromyographic data;
the data framing unit is used for framing the abnormal driving behavior electromyographic data subjected to the down sampling;
and the characteristic extraction unit is used for extracting the characteristics of the data subjected to the framing processing to obtain the characteristic information of the electromyographic data of the abnormal driving behaviors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111518134.9A CN114081513B (en) | 2021-12-13 | 2021-12-13 | Electromyographic signal-based abnormal driving behavior detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111518134.9A CN114081513B (en) | 2021-12-13 | 2021-12-13 | Electromyographic signal-based abnormal driving behavior detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114081513A CN114081513A (en) | 2022-02-25 |
| CN114081513B true CN114081513B (en) | 2023-04-07 |
Family
ID=80307195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111518134.9A Active CN114081513B (en) | 2021-12-13 | 2021-12-13 | Electromyographic signal-based abnormal driving behavior detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114081513B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116269450B (en) * | 2023-03-21 | 2023-12-19 | 苏州海臻医疗器械有限公司 | Patient limb rehabilitation state evaluation system and method based on electromyographic signals |
| CN116211310A (en) * | 2023-05-09 | 2023-06-06 | 中国第一汽车股份有限公司 | Myoelectric sensor and detection method thereof |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170036428A (en) * | 2015-09-24 | 2017-04-03 | 삼성전자주식회사 | Driver monitoring method and driver monitoring apparatus using wearable device |
| CN109765823A (en) * | 2019-01-21 | 2019-05-17 | 吉林大学 | Ground crawler-type unmanned vehicle control method based on arm electromyography signal |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106073776B (en) * | 2016-08-29 | 2019-06-25 | 吉林大学 | CACC driver's limbs constant speed multichannel EMG Feature Extraction |
| CN106326873B (en) * | 2016-08-29 | 2019-04-16 | 吉林大学 | The manipulation Intention Anticipation method of CACC driver's limbs electromyography signal characterization |
| JP2018043580A (en) * | 2016-09-13 | 2018-03-22 | 株式会社東芝 | Information processing device for mobile body, information processing method for the same and information processing program for the same |
| CN106781283B (en) * | 2016-12-29 | 2019-04-05 | 东北大学秦皇岛分校 | A kind of method for detecting fatigue driving based on soft set |
| CN110399846A (en) * | 2019-07-03 | 2019-11-01 | 北京航空航天大学 | A kind of gesture identification method based on multichannel electromyography signal correlation |
| CN111184512B (en) * | 2019-12-30 | 2021-06-01 | 电子科技大学 | Method for recognizing rehabilitation training actions of upper limbs and hands of stroke patient |
| CN111976733B (en) * | 2020-08-27 | 2021-11-12 | 清华大学 | Method and system for continuously predicting steering intention of driver |
-
2021
- 2021-12-13 CN CN202111518134.9A patent/CN114081513B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170036428A (en) * | 2015-09-24 | 2017-04-03 | 삼성전자주식회사 | Driver monitoring method and driver monitoring apparatus using wearable device |
| CN109765823A (en) * | 2019-01-21 | 2019-05-17 | 吉林大学 | Ground crawler-type unmanned vehicle control method based on arm electromyography signal |
Non-Patent Citations (1)
| Title |
|---|
| 成娟 ; 陈香 ; 路知远 ; 张旭 ; 赵章琰 ; .基于表面肌电信号的手指按键动作识别研究.生物医学工程学杂志.2011,(02),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114081513A (en) | 2022-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114081513B (en) | Electromyographic signal-based abnormal driving behavior detection method and system | |
| US20220125322A1 (en) | Methods and Systems for Determining Abnormal Cardiac Activity | |
| Espinosa et al. | A vision-based approach for fall detection using multiple cameras and convolutional neural networks: A case study using the UP-Fall detection dataset | |
| Reddy et al. | Real-time driver drowsiness detection for embedded system using model compression of deep neural networks | |
| CN107580471B (en) | Wearable pulse sensing device signal quality estimation | |
| CN110944577B (en) | Method and system for detecting blood oxygen saturation | |
| Doukas et al. | Advanced patient or elder fall detection based on movement and sound data | |
| CN111166357A (en) | Fatigue monitoring device system with multi-sensor fusion and monitoring method thereof | |
| CN202257856U (en) | Driver fatigue-driving monitoring device | |
| KR20200011818A (en) | Method and device for estimating physical state of a user | |
| CN110728241A (en) | Driver fatigue detection method based on deep learning multi-feature fusion | |
| EP2967446A2 (en) | Determining activities from sensed motion signals | |
| CN111179552A (en) | Driver state monitoring method and system based on multi-sensor fusion | |
| Fan et al. | Safedriving: An effective abnormal driving behavior detection system based on emg signals | |
| Ma et al. | Real time drowsiness detection based on lateral distance using wavelet transform and neural network | |
| Martínez-Villaseñor et al. | Deep learning for multimodal fall detection | |
| CN114895301A (en) | Millimeter wave radar and video-assisted indoor fall detection method and device | |
| Zhang et al. | Sensors-based human activity recognition with convolutional neural network and attention mechanism | |
| CN112017155A (en) | Method, device and system for measuring health sign data and storage medium | |
| US8396825B2 (en) | Method and system to recognize temporal events using enhanced temporal decision trees | |
| CN107067649A (en) | A kind of typical behaviour real-time identification method based on wireless wearable aware platform | |
| CN115054218A (en) | Blood pressure measuring method and equipment | |
| KR20230139666A (en) | Devices and Methods for Recognizing Drowsy Driving based on Classification of Drowsy Driving Types and Ensemble Deep Learning | |
| Dong et al. | A Hybrid HMM/Kalman filter for tracking hip angle in gait cycle | |
| KR102596957B1 (en) | Multi sensor fusion-based driver monitoring device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |