CN117442159A

CN117442159A - Multi-mode physiological signal-based early Parkinson screening method and device

Info

Publication number: CN117442159A
Application number: CN202311310817.4A
Authority: CN
Inventors: 伍楷舜; 黄佳玲; 陈林; 黄彦道
Original assignee: Hong Kong University Of Science And Technology Guangzhou
Current assignee: Hong Kong University Of Science And Technology Guangzhou
Priority date: 2023-10-10
Filing date: 2023-10-10
Publication date: 2024-01-26

Abstract

The invention discloses a method and a device for early Parkinson screening based on multi-mode physiological signals. The method comprises the following steps: acquiring a respiration signal and an electrocardiogram signal of a target; and inputting the respiratory signal and the electrocardiogram signal into a trained deep learning model to obtain a parkinsonism screening result. The deep learning model comprises a first time feature extraction module, a second time feature extraction module, a first global feature extraction module, a second global feature extraction module and a classification module, wherein the first time feature extraction module extracts different scale features of respiratory signals, the second time feature extraction module extracts different scale features of electrocardiographic signals and performs feature fusion, the first global feature extraction module is used for extracting global features of respiratory signals, the second global feature extraction module is used for extracting global features of electrocardiographic signals, and the classification module is used for outputting Parkinson screening results. The invention improves the reliability and the robustness of the parkinsonism detection.

Description

Multi-mode physiological signal-based early Parkinson screening method and device

Technical Field

The invention relates to the technical field of medical physiological signal analysis, in particular to a method and a device for early Parkinson screening based on multi-mode physiological signals.

Background

Parkinson's Disease (PD) is a neurodegenerative disease. Characteristic manifestations of parkinson's disease include neuronal loss of specific areas of substantia nigra and extensive intracellular protein accumulation. Parkinson's disease is caused by progressive degeneration and death of dopamine-producing neurons in the substantia nigra of the brain, but when used in combination with these two major neuropathologies, is specific for the definitive diagnosis of idiopathic parkinson's disease.

Numerous other cell types of the entire central and peripheral autonomic nervous system have also been implicated from the early stages of the disease. For example, neurons of the substantia nigra part of the brain are lost. These neurons are responsible for the production of a chemical called dopamine. Dopamine helps transfer information from the substantia nigra to other parts of the body, thereby controlling its movement. This degenerative process results in reduced dopamine levels, which in turn may lead to motor symptoms commonly associated with PD, including tremors, rigidity, bradykinesia, and the like.

In addition to motor symptoms, most parkinsonian patients also have non-motor symptoms. Non-motor symptoms are involved in a variety of functions including sleep-wake cycle regulation disorders, cognitive impairment (including frontal lobe executive dysfunction, memory extraction defects, dementia and hallucinations), mood and affective disorders, autonomic dysfunction, and sensory symptoms and pain. Some of these symptoms may occur years or decades before the onset of classical motor symptoms. Non-motor symptoms are becoming more common in the course of the disease and are a major determinant of quality of life and overall disability progression.

In the prior art, there are a number of methods for diagnosing parkinson's disease. On motor symptoms, clinicians may evaluate the progression of parkinson's disease using the unified parkinsonism rating scale (MDS-UPDRS) of the dyskinesia association, analyzing the differences in gait of PD patients versus healthy persons to evaluate the severity and progression of the disease. However, motor symptom-based diagnosis may not be able to discover parkinson's disease early and is time consuming. Moreover, motor symptoms associated with parkinson's disease often begin to manifest after years of onset, and thus cannot be used for early screening. Furthermore, since the assessment of motor symptoms is subjective, it is necessary to see a doctor or expert multiple times before a definitive diagnosis is obtained.

In addition to a direct medical history, imaging techniques can diagnose parkinson's disease by providing objective markers of neurodegeneration. Neuroimaging techniques such as Magnetic Resonance Imaging (MRI), functional magnetic resonance imaging (fMRI), single Photon Emission Computed Tomography (SPECT), and transcranial B-mode ultrasound examination (TCS). DaTscan is an imaging technique for diagnosing PD by measuring dopamine transporter levels in the brain. Biochemical markers, including cerebrospinal fluid and alpha-synuclein, are also used for diagnosis of PD due to various biochemical changes associated with PD.

The prior art, as analyzed, had the following drawbacks:

1) Traditional methods of parkinson detection rely on subjective judgment by the clinician, are subjective and unstable, and require significant time and cost.

2) Existing automated parkinson's disease detection methods typically use only a single biological signal, such as an ECG (electrocardiogram) or EEG (electroencephalogram) signal, which alone does not provide a sufficient amount of information.

3) Existing automated parkinson's disease detection methods typically use a single scale or feature extraction method, so that key feature information cannot be captured.

4) The existing automatic parkinsonism detection method generally uses shallow machine learning models, and the models cannot process a large amount of complex biological signal data, so that the accuracy and the robustness of detection are limited.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a method and a device for early Parkinson screening based on multi-mode physiological signals.

According to a first aspect of the present invention there is provided a method of early parkinson's screening based on multimodal physiological signals. The method comprises the following steps:

acquiring a respiration signal and an electrocardiogram signal of a target;

inputting the respiratory signal and the electrocardiogram signal into a trained deep learning model to obtain a parkinsonism screening result;

the deep learning model comprises a first time feature extraction module, a second time feature extraction module, a first global feature extraction module, a second global feature extraction module and a classification module, wherein the first time feature extraction module is used for extracting different scale features of respiratory signals, the second time feature extraction module is used for extracting different scale features of electrocardiogram signals and fusing the different features, the first global feature extraction module is used for extracting global features of the respiratory signals by using a self-attention mechanism, the second global feature extraction module is used for extracting global features of the electrocardiogram signals by using the self-attention mechanism, and the classification module is used for outputting Parkinson screening results by combining the features of the respiratory signals and the electrocardiogram signals.

According to a second aspect of the present invention there is provided an early parkinson's screening device based on a multimodal physiological signal. The device comprises:

a signal acquisition unit: for acquiring a respiration signal and an electrocardiogram signal of the subject;

prediction unit: the method comprises the steps of inputting the respiratory signal and the electrocardiogram signal into a trained deep learning model to obtain a parkinsonism screening result;

Compared with the prior art, the method for screening the parkinsonism early stage based on the multi-mode physiological signals has the advantages that the method for screening the parkinsonism early stage based on the multi-mode multi-scale deep learning is based on the multi-mode multi-scale deep learning, the ECG and the respiratory signals are combined, automatic detection of the parkinsonism is achieved, and the accuracy and the robustness of detection are improved.

Other features of the present invention and its advantages will become apparent from the following detailed description of exemplary embodiments of the invention, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a flow chart of a method of early Parkinson screening based on a multi-modal physiological signal in accordance with one embodiment of the present invention;

FIG. 2 is a process schematic diagram of a multi-modal, multi-scale time-global fusion network according to one embodiment of the invention;

FIG. 3 is a particular network architecture diagram of a respiratory signal temporal feature extraction module according to one embodiment of the invention;

FIG. 4 is a schematic diagram of an SRU-based deep circulation unit according to one embodiment of the present invention;

FIG. 5 is a particular network architecture of an electrocardiographic signal temporal feature extraction module according to one embodiment of the invention;

FIG. 6 is a structural comparison of a bottleneck layer, an inverted bottleneck layer, and a nonlinear feature fusion module according to one embodiment of the invention;

FIG. 7 is a network block diagram of a global feature extraction module according to one embodiment of the invention;

FIG. 8 is a network architecture diagram of a Parkinson's disease classification module according to one embodiment of the invention;

FIG. 9 is a specific network architecture diagram of dual assistance tasks according to one embodiment of the invention;

in the drawings, an ECG Encoder-electrocardiogram signal Encoder; breathing Encoder-respiratory signal encoder; QEEG predictor-quantitative electroencephalogram predictor; breathing Signal-respiratory Signal; similarity Matching-similarity matching; block-Block; conv Block-convolution Block; attention Weights-attention weight.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.

The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.

Referring to fig. 1, the provided method for early parkinson's screening based on multimodal physiological signals comprises the following steps:

step S110, constructing a multi-mode multi-scale deep learning model.

Parkinson's disease is a chronic neurodegenerative brain disease that affects a person's ability to perform daily activities. In view of the potential for electrocardiographic changes prior to parkinson's disease, respiratory symptoms also often occur years earlier than motor symptoms, and thus respiratory and electrocardiographic signals may be used as biomarkers for the detection of parkinson's disease. Therefore, the invention provides a multi-mode multi-scale deep learning-based early detection method for parkinsonism.

Referring to fig. 2, the provided deep learning model mainly includes a multi-modal feature extraction module and a parkinson classification module, and forms a multi-modal backbone network. The multi-modal feature extraction module comprises a time feature extraction module and a global feature extraction module of two physiological signals, namely an electrocardiogram time feature extraction module (or a called electrocardio signal feature extraction module), a respiratory signal feature extraction module, a respiratory signal global feature extraction module and an electrocardiogram global feature extraction module. In one embodiment, the respiratory signal temporal feature extraction module uses a residual network to downsample extract different features of the respiratory signal. The electrocardiogram time feature extraction module performs parallel extraction on different features of an electrocardiogram signal, and utilizes a nonlinear feature fusion module to effectively fuse the different features. In addition, to address sparse supervision of parkinsonism, deep learning models have introduced a dual auxiliary task, namely to use both signals to predict quantitative electroencephalograms during sleep of a subject, respectively, to assist the task in providing additional labels, together with regularizing the model during training.

Specifically, for the deep learning model architecture of fig. 2, first, a night respiration signal and a night electrocardiogram signal of a subject at night are taken as inputs, and the night respiration signal and the night electrocardiogram signal are subjected to feature extraction in time and global, and finally classified. The respiratory signal time feature extraction module extracts features through a residual information structure, the electrocardiogram time feature extraction module carries out convolution parallel extraction on different features of an electrocardiogram signal, and a nonlinear feature fusion module is utilized to carry out nonlinear effective fusion on different features of the electrocardiogram signal. And the latter two extract global features of the respective signals by means of a self-attention mechanism. In addition, to address sparse supervision of parkinsonism, an auxiliary task was introduced in the feature extraction process of the two signals separately, i.e., quantitative electroencephalogram (qEEG) of the subject during sleep was predicted with the two signals separately to assist the task in providing additional labels, together with regularizing the model during training.

Hereinafter, embodiments of the model data processing, the respiratory signal temporal feature extraction module, the electrocardiogram temporal feature extraction module, the respiratory signal global feature extraction module, the electrocardiogram global feature extraction module, the parkinson's disease classification module, and the dual auxiliary tasks will be described in detail, respectively.

(1) Model data processing

In one embodiment, the data is processed to filter out nights shorter than 2 hours. Furthermore, the night when the respiratory signal and the electrocardiogram signal are distorted or not present is filtered out. In data processing, the electrocardiogram is downsampled from 512Hz to 128Hz, and signals other than 0.5 to 10Hz are filtered out by bandpass filtering to reduce noise, and the zscore regularization is also performed on the data because if the variance of some features is too large, the objective function will be dominated, so that the parameter estimator cannot learn other features correctly. The respiratory signal is downsampled to 10Hz, and zscore regularization is performed to eliminate inconvenience brought by data analysis, so that the influence of the data magnitude is avoided, and the characteristic that information is rich in the subsequent characteristic extraction can be ensured.

Table 1 is a representation of symbols commonly used in the implementation.

Table 1: common symbology for M2TGNet implementation

(2) Respiratory signal time feature extraction module

In one embodiment, referring to FIG. 3, the respiratory signal temporal feature extraction module first uses a 3x1 convolution kernel to perform channel addition on the respiratory signal, providing conditions for extracting more features. The inputs to the module are then normalized using the BN () function and the linear rectification function RELU (). And then eight layers of conventional residual blocks are used for carrying out downsampling extraction on different characteristics of the respiratory signals. For example, each block of conventional residual blocks is represented as follows:

M ^(out) ＝BN(Conv ^(k) (Relu(BN(Conv ^(k) (M ⁽ⁱⁿ⁾ ))))) (1)

wherein M is ^(out) Is a characteristic signal of a conventional residual block, M ⁽ⁱⁿ⁾ Is the input of a conventional residual block, k represents the convolution kernel. The BN function is used to activate functions RELU () and the like, so that the convergence rate of the model can be accelerated, the risk of overfitting is reduced, and the generalization capability of the model is improved.

The respiratory signal time feature extraction module extracts respiratory signal time features of different scales by continuously increasing the number of channels of the conventional residual blocks, so that the nonlinear fitting capacity is enhanced, and in addition, in order to solve gradient disappearance, a layer of simple circulation units (SRU) are respectively arranged behind the final three layers of conventional residual blocks. For example, referring to fig. 4, the SRU calculation process is expressed as:

wherein f _t To forget the door z _t To reset the gate c _t For an internal memory cell, the final output state is h _t 。To input a sequence, W _f 、W _z And W is the parameter matrix in the SRU, b _f And b _z Is the offset cell vector.

The SRU has the same parallelism as the convolution and feed forward networks. SRU replaces the use of convolution. As with QRNN and KNN, there are more cyclic connections. This allows the respiratory signal temporal feature extraction module to retain good modeling capabilities, to extract abnormal time and respiratory conditions in the signal, and also allows for faster training speeds.

(3) Electrocardiogram time feature extraction module

The amount of information acquired during night sleep is greater due to the higher sampling rate of the electrocardiogram signal. The electrocardiographic temporal feature extraction module therefore employs a different model design than the respiratory signal temporal feature extraction module. The electrocardiographic time feature extraction module also employs "deep convolution" because this effectively reduces the number of hops in the network.

Referring to fig. 5, in one embodiment, the number of blocks per depth is adjusted (3,3,9, 18) for the electrocardiogram temporal feature extraction module so that the model learns shallow temporal features in the early stages and deepens in the later stages. During this time, each time a convolution of depth is completed, it is downsampled to the appropriate feature map size. The characteristic signal is reduced to 1/16 of the original length. And each depth will input a size of C x l _E The number of channels is increased. Thereby increasing the complexity and the expression capacity of the electrocardiographic time feature extraction module, and enabling the module to more accurately recognize the input features. In each block, the electrocardiogram signals are firstly processed through three convolutions accompanied with BN operation in parallel to obtain a characteristic map of a plurality of receptive fields, the convolution kernels of the three branches are respectively 3x1,5x1 and 11x1, each branch can process the electrocardiogram signals with the same size into characteristic signals with the same size, and the characteristic signals are extracted to a plurality of time scales, so that the generalization capability of the module is improved.

Wherein,is represented as having C _in Input channel and C _out The k 1 size convolution kernel of the output channel operates. Mu (mu) ^k ，σ ^k ，γ ^k ，β ^k Representing following C ^(k) Accumulated mean, standard deviation, learned scale factor, deviation in BN layer, { } represents the splice operation of the three branches, +.>Representing the input and output of an electrocardiogram signal during the feature mapping process, BN () represents BN operations, and the implementation formula of BN () is expressed as:

wherein,

the 3 groups of characteristic signals with different receptive field characteristics obtained in the process are combined and input to a nonlinear characteristic fusion module (Nonlinear Feature Fusion Module) together so as to carry out nonlinear effective fusion on different characteristics of the electrocardiogram signals to obtain the time characteristics of the electrocardiogram signals.

The nonlinear feature fusion module is an improvement on the neck layer and reverse neck layer structure as shown in fig. 6. First, the earliest bottleneck layer appears to reduce the number of parameters and the calculation amount, and after dimension reduction, the data is more intuitively trained and extracted, which consists of three convolution parts, namely two point convolutions responsible for the dimension increase and decrease of the data feature and the depth convolution actually performed with the convolution operation, as shown in fig. 6 (a). Correspondingly, the data dimension of the deep convolution processing layer is four times of the input dimension by the reverse bottleneck layer structure appearing later, and as shown in fig. 6 (b), more valuable model performance improvement is obtained by using the operation on the premise of neglecting the calculated amount.

Compared with the structures shown in fig. 6 (a) and fig. 6 (b), the nonlinear feature fusion module performs special optimization processing to realize downward movement of the functional layer, and only nonlinear functions are reserved between the dimension increasing and decreasing operations, as shown in fig. 6 (c). Based on different feature sets in a rough splicing form, a fusion feature with stronger feature semantic representation capability is obtained by a series of nonlinear operations including feature dimension changing operations and inclusion operations. Specifically, 3 sets of characteristic signals with different receptive field characteristics are combined together, and enter a nonlinear characteristic fusion module together, and the characteristic signals are normalized along a characteristic dimension through a LayerNorm layer (layer normalization layer), wherein the LayerNorm layer has the following formula:

M ^(out) ＝LayerNorm(M ⁽ⁱⁿ⁾ ，ε，ω，δ) (5)

wherein the method comprises the steps ofRepresenting the input multi-scale stitching feature, ε is a constant added for numerical stabilization, ω is a learnable scaling parameter, and δ is a learnable offset parameter. The LayerNorm process can thus be expressed as:

wherein Var (M ⁽ⁱⁿ⁾ ) Is the feature variance input to the nonlinear feature fusion module, E (M ⁽ⁱⁿ⁾ ) Represents the mean value of the input, M ^(out) Representing the output after LayerNorm processing.

Then, the features processed by LayerNorm layers expand the number of channels by 4 times through convolution with a convolution kernel size of 1x1, further deepen nonlinearity of the features by a nonlinear activation function RELU, and compress the number of channels by 4 times through convolution with the convolution kernel size of 1x 1. Finally, the fully nonlinear operated feature is subjected to a third convolution kernel of size 1x1,final fusion along feature dimensions to scale the feature from the originalBecome->Finally, obtaining the time characteristic tensor of the electrocardiogram signal through the LayerNorm operation again.

(4) An electrocardiogram global feature extraction module and a respiratory signal global feature extraction module.

After the time feature modules of the two signals extract the respective time features, the invention utilizes the essence that the attention mechanism is similar to the observation mechanism of human beings on the external matters in the global feature extraction modules of the two signals, when the features are extracted, the important local information of the objects is firstly tended to be concerned, then the information of different areas is combined to form an overall impression on the observed objects, and the different-layer shallow feature information contained in the feature pool enables the global feature extraction modules to have more pertinence when the global feature extraction modules operate, so that a more reliable global dependency relationship is established.

Referring to fig. 7, for the global feature extraction module, the Keys, values, of the query, which implements the attention mechanism process, first select a feature map that is subjected to 1x1 convolution processing as a map of Keys, so as to minimize the difference between key information and input data in the attention operation process; the feature map which is subjected to 1x1 convolution and extracted is used as the mapping of Queries, so that more abstract features can be considered in the process of calculating the update weight with Keys; and finally, mapping the feature map with more abstract global information to Values, so that the feature map contains shallow global features at the starting point of the updating process, and providing a basis for the whole global feature extraction process. Then input to normalization layer to generate attention score of each signal feature, and calculate time average value of two signal features, and weighting by corresponding attention score to obtain global parkinsonism G (M) ^(out) )∈R ^d×1 Where d is the fixed dimension of the global feature. The calculation process is expressed as：

Wherein Q, K, V respectively represent Queries, keys and Values, d in matrix form _k Representing the dimension of the vector Keys.

(5) Parkinson disease classification module

Referring to FIG. 8, in one embodiment, the Parkinson's disease classification module includes 3 full connectivity layers and 1 Sigmoid layer. And the classifier output PD diagnostic score is a number between 0 and 1. If the score exceeds 0.5, the person is considered to have Parkinson's disease.

(6) Dual auxiliary tasks

To predict whether a parkinsonian label would require the use of about 10 hours of nocturnal breathing and electrocardiographic signals, the present invention also introduces a dual assistance task to predict quantitative electroencephalogram (qEEG) during sleep in a subject, such that the dual assistance task can provide an additional label help model regularizing model during training, the qEEG prediction being chosen as an assistance task because electroencephalogram and parkinsonism, respiration and electrocardiography are all relevant.

Specifically, to generate the qEEG tag, the base time series EEG signal is first converted to the frequency domain using a short time Fourier transform and Welch periodogram methods. Time-series electroencephalogram signals are extracted from C4-M1 channels, which are a common method in sleep studies. The EEG spectrum is then decomposed into the delta (0.5-4 Hz), theta (4-8 Hz), theta (8-13 Hz) and beta (13-30 Hz) bands and the power is normalized to obtain the relative power per second for each band.

The dual auxiliary tasks respectively take the coded signals as input, predict the relative power of each electroencephalogram signal wave band, and consist of three layers of one-dimensional deconvolution blocks (up-sampling the extracted respiratory features to the same time resolution as the qEEG signals) and two layers of complete connection. Each 1D deconvolution block contains three deconvolution layers, followed by batch normalization, rectifying linear cell activation and residual connection. The present invention also uses a skip connection that follows the UNet structure by connecting the output of the SRU layer in the respiratory signal temporal feature extraction module and the deep convolution features of the latter two layers in the electrocardiogram temporal feature extraction module to the deconvolution layer in the qEEG predictor. Fig. 9 is a specific network structure of the dual assist task, in which fig. 9 (a) is a process of corresponding respiratory signals in the dual assist task and fig. 9 (b) is a process of corresponding electrocardiographic signals in the dual assist task.

Step S120, training the deep learning model by using a training set, wherein the training set reflects the corresponding relation among the respiratory signal, the electrocardiogram signal and the Parkinson predictive label.

For example, with the set loss function minimized as an optimization target, training the deep learning model by using a training set to obtain the optimization parameters of the model. The training set reflects the correspondence between the respiratory signal, the electrocardiogram signal and whether or not the patient has parkinson's disease. The loss function may employ a cross entropy loss function or a square loss function.

Taking the introduction of a dual assistance task as an example, the overall loss function of training the deep learning model consists of two parts. The first part of parkinsonian classification employs a cross entropy loss function to measure the difference between the true probability distribution and the predicted probability distribution, expressed as:

wherein,representing the loss of Parkinson's disease classification, Y _PB Representing the true classification result,/->Representing the predicted classification result.

The double auxiliary task of the second part of electroencephalogram signals adopts a mean square error loss function to measure the square value of the average difference between the predicted output and the real label, and the square value is expressed as follows:

wherein,measuring the square value of the average difference between the electroencephalogram signal predicted by the auxiliary task and the real electroencephalogram signal in the respiratory signal time feature extraction module; />And measuring the square value of the average difference between the electroencephalogram signal predicted by the auxiliary task and the real electroencephalogram signal in the electrocardiographic time feature extraction module, wherein n represents the length of the electroencephalogram signal, T epsilon (1, T) represents the time of the signal, and T represents the total duration. X is X _qeeg Representing a real electroencephalogram signal->And respectively representing an electroencephalogram signal for assisting task prediction in the respiratory signal time feature extraction module and an electroencephalogram signal for assisting task prediction in the electrocardiographic signal time feature extraction module.

Thus, in one embodiment, the overall loss function is expressed as follows:

step S130, predicting the Parkinson diagnosis result of the target in real time by using the trained deep learning model.

After the deep learning model is trained, the deep learning model can be used as a parkinsonism detection model for actual parkinsonism diagnosis. For example, respiratory signals and electrocardiograms of the target are acquired for a period of time and input into a trained model to obtain a clinical indication of whether the patient has parkinsonism.

It is to be noted that those skilled in the art may make appropriate changes and modifications to the above-described embodiments without departing from the spirit and scope of the invention. For example, the convolution kernel sizes used by the respiratory signal temporal feature extraction module and the electrocardiogram temporal feature extraction module may vary in size. The nonlinear operations of the electrocardiogram time feature extraction module and the respiratory signal time feature extraction module can be replaced by other nonlinear operations, for example, the Layer Norm operation can be replaced by the Batch Normal operation, and the RELU activation function can be replaced by the GELU activation function. The multiscale used by the electrocardiogram temporal feature extraction module may be acquired with more or fewer convolutions. The functional layer in the nonlinear feature extraction module used by the electrocardiogram time feature extraction module can move upwards to the topmost layer, namely feature fusion is performed first, and nonlinear change is performed. In addition, the signals used may be replaced by other physiological signals associated with parkinson's disease. The number of skipped connections in the dual assistance task may be increased or decreased. The signal used for assistance in the dual assistance task may be replaced.

Correspondingly, the invention also provides a multi-mode physiological signal-based early parkinsonism screening device, which is used for realizing one or more aspects of the aspects. For example, the apparatus includes: a signal acquisition unit for acquiring a respiratory signal and an electrocardiogram signal of a target; and the prediction unit is used for inputting the respiratory signal and the electrocardiogram signal into a trained deep learning model to obtain a parkinsonism screening result. The deep learning model comprises a first time feature extraction module, a second time feature extraction module, a first global feature extraction module, a second global feature extraction module and a classification module, wherein the first time feature extraction module is used for extracting different scale features of respiratory signals, the second time feature extraction module is used for extracting different scale features of electrocardiogram signals and fusing the different features, the first global feature extraction module is used for extracting global features of the respiratory signals by using a self-attention mechanism, the second global feature extraction module is used for extracting global features of the electrocardiogram signals by using the self-attention mechanism, and the classification module is used for outputting Parkinson screening results by combining the features of the respiratory signals and the electrocardiogram signals. The functions of the signal acquisition unit and the prediction unit may be implemented by a general purpose processor, a special purpose processor, an FPGA, or the like.

To further verify the effect of the present invention, experiments were performed. Experimental results show that the model provided by the invention is reliable and stable, the learning capacity of the model is effectively improved, the learned model is more robust, and the robustness is stronger. In experimental verification, a verification method of 4-fold cross-validation was used to avoid overfitting. The data set is divided into a training set, a verification set and a test set, and the ratio is 13:5:3. The results were run with area under the curve (AUC) as the evaluation criterion, where the test AUC was 78.5%,84.4%,81.4% and 76%, respectively, and the average AUC was 80.1%.

In summary, the invention provides an early detection method for parkinsonism based on multi-mode multi-scale deep learning. According to the method, ECG and respiratory signals are combined, and the multi-mode multi-scale deep learning model is used for extracting and classifying features of ECG and respiratory signal data of a subject sleeping at night, so that richer biological signal information and key features can be captured, and accuracy and robustness are improved. Compared with the prior art, the invention has the following advantages:

1) The model provided by the invention can combine two night sleep physiological signal data for a long time to provide enough information quantity, and perform multi-mode and multi-scale signal feature extraction and classification for processing a large amount of complex biological signal data, thereby ensuring the accuracy and robustness of parkinsonism detection.

2) The model provided by the invention can utilize auxiliary tasks in long-time one-dimensional night sleep physiological signal data, and the problem of non-ideal results caused by sparse supervision is avoided.

3) The model provided by the invention can extract useful information from long-time one-dimensional night sleep physiological signal data to classify, has better classifying effect than training effect by only using respiratory signals, and provides an effective scheme for long-time sequence processing.

4) The invention follows the design of the UNet network structure and provides a design thought of dual auxiliary tasks for solving the data sparseness problem.

5) The invention provides a more convenient and reliable method for the detection of the Parkinson's disease and provides a better early screening tool for clinicians.

The present invention may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++, python, and the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which can execute the computer readable program instructions.

Various aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

The foregoing description of embodiments of the invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. An early parkinsonism screening method based on multi-modal physiological signals, comprising the steps of:

acquiring a respiration signal and an electrocardiogram signal of a target;

2. The method of claim 1, wherein the first temporal feature extraction module sequentially comprises a convolutional layer, a BN layer, an active layer, and a multi-layer residual block, and a simple cyclic unit connected to one layer after a last set number of residual blocks.

3. The method according to claim 2, characterized in that the residual block is set to 8 layers and a simple cyclic unit of one layer is connected after the last three layers of residual blocks, respectively, and each layer of residual blocks is represented as:

M ^(out) ＝BN(Conv ^(k) (Relu(BN(Conv ^(k) (M ⁽ⁱⁿ⁾ )))))

wherein M is ^(out) Is the characteristic signal extracted by the residual block, M ⁽ⁱⁿ⁾ Is the input of the residual block, conv represents the convolution process and k represents the convolution kernel.

4. The method of claim 1, wherein the second temporal feature extraction module comprises a plurality of depth blocks, and for each depth block, a plurality of parallel branches and a nonlinear feature fusion module, wherein the plurality of parallel branches obtain feature maps of a plurality of receptive fields for input electrocardiogram signals by convolution processing accompanied by BN operations, and each branch processes a corresponding electrocardiogram signal into feature signals of the same size to extract features of a plurality of time scales, and the nonlinear feature fusion module is used for fusing features of different time scales.

5. The method of claim 1, wherein for the first global feature extraction module and the second global feature extraction module, implementing an attention mechanism process is:

taking the feature map subjected to convolution processing as mapping of Keys, and taking the feature map subjected to convolution and extraction as mapping of Queries to obtain abstract global information;

mapping the obtained feature map of the abstract global information to Values, inputting the Values into a normalization layer, and generating attention scores of corresponding signal features;

a time average of the signal features is calculated and weighted as global features by the corresponding attention scores.

6. The method of claim 1, wherein the classification module comprises a plurality of fully connected layers and a single Sigmoid layer, wherein the classifier outputs a score of between 0 and 1, and further wherein the comparison with a set threshold is used to determine whether the target is suffering from parkinson's disease.

7. The method of claim 1, wherein during training the deep learning model, a dual assistance task is introduced to predict quantitative electroencephalograms during sleep of the subject, the overall loss function of training the deep learning model being set to:

wherein:

wherein L represents the total loss value,representing the loss of Parkinson's disease classification, Y _PB Representing the true classification result of Parkinson's disease, +.>Classification result representing parkinsonian disease prediction, +.>Measuring the square value of the average difference between the electroencephalogram signals predicted by the auxiliary task in the first time feature extraction module and the real electroencephalogram signals; />Is the square value of the average difference between the electroencephalogram signal predicted by the auxiliary task in the second time feature extraction module and the real electroencephalogram signal, n represents the length of the electroencephalogram signal, T epsilon (1, T) represents the time of the signal, T represents the total duration and X _qeeg Representing a real electroencephalogram signal->And respectively representing the electroencephalogram signals for assisting task prediction in the first time feature extraction module and the electroencephalogram signals for assisting task prediction in the second time feature extraction module.

8. The method of claim 4, wherein the nonlinear feature fusion module comprises, in order, a first LayerNorm layer for performing a layer normalization operation along a feature dimension, a first convolution layer for expanding a number of channels, a second convolution layer for scaling values on each channel by a learnable one-dimensional vector, a third convolution layer for performing feature fusion along the feature dimension, a third convolution layer for obtaining a temporal feature tensor of the electrocardiogram signal by the layer normalization operation, and a second LayerNorm layer.

9. An early parkinson's screening device based on a multimodal physiological signal, comprising:

10. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor realizes the steps of the method according to any of claims 1 to 8.