CN116712099A - Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium - Google Patents
Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116712099A CN116712099A CN202310696509.3A CN202310696509A CN116712099A CN 116712099 A CN116712099 A CN 116712099A CN 202310696509 A CN202310696509 A CN 202310696509A CN 116712099 A CN116712099 A CN 116712099A
- Authority
- CN
- China
- Prior art keywords
- data
- mode
- fetal heart
- features
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000002458 fetal heart Anatomy 0.000 title claims abstract description 72
- 238000001514 detection method Methods 0.000 title claims abstract description 22
- 238000013528 artificial neural network Methods 0.000 claims abstract description 39
- 239000013598 vector Substances 0.000 claims abstract description 38
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims description 43
- 230000015654 memory Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 230000008774 maternal effect Effects 0.000 claims description 15
- 230000000306 recurrent effect Effects 0.000 claims description 15
- 238000005070 sampling Methods 0.000 claims description 15
- WQZGKKKJIJFFOK-GASJEMHNSA-N Glucose Natural products OC[C@H]1OC(O)[C@H](O)[C@@H](O)[C@@H]1O WQZGKKKJIJFFOK-GASJEMHNSA-N 0.000 claims description 6
- 210000004369 blood Anatomy 0.000 claims description 6
- 239000008280 blood Substances 0.000 claims description 6
- 239000008103 glucose Substances 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 230000036772 blood pressure Effects 0.000 claims description 5
- 230000036760 body temperature Effects 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 230000001605 fetal effect Effects 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 3
- 210000003754 fetus Anatomy 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000006403 short-term memory Effects 0.000 description 3
- 210000001015 abdomen Anatomy 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 230000006793 arrhythmia Effects 0.000 description 1
- 206010003119 arrhythmia Diseases 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 230000036471 bradycardia Effects 0.000 description 1
- 208000006218 bradycardia Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/02—Measuring pulse or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/021—Measuring pressure in heart or blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
- A61B5/14532—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring glucose, e.g. by tissue impedance measurement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/48—Diagnostic techniques
- A61B8/488—Diagnostic techniques involving Doppler signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2131—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on a transform domain processing, e.g. wavelet transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/02—Foetus
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The application provides a fetal heart state detection method based on multi-mode data, electronic equipment and a storage medium, which comprise the following steps: acquiring multi-mode data; for each mode, preprocessing the data in the mode to obtain the data characteristics of the mode; respectively inputting the multi-mode data characteristics into corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks; and fusing a plurality of feature vectors into features to be detected, and inputting the features to be detected into a trained classifier to obtain fetal heart state category information. According to the scheme, related information from different modes is effectively extracted and integrated, and the accuracy and the robustness of fetal heart state detection are effectively improved.
Description
Technical Field
The present application relates to the field of medical data processing technologies, and in particular, to a method and apparatus for detecting a fetal heart state based on multi-modal data, an electronic device, and a computer readable storage medium.
Background
Fetal heart is the heart beat of the fetus, reflects the intrauterine state of the fetus, and when the fetus is lack of oxygen due to various reasons, the fetal heart can change correspondingly. Fetal heartbeat monitoring is therefore an important means of assessing fetal health. In the related art, the fetal heart monitoring methods generally only depend on single-mode data, and the methods may be affected by noise interference, poor data quality and other factors, so that the accuracy and the robustness of the fetal heart state detection result are limited.
Disclosure of Invention
The embodiment of the application aims to provide a fetal heart state detection method and device based on multi-mode data, electronic equipment and a computer readable storage medium, which are used for effectively extracting and integrating related information from different modes, and effectively improving the accuracy and robustness of fetal heart state detection.
In one aspect, the present application provides a method for detecting a fetal heart state based on multi-modal data, including:
acquiring multi-mode data;
for each mode, preprocessing the data in the mode to obtain the data characteristics of the mode;
respectively inputting the multi-mode data characteristics into corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks;
and fusing a plurality of feature vectors into features to be detected, and inputting the features to be detected into a trained classifier to obtain fetal heart state category information.
In an embodiment, the acquiring multi-modal data includes:
collecting multi-mode analog signals, and respectively converting the analog signals of each mode into digital signals;
and respectively carrying out coding processing on the multi-mode digital signals to obtain multi-mode data in a specified format.
In an embodiment, the multi-modal analog signal is a combination of at least two of a doppler waveform, a skin electrical signal waveform, an electrocardiographic signal waveform, maternal blood pressure data, maternal blood glucose data, and maternal body temperature.
In an embodiment, for each mode, preprocessing data under the mode to obtain data features of the mode includes:
for each mode, respectively carrying out normalization processing on the data under the mode to obtain a normalized data sequence;
aligning the multi-mode normalized data sequence in the data acquisition time to obtain a multi-mode aligned data sequence;
unifying the aligned data sequences in a multi-mode to a target sampling rate to obtain a multi-mode target data sequence;
and extracting the characteristics of the target data sequence of each mode to obtain corresponding data characteristics.
In an embodiment, the feature extraction is performed on the target data sequence of each mode to obtain corresponding data features, including:
performing Fourier transform on the target data sequence to obtain a first data characteristic;
and performing short-time Fourier transform on the target data sequence to obtain a second data characteristic.
In an embodiment, the method further comprises:
inputting a plurality of groups of multi-mode sample data features in the sample data set into a corresponding cyclic neural network to obtain a sample feature vector corresponding to each data feature; wherein each set of multi-modal sample data features in the sample dataset carries a fetal heart status category tag;
fusing a plurality of sample feature vectors of each group into sample features to be detected, and inputting the sample features to be detected into a classifier to obtain a predicted fetal heart state category;
based on the difference between the predicted fetal heart state category and the fetal heart state category label, adjusting model parameters of the classifier and the recurrent neural network;
repeating the above process until the classifier and the recurrent neural network converge, thereby obtaining a trained classifier and a trained recurrent neural network.
In one embodiment, the recurrent neural network is a long and short term memory network.
In another aspect, the present application provides a fetal heart condition detection device based on multi-modal data, including:
the acquisition module is used for acquiring multi-mode data;
the preprocessing module is used for preprocessing data in each mode to obtain data characteristics of the mode;
the input module is used for respectively inputting the multi-mode data characteristics to the corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks;
the classification module is used for fusing a plurality of feature vectors into features to be detected, inputting the features to be detected into a trained classifier and obtaining fetal heart state category information.
Further, the present application provides an electronic device, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the above-described multi-modal data-based fetal heart condition detection method.
Furthermore, the present application provides a computer-readable storage medium storing a computer program executable by a processor to perform the above-described multi-modal data-based fetal heart condition detection method.
According to the scheme, more abundant fetal heartbeat related information can be obtained by fusing the multi-modal data, so that the prediction accuracy is improved, and as a plurality of data sources are comprehensively utilized, the higher prediction performance can be maintained under the condition of poor quality of single-modal data, and the robustness of fetal heart state detection is improved; in addition, the deep learning model is adopted for feature extraction and fusion, potential classification in the data can be automatically learned, and the method has strong adaptability; by carrying out real-time processing and prediction on the multi-mode data, fetal heart beat abnormality can be found in time.
Drawings
In order to more clearly illustrate the technical solution of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic diagram of an application scenario of a method for detecting a fetal heart state based on multi-modal data according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for detecting a fetal heart condition based on multi-modal data according to an embodiment of the present application;
FIG. 4 is a detailed flowchart of step 320 in FIG. 3 according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a model architecture according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a model training process according to an embodiment of the present application;
fig. 7 is a block diagram of a fetal heart condition detection device based on multi-modal data according to an embodiment of the application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.
Like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a schematic diagram of an application scenario of a fetal heart state detection method based on multi-mode data according to an embodiment of the present application. As shown in fig. 1, the application scenario includes a client 20 and a server 30; the client 20 may be a data acquisition device, such as a doppler ultrasound device, a skin sensor, a dynamic cardiograph, etc., for transmitting acquired data to the server 30; the server 30 may be a host, a server cluster, or a cloud computing center, and may detect a fetal heart status based on multi-modal data.
As shown in fig. 2, the present embodiment provides an electronic apparatus 1 including: at least one processor 11 and a memory 12, one processor 11 being exemplified in fig. 2. The processor 11 and the memory 12 are connected by a bus 10, and the memory 12 stores instructions executable by the processor 11, which instructions are executed by the processor 11, so that the electronic device 1 may perform all or part of the flow of the method in the embodiments described below. In an embodiment, the electronic device 1 may be the server 30, for executing the fetal heart status detection method based on the multi-modal data.
The Memory 12 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as static random access Memory (Static Random Access Memory, SRAM), electrically erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk.
The present application also provides a computer readable storage medium storing a computer program executable by the processor 11 to perform the multi-modal data-based fetal heart condition detection method provided by the present application.
Referring to fig. 3, a flowchart of a method for detecting a fetal heart condition based on multi-modal data according to an embodiment of the present application is shown in fig. 3, and the method may include the following steps 310-340.
Step 310: and acquiring multi-mode data.
In the scheme of the application, multi-mode data can be adopted for fetal heart state detection. By way of example, the multimodal data may be a combination of at least two of doppler waveforms, skin electrical signal waveforms, electrocardiographic signal waveforms, maternal blood pressure data, maternal blood glucose data, maternal body temperature.
In one embodiment, multi-modal analog signals may be acquired while multi-modal data is acquired. For example, if the multimodal data includes Doppler waveforms, fetal heart and blood flow information may be acquired by a specialized Doppler ultrasound device to obtain Doppler waveforms that characterize the original waveform and frequency information of the fetal heart beat. If the multimodal data includes a skin electrical signal waveform, the fetal heart electrical signal may be acquired by placing a skin electrical sensor on the abdomen of the pregnant woman, thereby obtaining the skin electrical signal waveform. If the multi-mode data comprises electrocardiosignal waveforms, corresponding electrodes can be placed on the chest and the abdomen of the pregnant woman to capture electrocardiosignals in real time, so that electrocardiosignal waveforms can be obtained, and the electrocardiosignal waveforms can represent indexes such as fetal heart rate variability and the like, and are helpful for evaluating the stability of fetal heart beat. If the multimodal data includes maternal blood pressure data, maternal blood pressure can be measured at multiple time points by a sphygmomanometer. If the multimodal data includes maternal blood glucose data, maternal blood glucose can be measured by a blood glucose meter at multiple time points. If the multimodal data includes a maternal body temperature, the maternal body temperature can be measured at multiple time points by a thermometer.
Because the acquired data are Analog signals, the Analog signals of each mode can be respectively converted into digital signals through a digital-to-Analog converter (Analog-to-Digital Converter), and the subsequent processing is convenient.
Further, the multi-mode digital signals can be respectively encoded to obtain multi-mode data with a specified format. Here, the specified format may be a CSV (Comma-Separated Values) format. Because each piece of multi-mode data is actually a data sequence, the corresponding time stamp and equipment information can be added into the data in the appointed format, and the subsequent processing is convenient.
Step 320: and preprocessing data in the modes according to each mode to obtain data characteristics of the modes.
And respectively preprocessing the data in each mode so as to generate the data characteristics of the mode.
In an embodiment, referring to fig. 4, a detailed flowchart of step 320 in fig. 3 is provided for an embodiment of the present application, and as shown in fig. 4, the preprocessing process may be completed through the following steps 321 to 324.
Step 321: and respectively carrying out normalization processing on the data under the modes aiming at each mode to obtain a normalized data sequence.
The data in any mode are all data sequences, and normalization processing can be carried out on the data sequences in each mode respectively. By way of example, the normalization process may be represented by the following equation (1):
wherein x' is a value in the normalized data sequence; x is a value in the original data sequence; μ represents the mean of the values in the original data sequence; σ represents the standard deviation of the values in the original data sequence.
And (3) respectively executing the normalization process of the formula (1) on the data sequence of each mode to obtain the normalized data sequence of each mode.
Step 322: and aligning the multi-mode normalized data sequence in the data acquisition time to obtain the multi-mode aligned data sequence.
For the multi-mode normalized data sequences, alignment can be performed on the data acquisition time based on the time stamp corresponding to each numerical value in each normalized data sequence, so as to obtain the multi-mode aligned data sequences. Here, in the data sequences of different modalities, if the timestamp difference of the values is smaller than a preset duration threshold (for example, 0.01 seconds), the values can be considered to be acquired at the same time.
Step 323: and unifying the aligned data sequences in multiple modes to a target sampling rate to obtain the target data sequences in multiple modes.
Because the data acquisition equipment is different, the data of the multiple modes can correspond to different sampling frequencies, and in order to facilitate subsequent processing, the data sequences after the alignment of the multiple modes can be unified to a target sampling rate. Here, the target sampling rate may be set as needed. The target sampling rate may be, for example, a maximum sampling rate that the plurality of data acquisition devices have.
For low frequency data, linear interpolation may be performed to increase to the target sampling rate. By way of example, the linear interpolation process can be represented by the following equation (2):
wherein x 'represents the value of the t' moment obtained by interpolation; t is t i And t i+1 Two sampling point moments; x is x i And x i+1 Is the value of the time of the two sampling points.
And unifying the data sequence of any mode to the target sampling rate to obtain the target data sequence. Thus, a plurality of target data sequences of multiple modes can be obtained.
Step 324: and extracting the characteristics of the target data sequence of each mode to obtain corresponding data characteristics.
After the target data sequence of any mode is obtained, the characteristic extraction can be carried out in an aligned mode, so that the data characteristic under the mode is obtained.
In one embodiment, for each modality of the target data sequence, a fourier transform (Fourier Transform) may be performed on the target data sequence to obtain the first data characteristic. The first data characteristic is a frequency domain characteristic.
By way of example, the fourier transform process can be represented by the following equation (3):
wherein N is the signal length, i.e. the number of sampling points; x (n) is the sampling point value of the target data sequence; x (k) is a complex value of the frequency domain signal.
For each mode of the target data sequence, short-time Fourier transform (short-time Fourier transform) can be further performed on the target data sequence to obtain a second data characteristic. The second data characteristic is a time-frequency domain characteristic. The short-time fourier transform is implemented by sliding a window function (e.g., hamming window, hanning window, etc.) over the time-domain signal (target data sequence) to fourier transform the signal segments within the window.
By way of example, the short-time fourier transform process can be represented by the following equation (4):
wherein w (n) is a window function; m is a time index; k is the frequency index; n is the window length; x (n) is the sampling point value of the target data sequence; x (m, k) is a complex matrix of time-frequency domain signals, representing the signal distribution over different times and frequencies.
Step 330: and respectively inputting the multi-mode data characteristics into the corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks.
Wherein the recurrent neural network (Recurrent Neural Network, RNN) is used for further processing of the data features. In one embodiment, the recurrent neural network used in the present application may be a Long short-term memory network (LSTM). The long and short term memory network may include several components, namely a forget gate, an input gate, a cell status update and an output gate.
Each data characteristic distribution of each mode is preset with a corresponding cyclic neural network. For example, if 5 modes of data are selected for fetal heart condition detection, and 2 data features are extracted from each mode of data, 10 recurrent neural networks need to be set.
The data characteristics are respectively processed through a plurality of cyclic neural networks, so that characteristic vectors can be output, and a plurality of characteristic vectors are obtained.
Step 340: and fusing the plurality of feature vectors into features to be detected, and inputting the features to be detected into a trained classifier to obtain fetal heart state category information.
After a plurality of feature vectors are obtained, the feature vectors can be connected according to a preset arrangement sequence, so that the feature to be detected is obtained through fusion. The feature to be detected may be expressed as h= [ h ] 1 ,h 2 ,h 3 ,h 4 ,h 5 ,h 6 ]Here, h 1 、h 2 、h 3 、h 4 、h 5 、h 6 Representing 6 feature vectors.
The feature to be detected is input into a classifier, which may be a fully connected layer, for example. Within the classifier, the feature to be detected may be linearly transformed. By way of example, the linear transformation process can be represented by the following equation (5):
z＝hW+b (5)
wherein h is the feature to be detected; w is a weight matrix; b is a bias vector; z is the output vector.
Further, the output vector after the linear transformation is transferred to an activation function, and a final output vector of the full connection layer is obtained, wherein the final output vector can represent fetal heart state category information.
Illustratively, using softmax as the activation function, the calculation process is represented by the following equation (6):
wherein z is i Representing an ith element in the output vector; y is i Representing the ith element in the final output vector, and representing the probability that the feature h to be detected belongs to the ith category.
After the final output vector is obtained, determining the category corresponding to the largest element in the final output vector, namely the fetal heart state category information. The fetal heart status category information which can be detected by the scheme of the application can include, but is not limited to, normal, overspeed, bradycardia, irregular, sinus rhythm, arrhythmia, fetal heart pacing abnormality and the like, and can be set according to requirements.
For example, referring to fig. 5, a schematic diagram of a model architecture provided for an embodiment of the present application is shown in fig. 5, where a corresponding long-short-time memory network is set for the frequency domain feature of the doppler waveform, the time-frequency domain feature of the doppler waveform, the frequency domain feature of the skin electric signal waveform, the time-frequency domain feature of the skin electric signal waveform, the frequency domain feature of the electrocardiograph signal waveform, and the time-frequency domain feature of the electrocardiograph signal waveform. And fusing the feature vectors input by the long-short-time memory network to obtain the feature to be detected. And inputting the characteristics to be detected into the full-connection layer, and obtaining a classification result as fetal heart state type information.
According to the scheme, more abundant fetal heartbeat related information can be obtained by fusing the multi-modal data, so that the prediction accuracy is improved, and as a plurality of data sources are comprehensively utilized, the higher prediction performance can be maintained under the condition of poor quality of single-modal data, and the robustness of fetal heart state detection is improved; in addition, the deep learning model is adopted for feature extraction and fusion, potential classification in the data can be automatically learned, and the method has strong adaptability; by carrying out real-time processing and prediction on the multi-mode data, fetal heart beat abnormality can be found in time.
In an embodiment, referring to fig. 6, which is a schematic diagram of a model training process according to an embodiment of the present application, as shown in fig. 6, the method may include the following steps 610 to 640.
Step 610: inputting a plurality of groups of multi-mode sample data characteristics in the sample data set into a corresponding cyclic neural network to obtain sample characteristic vectors corresponding to each sample data characteristic; wherein each set of multi-modal sample data features in the sample dataset carries a fetal heart condition category tag.
The sample data set includes sample data features that pre-collect multi-modal data and are generated in the multi-modal data. Each group of multi-mode data features are generated by multi-mode data acquired at the same moment, and can represent the fetal heart state at the moment. Based on the manually judged fetal heart state category, a fetal heart state category label can be marked for each group of multi-mode sample data features.
After each group of multi-mode sample data characteristics is input to the corresponding cyclic neural network, corresponding sample characteristic vectors can be calculated for sample data characteristic distribution through the cyclic neural network. Thus, a plurality of sample feature vectors may be obtained for each set of multi-modal sample data features.
Step 620: and fusing the plurality of sample feature vectors of each group into sample features to be detected, and inputting the sample features to be detected into a classifier to obtain the predicted fetal heart state category.
And connecting the plurality of sample feature vectors of each group through a preset arrangement sequence, so as to obtain the sample feature to be detected corresponding to the group. After the sample to-be-detected characteristics are input into the classifier, the predicted fetal heart state category can be calculated. Here, the predicted fetal heart state category is fetal heart state category information output by the classifier in the training process.
Step 630: model parameters of the classifier and the recurrent neural network are adjusted based on the difference between the predicted fetal heart status category and the fetal heart status category label.
Step 640: repeating the above process until the classifier and the cyclic neural network converge, and obtaining the trained classifier and the trained cyclic neural network.
And evaluating and predicting the difference between the fetal heart state type and the fetal heart state type label through a preset loss function, and adjusting model parameters of the classifier and the cyclic neural network through a back propagation method based on the function value of the loss function.
After the model parameters are adjusted, the process may return to step 610, where the above steps are repeated. After repeated iteration, when the training times reach a preset time threshold or when the function value of the loss function tends to be stable, the classifier and the circulating neural network can be determined to converge, and the trained classifier and the trained circulating neural network are obtained.
FIG. 7 is a block diagram of a multi-modal data-based fetal heart condition detection apparatus according to an embodiment of the present application, as shown in FIG. 7, the apparatus may include:
an acquisition module 710, configured to acquire multi-modal data;
the preprocessing module 720 is configured to, for each mode, respectively preprocess data in the mode to obtain data features of the mode;
the input module 730 is configured to input the multi-mode data features to the corresponding recurrent neural networks respectively, so as to obtain feature vectors output by the multiple recurrent neural networks respectively;
the classification module 740 is configured to fuse a plurality of feature vectors into features to be detected, and input the features to be detected into a trained classifier to obtain fetal heart status category information.
The implementation process of the functions and actions of each module in the device is specifically detailed in the implementation process of corresponding steps in the method for detecting the fetal heart state based on the multi-mode data, and is not repeated here.
In the several embodiments provided in the present application, the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, of the flowcharts and block diagrams in the figures that illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored on a computer readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Claims (10)
1. A method for detecting a fetal heart condition based on multi-modal data, comprising:
acquiring multi-mode data;
for each mode, preprocessing the data in the mode to obtain the data characteristics of the mode;
respectively inputting the multi-mode data characteristics into corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks;
and fusing a plurality of feature vectors into features to be detected, and inputting the features to be detected into a trained classifier to obtain fetal heart state category information.
2. The method of claim 1, wherein the acquiring multi-modal data comprises:
collecting multi-mode analog signals, and respectively converting the analog signals of each mode into digital signals;
and respectively carrying out coding processing on the multi-mode digital signals to obtain multi-mode data in a specified format.
3. The method of claim 2, wherein the multi-modal analog signal is a combination of at least two of doppler waveform, skin electrical signal waveform, electrocardiographic signal waveform, maternal blood pressure data, maternal blood glucose data, and maternal body temperature.
4. The method according to claim 1, wherein the preprocessing the data in each mode to obtain the data characteristics of the mode includes:
for each mode, respectively carrying out normalization processing on the data under the mode to obtain a normalized data sequence;
aligning the multi-mode normalized data sequence in the data acquisition time to obtain a multi-mode aligned data sequence;
unifying the aligned data sequences in a multi-mode to a target sampling rate to obtain a multi-mode target data sequence;
and extracting the characteristics of the target data sequence of each mode to obtain corresponding data characteristics.
5. The method of claim 4, wherein the feature extraction of the target data sequence of each modality to obtain the corresponding data feature comprises:
performing Fourier transform on the target data sequence to obtain a first data characteristic;
and performing short-time Fourier transform on the target data sequence to obtain a second data characteristic.
6. The method according to claim 1, wherein the method further comprises:
inputting a plurality of groups of multi-mode sample data features in the sample data set into a corresponding cyclic neural network to obtain a sample feature vector corresponding to each data feature; wherein each set of multi-modal sample data features in the sample dataset carries a fetal heart status category tag;
fusing a plurality of sample feature vectors of each group into sample features to be detected, and inputting the sample features to be detected into a classifier to obtain a predicted fetal heart state category;
based on the difference between the predicted fetal heart state category and the fetal heart state category label, adjusting model parameters of the classifier and the recurrent neural network;
repeating the above process until the classifier and the recurrent neural network converge, thereby obtaining a trained classifier and a trained recurrent neural network.
7. The method of claim 1 or 6, wherein the recurrent neural network is a long-short-term memory network.
8. A multi-modal data-based fetal heart condition detection apparatus, comprising:
the acquisition module is used for acquiring multi-mode data;
the preprocessing module is used for preprocessing data in each mode to obtain data characteristics of the mode;
the input module is used for respectively inputting the multi-mode data characteristics to the corresponding cyclic neural networks to obtain characteristic vectors respectively output by the cyclic neural networks;
the classification module is used for fusing a plurality of feature vectors into features to be detected, inputting the features to be detected into a trained classifier and obtaining fetal heart state category information.
9. An electronic device, the electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the multi-modal data-based fetal heart condition detection method of any one of claims 1-7.
10. A computer readable storage medium, wherein the storage medium stores a computer program executable by a processor to perform the multi-modal data-based fetal heart condition detection method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310696509.3A CN116712099A (en) | 2023-06-13 | 2023-06-13 | Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310696509.3A CN116712099A (en) | 2023-06-13 | 2023-06-13 | Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116712099A true CN116712099A (en) | 2023-09-08 |
Family
ID=87874797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310696509.3A Pending CN116712099A (en) | 2023-06-13 | 2023-06-13 | Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116712099A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392396A (en) * | 2023-12-08 | 2024-01-12 | 安徽蔚来智驾科技有限公司 | Cross-modal target state detection method, device, intelligent device and medium |
-
2023
- 2023-06-13 CN CN202310696509.3A patent/CN116712099A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392396A (en) * | 2023-12-08 | 2024-01-12 | 安徽蔚来智驾科技有限公司 | Cross-modal target state detection method, device, intelligent device and medium |
| CN117392396B (en) * | 2023-12-08 | 2024-03-05 | 安徽蔚来智驾科技有限公司 | Cross-modal target state detection method, device, intelligent device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gao et al. | An effective LSTM recurrent network to detect arrhythmia on imbalanced ECG dataset | |
| Sangaiah et al. | An intelligent learning approach for improving ECG signal classification and arrhythmia analysis | |
| Übeyli | Combining recurrent neural networks with eigenvector methods for classification of ECG beats | |
| CN107714023B (en) | Static electrocardiogram analysis method and device based on artificial intelligence self-learning | |
| CN110020623B (en) | Human body activity recognition system and method based on conditional variation self-encoder | |
| US20200337580A1 (en) | Time series data learning and analysis method using artificial intelligence | |
| US20180260706A1 (en) | Systems and methods of identity analysis of electrocardiograms | |
| CN107832737B (en) | Artificial intelligence-based electrocardiogram interference identification method | |
| US10368792B2 (en) | Method for detecting deception and predicting interviewer accuracy in investigative interviewing using interviewer, interviewee and dyadic physiological and behavioral measurements | |
| CN110598676B (en) | Deep learning gesture electromyographic signal identification method based on confidence score model | |
| CN113052113B (en) | Depression identification method and system based on compact convolutional neural network | |
| CN112426160A (en) | Electrocardiosignal type identification method and device | |
| CN116602642B (en) | Heart rate monitoring method, device and equipment | |
| CN116503673B (en) | Arrhythmia identification and detection method and system based on electrocardiogram | |
| CN116712099A (en) | Fetal heart state detection method based on multi-mode data, electronic equipment and storage medium | |
| CN111956214A (en) | QRS wave automatic detection method based on U-net end-to-end neural network model | |
| Gao et al. | Real time ECG characteristic point detection with randomly selected signal pair difference (RSSPD) feature and random forest classifier | |
| Lu et al. | KecNet: a light neural network for arrhythmia classification based on knowledge reinforcement | |
| CN111053552A (en) | QRS wave detection method based on deep learning | |
| CN115486849A (en) | Electrocardiosignal quality evaluation method, device and equipment | |
| Liu et al. | Semantic segmentation of qrs complex in single channel ecg with bidirectional lstm networks | |
| CN115177260A (en) | Intelligent electrocardio-heart sound diagnosis method and device based on artificial neural network | |
| CN112957018A (en) | Heart state detection method and device based on artificial intelligence | |
| Vimala et al. | Classification of cardiac vascular disease from ECG signals for enhancing modern health care scenario | |
| Wen et al. | ECG beat classification using GreyART network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |