CN117982116A

CN117982116A - Data processing method, device and computer readable storage medium

Info

Publication number: CN117982116A
Application number: CN202211358814.3A
Authority: CN
Inventors: 赵咏豪; 殷潜; 刘翔飞
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2022-11-01
Filing date: 2022-11-01
Publication date: 2024-05-07
Also published as: WO2024093309A1

Abstract

The application relates to a data processing method, a device and a computer readable storage medium, wherein the method comprises the following steps: when a detection request is received, acquiring various physiological data acquired by a plurality of sensors in a first time period; processing the plurality of physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and the number of data points of the different physiological data is the same; inputting the processed physiological data into a first neural network model; and acquiring a first detection result output by the first neural network model, wherein the first detection result indicates the blood pressure waveform of the user. According to the data processing method provided by the embodiment of the application, more perfect blood pressure waveforms can be obtained, and the blood pressure waveforms are ensured to have certain accuracy. The method can also be applied to the portable equipment to improve the use experience of the user.

Description

Data processing method, device and computer readable storage medium

Technical Field

The present application relates to the field of medical health, and in particular, to a data processing method, apparatus, and computer readable storage medium.

Background

With aging population, cardiovascular health is receiving more and more attention, and miniaturization and convenience of equipment become the latest appeal of consumers. At present, intelligent wearable equipment is commonly provided with heart rate, blood oxygen saturation, electrocardiogram and other cardiovascular data measurement functions, and partial products support blood pressure measurement functions, but generally only support calculation of systolic pressure and diastolic pressure. Although systolic and diastolic pressures can satisfy basic analysis of blood pressure, their information amount is limited.

The systolic pressure and the diastolic pressure are respectively the peak value and the valley value in the blood pressure waveform, and the complete blood pressure waveform can provide richer blood vessel pressure fluctuation information, so that compared with the systolic pressure and the diastolic pressure, the blood pressure waveform data information is richer and more accurate.

Therefore, how to accurately obtain a more perfect blood pressure waveform becomes a research hotspot in the field.

Disclosure of Invention

In view of this, a data processing method, a data processing device and a computer readable storage medium are provided, and according to the data processing method of the embodiment of the application, a more perfect blood pressure waveform can be obtained, and the blood pressure waveform is ensured to have a certain accuracy. The method can also be applied to the portable equipment to improve the use experience of the user.

In a first aspect, an embodiment of the present application provides a data processing method, including: when a detection request is received, acquiring various physiological data acquired by a plurality of sensors in a first time period; processing the plurality of physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and the number of data points of the different physiological data is the same; inputting the processed physiological data into a first neural network model; and acquiring a first detection result output by the first neural network model, wherein the first detection result indicates the blood pressure waveform of the user.

According to the data processing method provided by the embodiment of the application, when the detection request is received, multiple physiological data acquired by the sensors in the first time period are acquired, the multiple physiological data are processed to obtain the processed physiological data, so that the different physiological data in the processed physiological data are synchronous and the number of data points of the different physiological data is the same, each sampling point in the processed physiological data corresponds to the multiple physiological data, and the establishment of the association relation among the multiple physiological data is completed; the processed physiological data is input into the first neural network model, so that a first detection result which is output by the first neural network model and indicates the blood pressure waveform of the user can be obtained. Compared with the prior art only obtaining the diastolic pressure and the systolic pressure, the method provided by the embodiment of the application can obtain the complete blood pressure waveform, the obtained blood pressure information is more perfect, the blood pressure waveform is obtained by synthesizing physiological data of a plurality of sensors, and compared with the prior art, the blood pressure waveform is more accurate; the method can be executed by wearable equipment or miniaturized portable equipment, and compared with large equipment with large volume and inconvenient movement, the method can enable the blood pressure waveform detection of the user to be more convenient and the user experience to be improved.

In a first possible implementation manner of the data processing method according to the first aspect, the plurality of physiological data includes at least one of electrocardiographic data, photoplethysmographic data, heart sound data, heart vibration data, heart impedance data, blood flow velocity data, and at least one of pulse vibration mechanical wave data, pulse pressure wave data.

In a second possible implementation manner of the data processing method according to the first aspect or the first possible implementation manner of the first aspect, the processed physiological data is multi-channel one-dimensional data, and the one-dimensional data of each channel is obtained according to physiological data acquired by a corresponding one of the sensors.

Each channel may serve as a unit of storage for the processed physiological data, in which case the processed physiological data is more computationally efficient in a hardware module supporting one-dimensional data operations.

In a third possible implementation manner of the first aspect or the first possible implementation manner of the first aspect, the processed physiological data is single-channel two-dimensional data, wherein the data in the first dimension is sampling time of the physiological data, and the data in the second dimension is physiological data acquired by a plurality of sensors.

In this case, the processed physiological data is more computationally efficient in the hardware module supporting the two-dimensional data operation.

In a fourth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the inputting the processed physiological data into the first neural network model includes: determining a plurality of time windows according to a preset first window length and window step length, wherein two adjacent time windows are not completely overlapped; and sequentially inputting the processed physiological data in the plurality of time windows into the first neural network model.

In this way, it is ensured that the physiological data input into the first neural network model in two adjacent times are consecutive in time, so that the blood pressure waveform indicated by the first detection result output from the first neural network model is also consecutive.

In a fifth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the method further includes: obtaining sample data, the sample data comprising a plurality of physiological data acquired by a plurality of sensors over a second time period; processing the sample data to obtain processed sample data, wherein the processed sample data comprise different types of physiological data which are synchronous and have the same number of data points; inputting the processed sample data into a second neural network model, the second neural network model being an untrained model; according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain the first neural network model; and the data points output by the second neural network model each time are consistent with the data points of the real blood pressure waveform corresponding to the sample data.

In this way, the accuracy of the first detection result obtained when the processed physiological data in the same manner as the input data obtained when the first neural network model is used to detect the blood pressure waveform is ensured.

In a fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the data processing method, the adjusting parameters of the second neural network model according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data to obtain the first neural network model includes: determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the second neural network model; and adjusting parameters of the second neural network model according to the regression loss to obtain the first neural network model.

In this way, a first neural network model that meets the use requirements can be obtained. Thereby ensuring the accuracy of blood pressure waveform detection using the first neural network model.

In a seventh possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the first neural network model includes at least one inference unit, and when each inference unit is connected in series and/or parallel, and a first layer of the first neural network model is one inference unit, the processed physiological data is input to the inference unit; when the first layer of the first neural network model is a plurality of inference units connected in parallel, the processed physiological data are respectively input into the plurality of inference units; for two inference units in a series connection relationship, the output of the former inference unit is used as the input of the latter inference unit; for a plurality of inference units connected in parallel, wherein a series or weighted summation result of the outputs of each of the inference units is taken as a common output of the plurality of inference units; when the last layer of the first neural network model is an inference unit, the output of the inference unit is used as the first detection result; when the last layer of the first neural network model is a plurality of inference units connected in parallel, the common output of the plurality of inference units is used as the first detection result.

The plurality of reasoning units are connected in series and/or in parallel to improve the accuracy of the result, and in this way, the flexibility of the structure of the first neural network model can be effectively improved.

In an eighth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the method is applied to a wearable device, and the processing the plurality of physiological data to obtain processed physiological data includes: synchronizing physiological data acquired by the plurality of sensors based on a clock of the wearable device when the plurality of sensors are disposed on the wearable device; when the plurality of sensors are not arranged on the wearable device, physiological data acquired by the plurality of sensors are collected through mutual communication with the device where the plurality of sensors are arranged, and the physiological data acquired by the plurality of sensors are synchronized through communication time stamps.

In this way, the synchronism of the collected physiological data can be ensured, and the accuracy of the first detection result can be improved.

In a ninth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the obtaining the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data includes: and performing filtering processing and/or baseline drift removal processing on the first physiological data, the second physiological data and the third physiological data by using at least one of a band-pass filtering algorithm, a wavelet decomposition algorithm and a mode decomposition algorithm to obtain processed physiological data.

In this way, the processed physiological data is made more accurate.

In a tenth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the obtaining the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data further includes: and screening each data point in the first physiological data, the second physiological data and the third physiological data according to the threshold value corresponding to the physiological data to obtain processed physiological data.

In this way, the processed physiological data is made more accurate.

In an eleventh possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the obtaining the first detection result output by the first neural network model includes: and acquiring first detection results sequentially output by the first neural network model, wherein each output first detection result indicates a blood pressure waveform in a preset second window length.

When the second window length is equal to the first window length, the first neural network model may be enabled to detect a blood pressure waveform synchronized with data input to the first neural network model. When the second window length is greater than the first window length, the first neural network model can be enabled to predict the blood pressure waveform in a future period of time, and the capability of the first neural network model is improved.

In a twelfth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, when the second neural network model is implemented based on generating the antagonistic neural network model, adjusting parameters of the second neural network model according to a real blood pressure waveform corresponding to sample data output by the second neural network model, to obtain the first neural network model, including: determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the generator in the generated antagonistic neural network model; determining cross entropy loss of sample data according to a discrimination result output by a discriminator in the generated countermeasure neural network model; determining the weighted loss of the sample data according to the regression loss, the first weight corresponding to the regression loss, the cross entropy loss and the second weight corresponding to the cross entropy loss; and adjusting parameters for generating the antagonistic neural network model according to at least one of the regression loss and the weighted loss, and taking the obtained generator as a first neural network model.

By the method, on the premise that the first neural network model meeting the requirements is obtained through training, the structure of the second neural network model is more likely, the flexibility of the structure of the second neural network model is improved, and the accuracy of the first neural network model is improved by generating the antagonistic neural network model.

In a thirteenth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the processing the plurality of physiological data to obtain processed physiological data includes: when the sampling rate of any one physiological data is smaller than the sampling rate reference, adopting an interpolation algorithm to process the physiological data to obtain first physiological data with the sampling rate equal to the sampling rate reference; when the sampling rate of any one physiological data is larger than the sampling rate reference, adopting a downsampling algorithm to process the physiological data to obtain second physiological data with the sampling rate equal to the sampling rate reference; when the sampling rate of any one physiological data is equal to the sampling rate reference, taking the physiological data as third physiological data; and obtaining the processed physiological data according to the first physiological data, the second physiological data and the third physiological data.

The sample rate criterion may be manually selected, for example, 100 sample points/second.

In this way, each data point in each kind of physiological data can be guaranteed to correspond to a unique one of the data points of other physiological data in the processed physiological data, so that the detection accuracy of the first neural network model on the blood pressure waveform is guaranteed when the processed physiological data is input into the first neural network model.

In a fourteenth possible implementation manner of the data processing method according to the first aspect or any one of the possible implementation manners of the first aspect, the processing the plurality of physiological data to obtain processed physiological data includes: determining a first number of data points at a sampling rate reference sampling over the first period of time; adding zero data points to any physiological data when the number of data points of the physiological data is smaller than the first data point number until the sum of the number of data points of the physiological data and the added data point number is equal to the first data point number; the processed physiological data is obtained from such physiological data after adding a zero-valued data point.

The first number of data points may also be selected manually, for example based on the number of neurons of the first neural network output layer.

In this way, the flexibility of the acquisition mode of the processed physiological data can be improved. And the method is simpler and the data processing cost is lower.

In a second aspect, embodiments of the present application provide a data processing apparatus, the apparatus comprising: the first acquisition module is used for acquiring various physiological data acquired by the plurality of sensors in a first time period when a detection request is received; the first processing module is used for processing the plurality of physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and have the same number of data points of different physiological data; the first input module is used for inputting the processed physiological data into a first neural network model; the second acquisition module is used for acquiring a first detection result output by the first neural network model, and the first detection result indicates the blood pressure waveform of the user.

According to a second aspect, in a first possible implementation manner of the data processing apparatus, the plurality of physiological data includes at least one of electrocardiographic data, photoplethysmographic data, heart sound data, heart vibration data, heart impedance data, blood flow velocity data, and at least one of pulse vibration mechanical wave data, pulse pressure wave data.

In a second possible implementation manner of the data processing apparatus according to the second aspect or the first possible implementation manner of the second aspect, the processed physiological data is multi-channel one-dimensional data, and the one-dimensional data of each channel is obtained according to physiological data acquired by a corresponding one of the sensors.

In a third possible implementation manner of the data processing apparatus according to the second aspect or the first possible implementation manner of the second aspect, the processed physiological data is single-channel two-dimensional data, wherein the data in the first dimension is a sampling time of the physiological data, and the data in the second dimension is physiological data acquired by a plurality of sensors.

In a fourth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the first input module includes: the first determining unit is used for determining a plurality of time windows according to a preset first window length and window step length, wherein two adjacent time windows are not completely overlapped; and the input unit is used for sequentially inputting the processed physiological data in the time windows into the first neural network model.

In a fifth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the apparatus further includes: a third acquisition module for acquiring sample data comprising a plurality of physiological data acquired by a plurality of sensors over a second time period; the second processing module is used for processing the sample data to obtain processed sample data, wherein the processed sample data comprise different types of physiological data which are synchronous and have the same number of data points; a second input module for inputting the processed sample data into a second neural network model, the second neural network model being an untrained completed model; the first adjustment module is used for adjusting parameters of the second neural network model according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data to obtain the first neural network model; and the data points output by the second neural network model each time are consistent with the data points of the real blood pressure waveform corresponding to the sample data.

In a sixth possible implementation manner of the data processing apparatus according to the fifth possible implementation manner of the second aspect, the first adjusting module includes: the second determining unit is used for determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the second neural network model; and a third determining unit, configured to adjust parameters of the second neural network model according to the regression loss, so as to obtain the first neural network model.

In a seventh possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the first neural network model includes at least one inference unit, and when the inference units are connected in series and/or in parallel, the first layer of the first neural network model is an inference unit, the processed physiological data is input to the inference unit; when the first layer of the first neural network model is a plurality of inference units connected in parallel, the processed physiological data are respectively input into the plurality of inference units; for two inference units in a series connection relationship, the output of the former inference unit is used as the input of the latter inference unit; for a plurality of inference units connected in parallel, wherein a series or weighted summation result of the outputs of each of the inference units is taken as a common output of the plurality of inference units; when the last layer of the first neural network model is an inference unit, the output of the inference unit is used as the first detection result; when the last layer of the first neural network model is a plurality of inference units connected in parallel, the common output of the plurality of inference units is used as the first detection result.

In an eighth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the apparatus is applied to a wearable device, and the first processing module includes: the first synchronization unit is used for synchronizing physiological data acquired by the plurality of sensors based on clocks of the wearable device when the plurality of sensors are arranged on the wearable device; and the second synchronization unit is used for collecting physiological data acquired by the plurality of sensors through mutual communication with equipment where the plurality of sensors are positioned when the plurality of sensors are not arranged on the wearable equipment, and synchronizing the physiological data acquired by the plurality of sensors through communication time stamps.

In a ninth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the obtaining the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data includes: and performing filtering processing and/or baseline drift removal processing on the first physiological data, the second physiological data and the third physiological data by using at least one of a band-pass filtering algorithm, a wavelet decomposition algorithm and a mode decomposition algorithm to obtain processed physiological data.

In a tenth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the obtaining the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data further includes: and screening each data point in the first physiological data, the second physiological data and the third physiological data according to the threshold value corresponding to the physiological data to obtain processed physiological data.

In an eleventh possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the second obtaining module includes:

The first acquisition unit is used for acquiring first detection results sequentially output by the first neural network model, and the first detection results output each time indicate the blood pressure waveform in the preset second window length.

In a twelfth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the first adjustment module includes: a seventh determining unit, configured to determine a regression loss according to a real blood pressure waveform corresponding to the sample data generated by the generator in the antagonistic neural network model; an eighth determining unit for determining cross entropy loss of the sample data according to the discrimination result output by the discriminator in the generated countermeasure neural network model; a ninth determining unit, configured to determine a weighted loss of the sample data according to the regression loss, a first weight corresponding to the regression loss, the cross entropy loss, and a second weight corresponding to the cross entropy loss; and a tenth determination unit for adjusting parameters for generating an antagonistic neural network model according to at least one of the regression loss and the weighted loss, the resulting generator being the first neural network model.

In a thirteenth possible implementation manner of the data processing apparatus according to the second aspect or any one of the possible implementation manners of the second aspect, the first processing module includes: the first processing unit is used for processing any physiological data by adopting an interpolation algorithm when the sampling rate of the physiological data is smaller than a sampling rate reference to obtain first physiological data with the sampling rate equal to the sampling rate reference; the second processing unit is used for processing any physiological data by adopting a downsampling algorithm when the sampling rate of the physiological data is larger than the sampling rate reference to obtain second physiological data with the sampling rate equal to the sampling rate reference; a third processing unit configured to take any one of the physiological data as third physiological data when a sampling rate of the physiological data is equal to a sampling rate reference; and a fourth determining unit, configured to obtain the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data.

In a fourteenth possible implementation form of the data processing apparatus according to the second aspect as such or any one of the possible implementation forms of the second aspect, the first processing module comprises: a fifth determining unit configured to determine a first number of data points when sampling at a sampling rate reference in the first period; an adding unit, configured to add a zero-valued data point to any one of the physiological data when the number of data points of the physiological data is smaller than the first number of data points, until the sum of the number of data points of the physiological data and the added number of data points is equal to the first number of data points; a sixth determining unit for obtaining the processed physiological data from the physiological data after adding the data point with zero value.

In a third aspect, embodiments of the present application provide a model training method, the method comprising: obtaining sample data, the sample data comprising a plurality of physiological data acquired by a plurality of sensors over a second time period; processing the sample data to obtain processed sample data, wherein the processed sample data comprise different types of physiological data which are synchronous and have the same number of data points; inputting the processed sample data into a second neural network model, the second neural network model being an untrained model; according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain the first neural network model; and the data points output by the second neural network model each time are consistent with the data points of the real blood pressure waveform corresponding to the sample data.

According to a third aspect, in a first possible implementation manner of the model training method, the adjusting parameters of the second neural network model according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data to obtain the first neural network model includes: determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the second neural network model; and adjusting parameters of the second neural network model according to the regression loss to obtain the first neural network model.

According to a third aspect, in a second possible implementation manner of the model training method, when the second neural network model is implemented based on generating the antagonistic neural network model, according to a real blood pressure waveform corresponding to the sample data output by the second neural network model, parameters of the second neural network model are adjusted to obtain the first neural network model, including: determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the generator in the generated antagonistic neural network model; determining cross entropy loss of sample data according to a discrimination result output by a discriminator in the generated countermeasure neural network model; determining the weighted loss of the sample data according to the regression loss, the first weight corresponding to the regression loss, the cross entropy loss and the second weight corresponding to the cross entropy loss; and adjusting parameters for generating the antagonistic neural network model according to at least one of the regression loss and the weighted loss, and taking the obtained generator as a first neural network model.

In a fourth aspect, an embodiment of the present application provides a data processing apparatus, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the data processing method of the first aspect or one or several of the possible implementations of the first aspect when executing the instructions.

In a fifth aspect, embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a data processing method of the above-described first aspect or one or more of the possible implementations of the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device performs the data processing method of the first aspect or one or more of the possible implementations of the first aspect.

These and other aspects of the application will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.

Fig. 1 shows an example of a wearable device to which the scheme of the first prior art is applied.

Fig. 2 shows an example of the way in which a wearable device provided with an electrocardiogram sensor is used.

Fig. 3 shows an example of electrocardiographic data acquired by a prior art second approach.

Fig. 4 shows an example of a wearable device provided with a photoplethysmograph sensor.

Fig. 5 shows an example of a complete blood pressure waveform.

Fig. 6 shows an exemplary application scenario of a data processing method according to an embodiment of the present application.

Fig. 7 shows a schematic diagram of a flow of a data processing method according to an embodiment of the application.

Fig. 8 shows a schematic diagram of a data format of processed physiological data according to an embodiment of the present application.

Fig. 9 shows a schematic diagram of a data format of processed physiological data according to an embodiment of the present application.

Fig. 10 shows an exemplary structural diagram of a first neural network model according to an embodiment of the present application.

Fig. 11 shows an exemplary structural diagram of a data processing apparatus according to an embodiment of the present application.

Fig. 12 shows an exemplary structural diagram of a data processing apparatus according to an embodiment of the present application.

Detailed Description

Various exemplary embodiments, features and aspects of the application will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, numerous specific details are set forth in the following description in order to provide a better illustration of the application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application.

Several prior art blood pressure information detection schemes are described below.

Prior art one proposes a blood pressure information detection scheme based on a micro air bag, and fig. 1 shows an example of a wearable device to which the scheme of prior art one is applied. As shown in fig. 1, taking a smart watch as an example, the micro air bag and the watch body form a wearable smart watch, the air bag is inflated to press the blood vessel of the wrist of the user, and the diastolic pressure and the systolic pressure of the user are determined through the change of the air bag pressure detected in the slow deflation process. The disadvantage of this solution is that the inflation pressure makes the user uncomfortable and that only the systolic and diastolic blood pressure information amounts are relatively limited.

The second prior art proposes a blood pressure information detection scheme based on an electrocardiogram (electrocardiograph, ECG) sensor and a photoplethysmograph (photoplethysmograph, PPG) sensor. The electrocardiogram sensor and the photoplethysmograph sensor may be provided on the same wearable device.

Fig. 2 shows an example of the way in which a wearable device provided with an electrocardiogram sensor is used. As shown in fig. 2, the electrocardiogram sensor generally includes two electrodes E1 and E2, one is located at the bottom of the wearable device and contacts with the skin (E1), and the other is located at the side (E2) of the wearable device, and when detecting electrocardiographic data, a user needs to contact the electrode E2 at the side of the wearable device with a finger, so that a human body and the two electrodes form a loop arm path, and thus an electrical signal released by heart activity of the user, namely electrocardiographic data, can be detected through the electrodes. Fig. 3 shows an example of electrocardiographic data acquired by a prior art second approach. As shown in fig. 2, the electrocardiographic data may be periodic, and may include a plurality of feature segments within a period, such as RR intervals, etc., each corresponding to a particular cardiac activity.

The photoplethysmograph sensor typically contains multiple sets of light emitting diodes (LIGHT EMITTING diodes, LEDs) and Photo Diodes (PDs), fig. 4 shows an example of a wearable device provided with the photoplethysmograph sensor.

As shown in fig. 4, the light emitting diode LED emits visible light to penetrate the skin and blood vessels to reach the red blood cells, the light reflected by the red blood cells (photoplethysmography data) is received by the photodiode PD, and the absorption and reflectance of light by the oxygenated hemoglobin and the anoxic hemoglobin in the red blood cells are different, and the difference is present in the photoplethysmography data, so that pulse fluctuation, heart rate and blood oxygen saturation can be calculated based on the photoplethysmography data.

Pulse TRANSIT TIME (PTT) can be calculated based on the synchronized electrocardiographic data and photoplethysmographic data, which has a potential correlation with blood pressure, and systolic and diastolic pressures can be derived based on the pulse transit time.

The disadvantage of this solution is that the electrocardiographic data and photoplethysmographic data are not completely synchronized at the sampling point when determining the pulse transit time, and therefore the accuracy of the pulse transit time cannot be guaranteed. And only systolic and diastolic blood pressure information amounts are relatively limited.

Fig. 5 shows an example of a complete blood pressure waveform. As shown in fig. 5, the abscissa of the complete blood pressure waveform is time, the ordinate is pressure (mmHG), and the systolic pressure and the diastolic pressure are two points of peak value and valley value in the blood pressure waveform, respectively, and the complete blood pressure waveform can provide richer blood pressure fluctuation information, so that compared with the systolic pressure and the diastolic pressure, the blood pressure waveform data information is richer and more accurate.

In this regard, the embodiment of the application provides a data processing method, a data processing device and a computer readable storage medium. The method can also be applied to the portable equipment to improve the use experience of the user.

As shown in fig. 6, the data processing method of the embodiment of the present application may be applied to an electronic device, for example, the electronic device of the present application may be a wearable electronic device (such as a smart bracelet, a smart watch, etc.), and a small electronic device that is portable, such as a smart phone, etc.

The user may wear a wearable electronic device on which sensors for acquiring physiological data, such as an electrocardiograph sensor acquiring electrocardiographic data and/or a photoplethysmograph sensor acquiring photoplethysmographic data, and at least one of a Pressure Sensor (PS), a Magnetic Sensor (MS), a resonant probe sensor (resonant probe sensor, RPS) or the like, which acquire pulse vibration mechanical wave data, may be provided. Alternatively, the sensor for acquiring the pulse vibration mechanical wave data may be a millimeter wave radar or the like provided on a device in the environment where the user is located. The principle of the sensors for acquiring pulse vibration mechanical wave data is to detect micro mechanical waves caused by the contraction and expansion of blood vessels on the body surface, so that the acquired pulse vibration mechanical wave data has correlation with blood vessel pressure to a certain extent.

Optionally, there may be more kinds of physiological data, for example, heart sound data, heart vibration data, heart impedance data, blood flow velocity data, pulse pressure wave data, etc., and the sensor for collecting the above physiological data may be implemented based on the prior art, which is not described herein.

The data processing method according to the embodiment of the present application may be executed on the same electronic device where the sensor is located, or may be executed on a different electronic device, which is not limited by the present application.

With the development of artificial intelligence algorithms, the development of artificial intelligence algorithms is mature and applied to a plurality of fields, such as: face recognition, voice recognition, intelligent driving, and the like. Different fields focus on different algorithm categories, such as a main face recognition method based on a convolutional neural network and a main voice recognition method based on a recurrent neural network. The artificial intelligence algorithm can automatically discover the complex relation between input data and output data through a large amount of data training, so that the difficulty of manual processing and analysis is reduced. Practice in some fields proves that the accuracy and reliability of the artificial intelligence algorithm can meet the actual scene requirements. Based on the above, the embodiment of the application sets a trained first neural network model on the electronic device executing the data processing method.

The request for detection of the blood pressure waveform may be initiated by the electronic device performing the data processing method, or by any electronic device capable of communicating with the electronic device performing the data processing method, and may be initiated actively by the user, a wearable electronic device being exemplified in fig. 7. The initiation condition may be preset, for example, automatically initiated after the wearable electronic device detects that the user movement is finished, or automatically initiated at a preset time point, etc., which the present application is not limited to. When receiving the detection request, the electronic device executing the data processing method can acquire various physiological data acquired by the electronic device from each sensor, and obtain a first detection result indicating the blood pressure waveform of the user according to the various physiological data and the first neural network model. The first detection result may be displayed on a display screen of the electronic device performing the data processing method or transmitted to any electronic device capable of communicating with the electronic device performing the data processing method and then displayed on the display screen. The application is not limited in this regard.

As shown in fig. 7, in one possible implementation manner, an embodiment of the present application provides a data processing method, which includes steps S81-S84:

In step S81, upon receiving the detection request, a plurality of physiological data acquired by the plurality of sensors during the first period of time is acquired.

Examples of sources and multiple sensors for detection requests are described above and in relation to fig. 7. The starting point of the first time period may be determined according to the time of receiving the detection request, or according to a timestamp carried by the detection request itself, the duration of the first time period may be preset, and the end point of the first time period may be the sum of the starting point of the first time period and the duration, that is, a period of time from the starting point of the first time period to the end point of the first time period. Each sensor may be considered to acquire one physiological data, and in one possible implementation, the plurality of physiological data includes at least one of electrocardiographic data, photoplethysmographic data, heart sound data, heart vibration data, heart impedance data, blood flow velocity data, and at least one of pulse vibration mechanical wave data, pulse pressure wave data. The acquisition method can be seen from the description of fig. 7.

Step S82, processing the multiple physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and the number of data points of the different physiological data is the same.

Because the multiple physiological data are collected in the first time period, when the physiological data of different types in the processed physiological data are synchronous and the number of the data points of the physiological data of different types is the same, each data point in each physiological data can be considered to correspond to a unique one of the data points of other physiological data, so that the association relation of the multiple physiological data can be established. Exemplary implementations thereof may be seen in fig. 8, 9 and further description of step S82 below.

Step S83, the processed physiological data is input into the first neural network model. The first neural network model may be a pre-trained neural network model, an exemplary acquisition of which may be found in the related description below. The processed physiological data may be input in a sliding window manner as it is input to the first neural network model, an exemplary implementation of which may be seen in the further description of step S83 below.

Step S84, a first detection result output by the first neural network model is obtained, and the first detection result indicates a blood pressure waveform of the user. The first detection result is displayed through the display screen, so that the user can acquire the blood pressure waveform information of the user.

An exemplary implementation of processing the plurality of physiological data to obtain processed physiological data (step S82) is described below.

Optionally, because the method of the embodiment of the application uses the first neural network model, when acquiring multiple physiological data acquired by the multiple sensors in the first time period, the multiple physiological data acquired by the multiple sensors can be synchronized, so that the synchronism of the processed multiple physiological data is ensured.

For example, in one possible implementation, the method is applied to a wearable device, step S82 includes:

synchronizing physiological data acquired by the plurality of sensors based on a clock of the wearable device when the plurality of sensors are disposed on the wearable device;

When the plurality of sensors are not arranged on the wearable device, physiological data acquired by the plurality of sensors are collected through mutual communication with the device where the plurality of sensors are arranged, and the physiological data acquired by the plurality of sensors are synchronized through communication time stamps.

For example, a hardware clock is typically provided on an electronic device, so if multiple sensors are provided on a wearable device that performs the data processing method, physiological data collected by the sensors may be synchronized based on timing of the hardware clock on the wearable device. If the plurality of sensors are not arranged on the wearable device executing the data processing method, the wearable device executing the data processing method can communicate with the device where the plurality of sensors are arranged, collect physiological data collected by the plurality of sensors, assign communication time stamps (such as receiving time stamps of the physiological data) to the physiological data in the communication process, and synchronize the physiological data collected by the plurality of sensors by using the communication time stamps. The specific manner may be implemented based on the prior art, and will not be described herein.

When the physiological data collected by the sensor is obtained, the physiological data collected by the sensor can be calibrated according to the received time stamp of the physiological data and the time stamp of the detection request. In an ideal case, the time delay for communication of physiological data between the two devices is typically less than the minimum sampling time interval of the sensor. The specific manner of calibration may be implemented based on the prior art and will not be described in detail herein.

An exemplary method for implementing the same number of data points for different types of physiological data in the processed physiological data according to an embodiment of the present application is described below.

In one possible implementation, step S82 includes:

When the sampling rate of any one physiological data is smaller than the sampling rate reference, adopting an interpolation algorithm to process the physiological data to obtain first physiological data with the sampling rate equal to the sampling rate reference;

when the sampling rate of any one physiological data is larger than the sampling rate reference, adopting a downsampling algorithm to process the physiological data to obtain second physiological data with the sampling rate equal to the sampling rate reference;

When the sampling rate of any one physiological data is equal to the sampling rate reference, taking the physiological data as third physiological data;

And obtaining the processed physiological data according to the first physiological data, the second physiological data and the third physiological data.

For example, the sampling rate of multiple sensors is typically fixed at the factory and may be non-uniform. That is, within the same first time period, if not processed, the number of data points for each of the plurality of physiological data acquired by the plurality of sensors may be different. In this case, in order to establish correspondence of various physiological data, the following processing may be performed on the various physiological data:

A sampling rate reference may be preset and the number of data points obtained when sampling at the sampling rate reference during the first period of time may be considered a desired number of data points. The sample rate criterion may be manually selected, for example, 100 sample points/second. The sampling rate reference may be set according to the number of data points that the first neural network model allows to be input at a time, which is not limited by the present application. Processing the plurality of physiological data may include aligning the sampling rate of the plurality of physiological data with the sampling rate reference, in which case the plurality of physiological data includes a number of data points equal to the number of data points required.

For example, when the sampling rate of any one of the physiological data is less than the sampling rate reference, the number of data points of the physiological data in the first period can be considered to be less than the required number of data points, and the physiological data can be processed by adopting an interpolation algorithm in the prior art, so that the number of data points of the physiological data in the first period is increased until the required number of data points is reached, and the first physiological data with the sampling rate equal to the sampling rate reference is obtained. Conversely, when the sampling rate of any one physiological data is greater than the sampling rate reference, the number of data points of the physiological data in the first time period can be considered to be greater than the required number of data points, and the physiological data can be processed by adopting a downsampling algorithm in the prior art, so that the number of data points of the physiological data in the first time period is reduced until the required number of data points is reached, and the second physiological data with the sampling rate equal to the sampling rate reference is obtained. Any kind of physiological data with the sampling rate equal to the sampling rate reference can be no longer processed, and the physiological data is taken as third physiological data. The processed physiological data can be obtained according to the first physiological data, the second physiological data and the third physiological data.

Another exemplary method for implementing the same number of data points for different types of physiological data in the processed physiological data according to an embodiment of the present application is described below.

In one possible implementation, step S82 includes:

Determining a first number of data points at a sampling rate reference sampling over a first period of time;

Adding zero data points to any physiological data when the number of data points of the physiological data is smaller than the first data point number until the sum of the number of data points of the physiological data and the added data point number is equal to the first data point number;

the processed physiological data is obtained from such physiological data after adding a zero-valued data point.

For example, as described above, based on a preset sampling rate reference, it may be determined that the number of data points obtained when sampling at the sampling rate reference during the first period of time is the required number of data points (i.e., the first number of data points). Processing the plurality of physiological data may include directly aligning the number of data points of the plurality of physiological data with the first number of data points regardless of the sampling rate. The first number of data points may also be selected manually, for example based on the number of neurons of the first neural network output layer.

For example, when the number of data points of any one physiological data is smaller than the first number of data points, zero-valued data points may be added to the physiological data until the sum of the number of data points of the physiological data and the added (zero-valued) number of data points is equal to the first number of data points. Accordingly, if the number of data points of any one of the physiological data is greater than the first number of data points, a portion of the data points may be deleted until the number of data points after deletion is equal to the first number of data points. For physiological data having a number of data points equal to the first number of data points, no processing may be performed. After the above processing is performed on the plurality of kinds of physiological data, the processed physiological data can be obtained according to the physiological data after adding the data point with zero value, the physiological data after deleting part of the data points and the physiological data which is not processed. The specific location of the addition of zeros, or the specific location of the deletion of data points, may be set as desired.

It should be understood by those skilled in the art that the method for achieving the same number of data points of different kinds of physiological data in processed physiological data, which can be achieved based on the prior art, can be used in the embodiment of the present application, which is not limited thereto.

In one possible implementation, the obtaining the processed physiological data according to the first physiological data, the second physiological data and the third physiological data includes:

And performing filtering processing and/or baseline drift removal processing on the first physiological data, the second physiological data and the third physiological data by using at least one of a band-pass filtering algorithm, a wavelet decomposition algorithm and a mode decomposition algorithm to obtain processed physiological data.

For example, there may be motion noise and hardware noise in the physiological data collected by the sensor, and there may also be baseline drift, for which, a band-pass filtering algorithm, a wavelet decomposition algorithm, a mode decomposition algorithm, and so on in the prior art may be used to perform filtering processing and/or baseline drift removal processing on the first physiological data, the second physiological data, and the third physiological data, to remove noise and/or baseline drift, and then to use the processed physiological data.

It should be understood by those skilled in the art that the filtering processing and/or the baseline shift removing processing may be performed on the physiological data after the data points with zero added value and the physiological data after the partial data points are deleted and the physiological data which is not processed, or the processing of aligning the sampling rate standard or aligning the number of the first data points may be performed on the physiological data after the filtering processing and/or the baseline shift removing processing, which are performed on the physiological data after the filtering processing and/or the baseline shift removing processing, so long as each data point in the processed physiological data can correspond to only one of the data points of other physiological data, and the number and the sequence of the processing steps in the processing process of the physiological data are not limited.

In this way, the processed physiological data is made more accurate.

In one possible implementation manner, the obtaining the processed physiological data according to the first physiological data, the second physiological data and the third physiological data further includes:

And screening each data point in the first physiological data, the second physiological data and the third physiological data according to the threshold value corresponding to the physiological data to obtain processed physiological data.

For example, various physiological data are associated with physiological activities of the human body, and the physiological activities of the human body are subject to a limit value, so that if the acquired physiological data exceed the limit value, the acquired data exceeding the limit value may be considered to be inaccurate. For this, each data point in the physiological data may be screened for each physiological data in the plurality of physiological data according to a threshold value corresponding to the physiological data, for example, a data point exceeding a limit value is removed, and the processed physiological data is obtained.

For example, for electrocardiographic data, the corresponding threshold may be a human heart rate limit value, heart rate information corresponding to electrocardiographic data may be identified according to an R-R interval in electrocardiographic data, electrocardiographic data may be screened according to the human heart rate limit value, and data greater than the human heart rate limit value may be screened. In this way, the processed physiological data is made more accurate.

It should be understood by those skilled in the art that, after screening multiple physiological data, the screened multiple physiological data may be subjected to the above-mentioned process of aligning the sampling rate reference or aligning the number of the first data points, and/or the filtering process and the baseline drift removal process, so long as each data point in each physiological data in the processed physiological data can correspond to a unique one of the data points of other physiological data.

An example of the data format of the processed physiological data is described below. Fig. 8 shows a schematic diagram of a data format of processed physiological data according to an embodiment of the present application.

In one possible implementation, as shown in fig. 8, the processed physiological data is multi-channel one-dimensional data, and the one-dimensional data of each channel is obtained according to the physiological data acquired by a corresponding one of the sensors.

For example, the data format of the processed physiological data may be similar to the data format of the image data, for example, the RGB image data includes three-channel two-dimensional data, and the processed physiological data of the embodiment of the present application may be multi-channel one-dimensional data, where the number of channels is equal to the number of sensors, and the plurality of channels corresponds to the plurality of sensors one by one. In this case, the one-dimensional data of each channel can be obtained from the physiological data acquired by the corresponding one of the sensors. The data dimension of the one-dimensional data may be a time dimension. In the example of fig. 8, each square represents one data point of physiological data, one-dimensional data of channel 1 can be obtained from physiological data acquired by sensor 1, one-dimensional data of channel 2 can be obtained from physiological data acquired by sensor 2, and one-dimensional data of channel 3 can be obtained from physiological data acquired by sensor 3. The data points of the physiological data in the channels may be ordered in order of sampling time from low to high.

In one possible implementation, as shown in fig. 9, the processed physiological data is single-channel two-dimensional data, where the data in the first dimension is the sampling time of the physiological data and the data in the second dimension is the physiological data acquired by the plurality of sensors.

For example, the processed physiological data may be synthesized into two-dimensional data with a single channel, the data in the first dimension is the sampling time of the physiological data, and the data in the second dimension is the physiological data acquired by the plurality of sensors. In the example of fig. 9, each square represents one data point of physiological data, each column of data points having the same sampling time, the columns of data points being ordered in order of sampling time from low to high. The data points of each row are from the same sensor, and the application is not limited to the order of the data points of each row.

An exemplary method for inputting the processed physiological data into the first neural network model in a sliding window manner (step S83) is described below.

In one possible implementation, step S83 includes:

determining a plurality of time windows according to a preset first window length and window step length, wherein two adjacent time windows are not completely overlapped;

And sequentially inputting the processed physiological data in a plurality of time windows into the first neural network model.

For example, a first window length and window step size may be preset, e.g. the first window length equals 10 seconds and the window step size equals 5 seconds. Assuming that the first neural network model allows 2000 data points to be input (i.e., the input layer of the first neural network model has 2000 neurons), the first time period is 30 seconds long, and when the number of data points of physiological data per second is 200 seconds, the plurality of time windows determined according to the preset first window length and window step length may be, for example, 0-10 seconds, 5-15 seconds, and 10-20 seconds … … -30 seconds. It can be seen that adjacent two time windows are not completely overlapping. And then, sequentially inputting the processed physiological data in a plurality of time windows into the first neural network model.

Accordingly, in one possible implementation, step S84 includes:

And acquiring first detection results sequentially output by the first neural network model, wherein each output first detection result indicates a blood pressure waveform in a preset second window length.

As described above, since the processed physiological data is sequentially input into the first neural network model, when the first detection result shown by the first neural network model is acquired, the first detection result may be sequentially acquired, where each acquired first detection result may also correspond to a time window, and the time window may correspond to the time window in which the physiological data of the first neural network model is input. The length of the time window corresponding to the first detection result may be equal to the second window length, and the second window length may be equal to the first window length, or may be greater than or less than the first time window length. The first detection result also corresponds to a window step length, and the window step length can be the same as or different from the window step length of the processed physiological data, so long as the time windows corresponding to the adjacent first detection results also meet the condition of incomplete overlapping (namely partial overlapping).

An exemplary training method of the first neural network model of an embodiment of the present application is described below.

In one possible implementation, the method further includes:

Acquiring sample data, wherein the sample data comprises a plurality of physiological data acquired by a plurality of sensors in a second time period;

Processing the sample data to obtain processed sample data, wherein the processed sample data comprise different types of physiological data which are synchronous and have the same number of data points;

inputting the processed sample data into a second neural network model, wherein the second neural network model is an untrained model;

According to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain a first neural network model;

The data points output by the second neural network model each time are consistent with the data points of the real blood pressure waveform corresponding to the sample data.

For example, the sample data may include a plurality of physiological data acquired by a plurality of sensors during the second time period, wherein the plurality of sensors may be different from the plurality of sensors in step S81. The duration of the second time period and the first time period may also be different. The sample data acquisition mode may be the same as the acquisition mode of various physiological data when the detection request is received, and an exemplary implementation thereof may be referred to the above description of step S81. The actual blood pressure waveform to which the sample data corresponds may be known.

For the manner of processing the sample data to obtain processed sample data, reference may be made to the manner of processing the plurality of physiological data to obtain processed physiological data in step S82 above. The data format of the processed sample data may be referred to as examples of fig. 8 or 9.

The processed sample data may be input to a second neural network model, which may be an untrained model. The processed sample data may also be input to the second neural network model in a sliding window manner, and an exemplary implementation thereof may be referred to above as a sliding window manner in which the processed physiological data is input to the first neural network model.

According to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, parameters of the second neural network model can be adjusted to obtain the first neural network model. Exemplary implementations thereof may be found in the related description below.

The second neural network model may include a variety of neural network models, such as a convolutional neural network model, a generative neural network model, and so forth. There are various ways of adjusting the parameters of the second neural network model according to the structure of the second neural network model. One exemplary manner of adjusting the parameters of the second neural network model is described below.

In one possible implementation manner, according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain the first neural network model, including:

determining regression loss according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data;

and adjusting parameters of the second neural network model according to the regression loss to obtain a first neural network model.

For example, the second neural network model is an untrained model, but also has a certain detection capability, and by continuously adjusting parameters of the second neural network model, the output of the second neural network model can be more and more close to the real blood pressure waveform. For example, in each parameter adjustment process, the regression loss of the sample data may be determined according to the difference between the current output of the second neural network model and the actual blood pressure waveform corresponding to the sample data input to the second neural network model, and the specific determination manner may be implemented based on the prior art. The parameters of the second neural network model may be adjusted according to the regression loss, and the above-described process may be repeated until the degree of similarity of the output of the second neural network model and the actual blood pressure waveform corresponding to the sample data meets the requirement (e.g., the regression loss is less than a certain threshold), and the first neural network model is obtained according to the newly adjusted parameters.

The generation of an antagonistic neural network model (GENERATIVE ADVERSARIAL network, GAN) is a more emerging neural network model, and can generate output data with potential correlation with the input data based on the input data, so far, the method has been widely applied to the fields of image restoration, super resolution, data denoising and the like. The generation of the antagonistic neural network model comprises a generator and a discriminator, wherein the generator is used for generating false samples close to real samples according to input data. The false sample output by the generator and the corresponding real sample are input into a discriminator, and the discriminator gives a discrimination result. By alternately training the generator and the arbiter, the false samples generated by the generator may be more nearly real samples. One exemplary manner in which embodiments of the present application adjust parameters of a second neural network model based on generating an antagonistic neural network model is described below.

In one possible implementation manner, when the second neural network model is implemented based on generating the antagonistic neural network model, according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain the first neural network model, including:

determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the generator in the generated antagonistic neural network model;

Determining cross entropy loss of sample data according to a discrimination result output by a discriminator in the generated countermeasure neural network model;

Determining the weighted loss of the sample data according to the regression loss, the first weight corresponding to the regression loss, the cross entropy loss and the second weight corresponding to the cross entropy loss;

And adjusting parameters for generating the antagonistic neural network model according to at least one of the regression loss and the weighted loss, and taking the obtained generator as a first neural network model.

For example, when the second neural network model is implemented based on generating the antagonistic neural network model, the output of the generator may be a blood pressure waveform corresponding to the sample data that is not accurate enough, and the output of the discriminator may be a discrimination result of the output of the second neural network model to a real blood pressure waveform corresponding to the sample data. In each parameter adjustment process, the regression loss of the sample data can be determined according to the difference of the current output of the generator and the actual blood pressure waveform corresponding to the sample data input into the generator, and the specific determination mode can be realized based on the prior art. And then determining the cross entropy loss of the sample data according to the discrimination result output by the discriminator, wherein the specific determination mode can be realized based on the prior art. The first weight corresponding to the regression loss and the second weight corresponding to the cross entropy loss may be preset, the weighted loss of the sample data may be determined according to the regression loss, the first weight corresponding to the regression loss, the cross entropy loss, and the second weight corresponding to the cross entropy loss, for example, the product of the regression loss and the first weight corresponding to the regression loss and the product of the cross entropy loss and the second weight corresponding to the cross entropy loss may be determined, and the weighted loss of the sample data may be obtained according to the sum of the two products. The sum of the first weight and the second weight may be preset to 1 or may be preset to other values, which is not limited in the present application.

Parameters for generating the antagonistic neural network model may be adjusted according to at least one of the regression loss and the weighting loss, and the above-described process may be repeated until the degree of similarity of the output of the generator to the actual blood pressure waveform corresponding to the sample data reaches the requirement (e.g., the regression loss and/or the weighting loss is less than a certain threshold), and the generator obtained according to the newly adjusted parameters is used as the first neural network model.

Fig. 10 shows an exemplary structural diagram of a first neural network model according to an embodiment of the present application. An exemplary method for obtaining the first detection result by the first neural network model according to the embodiment of the present application is described below with reference to fig. 10.

In one possible implementation, the first neural network model includes at least one inference unit, each inference unit being connected in series and/or parallel,

When the first layer of the first neural network model is an inference unit, the processed physiological data is input into the inference unit;

when the first layer of the first neural network model is a plurality of inference units connected in parallel, the processed physiological data are respectively input into the plurality of inference units;

For two inference units in a series connection relationship, the output of the former inference unit is used as the input of the latter inference unit;

For a plurality of inference units connected in parallel, wherein a series or weighted summation result of the outputs of each of the inference units is taken as a common output of the plurality of inference units;

when the last layer of the first neural network model is an inference unit, the output of the inference unit is used as the first detection result;

When the last layer of the first neural network model is a plurality of inference units connected in parallel, the common output of the plurality of inference units is used as the first detection result.

For example, the first neural network model may include at least one layer, and each layer may be one inference unit, or may be a plurality of inference units connected in parallel. One or more of these inference units may correspond to a complete neural network model, such as the convolutional neural network model described above, a generator in generating an antagonistic neural network model, and so forth.

When the first layer of the first neural network model is an inference unit, the processed physiological data is input to the inference unit. When the last layer of the first neural network model is an inference unit, the output of the inference unit is used as the first detection result. As shown in fig. 10, in example (a), assuming that the first neural network model includes all of the first inference unit, the second inference unit, … …, and the nth inference unit (N is an integer greater than 2) connected in series, in this case, it is equivalent to the first neural network model including N layers, where the first layer is one inference unit (first inference unit) and the last layer is one inference unit (nth inference unit), the processed physiological data may be input to the first inference unit, and the output of the nth inference unit is the first detection result.

Wherein, for two inference units in a series connection, the output of the former inference unit is used as the input of the latter inference unit.

Similarly, in example (b), corresponding to the first neural network model including layer 1, the processed physiological data may be input to the first inference unit, and the output of the first inference unit may be used as the first detection result.

In example (c), assuming that only the third inference unit and the fourth inference unit are connected in parallel, the other inference units are connected in series, which corresponds to the first neural network model including N-1 layers, wherein layer 3 includes the third inference unit and the fourth inference unit connected in parallel. The processed physiological data may be input to a first inference unit, and the output of the nth inference unit may be used as the first detection result. Wherein, for two inference units in a series connection relationship, the output of the former inference unit is used as the input of the latter inference unit; for a plurality of inference units in a parallel connection relationship, the same input (i.e., the output of the second inference unit) is respectively received, and a plurality of prediction results are respectively output, where the plurality of prediction results may be first connected in series (such as adopting an end-to-end mode of every two prediction results to obtain a sequence or an array, and then used as the input of the next inference unit (a fifth inference unit, not shown); or the multiple prediction results may be weighted and summed according to weights corresponding to the multiple inference units, so as to obtain a weighted and summed result as an input of a next inference unit (a fifth inference unit, not shown).

When the first layer of the first neural network model is a plurality of reasoning units connected in parallel, the processed physiological data are respectively input into the plurality of reasoning units; when the last layer of the first neural network model is a plurality of inference units connected in parallel, the common output of the plurality of inference units is used as a first detection result. For example, in example (d), it is equivalent to the first neural network model including layer 1, and the layer 1 includes a first inference unit and a second inference unit connected in parallel. In this case, for the plurality of inference units connected in parallel, the same input (i.e., the processed physiological data) is received respectively, and a plurality of prediction results are output respectively, and the plurality of prediction results may be first connected in series or weighted and summed to obtain a weighted and summed result, which is then output as the first detection result.

Correspondingly, the second neural network model can also comprise at least one unit to be trained, and after training is completed, the units to be trained can respectively become inference units. The inference units are connected in series and/or in parallel, so that the units to be trained are also connected in series and/or in parallel, the connection mode can realize the respective training of the units to be trained, and the process of adjusting the parameters of the second neural network model each time can be a process of adjusting the parameters of part or all of the units to be trained in the units to be trained. Optionally, multiple units to be trained can also be trained together, and the specific training mode of the multiple units to be trained is not limited by the application.

An embodiment of the present application provides a data processing apparatus, and fig. 11 shows an exemplary structural diagram of the data processing apparatus according to an embodiment of the present application.

As shown in fig. 11, the apparatus includes:

A first acquisition module 91, configured to acquire, when a detection request is received, multiple physiological data acquired by a plurality of sensors during a first period of time;

The first processing module 92 is configured to process the multiple types of physiological data to obtain processed physiological data, where the processed physiological data are synchronous among different types of physiological data and the number of data points of the different types of physiological data is the same;

a first input module 93 for inputting the processed physiological data into a first neural network model;

The second obtaining module 94 is configured to obtain a first detection result output by the first neural network model, where the first detection result indicates a blood pressure waveform of the user.

In one possible implementation, the plurality of physiological data includes at least one of electrocardiographic data, photoplethysmographic data, heart sound data, heart vibration data, heart impedance data, blood flow rate data, and at least one of pulse vibration mechanical wave data, pulse pressure wave data.

In one possible implementation, the processed physiological data is multi-channel one-dimensional data, and the one-dimensional data of each channel is obtained according to the physiological data acquired by a corresponding sensor.

In one possible implementation, the processed physiological data is single-channel two-dimensional data, wherein the data in the first dimension is the sampling time of the physiological data and the data in the second dimension is the physiological data acquired by the plurality of sensors.

In one possible implementation, the first input module includes:

The first determining unit is used for determining a plurality of time windows according to a preset first window length and window step length, wherein two adjacent time windows are not completely overlapped;

and the input unit is used for sequentially inputting the processed physiological data in the time windows into the first neural network model.

In one possible implementation, the apparatus further includes:

A third acquisition module for acquiring sample data comprising a plurality of physiological data acquired by a plurality of sensors over a second time period;

the second processing module is used for processing the sample data to obtain processed sample data, wherein the processed sample data comprise different types of physiological data which are synchronous and have the same number of data points;

A second input module for inputting the processed sample data into a second neural network model, the second neural network model being an untrained completed model;

The first adjustment module is used for adjusting parameters of the second neural network model according to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data to obtain the first neural network model;

And the data points output by the second neural network model each time are consistent with the data points of the real blood pressure waveform corresponding to the sample data.

In one possible implementation manner, the first adjusting module includes:

the second determining unit is used for determining regression loss of the sample data according to the real blood pressure waveform corresponding to the sample data and output by the second neural network model;

And a third determining unit, configured to adjust parameters of the second neural network model according to the regression loss, so as to obtain the first neural network model.

In one possible implementation, the first processing module includes:

The first processing unit is used for processing any physiological data by adopting an interpolation algorithm when the sampling rate of the physiological data is smaller than a sampling rate reference to obtain first physiological data with the sampling rate equal to the sampling rate reference;

the second processing unit is used for processing any physiological data by adopting a downsampling algorithm when the sampling rate of the physiological data is larger than the sampling rate reference to obtain second physiological data with the sampling rate equal to the sampling rate reference;

A third processing unit configured to take any one of the physiological data as third physiological data when a sampling rate of the physiological data is equal to a sampling rate reference;

And a fourth determining unit, configured to obtain the processed physiological data according to the first physiological data, the second physiological data, and the third physiological data.

In one possible implementation, the first processing module includes:

A fifth determining unit configured to determine a first number of data points when sampling at a sampling rate reference in the first period;

An adding unit, configured to add a zero-valued data point to any one of the physiological data when the number of data points of the physiological data is smaller than the first number of data points, until the sum of the number of data points of the physiological data and the added number of data points is equal to the first number of data points;

A sixth determining unit for obtaining the processed physiological data from the physiological data after adding the data point with zero value.

In one possible implementation, the apparatus is applied to a wearable device, and the first processing module includes:

the first synchronization unit is used for synchronizing physiological data acquired by the plurality of sensors based on clocks of the wearable device when the plurality of sensors are arranged on the wearable device;

And the second synchronization unit is used for collecting physiological data acquired by the plurality of sensors through mutual communication with equipment where the plurality of sensors are positioned when the plurality of sensors are not arranged on the wearable equipment, and synchronizing the physiological data acquired by the plurality of sensors through communication time stamps.

In one possible implementation manner, the second obtaining module includes:

In one possible implementation, the first adjustment module includes:

a seventh determining unit, configured to determine a regression loss of the sample data according to a real blood pressure waveform corresponding to the sample data generated by the output of the generator in the antagonistic neural network model;

An eighth determining unit for determining cross entropy loss of the sample data according to the discrimination result output by the discriminator in the generated countermeasure neural network model;

a ninth determining unit, configured to determine a weighted loss of the sample data according to the regression loss, a first weight corresponding to the regression loss, the cross entropy loss, and a second weight corresponding to the cross entropy loss;

And a tenth determination unit for adjusting parameters for generating an antagonistic neural network model according to at least one of the regression loss and the weighted loss, the resulting generator being the first neural network model.

An embodiment of the present application provides a data processing apparatus including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions.

Embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

Embodiments of the present application provide a computer program product comprising a computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.

As shown in fig. 12, the data processing apparatus may include at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a Personal Digital Assistant (PDA), an augmented reality (augmented reality, AR) device, a Virtual Reality (VR) device, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) device, a wearable device, a vehicle-mounted device, a smart home device, or a smart city device, a server device. The embodiment of the present application is not particularly limited as to the specific type of the data processing apparatus.

The data processing apparatus may include a processor 110, a memory 121, and a communication module 160. It will be appreciated that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the data processing apparatus. In other embodiments of the application, the data processing apparatus may include more or less components than those illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural-Network Processor (NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The processor can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 may be a cache memory. The memory may hold instructions or data that are used or used more frequently by the processor 110, such as processed physiological data in embodiments of the present application. If the processor 110 needs to use the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

Memory 121 may be used to store computer-executable program code that includes instructions. The memory 121 may include a stored program area and a stored data area. The storage program area may store an application program (such as a first neural network model) required for at least one function, etc. of the operating system. The storage data area may store data (such as a first detection result, etc.) acquired or created during use of the data processing apparatus, and the like. In addition, the memory 121 may include a high-speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like. The processor 110 performs various functional methods of the data processing apparatus or the above-described data processing method by executing instructions stored in the memory 121 and/or instructions stored in a memory provided in the processor.

The communication module 160 may be used to receive data (e.g., physiological data in embodiments of the present application) from other devices or apparatuses and to output data to other devices or apparatuses by way of wireless/wired communication. Solutions may be provided for wireless communication including WLAN (e.g., wi-Fi network), bluetooth (BT), global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), near Field Communication (NFC), infrared (IR), etc.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disk, hard disk, random Access Memory (Random Access Memory, RAM), read Only Memory (ROM), erasable programmable Read Only Memory (ELECTRICALLY PROGRAMMABLE READ-Only-Memory, EPROM or flash Memory), static Random Access Memory (SRAM), portable compact disk Read Only Memory (Compact Disc Read-Only Memory, CD-ROM), digital versatile disk (Digital Video Disc, DVD), memory stick, floppy disk, mechanical coding devices, punch cards or in-groove bump structures such as instructions stored thereon, and any suitable combination of the foregoing.

The computer readable program instructions or code described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present application may be assembler instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as SMALLTALK, C ++ or the like and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN) or a wide area network (Wide Area Network, WAN), or may be connected to an external computer (e.g., through the internet using an internet service provider). In some embodiments, aspects of the application are implemented by personalizing electronic circuitry, such as Programmable logic circuitry, field-Programmable gate arrays (GATE ARRAY, FPGA), or Programmable logic arrays (Programmable Logic Array, PLA), with state information for computer-readable program instructions.

Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by hardware, such as circuits or ASIC (Application SPECIFIC INTEGRATED circuits) which perform the corresponding functions or acts, or combinations of hardware and software, such as firmware and the like.

Although the invention is described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

The foregoing description of embodiments of the application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of data processing, the method comprising:

when a detection request is received, acquiring various physiological data acquired by a plurality of sensors in a first time period;

Processing the plurality of physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and the number of data points of the different physiological data is the same;

inputting the processed physiological data into a first neural network model;

And acquiring a first detection result output by the first neural network model, wherein the first detection result indicates the blood pressure waveform of the user.

2. The method of claim 1, wherein the plurality of physiological data comprises at least one of electrocardiographic data, photoplethysmographic data, heart sound data, heart vibration data, heart impedance data, blood flow rate data, and at least one of pulse vibration mechanical wave data, pulse pressure wave data.

3. A method according to claim 1 or 2, wherein the processed physiological data is multi-channel one-dimensional data, the one-dimensional data for each channel being derived from physiological data acquired by a corresponding one of the sensors.

4. The method of claim 1 or 2, wherein the processed physiological data is single-channel two-dimensional data, wherein the data in a first dimension is a sampling time of the physiological data and the data in a second dimension is physiological data acquired by a plurality of sensors.

5. The method of any one of claims 1-4, wherein the inputting the processed physiological data into a first neural network model comprises:

and sequentially inputting the processed physiological data in the plurality of time windows into the first neural network model.

6. The method according to any one of claims 1-5, further comprising:

Obtaining sample data, the sample data comprising a plurality of physiological data acquired by a plurality of sensors over a second time period;

inputting the processed sample data into a second neural network model, the second neural network model being an untrained model;

According to the output of the second neural network model and the real blood pressure waveform corresponding to the sample data, adjusting parameters of the second neural network model to obtain the first neural network model;

7. The method of claim 6, wherein adjusting parameters of the second neural network model based on the output of the second neural network model and the actual blood pressure waveform corresponding to the sample data to obtain the first neural network model comprises:

determining regression loss according to the real blood pressure waveform corresponding to the sample data output by the second neural network model;

And adjusting parameters of the second neural network model according to the regression loss to obtain the first neural network model.

8. The method according to any of the claims 1-7, characterized in that the first neural network model comprises at least one inference unit, each inference unit being connected in series and/or in parallel,

9. A data processing apparatus, the apparatus comprising:

The first acquisition module is used for acquiring various physiological data acquired by the plurality of sensors in a first time period when a detection request is received;

The first processing module is used for processing the plurality of physiological data to obtain processed physiological data, wherein the processed physiological data are synchronous among different physiological data and have the same number of data points of different physiological data;

The first input module is used for inputting the processed physiological data into a first neural network model;

The second acquisition module is used for acquiring a first detection result output by the first neural network model, and the first detection result indicates the blood pressure waveform of the user.

10. A data processing apparatus, comprising:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the method of any of claims 1-8 when executing the instructions.

11. A non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-8.