WO2021258882A1

WO2021258882A1 - Recurrent neural network-based data processing method, apparatus, and device, and medium

Info

Publication number: WO2021258882A1
Application number: PCT/CN2021/093152
Authority: WO
Inventors: 康焱; 张天豫; 刘洋; 陈天健
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2020-06-24
Filing date: 2021-05-11
Publication date: 2021-12-30
Also published as: CN111737922A

Abstract

A recurrent neural network-based data processing method, apparatus, and device, and a medium. The method comprises: obtaining time sequence data to be processed, and inputting the time sequence data to be processed to a data processing model, the data processing model being obtained by performing federated forward training on recurrent neural network models at difference moments on the basis of time sequence data of a user at different moments and then performing federated backward training of the moments on the recurrent neural network models at difference moments after the federated forward training is completed (S10); and executing, on the basis of the data processing model, a preset data processing process on the time sequence data to be processed to obtain a target prediction label of the time sequence data to be processed (S20).

Description

Data processing method, device, equipment and medium based on cyclic neural network

Priority information

This application claims the priority of the Chinese patent application filed on June 24, 2020 with the application number 202010596089.8, the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a data processing method, device, equipment and medium based on a recurrent neural network.

Background technique

With the continuous development of the Internet, more and more technologies (such as distributed, blockchain, artificial intelligence, etc.) have been applied, but the Internet has also put forward higher requirements for technology, such as data processing. Requirements.

Recurrent Neural Network (RNN) models are often used to process time series data of users at different moments. Time series data refers to data arranged based on time series. Time series data of users at different moments includes the same user at different moments. Characteristic data on different data sources such as different parties or different devices. For example, the same patient may have visited several different medical institutions at different times to diagnose the same (or similar) condition. The same patient is in different conditions. Time (different moments) access to different medical institutions (different participants or different equipment) for the same disease diagnosis time series data, is the same user at different times, in different data sources such as different server characteristic data.

At present, the time series data of users at different moments are stored in different parties or different devices. For privacy protection, different participants or different devices cannot directly interact with the time series data of users at different moments, that is, each participant or Different devices cannot share user timing data at different moments for joint modeling. As a result, each participant or different device can only model based on a small amount of user timing data at different moments, and can only be based on a small amount of timing data. The data is modeled, so it takes longer to train to make the trained model reach the target performance, which in turn causes the computer to consume a lot of resources and computing power, resulting in low utilization of computer computing resources.

Summary of the invention

The main purpose of this application is to provide a data processing method, device, equipment, and medium based on a recurrent neural network, which aims to solve the problem that in the prior art, different participants or different equipment cannot directly perform user timing data at different moments. The interaction between different parties or different devices can only perform modeling based on a small amount of time-series data of users at different moments, which makes it difficult for the model obtained after modeling to achieve the expected prediction effect.

To achieve the above objective, this application provides a data processing method based on a cyclic neural network. The preset time series data refers to the time series data of a user at a different time. The data processing method based on the cyclic neural network includes:

Acquiring time series data to be processed, and inputting the time series data to be processed into a data processing model;

The data processing model is to perform federated forward training of the cyclic neural network model at different moments based on the time-series data of the user at different moments, and then perform federated feedback on the cyclic neural network model at different moments when the federated forward training is completed. Obtained after training;

Based on the data processing model, a preset data processing procedure is executed on the to-be-processed time-series data to obtain a target prediction tag of the to-be-processed time-series data.

This application also provides a data processing device based on a cyclic neural network. The preset time series data refers to the time series data of a user at a different time. The data processing device based on the cyclic neural network includes:

The first acquisition module is configured to acquire time series data to be processed, and input the time series data to be processed into the data processing model;

The second acquisition module is configured to execute a preset data processing procedure on the to-be-processed time-series data based on the data processing model to obtain the target prediction tag of the to-be-processed time-series data.

This application also provides a data processing device based on a recurrent neural network. The data processing device based on a recurrent neural network is a physical device. The data processing device based on a recurrent neural network includes: a memory, a processor, and The program of the data processing method based on the recurrent neural network that can be executed on the memory and can be run on the processor, and when the program of the data processing method based on the recurrent neural network is executed by the processor, the recurrent neural network based Steps of network data processing method.

The application also provides a medium storing a program for realizing the above-mentioned data processing method based on cyclic neural network. When the program of the data processing method based on cyclic neural network is executed by a processor, the above-mentioned loop-based Steps of neural network data processing method.

This application obtains time series data to be processed, and inputs the time series data to be processed into a data processing model; the data processing model is fed forward training of the recurrent neural network model at different times based on the time series data of the user at different times After that, the cyclic neural network model at different moments when the federated forward training is completed is obtained after federated reverse training is performed at each moment; based on the data processing model, the preset data processing flow is executed on the to-be-processed time series data to obtain The target prediction label of the time series data to be processed. Compared with modeling based on a small amount of time series data in the prior art, and then processing the time series data to be processed, in this application, after the time series data to be processed is obtained, based on the training completed, the data processing model The time series data executes the preset data processing procedure to obtain the technical means of the target prediction label of the time series data to be processed, because the data processing model is based on the time series data of the user at different times to federate the cyclic neural network models at different times After forward training, the cyclic neural network model at different moments of the federated forward training is obtained after federated reverse training at each moment, and then while protecting privacy, it realizes the federation construction model based on time series data at different moments. , Which overcomes the defects in the prior art that consumes a large number of computer computing resources, reduces the time it takes for the trained model to reach the target performance, and improves the utilization of computer computing resources. In this application, it is based on the federation of time series data at different times. The model is constructed, thus improving the accuracy of predicting the types of time series data to be processed, and solves the problem that in the prior art, different participants or different devices cannot directly interact with the time series data of users at different moments, causing each participant to or Different devices can only perform modeling based on a small number of users' time series data at different moments, which makes it difficult for the model obtained after modeling to achieve the expected prediction effect.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments that conform to the application, and are used together with the specification to explain the principle of the application.

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic flowchart of a first embodiment of a data processing method based on a recurrent neural network according to this application;

2 is a second embodiment of the application based on the recurrent neural network data processing method based on the intermediate parameters at the last moment, the tag timing data with preset tags, and the preset predictions of the preset recurrent neural network model Model, a detailed flow diagram of the step of determining the intermediate gradient corresponding to the predicted time at the last moment;

3 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the application;

FIG. 4 is a schematic diagram of the first scene in the data processing method based on the recurrent neural network of this application;

FIG. 5 is a schematic diagram of the second scene in the data processing method based on the recurrent neural network of this application.

The realization of the purpose, functional characteristics and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

detailed description

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

The embodiment of the application provides a data processing method based on a recurrent neural network. In the first embodiment of the data processing method based on the recurrent neural network of the present application, referring to FIG. 1, the data processing method based on the recurrent neural network includes:

Step S10, acquiring time series data to be processed, and inputting the time series data to be processed into a data processing model;

Step S20: Perform a preset data processing procedure on the to-be-processed time-series data based on the data processing model to obtain a target prediction tag of the to-be-processed time-series data.

Specific steps are as follows:

In this embodiment, the data processing method based on the recurrent neural network can be applied to a data processing system based on the recurrent neural network. The data processing system based on the recurrent neural network includes a plurality of different parties or devices, and the plurality of different Among the participants or devices, the participant containing the tag timing data with preset tags is the main participant (or the main device), and the data processing method based on the recurrent neural network can also be applied to the main participant or the main device In this embodiment, the data processing method based on the recurrent neural network is applied to a data processing system based on the recurrent neural network instead of a certain device as an example for specific description. Specifically, after acquiring the time series data to be processed, the time series data to be processed is input into the data processing model of the data processing system based on the cyclic neural network, so that the data processing model processes the time series data to be processed, that is, in this embodiment In, the time series data is processed through the trained data processing model, where the time series data refers to the data arranged based on time series. Specifically, the time series data in this embodiment refers to the time series data of the user at different moments. In particular, it refers to the time series data of users at different moments in the preset sample set. The time series data of users at different moments includes different moments of the same user. The characteristic data on different data sources such as different participants or different devices needs to be explained. However, the different data sources can be of the same type. For example, the same patient may have visited several different medical institutions at different times (although the specific institutions are different, they are all types of medical institutions) to diagnose the same condition. Different medical institutions (different parties or different devices) visited by the same patient at different times (different moments) to diagnose time series data of the same condition, which is the characteristic data of the same user at different times and on different data sources such as different servers . It should be noted that, in this embodiment, the time series data of users at different moments does not include the feature data of the same user at the same time at different data sources (participants or equipment). For example, user m is in a medical institution. (Participants or equipment) purchased drug A, and a new sales record was generated on the server of the medical institution. At the same time, a new expenditure record was also generated by user m on the server of the banking system (participant or equipment). The time series data includes the characteristic data of the same user at the same time and in different data sources (participants or devices). The time series data in this embodiment does not include this situation.

In this embodiment, the data processing model is based on the user's time series data at different moments after fed forward training of the cyclic neural network model at different moments, and then the cyclic neural network model at different moments when the federated forward training is completed The target model obtained after the federated reverse training at each time is carried out. Among them, the recurrent neural network model at different moments is fed forward based on the time series data of the user at different moments, and then the federated forward training is completed at different moments. The federated reverse training of the cyclic neural network model at each moment specifically refers to: based on the user's preset time series data at different moments, such as the user's preset time series data at time t1, before the federation of the preset cyclic neural network model at time t1 Forward training to obtain the preset cyclic neural network model of the federated forward propagation at the first moment (it can be the preset cyclic neural network model at time t2), and then based on the user’s preset timing data at different moments, such as the user’s prediction at time t2 Set time series data, perform federated forward training on the preset recurrent neural network model at time t2, and obtain the preset recurrent neural network model of federated forward propagation at the second time (it can be the preset recurrent neural network model at time t3), Based on this, the federated forward training is continuously performed until the last-minute cyclic neural network model is obtained. After the last-minute cyclic neural network model is obtained, the preset time series data based on the user at the last time is not the same time, such as user t(N) time Perform the federated reverse training of the recurrent neural network model at the last moment to obtain the recurrent neural network model of the federated reverse update at the last moment, based on the preset time sequence data of the user at different moments, such as user t(N- 1) The preset time sequence data at the time, the federated reverse update of the cyclic neural network model at the last moment, and the federated reverse update of the cyclic neural network model at each subsequent time, until the preset cycle at the initial moment The neural network model is updated. It should be noted that, in this embodiment, in addition to the time series data with preset tags, the time series data involved without the preset tags is the preset time series data at different moments of the user. The data is in different devices or participants in order to obtain the parameters for federation based on the preset timing data of the users at different moments. Of course, the preset timing data of the users at different moments at different moments can also be in the same Participating parties.

In this embodiment, it needs to be emphasized that for each federated forward training and federated reverse training, they are performed in time sequence.

Wherein, before the step of executing a preset data processing procedure on the to-be-processed time-series data based on the data processing model to obtain the target prediction tag of the to-be-processed time-series data, the method includes:

Step S11, based on the user's time series data at different moments, after federated forward training is performed on the recurrent neural network model at different moments, then federated reverse training is performed on the recurrent neural network model at different moments when the federated forward training is completed. To get the joint model;

In this embodiment, the tag timing data with preset tags is acquired, where the preset tags include types such as type tags, and the tag timing data with preset tags refers to data arranged in time with preset tags.

After the tag time series data is obtained, based on the tag time series data and the time series data of the user at different moments, the cyclic neural network model at different moments is fed forward training, and then the fed forward training is completed at different moments. The recurrent neural network model performs federated reverse training at each time to obtain a joint model. Specifically, it refers to the recurrent neural network model at different times based on multiple data sources (participants or devices) at different user moments After the federated forward training is performed, based on the tag timing data and the preset timing data of users of the multiple data sources (participants or devices) at different moments, the federated forward The cyclic neural network models at different moments after the training are completed perform federated reverse training at each moment to obtain a joint model, that is, the time is different, and the preset time series data of the user at different moments are the time series data of different participants, for example, the first The preset time series data at one time is in the first medical institution, and the preset time series data at the second time is in the second medical institution.

Wherein, preset federated communication is performed between each participant, and the preset time sequence data of the user at different times includes the preset time sequence data at each time;

The preset time series data of the user at different moments include preset time series data at various moments;

After the forward federated training is performed on the cyclic neural network model at different moments based on the time series data of the user at different moments, the federated reverse training is performed on the cyclical neural network model at different moments when the federated forward training is completed, The steps to get the joint model include:

Step a1, based on the received intermediate parameters of the forward propagation corresponding to the previous time at the target time, the preset time series data of the target time and the model parameters of the preset recurrent neural network model at the target time, determine the target time The forward propagation intermediate parameters of, and the federated forward training for each subsequent time based on the intermediate parameters of forward propagation at the target time, until the intermediate parameters at the final time are obtained;

In this embodiment, the intermediate parameter is the state information h, and the initial parameter is the initialization state information

The various intermediate parameters include

In this embodiment, the preset time series data of users at different moments are in different data sources or participants. Specifically, as shown in FIG. 4, the data sets X1, X2, and X3 respectively record the user k in the sample set. The timing feature generated at time t1 (in participant 1), the timing feature generated by user k at time t2 (in participant 2), and the timing feature generated by user k at time t3 (in participant 3) For the sample set, it includes multiple users. The data sets X1, X2, and X3 also record the time series characteristics of user m in the sample set at time t1 (in participant 1), and user k at time t2 Time sequence features generated at time (in participant 2), and user k generated at time t3 (in participant 3). It should be noted that based on the time sequence data of users at different moments, the cycle of different moments After the neural network model undergoes federated forward training, the cyclic neural network model at different moments when the federated forward training is completed is then federated backward training at each moment to obtain the joint model. The essence of the steps in this embodiment is: The preset time series data of the participants at the target time at the target time is the preset time series data at the target time. After the federated forward training of the preset recurrent neural network model at the target time is performed, the target time corresponds to the participation at the next time. The preset time sequence data corresponding to the next time at the target time is the preset time sequence data corresponding to the next time at the target time, and the forward training process of the federation at the target time corresponding to the next time is carried out, and the federation at different times is continued. The forward training process is to obtain the preset cyclic neural network model at the last moment. After obtaining the preset cyclic neural network model at the last moment, based on the tag timing data with preset tags, the preset cyclic neural network model at the last moment, The preset time sequence data at the last moment, such as the preset time sequence data at the user t(N), update the preset loop neural network model at the last moment, and continue to perform the federal reverse update of the preset loop neural network model at different moments. Until the initial moment, the preset cyclic neural network model is updated.

Specifically, the target time participant receives the intermediate parameter of the forward propagation at the target time corresponding to the previous time, where the intermediate parameter of the forward propagation at the target time corresponding to the previous time is determined by the intermediate parameter corresponding to the previous time at the target time Or the initial parameters, the preset time series data corresponding to the previous time at the target time and the target time corresponding to the previous time of the preset cyclic neural network model are determined by the model parameters of the previous time, as shown in Figure 5, based on Intermediate parameters or initialization state information at the initial time t0 (the target time corresponds to the previous time)

The preset time series data at the first time t1, namely X1 and the model parameters at the first time t1, determine the intermediate parameters of the forward propagation at the second time t2

In this embodiment, the target moment corresponds to the intermediate parameter of the forward propagation by the participant at the previous moment

And the target moment corresponds to the participant's local model at the previous moment and is sent to the target moment participant.

Step a2, calculating the last-minute loss gradient based on the last-minute intermediate parameters, the last-minute preset time series data, and the last-minute preset prediction model to calculate the last-minute prediction model intermediate gradient;

Step a3, based on the intermediate gradient of the prediction model at the last moment and the preset timing data at the last moment, calculate the intermediate gradient corresponding to the last moment, and update the preset recurrent neural network based on the intermediate gradient at the last moment The model parameters at the last moment of the model and the intermediate gradient corresponding to the last moment in the calculation of the last moment;

Step a4: Perform federated reverse training based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network model at the first moment is updated.

Specifically, based on the target moment corresponding to the previous moment the participants will forward the intermediate parameters

The preset time series data X2 of the target time and the target time model parameters of the preset cyclic neural network model at the target time determine the intermediate parameters of the forward propagation at the target time

And so on, and based on the intermediate parameters of the forward propagation at the target moment

Carry out the federal forward training at each subsequent moment, and get

And so on, and get the intermediate parameters corresponding to the last moment of the target moment

Intermediate parameters based on the last moment

The tag timing data Y ⁿ with preset tags at the last moment and the preset prediction model V0 at the last moment are calculated, the loss gradient at the last moment is calculated, and the preset is updated based on the loss gradient at the last moment The model parameters of the prediction model, specifically, calculate the loss gradient y _t ⁿ at the last moment, and update the model parameters of the preset prediction model V0 _{based on the intermediate gradient y t} ⁿ corresponding to the prediction moment at the last moment.

It should be noted that the model parameters corresponding to each moment in the federated forward training may be the same, that is, the model parameters corresponding to each moment in the federated forward training process may be the same or different, for example, the preset cyclic nerve The model parameters of the network model at the last time of the target time, the model parameters of the preset recurrent neural network model at the last time of the last time, and the like can all be the same.

Calculate the intermediate gradient of the prediction model at the last moment based on the loss gradient y _t ⁿ corresponding to the last moment and the model parameters of the updated preset prediction model

Loss gradient at the last moment

Based on the intermediate gradient of the prediction model at the last moment and the preset time series data X ⁿ at the last moment, calculate the intermediate gradient corresponding to the last moment

And based on the intermediate gradient at the last moment

According to the preset time series data X ⁿ at the last moment, the model parameters of the preset cyclic neural network model at the last moment are updated and the intermediate gradient corresponding to the last moment at the last moment is calculated.

Specifically, based on the updated preset recurrent neural network model, the intermediate gradient at the last moment

And the preset time series data X ^n-1 corresponding to the last moment at the last moment, and calculate the intermediate gradient corresponding to the last moment at the last moment

And based on the intermediate gradient of the last moment corresponding to the previous moment

The preset time series data X ^n-1 corresponding to the last moment at the last moment is updated, and the preset cyclic neural network model corresponding to the last moment is updated until the preset cyclic neural network model at the initial moment is updated.

Step S12: Set the joint model as the data processing model.

The target model is set as the data processing model.

After the data processing model is obtained, a preset data processing procedure is performed on the to-be-processed time-series data based on the data-processing model to obtain the target prediction tag of the to-be-processed time-series data.

In this embodiment, since the data processing model has been trained to obtain an accurate model, after the time series data to be processed is obtained, the target prediction label of the time series data to be processed can be accurately obtained, and the target prediction The label may be a classification label for classifying the time sequence data to be processed, for example, the user corresponding to the time sequence data to be processed is a high-consumption user in the corresponding time sequence period.

Further, referring to FIG. 2, based on the first embodiment of the present application, in another embodiment of the present application, the intermediate parameters based on the last moment, the preset time sequence data at the last moment, and the last moment preset Set a prediction model, calculate the loss gradient at the last moment, and update the model parameters of the preset prediction model based on the loss gradient at the last moment, including:

Step b1, based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment, determining the prediction result of the preset recurrent neural network model at the last moment;

Step b2, based on the prediction result of the preset cyclic neural network model at the last moment, the real result of the preset time series data at the last moment, and the preset loss function at the last moment, determine the loss gradient at the last moment.

In this embodiment, it is the step of determining the loss gradient at the last moment. Specifically, based on the intermediate parameters at the last moment, the tag timing data with preset tags at the last moment and the last moment preset prediction The model determines the prediction result of the preset cyclic neural network model at the last moment, based on the prediction result of the preset cyclic neural network model at the last moment, the real result of the preset time series data at the last moment, and the preset at the last moment The loss function determines the loss gradient at the last moment, and based on the loss gradient at the last moment, determines the loss gradient at the last moment through a preset calculation formula to update the model parameters of the preset prediction model.

In this embodiment, the prediction result of the preset recurrent neural network model at the last time is determined based on the intermediate parameters at the last time, the preset time series data at the last time, and the preset prediction model at the last time; The prediction result of the preset cyclic neural network model at the moment, the real result of the preset time series data at the last moment, and the preset loss function at the last moment, determine the loss gradient at the last moment, and based on the loss gradient at the last moment, Update the model parameters of the preset prediction model. And then lay the foundation for accurately obtaining the target model.

Further, based on the first and second embodiments of the present application, in another embodiment of the present application, the preset recurrent neural network model based on the update at the last moment and the last moment correspond to the middle of the previous moment The gradient performs federated reverse training until the preset recurrent neural network model at the initial moment is updated, including:

Step c1, based on the preset recurrent neural network model updated at the last moment, the intermediate gradient corresponding to the last moment at the last moment and the preset time series data at the last moment corresponding to the last moment, update the preset recurrent neural network model to correspond to the last moment The model parameters at a moment and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

Step c2, based on the preset recurrent neural network model updated at the last moment corresponding to the last moment and the intermediate gradient of the last moment corresponding to the last moment and the last moment of the last moment, perform the federated reverse training of the previous moments until the initial moment. Suppose the recurrent neural network model is updated.

It should be noted that in this embodiment, it is the specific process of obtaining federated reverse training, that is, based on the preset recurrent neural network model updated at the last moment, the last moment corresponds to the intermediate gradient of the previous moment and the last moment corresponds to The preset time series data at the last moment, update the model parameters of the preset loop neural network model at the last moment corresponding to the previous moment, and calculate the intermediate gradient of the last moment corresponding to the previous moment, based on the last moment corresponding to the previous moment The preset recurrent neural network model updated at all times and the intermediate gradient corresponding to the last time at the last moment are used to perform the federated reverse training of the previous moments until the preset recurrent neural network model at the first moment is updated. In the embodiment, the federated reverse training is accurately performed.

Further, based on the first embodiment of the present application, the time series data at different times of the user is medical time series data at the different times of the user, the time series data to be processed is medical time series data to be processed, and the data processing model is medical time series data. Attribute prediction model,

The medical attribute prediction model is based on the user's medical time series data at different moments after the fed forward training of the cyclic neural network model at different moments, and then the cyclic neural network model at different moments when the federated forward training is completed. Obtained after federated reverse training;

The step of executing a preset data processing procedure on the time series data to be processed based on the data processing model to obtain the target prediction tag of the time series data to be processed includes:

Step M1: Perform medical attribute prediction processing on the medical time series data to be processed based on the medical attribute prediction model to obtain a medical attribute prediction result of the medical time series data to be processed.

In this embodiment, a data processing method based on recurrent neural network is provided in a medical institution application scenario. In this application scenario, if the same patient has visited several different medical institutions at different times to perform data processing on the same type of data After labeling and obtaining preset medical time series data at different times, a data processing model is obtained based on the medical time series data at different times.

Specifically, the medical time series data of the user at different moments may be the currently detected blood pressure, blood lipids, blood glucose, uric acid, cholesterol, the number of times of detection, the length of the detection, etc., and the medical time series data of the user at different moments are input to the training. In the medical attribute prediction model (the medical attribute prediction model is based on the user's medical time series data at different moments after the fed forward training of the cyclic neural network model at different moments, and then the cycle of different moments when the fed forward training is completed After the neural network model performs the federated reverse training at each time), it performs medical attribute prediction processing on the medical time series data to be processed based on the medical attribute prediction model to obtain the medical attribute prediction results of the medical time series data to be processed , The medical attribute prediction result includes the first medical attribute prediction result that is greater than the first preset label value (the probability of a certain medical attribute data is greater than 90%), or is less than the first preset label value and greater than the second preset label value The second medical attribute prediction result, or the third medical attribute prediction result less than the second preset label value, the medical attribute prediction result is different, and the associated loan amount obtained based on the medical attribute prediction result is different.

Referring to FIG. 3, FIG. 3 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.

As shown in FIG. 3, the data processing device based on the cyclic neural network may include a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

In an embodiment, the data processing device based on a recurrent neural network may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on. The rectangular user interface may include a display screen (Display) and an input sub-module such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

Those skilled in the art can understand that the structure of the data processing device based on the recurrent neural network shown in FIG. 3 does not constitute a limitation on the data processing device based on the recurrent neural network, and may include more or less components than shown in the figure. Or some parts are combined, or different parts are arranged.

As shown in FIG. 3, the memory 1005 as a computer medium may include an operating system, a network communication module, and a data processing program based on a cyclic neural network. The operating system is a program that manages and controls the hardware and software resources of the data processing equipment based on the cyclic neural network, and supports the operation of the data processing program based on the cyclic neural network and other software and/or programs. The network communication module is used to realize the communication between the components in the memory 1005 and the communication with other hardware and software in the data processing system based on the cyclic neural network.

In the data processing device based on the recurrent neural network shown in FIG. 3, the processor 1001 is used to execute the data processing program based on the recurrent neural network stored in the memory 1005 to realize the data based on the recurrent neural network described in any one of the above Processing method steps.

The specific implementation of the data processing device based on the recurrent neural network of this application is basically the same as the foregoing embodiments of the data processing method based on the recurrent neural network, and will not be repeated here.

In an embodiment, the data processing device based on recurrent neural network further includes:

The training module is used to perform federated forward training on the cyclic neural network model at different moments based on the time series data of the user at different moments, and then perform federated feedback on the cyclic neural network model at different moments when the federated forward training is completed. To train to obtain a joint model;

The setting module is used to set the joint model as the data processing model.

In an embodiment, the preset time sequence data of the user at different moments includes preset time sequence data at various moments;

The training module includes:

The first receiving unit is configured to receive, based on the received intermediate parameters of the forward propagation corresponding to the previous time at the target time, the preset time series data of the target time and the model parameters of the preset cyclic neural network model at the target time, Determine the intermediate parameters of the forward propagation at the target time, and perform federated forward training at each subsequent time based on the intermediate parameters of the forward propagation at the target time, until the intermediate parameters at the final time are obtained;

The first calculation unit is configured to calculate the loss gradient at the last moment based on the intermediate parameters at the last moment, the preset timing data at the last moment, and the preset prediction model at the last moment to calculate the prediction at the last moment The intermediate gradient of the model;

The second calculation unit is configured to calculate the intermediate gradient corresponding to the final time based on the intermediate gradient of the prediction model at the final time and the preset time series data at the final time, and update the prediction based on the intermediate gradient at the final time Suppose the model parameters of the recurrent neural network model at the last moment and the intermediate gradient of the last moment corresponding to the last moment are calculated;

The first update unit is used to perform federated reverse training based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network model at the first moment is updated.

In an embodiment, the second determining unit includes:

The first determining subunit is configured to determine the prediction result of the preset cyclic neural network model at the last moment based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment;

The second determining subunit is used to determine the loss at the last moment based on the prediction result of the preset recurrent neural network model at the last moment, the real result of the preset time series data at the last moment, and the preset loss function at the last moment gradient.

In an embodiment, the second update unit includes:

The update subunit is used to update the preset recurrent neural network model based on the preset recurrent neural network model updated at the last moment, the intermediate gradient corresponding to the last time at the last time and the preset time series data at the last time corresponding to the last time, to update the preset recurrent neural network model at the end Time corresponds to the model parameters of the previous time and calculates the intermediate gradient of the last time corresponding to the previous time;

The federated reverse training unit is used to perform federated reverse training for each previous moment based on the preset recurrent neural network model updated at the last moment corresponding to the previous moment and the intermediate gradient of the last moment corresponding to the previous moment and the last moment. Until the initial moment, the preset cyclic neural network model is updated.

In an embodiment, the model parameters corresponding to each moment in the federated forward training are the same.

In an embodiment, the preset time series data refers to the medical time series data of the user at different moments, and the data processing device based on the recurrent neural network further includes:

The third acquisition module is used to acquire the medical time series data to be processed, and input the medical time series data to be processed into the data processing model;

The data processing model is based on the user's medical time series data at different moments after fed forward training of the cyclic neural network model at different moments, and then federation of the cyclic neural network model at different moments when the federated forward training is completed. Obtained after reverse training;

The processing module is configured to execute a preset data processing procedure on the medical time series data to be processed based on the data processing model to obtain the target predicted medical label of the medical time series data to be processed.

The specific implementation of the data processing device based on the recurrent neural network of the present application is basically the same as the foregoing embodiments of the data processing method based on the recurrent neural network, and will not be repeated here.

The embodiment of the present application provides a medium, and the medium stores one or more programs, and the one or more programs may also be executed by one or more processors to implement any one of the foregoing The steps of the data processing method based on recurrent neural network.

The specific implementation of the medium of the present application is basically the same as the foregoing embodiments of the data processing method based on the recurrent neural network, and will not be repeated here.

The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent processing of this application.

Claims

A data processing method based on a cyclic neural network, wherein the preset time series data refers to the time series data of a user at a different time, and the data processing method based on the cyclic neural network includes:

Acquiring time series data to be processed, and inputting the time series data to be processed into a data processing model;

The data processing model is based on the user's time series data at different moments after the federated forward training of the cyclic neural network model at different moments, and then the federated feedback of the cyclic neural network model at different moments when the federated forward training is completed. Obtained after training;

Based on the data processing model, a preset data processing procedure is executed on the to-be-processed time-series data to obtain a target prediction tag of the to-be-processed time-series data.
The data processing method based on recurrent neural network according to claim 1, wherein said performing a preset data processing procedure on said to-be-processed time-series data based on said data processing model to obtain a target prediction of said to-be-processed time-series data Before the labeling step, the method includes:

Based on the user's time series data at different moments, after federated forward training is performed on the cyclic neural network model at different moments, the federated reverse training is performed on the cyclic neural network model at different moments when the federated forward training is completed to obtain Joint model

The joint model is set as the data processing model.
3. The data processing method based on recurrent neural network according to claim 2, wherein the preset time series data of the user at different moments include preset time series data at various moments;

After the forward federated training is performed on the cyclic neural network model at different moments based on the time series data of the user at different moments, the federated reverse training is performed on the cyclical neural network model at different moments when the federated forward training is completed, The steps to get the joint model include:

Based on the received intermediate parameters of the forward propagation at the target time corresponding to the previous time, the preset time series data of the target time and the model parameters of the preset recurrent neural network model at the target time, the forward direction at the target time is determined Propagate intermediate parameters, and perform federated forward training at each subsequent time based on the intermediate parameters of forward propagation at the target time, until the intermediate parameters at the final time are obtained;

Based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment, calculating the loss gradient at the last moment to calculate the intermediate gradient of the prediction model at the last moment;

Based on the intermediate gradient of the prediction model at the last moment and the preset time series data at the last moment, calculate the intermediate gradient corresponding to the last moment, and update the preset recurrent neural network model based on the intermediate gradient at the last moment. The model parameters at time and the intermediate gradient at the last time corresponding to the last time in the calculation;

The federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network model at the first moment is updated.
The data processing method based on the recurrent neural network according to claim 3, wherein the intermediate parameters based on the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment are calculated to calculate the last The steps of the loss gradient at time include:

Determine the prediction result of the preset recurrent neural network model at the last moment based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment;

Based on the prediction result of the preset cyclic neural network model at the last moment, the actual result of the preset time series data at the last moment, and the preset loss function at the last moment, the loss gradient at the last moment is determined.
The data processing method based on a recurrent neural network according to claim 3, wherein the federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last moment at the last moment, until the initial The steps for updating the preset recurrent neural network model at all times include:

Based on the preset loop neural network model updated at the last moment, the last moment corresponds to the intermediate gradient of the previous moment and the last moment corresponds to the preset timing data of the previous moment, and the preset loop neural network model is updated at the last moment corresponding to the previous moment. The model parameters and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

Based on the preset recurrent neural network model updated at the last moment corresponding to the previous moment and the intermediate gradient of the last moment corresponding to the last moment and the last moment, the federated reverse training of the previous moments is performed until the preset recurrent neural network at the first moment The network model is updated.
The data processing method based on recurrent neural network according to any one of claims 1 to 5, wherein the model parameters corresponding to each moment in the federated forward training are the same.
The data processing method based on the recurrent neural network according to claim 1, wherein the time series data of the user at a different time is medical time series data of the user at a different time, and the time series data to be processed is the medical time series data to be processed. The data processing model is a medical attribute prediction model,

The medical attribute prediction model is based on the user's medical time series data at different moments after the federated forward training of the recurrent neural network model at different moments, and then the recurrent neural network model at different moments when the federated forward training is completed. Obtained after federated reverse training;

The step of executing a preset data processing procedure on the time series data to be processed based on the data processing model to obtain the target prediction tag of the time series data to be processed includes:

Performing medical attribute prediction processing on the medical time series data to be processed based on the medical attribute prediction model to obtain the medical attribute prediction result of the medical time series data to be processed.
A data processing device based on a cyclic neural network, wherein the preset time series data refers to the time series data of a user at a different time, and the data processing device based on the cyclic neural network includes:

The first acquisition module is configured to acquire time series data to be processed, and input the time series data to be processed into the data processing model;

The data processing model is based on the user's time series data at different moments after the federated forward training of the cyclic neural network model at different moments, and then the federated feedback of the cyclic neural network model at different moments when the federated forward training is completed. Obtained after training;

The second acquisition module is configured to execute a preset data processing procedure on the to-be-processed time-series data based on the data processing model to obtain the target prediction tag of the to-be-processed time-series data.
A data processing device based on a recurrent neural network, wherein the data processing device based on a recurrent neural network includes: a memory, a processor, and a program stored in the memory for implementing the data processing method based on the recurrent neural network ,

The memory is used to store a program for realizing a data processing method based on a cyclic neural network;

The processor is configured to execute a program for realizing the data processing method based on the recurrent neural network, so as to implement the following steps:

Acquiring time series data to be processed, and inputting the time series data to be processed into a data processing model;

The data processing model is based on the user's time series data at different moments after the federated forward training of the cyclic neural network model at different moments, and then the federated feedback of the cyclic neural network model at different moments when the federated forward training is completed. Obtained after training;

Based on the data processing model, a preset data processing procedure is executed on the to-be-processed time-series data to obtain a target prediction tag of the to-be-processed time-series data.
The data processing device based on a recurrent neural network according to claim 9, wherein the data processing model performs a preset data processing procedure on the to-be-processed time-series data to obtain the target prediction of the to-be-processed time-series data Before the labeling step, the method includes:

Based on the user's time series data at different moments, after federated forward training is performed on the cyclic neural network model at different moments, the federated reverse training is performed on the cyclic neural network model at different moments when the federated forward training is completed to obtain Joint model

The joint model is set as the data processing model.
10. The data processing device based on a recurrent neural network according to claim 10, wherein the preset time series data of the user at different moments include preset time series data at various moments;

After the forward federated training is performed on the cyclic neural network model at different moments based on the time series data of the user at different moments, the federated reverse training is performed on the cyclical neural network model at different moments when the federated forward training is completed, The steps to get the joint model include:

Based on the received intermediate parameters of the forward propagation at the target time corresponding to the previous time, the preset time series data of the target time and the model parameters of the preset recurrent neural network model at the target time, the forward direction at the target time is determined Propagate intermediate parameters, and perform federated forward training at each subsequent time based on the intermediate parameters of forward propagation at the target time, until the intermediate parameters at the final time are obtained;

Based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment, calculating the loss gradient at the last moment to calculate the intermediate gradient of the prediction model at the last moment;

Based on the intermediate gradient of the prediction model at the last moment and the preset time series data at the last moment, calculate the intermediate gradient corresponding to the last moment, and update the preset recurrent neural network model based on the intermediate gradient at the last moment. The model parameters at the moment and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

The federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network model at the first moment is updated.
The data processing device based on the recurrent neural network according to claim 11, wherein the intermediate parameters based on the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment are calculated to calculate the last The steps of the loss gradient at time include:

Determine the prediction result of the preset recurrent neural network model at the last moment based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment;

Based on the prediction result of the preset cyclic neural network model at the last moment, the actual result of the preset time series data at the last moment, and the preset loss function at the last moment, the loss gradient at the last moment is determined.
The data processing device based on a recurrent neural network according to claim 11, wherein the federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last moment at the last moment, until the initial The steps for updating the preset cyclic neural network model at all times include:

Based on the preset loop neural network model updated at the last moment, the last moment corresponds to the intermediate gradient of the previous moment and the last moment corresponds to the preset timing data of the previous moment, and the preset loop neural network model is updated at the last moment corresponding to the previous moment. The model parameters and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

Based on the preset recurrent neural network model updated at the last moment corresponding to the previous moment and the intermediate gradient of the last moment corresponding to the last moment and the last moment, the federated reverse training of the previous moments is performed until the preset recurrent neural network at the first moment The network model is updated.
The data processing method based on the recurrent neural network according to claim 9, wherein the time series data of the user at a different time is medical time series data of the user at a different time, and the time series data to be processed is the medical time series data to be processed. The data processing model is a medical attribute prediction model,

The medical attribute prediction model is based on the user's medical time series data at different moments after the federated forward training of the recurrent neural network model at different moments, and then the recurrent neural network model at different moments when the federated forward training is completed. Obtained after federated reverse training;

The step of executing a preset data processing procedure on the time series data to be processed based on the data processing model to obtain the target prediction tag of the time series data to be processed includes:

Performing medical attribute prediction processing on the medical time series data to be processed based on the medical attribute prediction model to obtain the medical attribute prediction result of the medical time series data to be processed.
A medium, wherein a program for realizing a data processing method based on a recurrent neural network is stored on the medium, and the program for realizing a data processing method based on a recurrent neural network is executed by a processor to implement the following steps:

Acquiring time series data to be processed, and inputting the time series data to be processed into a data processing model;

The data processing model is based on the user's time series data at different moments after the federated forward training of the cyclic neural network model at different moments, and then the federated feedback of the cyclic neural network model at different moments when the federated forward training is completed. Obtained after training;

Based on the data processing model, a preset data processing procedure is executed on the to-be-processed time-series data to obtain a target prediction tag of the to-be-processed time-series data.
The medium of claim 15, wherein before the step of executing a preset data processing procedure on the to-be-processed time-series data based on the data processing model to obtain the target prediction tag of the to-be-processed time-series data, the Methods include:

Based on the user's time series data at different moments, after federated forward training is performed on the cyclic neural network model at different moments, the federated reverse training is performed on the cyclic neural network model at different moments when the federated forward training is completed to obtain Joint model

The joint model is set as the data processing model.
The medium according to claim 16, wherein the preset time sequence data of the user at different moments includes preset time sequence data at various moments;

After the forward federated training is performed on the cyclic neural network model at different moments based on the time series data of the user at different moments, the federated reverse training is performed on the cyclical neural network model at different moments when the federated forward training is completed, The steps to get the joint model include:

Based on the received intermediate parameters of the forward propagation at the target time corresponding to the previous time, the preset time series data of the target time and the model parameters of the preset recurrent neural network model at the target time, the forward direction at the target time is determined Propagate intermediate parameters, and perform federated forward training at each subsequent time based on the intermediate parameters of forward propagation at the target time, until the intermediate parameters at the final time are obtained;

Based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment, calculating the loss gradient at the last moment to calculate the intermediate gradient of the prediction model at the last moment;

Based on the intermediate gradient of the prediction model at the last moment and the preset time series data at the last moment, calculate the intermediate gradient corresponding to the last moment, and update the preset recurrent neural network model based on the intermediate gradient at the last moment. The model parameters at the moment and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

The federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network model at the first moment is updated.
The medium according to claim 17, wherein the step of calculating the loss gradient at the last moment based on the intermediate parameters at the last moment, the preset time series data at the last moment and the preset prediction model at the last moment, include:

Determine the prediction result of the preset recurrent neural network model at the last moment based on the intermediate parameters at the last moment, the preset time series data at the last moment, and the preset prediction model at the last moment;

Based on the prediction result of the preset cyclic neural network model at the last moment, the actual result of the preset time series data at the last moment, and the preset loss function at the last moment, the loss gradient at the last moment is determined.
The medium according to claim 17, wherein the federated reverse training is performed based on the preset recurrent neural network model updated at the last moment and the intermediate gradient corresponding to the last time at the last moment, until the preset recurrent neural network at the initial moment The steps to update the model include:

Based on the preset loop neural network model updated at the last moment, the last moment corresponds to the intermediate gradient of the previous moment and the last moment corresponds to the preset time series data of the previous moment, and the preset loop neural network model is updated at the last moment corresponding to the previous moment. The model parameters and the intermediate gradient of the last moment corresponding to the last moment in the calculation;

Based on the preset recurrent neural network model updated at the last moment corresponding to the previous moment and the intermediate gradient of the last moment corresponding to the last moment and the last moment, the federated reverse training of the previous moments is performed until the preset recurrent neural network at the first moment The network model is updated.
The medium according to claim 15, wherein the time series data at different times of the user is medical time series data at different times of the user, the time series data to be processed is medical time series data to be processed, and the data processing model is medical attribute prediction Model,

The medical attribute prediction model is based on the user's medical time series data at different moments after the fed forward training of the cyclic neural network model at different moments, and then the cyclic neural network model at different moments when the federated forward training is completed. Obtained after federated reverse training;

The step of executing a preset data processing procedure on the time series data to be processed based on the data processing model to obtain the target prediction tag of the time series data to be processed includes:

Performing medical attribute prediction processing on the medical time series data to be processed based on the medical attribute prediction model to obtain the medical attribute prediction result of the medical time series data to be processed.