CN117370472B - Data processing method, device, equipment and storage medium - Google Patents

Abstract

The invention relates to the technical field of computers, and discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: receiving local prediction data uploaded by a plurality of connected working machines, wherein the local prediction data is obtained by the corresponding working machine under attack risk; obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data; determining a target neighborhood based on the data distances of the plurality of local prediction data and the target local prediction data; the local prediction data in the target adjacent domain are aggregated to obtain global prediction data; iteratively updating the global prediction data based on the target precision to obtain updated global prediction data; updating the local prediction data based on the updated global prediction data; the invention can improve the robustness of each working machine in the cloud server network to external integrity attack and the reliability of data processing.

Description

Data processing method, device, equipment and storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and storage medium.

Background

With the advent of the internet and the big data age, the artificial intelligence performance needs of various fields are continuously improved. To improve data processing, computing, and storage efficiency, cloud server network distributed online machine learning has evolved. However, in the process of data prediction by the cloud server network, since a large number of working machines exist in the cloud server network, each working machine is vulnerable to external integrity attack, so that the working machine is easy to generate wrong local model parameters or predicted values.

Disclosure of Invention

In view of the above, the present invention provides a data processing method, apparatus, device and storage medium, so as to solve the problem that in the process of data prediction in the existing cloud server network, since each working machine in the cloud server network is vulnerable when facing to external integrity attack, the working machine is easy to generate wrong local model parameters or predicted values.

In a first aspect, the present invention provides a data processing method, applied to a cloud server in a cloud server network, where the method includes: receiving local prediction data uploaded by a plurality of connected working machines, wherein the local prediction data is generated by the corresponding working machines under attack risk; obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data; determining a target neighborhood based on data distances of the plurality of local prediction data and the target local prediction data; aggregating the local prediction data in the target adjacent domain to obtain global prediction data; iteratively updating the global prediction data based on the target precision to obtain updated global prediction data; and updating the local prediction data based on the updated global prediction data. Through the process, the robustness of each working machine in the cloud server network to external integrity attack can be improved, and the reliability of data processing is improved.

In some optional embodiments, the local prediction data includes local prediction expectations and corresponding local prediction variances, and the obtaining the target local prediction data with the lowest fault tolerance score in the local prediction data includes:

ranking a plurality of the local prediction expectations and the local prediction variances;

calculating fault tolerance scores of any one local prediction expected value and other local prediction expected values of the sequenced local prediction expected values, and obtaining a target local prediction expected value with the lowest score;

calculating fault-tolerant scores of any one local prediction variance and other local prediction variances in the sorted local prediction variances, and obtaining a target local prediction variance with the lowest score;

and determining target local prediction data with the lowest fault tolerance score in the local prediction data based on the target local prediction expectation and the target local prediction variance.

In some optional embodiments, the calculating the fault tolerance score of any one of the sorted local prediction expectations and other local prediction expectations to obtain the target local prediction expectations with the lowest score includes:

acquiring expected data distances between any one of the sequenced local prediction expected values and the local prediction expected values except for attack risks;

Summing squares of the expected data distances to obtain expected data scores;

and scoring the expected data to be the lowest local prediction expected as the target local prediction expected.

In some optional embodiments, the calculating the fault tolerance score of any one of the local prediction variances and other local prediction variances after sorting to obtain the target local prediction variance with the lowest score includes:

acquiring a variance data distance between any one of the local prediction variances after sequencing and the local prediction variance except for attack risk;

summing squares of the variance data distances to obtain a variance data score;

and taking the local prediction variance with the lowest score of the variance data as the target local prediction variance.

In some optional embodiments, the determining the target neighborhood based on the data distances of the plurality of the local prediction data from the target local prediction data includes:

determining a target expected neighborhood based on the data distance between the local prediction expectation and the target local prediction expectation;

determining a target variance neighborhood based on the data distance of the local prediction variance and the target local prediction variance;

The target neighborhood is determined based on the target expected neighborhood and the target variance neighborhood.

In some optional embodiments, the determining a target expected neighborhood based on the locally predicted expected data distance from the target locally predicted expected includes:

acquiring the data distance between each local prediction variance and the target local prediction expectation;

selecting the local prediction variance with the minimum target data distance;

acquiring the maximum local prediction expected value and the minimum local prediction expected value in the local prediction expected values with the minimum target data distance;

the target expected neighborhood is determined based on the target local prediction expectation, the maximum local prediction expectation, and the minimum local prediction expectation.

In some optional embodiments, the determining a target variance neighborhood based on the data distance of the local prediction variance from the target local prediction variance includes:

acquiring the data distance between each local prediction variance and the target local prediction variance;

selecting the local prediction variance with the minimum target data distance;

obtaining the maximum local prediction variance and the minimum local prediction variance in the local prediction variances with the minimum target data distance;

The target variance neighborhood is determined based on the target local prediction variance, the maximum local prediction variance, and the minimum local prediction variance.

In some alternative embodiments, the calculation model of the global prediction data is:

wherein,for working machine->For global prediction expectations, < >>For global prediction variance->For local prediction of expectations, ->For local prediction variance +.>Desired neighborhood for target->Is a target variance neighborhood.

In some optional embodiments, the target precision includes a target expected precision and a target variance precision, and the iteratively updating the global prediction data based on the target precision to obtain updated global prediction data includes:

iteratively updating the global prediction expectation, and obtaining an updated global prediction expectation when the updated global prediction expectation is in a first adjacent area of the target expectation precision;

iteratively updating the global prediction variance, and obtaining an updated global prediction variance when the updated global prediction variance is in a second adjacent region of the target variance accuracy;

and obtaining updated global prediction data based on the updated global prediction expectation and the updated global prediction variance.

In some alternative embodiments, the calculation model of the updated global prediction data is:

wherein,for the number of iterations->To update global prediction expectations +.>To update global prediction variance +.>For global prediction expectations, < >>Is the global prediction variance.

In some optional embodiments, the updating the local prediction data based on the updated global prediction data includes:

updating the local prediction expectation based on the updated global prediction expectation;

updating the local prediction variance based on the updated global prediction variance.

In some alternative embodiments, the updating the local prediction desire based on the updated global prediction desire includes:

comparing the updated global prediction expectation with the local prediction expectation to obtain an expectation comparison result;

and updating the updated global prediction expected to be the local prediction expected when the expected comparison result indicates that the updated global prediction expected is larger than the local prediction expected.

In some optional embodiments, the updating the local prediction variance based on the updated global prediction variance includes:

Comparing the updated global prediction variance with the local prediction variance to obtain a variance comparison result;

and updating the updated global prediction variance into the local prediction variance when the variance comparison result represents that the updated global prediction variance is smaller than the local prediction variance.

In some alternative embodiments, the local prediction expectations and the local prediction variances are calculated by the work machine based on a target training set and a kernel function, the target training set being determined based on stream data received by the work machine and a stored local training set.

In some optional implementations, determining the target training set based on the stream data and the local training set includes:

calculating the data distance between the stream data and any training data in the local training set to obtain a data distance list;

and sequencing all the data distances in the data distance list, and selecting the data with the minimum target data distance to form the target training set.

In a second aspect, the present invention provides a data processing apparatus, applied to a cloud server in a cloud server network, the apparatus mainly including: the data receiving module is used for receiving local prediction data uploaded by the connected multiple working machines, wherein the local prediction data is prediction data obtained by the corresponding working machines under attack risk; the data acquisition module is used for acquiring target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data; the neighborhood determining module is used for determining a target neighborhood based on the data distances between the local prediction data and the target local prediction data; the data aggregation module is used for aggregating the local prediction data in the target adjacent domain to obtain global prediction data; the iteration updating module is used for carrying out iteration updating on the global prediction data based on the target precision to obtain updated global prediction data; and the data processing module is used for updating the local prediction data based on the updated global prediction data. Through the process, the robustness of each working machine in the cloud server network to external integrity attack can be improved, and the reliability of data processing is improved.

In a third aspect, the present invention provides a computer device comprising: the data processing system comprises a memory and a processor, wherein the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions, so that the data processing method of the first aspect or any corresponding implementation mode of the first aspect is executed.

In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the data processing method of the first aspect or any of its corresponding embodiments.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic illustration of an application environment of an embodiment of the present invention;

FIG. 2 is a flow chart of a data processing method according to an embodiment of the present invention;

FIG. 3 is a flow chart of another data processing method according to an embodiment of the present invention;

FIG. 4 is a flow chart of a further data processing method according to an embodiment of the present invention;

FIG. 5 is a block diagram showing the structure of a data processing apparatus according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms first and second in the description and claims of the invention and in the above-mentioned figures are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The term "plurality" in the present invention may mean at least two, for example, two, three or more, and embodiments of the present invention are not limited.

Referring to fig. 1, fig. 1 is a schematic diagram of an application environment provided by an embodiment of the present invention, where a cloud server network of the schematic diagram includes a plurality of working machines, and the working machines may include a processor and a memory. The plurality of working machines can be in communication connection with a corresponding cloud server through a network, the cloud server can be used for providing services (such as aggregation services and the like) for computing programs installed on the client, and a database can be arranged on the cloud server or independent of the cloud server and used for providing data storage services for the cloud server. The number of the cloud servers is a plurality, namely, working machines in the cloud server network are grouped, and each group of working machines is provided with a cloud server for local prediction aggregation. Further, a processing engine may be run in the cloud server, which may be used to perform the steps performed by the cloud server.

Alternatively, the working machine may be, but not limited to, a terminal capable of calculating data, such as a mobile terminal (e.g., tablet computer), a notebook computer, a PC (Personal Computer ) or the like, and the network may include, but is not limited to: wide area networks, metropolitan area networks, local area networks, and the like. The cloud server may include, but is not limited to, any hardware device that may perform a calculation.

In addition, in this embodiment, the above-mentioned data processing method may be applied, but not limited to, to an independent processing device with a relatively high processing capability, without performing data interaction. For example, the processing device may be, but is not limited to, a more powerful terminal device, i.e. the individual operations of the data processing method described above may be integrated in a single processing device. The above is merely an example, and is not limited in any way in the present embodiment.

According to an embodiment of the present invention, there is provided a data processing method embodiment, it being noted that the steps shown in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.

In this embodiment, a data processing method is provided, which may be used for the cloud server applied to the cloud server network, and fig. 2 is a flowchart of the data processing method according to an embodiment of the present invention, as shown in fig. 2, where the flowchart includes the following steps:

step S201, local prediction data uploaded by a plurality of connected working machines is received, wherein the local prediction data is obtained by the corresponding working machine under attack risk.

As described above, by receiving the local prediction data uploaded by the connected multiple working machines, that is, the prediction data obtained by the cloud server corresponding to the working machine under the attack risk, a necessary condition is provided for determining the target local prediction data in the local prediction data.

In some alternative embodiments, the local prediction data includes local prediction expectations and corresponding local prediction variances. The local prediction expectation and the local prediction variance are calculated by the working machine based on a target training set and a kernel function, wherein the target training set is determined based on stream data received by the working machine and the stored local training set.

In some optional embodiments, when determining the target training set based on the stream data and the local training set, the data distance between the stream data and any training data in the local training set may be calculated to obtain a data distance list; and sequencing the data distances in the data distance list, and selecting the data with the minimum data distance of the target to form a target training set.

Specifically, the local training set is stored in a database of each working machine and is obtained from historical stream data received by the working machine. For any working machine, when the working machine receives corresponding stream data at the current moment, traversing the local training set to obtain the data distance between the stream data and each local training data in the local training set; then, based on the stream data and the data distance, such as Min distance and Manhattan distance, of each local training data in the local training set, a data distance list is generated, each data distance in the data distance list is ordered, and the data with the minimum target data distance is selected to form a target training set; and then, calculating Gaussian posterior probability distribution on a target training set through a kernel function, so as to obtain local prediction data and local prediction variance of the current working machine.

In some alternative embodiments, the computational model of the local prediction data is:

wherein,for local prediction of the expected attack risk of the working machine, the user is given a +.>For the local prediction variance of the working machine obtained at risk of variance attack, < >>To expect attack risk->Is the risk of variance attack.

It can be appreciated that when an unstable factor occurs in the network environment of the cloud server network, an attacker makes a worker pairThe machine locally predicts expected and variance tampering, such as data tampering caused by a Bayesian attack and a false data injection attack. The Bayesian attack directly controls the working machine to send false local prediction expectation and variance; the error data injection attack occurs in the stage of transmitting the local working machine to the cloud server, and the modification is directly carried out on the correct predicted value, so that the expected attack risk is the risk caused by the generation and transmission of the local prediction expected by the working machine under the Bayesian attack and the error data injection attack; the variance attack risk is the risk caused by the local prediction expected generation and transmission of the working machine under the Bayesian attack and the error data injection attack. And->Can be any value, otherwise +.>Which corresponds to a clean network environment.

Step S202, obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data.

As described above, the target local prediction data with the lowest fault tolerance score in the plurality of local prediction data is obtained, so that the target neighborhood is determined based on the relationship between the local prediction data and the target local prediction data, and the basis is provided for the aggregation of the subsequent data through the acquisition of the target precision.

In some optional embodiments, the local prediction data includes local prediction expectations and corresponding local prediction variances, and when target local prediction data with the lowest fault tolerance score in the plurality of local prediction data is obtained, the plurality of local prediction expectations and the local prediction variances may be ranked; calculating fault tolerance scores of any one local prediction expected value and other local prediction expected values of the sequenced local prediction expected values, and obtaining a target local prediction expected value with the lowest score; calculating fault-tolerant scores of any local prediction variance and other local prediction variances in the sorted local prediction variances, and obtaining a target local prediction variance with the lowest score; and determining the target local prediction data with the lowest fault tolerance score in the plurality of local prediction data based on the target local prediction expectation and the target local prediction variance.

In some optional embodiments, calculating fault tolerance scores of any one of the sorted local prediction expectations and other local prediction expectations to obtain a target local prediction expectation with the lowest score, and when the expected data distance between any one of the sorted local prediction expectations and the local prediction expectations except for attack risk is obtained first; summing squares of the plurality of expected data distances to obtain an expected data score; and taking the lowest local prediction expectation of the expected data score as a target local prediction expectation.

In some optional embodiments, calculating fault tolerance scores of any local prediction variance of the sorted local prediction variances and other local prediction variances, and obtaining a target local prediction variance with the lowest score, where the variance data distance between any local prediction variance of the sorted local prediction variances and the local prediction variance with attack risk removed can be obtained; summing squares of the multiple variance data distances to obtain a variance data score; and taking the lowest local prediction variance of the variance data score as a target local prediction variance.

Specifically, assume that The individual working machines each have data->Wherein->The individual working machines are attacked. For any->And->We use +.>Defining a fact: />Is->Distance->One of the most recent data. Then for each working machine +.>Defining fault tolerance score as +.>Here the sum in the fault tolerance score is taken to be the distance +.>Recently->Data. Finally, the final output isWherein->Corresponds to the working machine which can obtain the minimum fault tolerance score, namely +.>。

Illustratively, the cloud server sorts all received local prediction expectations and local prediction variances by magnitude, from small to large, respectively. Without loss of generality, assuming that the local prediction expectation and variance for the first work machine is the smallest and the local prediction expectation and variance for the nth work machine is the largest, the mathematical expression is as follows:

and respectively solving fault tolerance scores, namely a desired data score and a variance data score, according to the ordered local prediction expectations and variances.

Defining the desired data score and the variance data score as respectivelyAnd。

in some alternative embodiments, the target accuracy is determined from updated aggregate data at the previous time, and may also be determined based on user demand.

Step S203, determining a target neighborhood based on the data distances between the plurality of local prediction data and the target local prediction data.

As above, the target neighborhood is determined based on the data distances of the plurality of local prediction data from the target local prediction data, so as to facilitate the determination of global prediction data based on the target neighborhood.

In some optional embodiments, based on the data distances between the plurality of local prediction data and the target local prediction data, when determining the target neighborhood, the data distance between each local prediction variance and the target local prediction expectation can be obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction expected value and the minimum local prediction expected value in the local prediction expected values with the minimum target data distance; a target expected neighborhood is determined based on the target local prediction expectation, the maximum local prediction expectation, and the minimum local prediction expectation. The data distance between each local prediction variance and the target local prediction variance can be obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction variance and the minimum local prediction variance in the local prediction variances with the minimum target data distance; a target variance neighborhood is determined based on the target local prediction variance, the maximum local prediction variance, and the minimum local prediction variance. A target neighborhood is determined based on the target expected neighborhood and the target variance neighborhood.

Step S204, the local prediction data in the target neighborhood are aggregated to obtain global prediction data.

As described above, the global prediction data is obtained by aggregating the local prediction data in the target neighborhood, so that the local prediction data of the working machine is updated based on the global prediction data.

In some optional embodiments, when the local prediction data in the target neighbor is aggregated to obtain global prediction data, a weighted average calculation can be performed on a plurality of local prediction expectations in the target expectation neighbor to obtain the global prediction expectations; carrying out weighted average calculation on a plurality of local prediction variances in the target variance neighborhood to obtain a global prediction variance; and obtaining global prediction data based on the global prediction expectation and the global prediction variance.

In some alternative embodiments, the computational model of the global prediction data is:

wherein,for working machine->For global prediction expectations, < >>For global prediction variance->For local prediction of expectations, ->For local prediction variance +.>Desired neighborhood for target->Is a target variance neighborhood.

Step S205, performing iterative updating on the global prediction data based on the target precision to obtain updated global prediction data.

As described above, the global prediction data is iteratively updated based on the target accuracy, so as to obtain updated global prediction data, thereby further improving the prediction performance of the working machine.

In some optional embodiments, the global prediction data is iteratively updated based on the target precision, so that when updated global prediction data is obtained, the target precision can be decomposed into target expected precision and target variance precision, and the global prediction expected is iteratively updated to obtain updated global prediction expected and updated global prediction variance; and obtaining updated global prediction data based on the updated global prediction expectation and the updated global prediction variance.

Step S206, updating the local prediction data based on the updated global prediction data.

As described above, when the updated global prediction data is within the range corresponding to the target precision, it is determined that the updated global prediction data is the target global prediction data of the cloud server, so that robustness of the working machine against the integrity attack is further improved.

In some alternative embodiments, when the local prediction data is updated based on the updated global prediction data, the local prediction desire may be updated based on the updated global prediction desire; the local prediction variance is updated based on the updated global prediction variance.

In some optional embodiments, when the local prediction expectation is updated based on the updated global prediction expectation, the updated global prediction expectation and the local prediction expectation may be compared to obtain an expectation comparison result; and updating the updated global prediction expectation to be the local prediction expectation when the expected comparison result represents that the updated global prediction expectation is larger than the local prediction expectation.

In some optional embodiments, when the local prediction variance is updated based on the updated global prediction variance, the updated global prediction variance and the local prediction variance may be compared to obtain a variance comparison result; and when the variance comparison result represents that the updated global prediction variance is smaller than the local prediction variance, updating the updated global prediction variance into the local prediction variance.

It can be appreciated that the "benign" neighborhood of the local prediction is found by the lowest fault tolerance score, so that the global prediction expectation and the global prediction expectation calculated by the cloud server according to the "benign" neighborhood are robust to the integrity attack; by using all the predicted values in the benign neighborhood to carry out average calculation, the global prediction expectation and the global prediction variance can be enabled to more utilize the information of local prediction, and better global prediction can be obtained.

According to the data processing method provided by the embodiment, firstly, local prediction data uploaded by a plurality of connected working machines, namely, prediction data obtained by a cloud server corresponding to the working machines under attack risk, are received, so that a necessary condition is provided for determining target local prediction data in the local prediction data; the target local prediction data with the lowest fault tolerance score in the plurality of local prediction data is acquired so as to conveniently determine a target neighborhood based on the relation between the local prediction data and the target local prediction data, and a basis is provided for the aggregation of subsequent data through the acquisition of target precision; determining a target neighborhood by determining a data distance based on the plurality of local prediction data and the target local prediction data, so as to determine global prediction data based on the target neighborhood; the local prediction data in the target adjacent domain are aggregated to obtain global prediction data so as to update the local prediction data of the working machine based on the global prediction data; the global prediction data is iteratively updated based on the target precision, so that the updated global prediction data is obtained, and the predictability of the working machine is further improved; when the updated global prediction data is in the range corresponding to the target precision, the updated global prediction data is determined to be the target global prediction data of the cloud server, so that the robustness of the working machine against the integrity attack is further improved. Therefore, the method and the system can improve the robustness of each working machine to external integrity attack in the cloud server network and improve the reliability of data processing.

In this embodiment, a data processing method is provided, which may be applied to a cloud server in a cloud server network, and fig. 3 is a flowchart of the data processing method according to an embodiment of the present invention, as shown in fig. 3, where the flowchart includes the following steps:

step S301, receiving local prediction data uploaded by a plurality of connected working machines, where the local prediction data is prediction data obtained by a corresponding working machine under attack risk.

Please refer to step S201 in the embodiment shown in fig. 2 in detail, which is not described herein.

Step S302, obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data.

Please refer to step S202 in the embodiment shown in fig. 2, which is not described herein.

Step S303, determining a target neighborhood based on the data distances between the plurality of local prediction data and the target local prediction data.

Specifically, the step S303 includes:

in step S3031, a target expectation neighborhood is determined based on the data distance between the local prediction expectation and the target local prediction expectation.

As described above, by determining the target expectation neighborhood based on the data distance of the local prediction expectation from the target local prediction expectation, a necessary condition is provided for the determination of the target domain.

In some optional embodiments, when determining the target expected neighborhood based on the data distance between the local prediction expectation and the target local prediction expectation, the data distance between each local prediction variance and the target local prediction expectation may be obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction expected value and the minimum local prediction expected value in the local prediction expected values with the minimum target data distance; a target expected neighborhood is determined based on the target local prediction expectation, the maximum local prediction expectation, and the minimum local prediction expectation.

Step S3032, a target variance neighborhood is determined based on the data distance of the local prediction variance and the target local prediction variance.

As described above, the target variance neighborhood is determined by the data distance based on the local prediction variance and the target local prediction variance, providing a necessary condition for the determination of the target domain.

In some optional embodiments, when determining the target variance neighborhood based on the data distance between the local prediction variance and the target local prediction variance, the data distance between each local prediction variance and the target local prediction variance may be obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction variance and the minimum local prediction variance in the local prediction variances with the minimum target data distance; a target variance neighborhood is determined based on the target local prediction variance, the maximum local prediction variance, and the minimum local prediction variance.

Step S3033, a target neighborhood is determined based on the target expected neighborhood and the target variance neighborhood.

As above, the target neighborhood is determined based on the target expected neighborhood and the target variance neighborhood, so as to facilitate the determination of global prediction data based on the target neighborhood.

Illustratively, within a set, the resulting values are uncertain whether they were attacked, but these values may guarantee a robust global aggregation result. Therefore, the maximum number of attacks in the cloud server network is not more than half. Then for the firstA test input for scoring +.>Sum of variance data score->The present solution constructs a "benign" neighborhood: find and expect data score +.>Sum of variance data score->Nearest->Individual local prediction desire->And local prediction variance->。

Step S304, the local prediction data in the target neighborhood are aggregated to obtain global prediction data.

Please refer to step S204 in the embodiment shown in fig. 2 in detail, which is not described herein.

Step S305, performing iterative updating on the global prediction data based on the target precision to obtain updated global prediction data.

Please refer to step S205 in the embodiment shown in fig. 2 in detail, which is not described herein.

Step S306, updating the local prediction data based on the updated global prediction data.

Please refer to step S206 in the embodiment shown in fig. 2, which is not described herein.

According to the data processing method provided by the embodiment, firstly, local prediction data uploaded by a plurality of connected working machines, namely, prediction data obtained by a cloud server corresponding to the working machines under attack risk, are received, so that a necessary condition is provided for determining target local prediction data in the local prediction data; the target local prediction data with the lowest fault tolerance score in the plurality of local prediction data is acquired so as to conveniently determine a target neighborhood based on the relation between the local prediction data and the target local prediction data, and a basis is provided for the aggregation of subsequent data through the acquisition of target precision; determining a target neighborhood by determining a data distance based on the plurality of local prediction data and the target local prediction data, so as to determine global prediction data based on the target neighborhood; the local prediction data in the target adjacent domain are aggregated to obtain global prediction data so as to update the local prediction data of the working machine based on the global prediction data; the global prediction data is iteratively updated based on the target precision, so that the updated global prediction data is obtained, and the predictability of the working machine is further improved; when the updated global prediction data is in the range corresponding to the target precision, the updated global prediction data is determined to be the target global prediction data of the cloud server, so that the robustness of the working machine against the integrity attack is further improved. Therefore, the method and the system can improve the robustness of each working machine to external integrity attack in the cloud server network and improve the reliability of data processing.

In this embodiment, a data processing method is provided, which may be applied to a cloud server in a cloud server network, and fig. 4 is a flowchart of the data processing method according to an embodiment of the present invention, as shown in fig. 4, where the flowchart includes the following steps:

step S401, receiving local prediction data uploaded by a plurality of connected working machines, where the local prediction data is prediction data obtained by a corresponding working machine under attack risk.

The direct memory access request initiated by the target application is acquired, so that the initiation of the transport layer memory read request based on the direct memory access request is facilitated.

Please refer to step S201 in the embodiment shown in fig. 2 in detail, which is not described herein.

Step S402, obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data.

Please refer to step S202 in the embodiment shown in fig. 2, which is not described herein.

Step S403, determining a target neighborhood based on the data distances between the plurality of local prediction data and the target local prediction data.

Please refer to step S203 in the embodiment shown in fig. 2 in detail, which is not described herein.

Step S404, aggregating the local prediction data in the target neighborhood to obtain global prediction data.

Please refer to step S304 in the embodiment shown in fig. 3 in detail, which is not described herein.

And step S405, performing iterative updating on the global prediction data based on the target precision to obtain updated global prediction data.

Specifically, the step S405 includes:

in step S4051, the global prediction expectation is iteratively updated, and when the updated global prediction expectation is within the first neighborhood of the target expectation accuracy, the updated global prediction expectation is obtained.

Step S4052, iteratively updating the global prediction variance, and obtaining the updated global prediction variance when the updated global prediction variance is in the second neighborhood of the target variance accuracy.

Step S4053, based on the updated global prediction expectation and the updated global prediction variance, updated global prediction data is obtained.

In some alternative embodiments, all prediction expectations and variances in the target neighborhood are averaged separately to obtain a robust global prediction expectation and variance. We define the target neighborhood asAnd will be +.>Element assignment of (i.e.)>And->And then carrying out average operation to obtain a calculation model of the updated global prediction data, wherein the calculation model comprises the following steps:

wherein,for the number of iterations->To update global prediction expectations +. >To update global prediction variance +.>For global prediction expectations, < >>Is the global prediction variance.

Further, at the firstRound, aggregated local prediction expectation and local prediction variance is that the target neighborhood received by the cloud server is +.>Is expected->And local prediction variance->. Expressed mathematically as->And->. In->Round, cloud server expects global prediction +.>Sum of variances->And sent back to each work machine.

Specifically, at the firstRound, according to global prediction variance->And local prediction variance->Fusing each working machine so that the fused prediction expectation is more approximate to the function +.>Is a true value of (c). Constructing test data +.>Is as follows:

defining the prediction expectation and variance of each working machine after fusion asAnd->. If this set is not an empty set, i.e. +.>Then the global forecast from the cloud server expects +.>Global prediction varianceWill be used, is->And->. If this set is an empty set, then local predictions from the working machine will be used, +.>And->。/>Is a target training set.

To merge predictions for each work machine after a first round of work machine predictions, each work machine expects a merged prediction And prediction variance->And then sent to the cloud server. At->Round, cloud server receives all +.>And->Thereafter, the following polymerization was carried out:

wherein,and->。

In each round of iteration process, the target neighborhood structure with the lowest fault tolerance score is carried out, and robustness of the aggregation result to the integrity attack is ensured.

In step S406, the local prediction data is updated based on the updated global prediction data.

Please refer to step S206 in the embodiment shown in fig. 2, which is not described herein.

According to the data processing method provided by the embodiment, firstly, local prediction data uploaded by a plurality of connected working machines, namely, prediction data obtained by a cloud server corresponding to the working machines under attack risk, are received, so that a necessary condition is provided for determining target local prediction data in the local prediction data; the target local prediction data with the lowest fault tolerance score in the plurality of local prediction data is acquired so as to conveniently determine a target neighborhood based on the relation between the local prediction data and the target local prediction data, and a basis is provided for the aggregation of subsequent data through the acquisition of target precision; determining a target neighborhood by determining a data distance based on the plurality of local prediction data and the target local prediction data, so as to determine global prediction data based on the target neighborhood; the local prediction data in the target adjacent domain are aggregated to obtain global prediction data so as to update the local prediction data of the working machine based on the global prediction data; the global prediction data is iteratively updated based on the target precision, so that the updated global prediction data is obtained, and the predictability of the working machine is further improved; when the updated global prediction data is in the range corresponding to the target precision, the updated global prediction data is determined to be the target global prediction data of the cloud server, so that the robustness of the working machine against the integrity attack is further improved. Therefore, the method and the system can improve the robustness of each working machine to external integrity attack in the cloud server network and improve the reliability of data processing.

In this embodiment, a data processing device is further provided, and the device is used to implement the foregoing embodiments and preferred embodiments, and will not be described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The embodiment provides a data processing device, which is applied to a cloud server in a cloud server network, as shown in fig. 5, including:

the data receiving module 501 is configured to receive local prediction data uploaded by the connected multiple working machines, where the local prediction data is prediction data obtained by the corresponding working machine under attack risk.

The local prediction expectation and the local prediction variance are calculated by the working machine based on a target training set and a kernel function, wherein the target training set is determined based on stream data received by the working machine and the stored local training set.

The data obtaining module 502 is configured to obtain target local prediction data with a lowest fault tolerance score and target accuracy from the plurality of local prediction data.

A neighborhood determination module 503, configured to determine a target neighborhood based on data distances between the plurality of local prediction data and the target local prediction data.

The data aggregation module 504 is configured to aggregate the local prediction data in the target neighborhood to obtain global prediction data.

The iteration update module 505 is configured to iteratively update the global prediction data based on the target accuracy, and obtain updated global prediction data.

The data processing module 506 is configured to update the local prediction data based on the updated global prediction data.

In some alternative embodiments, the data receiving module 501 includes:

the data distance calculating unit is used for calculating the data distance between the stream data and any training data in the local training set to obtain a data distance list.

The training set determining unit is used for sorting the data distances in the data distance list and selecting the data with the minimum data distance of the target to form a target training set.

In some alternative embodiments, the local prediction data includes local prediction expectations and corresponding local prediction variances, and the data acquisition module 502 includes:

and the ordering unit is used for ordering the local prediction expectations and the local prediction variances.

The first calculation unit is used for calculating fault tolerance scores of any one local prediction expected value and other local prediction expected values in the sorted multiple local prediction expected values, and obtaining the target local prediction expected value with the lowest score.

Specifically, the expected data distance between any one of the sequenced local prediction expected values and the local prediction expected value except the attack risk is obtained; summing squares of the plurality of expected data distances to obtain an expected data score; and taking the lowest local prediction expectation of the expected data score as a target local prediction expectation.

The second calculating unit is used for calculating fault tolerance scores of any one local prediction variance and other local prediction variances in the sorted local prediction variances to obtain a target local prediction variance with the lowest score.

Specifically, obtaining a variance data distance between any one of the local prediction variances after sorting and the local prediction variance with attack risk removed; summing squares of the multiple variance data distances to obtain a variance data score; and taking the lowest local prediction variance of the variance data score as a target local prediction variance.

And the target local prediction data determining unit is used for determining target local prediction data with the lowest fault tolerance score in the plurality of local prediction data based on the target local prediction expectation and the target local prediction variance.

In some alternative embodiments, the computational model of the local prediction data is:

wherein,for local prediction of the expected attack risk of the working machine, the user is given a +.>For the local prediction variance of the working machine obtained at risk of variance attack, < >>To expect attack risk->Is the risk of variance attack.

In some alternative embodiments, neighborhood determination module 503 includes:

and the expected neighborhood determining unit is used for determining the expected neighborhood of the target based on the data distance between the local prediction expected and the target local prediction expected.

Specifically, the data distance between each local prediction variance and the target local prediction expectation is obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction expected value and the minimum local prediction expected value in the local prediction expected values with the minimum target data distance; a target expected neighborhood is determined based on the target local prediction expectation, the maximum local prediction expectation, and the minimum local prediction expectation.

The variance neighborhood determining unit is used for determining a target variance neighborhood based on the data distance between the local prediction variance and the target local prediction variance.

Specifically, the data distance between each local prediction variance and the target local prediction variance is obtained; selecting a local prediction variance with the minimum target data distance; obtaining the maximum local prediction variance and the minimum local prediction variance in the local prediction variances with the minimum target data distance; a target variance neighborhood is determined based on the target local prediction variance, the maximum local prediction variance, and the minimum local prediction variance.

The target neighborhood determining unit is used for determining the target neighborhood based on the target expected neighborhood and the target variance neighborhood.

In some alternative embodiments, the data aggregation module 504 includes:

and the global prediction expected calculation unit is used for carrying out weighted average calculation on a plurality of local prediction expected in the target expected neighbor to obtain the global prediction expected.

And the global prediction variance calculation unit is used for carrying out weighted average calculation on a plurality of local prediction variances in the target variance neighborhood to obtain the global prediction variance.

And the global prediction data determining unit is used for obtaining global prediction data based on the global prediction expectation and the global prediction variance.

In some alternative embodiments, the computational model of the global prediction data is:

wherein,for working machine->For global prediction expectations, < >>For global prediction variance->For local prediction of expectations, ->For local prediction variance +.>Desired neighborhood for target->Is a target variance neighborhood.

In some alternative embodiments, the iterative updating module 505 includes:

and the global prediction expected updating unit is used for carrying out iterative updating on the global prediction expected, and obtaining the updated global prediction expected when the updated global prediction expected is in the first adjacent region of the target expected precision.

And the global prediction variance updating unit is used for carrying out iterative updating on the global prediction variance, and obtaining the updated global prediction variance when the updated global prediction variance is in the second adjacent domain of the target variance precision.

And the global prediction data updating unit is used for obtaining updated global prediction data based on the updated global prediction expectation and the updated global prediction variance.

In some alternative embodiments, the updated calculation model of the global prediction data is:

wherein,for the number of iterations->To update global prediction expectations +.>To update global prediction variance +.>For global prediction expectations, < >>Is the global prediction variance.

In some alternative embodiments, the data processing module 506 includes:

and the expected updating unit is used for updating the local prediction expected based on the updated global prediction expected.

Specifically, comparing and updating global prediction expectation and local prediction expectation to obtain an expectation comparison result; and updating the updated global prediction expectation to be the local prediction expectation when the expected comparison result represents that the updated global prediction expectation is larger than the local prediction expectation.

And the variance updating unit is used for updating the local prediction variance based on the updated global prediction variance.

Specifically, comparing and updating the global prediction variance with the local prediction variance to obtain a variance comparison result; and when the variance comparison result represents that the updated global prediction variance is smaller than the local prediction variance, updating the updated global prediction variance into the local prediction variance.

Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.

The data processing apparatus in this embodiment is presented in the form of a functional unit, where a unit refers to an ASIC (application specific integrated circuit) circuit, a processor and a memory that execute one or more software or firmware programs, and/or other devices that can provide the above-described functions.

The embodiment of the invention also provides computer equipment, which is provided with the data processing device shown in the figure 5.

Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 6, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 6.

The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.

Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.

The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created from the use of the computer device of the presentation of a sort of applet landing page, and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, server clusters, mobile communication networks, and combinations thereof.

Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.

The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.

Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims (15)

1. A data processing method, applied to a cloud server in a cloud server network, the method comprising:

receiving local prediction data uploaded by a plurality of connected working machines, wherein the local prediction data is generated by the corresponding working machines under attack risk;

obtaining target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data;

determining a target neighborhood based on data distances of the plurality of local prediction data and the target local prediction data;

aggregating the local prediction data in the target adjacent domain to obtain global prediction data;

iteratively updating the global prediction data based on the target precision to obtain updated global prediction data;

updating the local prediction data based on the updated global prediction data;

the local prediction data includes local prediction expectations and corresponding local prediction variances, and the obtaining target local prediction data with the lowest fault tolerance score in the local prediction data includes:

Ranking a plurality of the local prediction expectations and the local prediction variances;

calculating fault tolerance scores of any one local prediction expected value and other local prediction expected values of the sequenced local prediction expected values, and obtaining a target local prediction expected value with the lowest score;

calculating fault-tolerant scores of any one local prediction variance and other local prediction variances in the sorted local prediction variances, and obtaining a target local prediction variance with the lowest score;

and determining target local prediction data with the lowest fault tolerance score in the local prediction data based on the target local prediction expectation and the target local prediction variance, wherein the fault tolerance score comprises an expected data score and a variance data score.

2. The method of claim 1, wherein calculating a fault tolerance score for any one of the ranked plurality of local prediction expectations and other local prediction expectations to obtain a scored lowest target local prediction expectation comprises:

acquiring expected data distances between any one of the sequenced local prediction expected values and the local prediction expected values except for attack risks;

Summing squares of the expected data distances to obtain expected data scores;

and scoring the expected data to be the lowest local prediction expected as the target local prediction expected.

3. The method of claim 1, wherein calculating the fault tolerance score for any one of the ranked plurality of local prediction variances and other local prediction variances to obtain the lowest scored target local prediction variance comprises:

acquiring a variance data distance between any one of the local prediction variances after sequencing and the local prediction variance except for attack risk;

summing squares of the variance data distances to obtain a variance data score;

and taking the local prediction variance with the lowest score of the variance data as the target local prediction variance.

4. The method of claim 1, wherein the determining a target neighborhood based on the data distances of the plurality of local prediction data from the target local prediction data comprises:

determining a target expected neighborhood based on the data distance between the local prediction expectation and the target local prediction expectation;

Determining a target variance neighborhood based on the data distance of the local prediction variance and the target local prediction variance;

the target neighborhood is determined based on the target expected neighborhood and the target variance neighborhood.

5. The method of claim 4, wherein the determining a target expected neighborhood based on the locally predicted expected data distance from the target locally predicted expected comprises:

acquiring the data distance between each local prediction variance and the target local prediction expectation;

selecting the local prediction variance with the minimum target data distance;

acquiring the maximum local prediction expected value and the minimum local prediction expected value in the local prediction expected values with the minimum target data distance;

the target expected neighborhood is determined based on the target local prediction expectation, the maximum local prediction expectation, and the minimum local prediction expectation.

6. The method of claim 4, wherein the determining a target variance neighborhood based on the data distance of the local prediction variance from the target local prediction variance comprises:

acquiring the data distance between each local prediction variance and the target local prediction variance;

Selecting the local prediction variance with the minimum target data distance;

obtaining the maximum local prediction variance and the minimum local prediction variance in the local prediction variances with the minimum target data distance;

the target variance neighborhood is determined based on the target local prediction variance, the maximum local prediction variance, and the minimum local prediction variance.

7. The method of claim 2, wherein the computational model of global prediction data is:

wherein,for working machine->For global prediction expectations, < >>For global prediction variance->For local prediction of expectations, ->For local prediction variance +.>Desired neighborhood for target->Is a target variance neighborhood.

8. The method of claim 7, wherein the target precision comprises a target expected precision and a target variance precision, the iteratively updating the global prediction data based on the target precision to obtain updated global prediction data, comprising:

iteratively updating the global prediction expectation, and obtaining an updated global prediction expectation when the updated global prediction expectation is in a first adjacent area of the target expectation precision;

Iteratively updating the global prediction variance, and obtaining an updated global prediction variance when the updated global prediction variance is in a second adjacent region of the target variance accuracy;

and obtaining updated global prediction data based on the updated global prediction expectation and the updated global prediction variance.

9. The method of claim 7, wherein the updated global prediction data is calculated using a model of:

wherein,for the number of iterations->To update global prediction expectations +.>In order to update the global prediction variance,for global prediction expectations, < >>Is the global prediction variance.

10. The method of claim 8, wherein updating the local prediction data based on the updated global prediction data comprises:

updating the local prediction expectation based on the updated global prediction expectation;

updating the local prediction variance based on the updated global prediction variance.

11. The method of claim 10, wherein the updating the local prediction desire based on the updated global prediction desire comprises:

comparing the updated global prediction expectation with the local prediction expectation to obtain an expectation comparison result;

And updating the updated global prediction expected to be the local prediction expected when the expected comparison result indicates that the updated global prediction expected is larger than the local prediction expected.

12. The method of claim 10, wherein the updating the local prediction variance based on the updated global prediction variance comprises:

comparing the updated global prediction variance with the local prediction variance to obtain a variance comparison result;

and updating the updated global prediction variance into the local prediction variance when the variance comparison result represents that the updated global prediction variance is smaller than the local prediction variance.

13. A data processing apparatus for use with a cloud server in a cloud server network, the apparatus comprising:

the data receiving module is used for receiving local prediction data uploaded by the connected multiple working machines, wherein the local prediction data is prediction data obtained by the corresponding working machines under attack risk;

the data acquisition module is used for acquiring target local prediction data with the lowest fault tolerance score and target precision in the plurality of local prediction data; the local prediction data includes local prediction expectations and corresponding local prediction variances, and the obtaining target local prediction data with the lowest fault tolerance score in the local prediction data includes:

Ranking a plurality of the local prediction expectations and the local prediction variances; calculating fault tolerance scores of any one local prediction expected value and other local prediction expected values of the sequenced local prediction expected values, and obtaining a target local prediction expected value with the lowest score; calculating fault-tolerant scores of any one local prediction variance and other local prediction variances in the sorted local prediction variances, and obtaining a target local prediction variance with the lowest score; determining target local prediction data with the lowest fault tolerance score in a plurality of local prediction data based on the target local prediction expectation and the target local prediction variance, wherein the fault tolerance score comprises an expected data score and a variance data score;

the neighborhood determining module is used for determining a target neighborhood based on the data distances between the local prediction data and the target local prediction data;

the data aggregation module is used for aggregating the local prediction data in the target adjacent domain to obtain global prediction data;

the iteration updating module is used for carrying out iteration updating on the global prediction data based on the target precision to obtain updated global prediction data;

And the data processing module is used for updating the local prediction data based on the updated global prediction data.

14. A computer device, comprising:

a memory and a processor in communication with each other, the memory having stored therein computer instructions which, upon execution, cause the processor to perform the method of any of claims 1 to 12.

15. A computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 12.