WO2021219080A1

WO2021219080A1 - Federated learning model-based view display method, apparatus and device, and medium

Info

Publication number: WO2021219080A1
Application number: PCT/CN2021/090983
Authority: WO
Inventors: 李�权; 魏锡光; 林焕彬; 陈天健
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2020-04-30
Filing date: 2021-04-29
Publication date: 2021-11-04
Also published as: CN111553485A

Abstract

A federated learning model-based view display method, apparatus and device, and a medium, which relate to the field of fintech. The federated learning model-based view display method comprises the following steps: acquiring run data of each client corresponding to a horizontal federated learning model in an iterative training process of the horizontal federated learning model (S10); according to the run data, constructing a visual view corresponding to the horizontal federated learning model, and determining a training process of the horizontal federated learning model (S20); and determining, according to the training process, content displayed in the visual view (S30).

Description

View display method, device, equipment and medium based on federated learning model

This application claims the priority of the Chinese patent application filed on April 30, 2020, the application number is 202010370699.6, and the name is "View display method, device, equipment and medium based on the federated learning model", which is hereby incorporated in its entirety as refer to.

Technical field

This application relates to the field of federated learning technology of Fintech, and in particular to a view display method, device, device and medium based on a federated learning model.

Background technique

With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually changing to Fintech. Artificial intelligence technology is no exception. However, due to the security and real-time requirements of the financial industry, It also places higher demands on artificial intelligence technology.

Traditional machine learning uses a centralized method to aggregate data from different sources for training in a computer or data center. However, this centralized traditional machine learning method easily exposes data privacy, and users have to share personal data. Sacrifice your privacy to train better machine learning models. In recent years, Federated Learning has enabled users to collaboratively train machine learning models while retaining their own data, especially private data containing private information, to stay locally. In this case, users can learn from well-trained machines Benefit from the model, and there is no need to share its sensitive personal data. At present, the focus of horizontal federated learning in federated learning is that the data sets of different clients have the same feature space, but the data samples are different. The operating mechanism of horizontal federated learning is more similar to a distributed learning framework. And a secure aggregation scheme is used to protect the privacy of users.

Although federated learning has performed well in industrial applications and medical applications, the practitioners of federated learning encountered the following problems when trying to perform joint modeling in their own scenarios: (1) The data available for viewing is limited. In the centralized machine learning framework, the data center or the server knows almost everything about the entire system, but the horizontal federated learning framework has no right to access the client's data, nor can it fully control the client's behavior due to the design of its data privacy mechanism. Therefore, potential risks such as malicious information in the client data are invisible to the federated learning server, and may cause unexpected results to the federated learning server, thereby reducing the success rate of horizontal federated learning model training, and will As a result, the accuracy of the trained horizontal federated learning model to recognize data is low.

It can be seen that, in the current process of training the horizontal federated learning model, the success rate is low, and the accuracy of the recognition data of the trained horizontal federated learning model is low.

technical problem

The main purpose of this application is to provide a view display method, device, device, and medium based on a federated learning model, which aims to solve the problem of low success rate in the process of training a horizontal federated learning model and the horizontal federated learning obtained from training. The model identifies the technical problem of low accuracy of the data.

Technical solutions

In order to achieve the above objective, the present application provides a view display method based on a federated learning model. The view display method based on a federated learning model includes the following steps:

Acquiring the running data of each client during the iterative training process of the horizontal federated learning model;

Construct a visual view corresponding to the horizontal federated learning model according to the operating data, and determine the training process of the horizontal federated learning model;

The content displayed in the visual view is determined according to the training process.

In addition, in order to achieve the above objective, the present application also provides a view display device based on a federated learning model, and the view display device based on a federated learning model includes:

An obtaining module, which is used to obtain the running data of each client corresponding to the horizontal federated learning model during the iterative training process of the horizontal federated learning model;

A building module for building a visual view corresponding to the horizontal federated learning model according to the operating data;

The determining module is used for determining the training process of the horizontal federated learning model; and determining the content displayed in the visual view according to the training process.

In addition, in order to achieve the above-mentioned object, the present application also provides a view display device based on a federated learning model. The view display device based on a federated learning model includes a memory, a processor, and a memory, a processor, and a memory that is stored in the memory and can be processed in the process. A view display program based on a federated learning model running on the processor, and when the view display program based on a federated learning model is executed by the processor, the steps of a view display method based on a federated learning model corresponding to the federated learning server are implemented.

In addition, in order to achieve the above object, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a view display program based on a federated learning model, and the view display program based on a federated learning model is processed When the processor is executed, the steps of the view display method based on the federated learning model as described above are implemented.

Beneficial effect

This application obtains the operating data of the horizontal federated learning model corresponding to each client during the iterative training process of the horizontal federated learning model, constructs a visual view corresponding to the horizontal federated learning model according to the running data, and determines the training process of the horizontal federated learning model, Control the content displayed in the visual view according to the training progress. The content displayed in the visual view is used to determine the training process data corresponding to the iterative training of the horizontal federated learning model. The training process data is used to determine various influencing factors in the iterative training process to avoid malicious information in the client data. The potential risks of, cause unexpected results to the federated learning server, thereby increasing the success rate of horizontal federated learning model training, and the accuracy of the recognition data of the trained horizontal federated learning model.

Description of the drawings

FIG. 1 is a schematic flowchart of a first embodiment of a view display method based on a federated learning model in this application;

Figure 2 is a schematic diagram of an overview view in an embodiment of the present application;

FIG. 3 is a schematic diagram of the visualized loss value, recognition accuracy, and number of training samples in an embodiment of the present application;

Fig. 4 is a schematic diagram of a projection view in an embodiment of the present application;

Fig. 5 is a schematic diagram of a contribution ranking view in an embodiment of the present application;

Fig. 6 is a functional schematic block diagram of a preferred embodiment of a view display device based on a federated learning model of the present application;

Fig. 7 is a schematic structural diagram of a hardware operating environment involved in a solution of an embodiment of the present application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

This application provides a view display method based on a federated learning model. Referring to FIG. 1, FIG. 1 is a schematic flowchart of a first embodiment of a view display method based on a federated learning model of this application.

The embodiment of this application provides an embodiment of the view display method based on the federated learning model. It should be noted that although the logical sequence is shown in the flowchart, in some cases, it can be executed in a different order than here. Steps shown or described.

The view display method based on the federated learning model is applied to the federated learning server. For ease of description, the execution subject is omitted to describe the various embodiments. View display methods based on the federated learning model include:

Step S10: Obtain the running data of each client corresponding to the horizontal federated learning model during the iterative training process of the horizontal federated learning model.

When a construction instruction to construct a visual view corresponding to the horizontal federated learning model is detected, the horizontal federated learning model is obtained according to the built instruction during the iterative training process of the horizontal federated learning model, and the horizontal federated learning model corresponds to the running data of each client. Among them, the construction instruction can be triggered by the user according to specific needs, and can also be triggered when the horizontal federated learning model starts to be constructed, that is, the construction instruction is automatically triggered when the horizontal federated learning model is iteratively trained for the first time. Each client has corresponding operating data. In this embodiment, the operating data includes at least one of the following: client identification, client name, start timestamp when the horizontal federated learning model starts iterative training, client The training times of the local model, the corresponding loss value of the local model after each iterative training, the recognition accuracy of the data recognition of the local model after each iterative training, the start time and end of each training of the local model by the client through the local data Point in time. The local model is a model that the federated learning server sends to each client after obtaining the horizontal federated learning model, and each client stores the model. After each client receives the horizontal federated learning model, it uses the received horizontal federated learning model as its own local model, and may use its own local data to adjust the local model. It is understandable that the operating data will not involve the private data of each client user.

Specifically, the client ID is used to uniquely indicate a certain client. During the horizontal federated learning iterative training process, the client ID and client name can be transmitted together; the start timestamp of each time the horizontal federated learning model starts iterative training can be Obtained by a timer when starting iterative training; the training times of the client's local model can be sent to the federated learning server during each iteration of the training process. It should be noted that the training times can be equal to the number of iterations or not equal to the number of iterations ;Each iteration of training can get the corresponding loss value; recognition accuracy can obtain the preset data to be tested after each iteration of training to obtain the local model, and input the data to be tested into the local model to obtain the horizontal local model The output result is compared with the correct result corresponding to the data to be tested to determine the recognition accuracy.

Step S20: Construct a visual view corresponding to the horizontal federated learning model according to the operating data, and determine the training process of the horizontal federated learning model.

When the operating data is obtained, a visual view corresponding to the horizontal federated learning model is constructed based on the running data, and the training process of the horizontal federated learning model is determined. Among them, the visual view includes at least one of the following: overview view, summary view, projection view, comparison view, and contribution ranking view. The overview view can display the overall running process of each client during the iterative training process of the horizontal federated learning model; The projection view can display the mapping relationship between the client ID and the two-dimensional distribution of each iteration of the training; the summary view is used to display the statistical information corresponding to various data during the iterative training process of the horizontal federated learning model; the comparison view is used to display the horizontal During the iterative training process of the federated learning model, the comparison of the corresponding indicator data of any two clients; the contribution ranking view is used to display the contribution of each client to the horizontal federated learning model in different dimensions. The training process of the horizontal federated learning model can be represented by the number of iterations of the horizontal federated learning model for iterative training. It is understandable that when the horizontal federated learning model is trained to the convergence state, a certain number of iterative training will be performed, and each iteration of training will Changing the model parameters corresponding to the horizontal model learning model. Correspondingly, during each iteration of the training process, some of the running data corresponding to each client will also change, such as the loss value and recognition accuracy. Therefore, in this embodiment, The training process of the horizontal federated learning model can be determined by the number of iterations. Specifically, in the iterative training process of the horizontal federated learning model, the number of iterations of the horizontal federated learning model is calculated by a timer. Each iteration of the horizontal federated learning model is trained, the value corresponding to the timer is increased by 1, so as to determine the horizontal federated learning model when needed. During the training process, the value corresponding to the timer is obtained, and the number of iterations of the horizontal federated learning is determined according to the value, thereby determining the training process of the horizontal federated learning model.

Further, the operating data includes at least one of the following: client identification, the start timestamp of each iteration of training, the number of training samples corresponding to the client, the loss value corresponding to the local model, and the local model trained by each client through the local data The visual view includes an overview view; the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data includes:

Step a: Perform visual coding on the operating data to obtain visualized operating data.

Further, when the visual view is the overview view, visual coding is performed on each operating data to obtain the visualized operating data. It should be noted that the process of processing the running data of each client is the same. Therefore, for ease of description, this embodiment takes the running data of a client as an example for description. Specifically, to visualize the change process of the horizontal federated learning network corresponding to the horizontal federated learning model, it needs to be explained that in order to measure the “network structure of the horizontal federated learning network when the client joins and exits during the iterative training process of the horizontal federated learning model” ”Changes, the current iterative training corresponding to the horizontal network seen by each client and federated learning server, and introduces the overall change rate of the network based on the centrality of the change to measure, this metric takes into account the horizontal federated learning network over a period of time The process of change.

In this embodiment, the position of each client in the horizontal federated learning network is determined during each iteration of the training process. Specifically, the client identifier and the start timestamp of the current iteration training can be used to determine that the corresponding client is in the horizontal federated learning network. The location in the network, specifically, the location of each client in the horizontal federated learning network during each iteration of training can be preset, and the mapping relationship between the client ID, the start timestamp, and the location ID can be established in advance. After the start timestamp and client identification of the current iterative training are determined, the position of the corresponding client in the horizontal federated learning network can be determined through the start timestamp, the client identification and the mapping relationship.

Visualize the start timestamp of each time the horizontal federated learning model starts iterative training, the loss value corresponding to the local model after each iterative training, the recognition accuracy of the data recognition by the local model after each iterative training, and the client through the local data every time The start time point and end time point of the local model are trained once, and the visualized start time stamp, loss value, recognition accuracy, start time point and end time point are correspondingly obtained. Specifically, a box-and-whisker graph is used to represent the identification accuracy and loss value distribution of each client during each iteration training process, and a curve is used to connect the average value of the identification accuracy and loss value during each iteration training process to obtain visualization After the recognition accuracy and the visualized loss value, the box and whisker chart is a statistical chart used to display a set of data dispersion information. Visualize the number of training samples corresponding to each client during each iteration of the training process as a curve to obtain the number of training samples after visualization. Further, you can add a corresponding bar graph to the corresponding area of the curve, and use the bar graph to indicate the corresponding iteration During the training process, the total number of training samples, it can be understood that the total number of training samples is equal to the sum of the number of training samples corresponding to each client. The start time point and end time point of training the local model through the client's local data can be calculated to obtain the training time of each training local model of the corresponding client. The slope of the connection between the start time point and the end time point represents the training of each client Time length, so as to get the end time point and start time point after visualization. Specifically, refer to Figure 3, which is a schematic diagram of the visualized loss value, recognition accuracy, and the number of training samples in an embodiment of the present application. The loss value of (loss), the second picture shows the recognition accuracy (Accuracy) after visualization, and the third picture shows the number of training samples after visualization (Sampie Number).

Step b: Construct an overview view corresponding to the horizontal federated learning model according to the visualized operating data.

When the visualized operating data is obtained, an overview view corresponding to the horizontal federated learning model is constructed based on the visualized operating data. It is understandable that the overview view is composed of various visualized operating data. Through the overview view, the operating conditions of each client during the iterative training process of the horizontal federated learning model can be determined.

It should be noted that through the overview view, it can be found that the clients participating in the iterative training of the horizontal federated learning model change with the evolution of the iterative training process. The changes in the federated learning network can be seen through the overview view. For example, through the overview view, it can be determined that in each iteration of the training process, the number of samples provided by each client participating in the iterative training is generally evenly distributed, that is, each client The difference between the number of samples provided is within a preset range, which can be set according to specific needs; through the overview view, you can also determine the start time point and end time point for each client to train the local model It is understandable that the start time of the local model training of each client may be different, which may be caused by network delays. Since each client uses local data to train the local model with different amounts of local data, the training time for each client to train the local model is also different. It is understandable that the training duration for each client to train the local model is determined through the overview view, and the start time of the next iteration training is adjusted according to the training duration, that is, the waiting time between two adjacent iteration training is adjusted to Better adapt to the training duration of each client. It is understandable that the waiting time should be greater than or equal to the maximum training time corresponding to the client. Specifically, through the overview view, it can be determined that as the number of iterations increases, the loss value is continuously reduced and the recognition accuracy is continuously improved. Furthermore, through the overview view, it is also possible to determine which iterations of training correspond to a larger change in recognition accuracy, and which iterations of training correspond to a relatively small change in recognition accuracy.

Specifically, refer to FIG. 2, which is a schematic diagram of an overview view in an embodiment of the present application. In FIG. 2, each rounded rectangular box represents an iterative training, and a small circle in each rounded rectangular box Indicates the client participating in the current iterative training. As shown in Figure 2, the number of iterations is aligned in the y (vertical axis) direction. The order in which the clients appear; the coordinates in the x (horizontal axis) direction in Figure 2 have been adjusted so that the lines in Figure 2 will not cross, in order to minimize the overall space utilization. It should be noted that the small solid circles in Figure 2 represent clients that only participate in part of the iterative training. It can be seen from Figure 2 that the overview view visualizes the clients involved in each iteration of the training process.

Step S30: Determine the content displayed in the visual view according to the training process.

When determining the training process of the horizontal federated learning model, the content displayed in the visual view is controlled according to the training process to determine the training process data of the iterative training of the horizontal federated learning model through the content displayed in the visual view. It is understandable that the training process data obtained from different visual views is also different. For example, the training process data corresponding to the overview view is the additional operating data of each client during the iterative training process of the horizontal federated learning model. A form of expression. As the training process of the horizontal federated learning model changes, the model parameters of the horizontal federated learning model will change, and the running data of the client will also change. Therefore, the content displayed in the corresponding visual view is different. It changes as the training process of the horizontal federated learning model changes, so that the client's running status during the training of the horizontal federated learning model can be viewed through the visual view. It is understandable that this embodiment can determine whether each client has an abnormal condition according to the content displayed in the corresponding visual view of each client. For example, there is a big difference between the visual view of one client and the visual view of other clients. It can be determined that there may be an abnormal situation in the client.

In this embodiment, by acquiring the running data of the horizontal federated learning model corresponding to each client during the iterative training process of the horizontal federated learning model, construct a visual view corresponding to the horizontal federated learning model according to the running data, and determine the training process of the horizontal federated learning model , According to the training process to control the content displayed in the visual view. The content displayed in the visual view is used to determine the training process data corresponding to the iterative training of the horizontal federated learning model. The training process data is used to determine various influencing factors in the iterative training process to avoid malicious information in the client data. The potential risks of, cause unexpected results to the federated learning server, thereby increasing the success rate of horizontal federated learning model training, and the accuracy of the recognition data of the trained horizontal federated learning model.

Further, traditional centralized machine learning usually conducts model training and inference in a separate manner, while a federated learning server usually couples the training and inference processes. In other words, the federated learning server is a data distribution that can be continuously updated to adapt to possible changes. For the federated learning server maintainers, they only rely on some simple logs and metrics to interpret the information at a given stage or moment. It is not enough to require quick and informed decisions in a short period of time. Therefore, there is an urgent need for a method that can effectively express the "temporal and spatial data" from different clients over time. Achieving this is helpful for phased adjustment of the federated learning aggregation strategy, and timely review of the iterative training process of the horizontal federated learning model in order to intervene more effectively. In this embodiment, the content displayed in the visual view is controlled according to the training process to determine the training process data corresponding to the iterative training of the horizontal federated learning model through the content displayed in the visible view, which changes as the training process of the horizontal federated learning model changes. The content displayed in the visual view is used to express the changes in the iterative training process of the horizontal federated learning model corresponding to the client through the visual view, so that relevant operation and maintenance personnel can review the iterative training process of the horizontal federated learning model in a timely manner, so as to be more effective Intervene to optimize the horizontal federated learning model obtained from training.

Further, a second embodiment of the view display method based on the federated learning model of this application is proposed. The difference between the second embodiment of the view display method based on the federated learning model and the first embodiment of the view display method based on the federated learning model is that the visual view includes a projection view, and the construction is based on the operating data. The steps of the visual view corresponding to the horizontal federated learning model include:

Step c: Determine indicator data corresponding to each client terminal according to the operating data.

When the visual view that needs to be constructed is a projection view, and the operating data is obtained, the indicator data corresponding to each client terminal is determined according to the operating data. It should be noted that when constructing the projection view, the running data can also include the gradient histogram corresponding to the local model and the weight histogram corresponding to the local model. The model corresponds to the gradient of the model parameters, so that the corresponding gradient histogram can be obtained according to the determined gradient. The weight is the weight corresponding to each model parameter in the local model. In this embodiment, the index data includes at least one of the following: loss value, recognition accuracy, training evolution number, weight histogram and gradient value histogram.

Step d: Construct a projection view corresponding to the horizontal federated learning model according to the indicator data, wherein each node in the projection view respectively represents a mapping relationship between a client identifier and the number of iterations.

When the index data is obtained, according to the index data, the projection view corresponding to the horizontal federated learning model is constructed based on the t-SNE (t-distributed stochastic neighbor embedding, t-distributed stochastic neighbor embedding) projection. It should be noted that the projection view is a 2D (two-dimensional) view, t-SNE is a dimensionality reduction technique used to create low-dimensional representations and retain local similarity to convey neighborhood structure. It will be appreciated that the present embodiment can also be employed PCA (Principal Component Analysis, Principal Component Analysis) and MDS (multidimensional scaling ^o, multidimensional scaling analysis) Construction of dimension reduction projection views. Through the projection view, you can view the potential clusters and outliers in the iterative training process of the horizontal federated learning model, so as to determine the abnormal clients during the iterative training of the horizontal federated learning model. In the projection view, there is at least one node, and each node represents a bunch of mapping relationships between "client identification-number of iterations", that is, in this embodiment, "client identification-number of iterations" is projected onto a two-dimensional view. middle. The corresponding projection views are different for different iteration times. In the projection view, the federated learning server is a special client in the projection view.

In the projection view, the federated learning server corresponding to the first iterative training is used as the starting point of the projection view, and the federated learning server corresponding to the last iterative training is used as the end point of the projection view. The nodes appearing in the middle are the clients involved in the iterative training process. Then all the nodes are connected, so that through the projection view, the evolution process of the client can be determined in the iterative training process of the horizontal federated learning model.

Specifically, referring to FIG. 4, FIG. 4 is a schematic diagram of a projection view in an embodiment of the present application. In Figure 4, the two small solid circles before and after the connection represent the first iterative training and the last iterative training of the horizontal federated learning model. The hollow small circles in the middle of the connection represent the clients of each client participating in the horizontal federated learning model. The mapping relationship between the identifier and the number of iterations. For example, a certain client identifier is A and the number of iterations is the 10th time. In Figure 4, a small hollow circle represents "A-10". It should be noted that if a small circle deviates far from the curve, it means that the small circle corresponds to the client's contribution to the iterative training process of the horizontal federated learning model, and it is more likely to be an abnormal client. The client corresponding to the small circle is determined to be an abnormal client.

Further, the visual view includes a summary view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data includes:

Step e: Determine statistical data corresponding to the operating data, where the statistical data includes at least one of the following: the number of clients corresponding to the horizontal federated learning model, the number of iterations, and the waiting time for training the horizontal federated learning model. The number of changes in the number of training samples, the reduction value corresponding to the loss value corresponding to the horizontal federated learning model, and the increase value of the recognition accuracy of the local model corresponding to each client;

Step f: Construct a summary view corresponding to the horizontal federated learning model according to the statistical data.

Further, when the visual view that needs to be constructed is a summary view, and after the operating data is obtained, the statistical data corresponding to the operating data is determined. The statistical data includes at least one of the following. The horizontal federated learning model corresponds to the number of clients of the client and the number of iterations. The number of changes in the number of samples to be trained for training the horizontal federated learning model, the reduction value corresponding to the loss value corresponding to the horizontal federated learning model, and the increase value of the recognition accuracy of each client corresponding to the local model, where the number of changes is two adjacent to each other. The second iterative training corresponds to the difference in the number of samples to be trained. The difference in data is equal to the number of samples to be trained in the next iterative training minus the number of samples to be trained in the previous iterative training; the reduced value is two adjacent ones. The loss difference between the corresponding loss values of the second iteration training. The loss difference is equal to the loss value corresponding to the previous iteration training minus the loss value corresponding to the next iteration training. It should be noted that the loss value of the local model is also learned by the horizontal federation Calculated by the model, in the same iterative training process, the loss value corresponding to the local model is equal to the loss value corresponding to the horizontal federated learning model; the increase value is equal to the recognition accuracy of the local model corresponding to the last iteration training minus the corresponding one of the previous iteration training The recognition accuracy of the local model.

When the statistical data is obtained, a summary view corresponding to the horizontal federated learning model is constructed based on the statistical data, where the summary view can display the statistical data in the form of a table or a graph.

Further, the step f includes:

Step f1: According to the statistical data, the number of iterations corresponding to the horizontal federated learning model is taken as the abscissa, and the corresponding statistical data is taken as the ordinate to construct a summary view of each statistical data corresponding to the horizontal federated learning model.

Specifically, in the process of constructing the summary view, the number of iterations corresponding to the horizontal federated learning model may be the abscissa, and the corresponding statistical data may be the ordinate to construct a summary view of each statistical data corresponding to the horizontal federated learning model. For example, in the process of constructing a summary view corresponding to the number of clients, the number of iterations is used as the abscissa, and the number of clients participating in the iterative training is the ordinate during each iteration training process, and the summary view corresponding to the number of clients is constructed. View, you can see the changes in the number of clients in the iterative training that has been carried out. It is understandable that for various statistical data, the principles of the construction process are similar, and will not be repeated in this embodiment. It is understandable that this embodiment can construct summary views corresponding to the number of clients, the number of changes in the number of samples to be trained, the decrease value corresponding to the loss value, and the increase value of the recognition accuracy.

Further, the visual view includes a comparison view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data includes:

Step g: Determine the index data corresponding to each client according to the operating data, and determine the target index corresponding to the last iterative training of the horizontal federated learning model in the index data.

Further, when the visual view that needs to be constructed is a comparison view, and after the operating data is obtained, the indicator data corresponding to each client terminal is determined according to the operating data, where the indicator data indicator includes at least one of the following: recognition accuracy, loss value, The training times, weight histogram and gradient histogram of the local model. It should be noted that the indicator data is part of the operating data. In this embodiment, it is possible to pre-set which operating data is the indicator data corresponding to the client. Specifically, a specific indicator identifier can be added to the indicator data. After the operating data is obtained, it is detected which operating data carries the indicator identifier, and The operating data carrying the indicator identifier is determined to be indicator data, and this embodiment does not limit the expression form of the indicator data.

It should be noted that the comparison view is at least constructed from indicator data between the two clients. After determining the index data, determine the target index corresponding to the last iterative training of the horizontal federated learning model in the index data. The target index is the comparison benchmark in the comparison view, that is, the target index is used to compare the relevant data of the two clients. It can be understood that there is at least one target indicator. In this embodiment, the weight histogram and the gradient histogram can be set as the target indicators.

Step h: Construct a comparison view corresponding to the horizontal federated learning model according to the target index.

When the target index is determined, the comparison view corresponding to the horizontal federated learning model is constructed according to the target index. Specifically, by comparing the views, it is possible to obtain the weight histogram and gradient histogram corresponding to each client in each iteration number, and then combine the weight histogram and the weight histogram between at least two clients with the same iteration number or different iteration numbers. The gradient histogram is compared to obtain the similarity between the weight histogram and the gradient histogram between at least two clients, and the similarity is used as a new indicator value, that is, the weight histogram and the gradient histogram are converted Therefore, in the iterative training process, the indicator data corresponding to each client can be converted into a numerical value, which is convenient for users to analyze each client in the iterative training process.

Specifically, step h includes:

Step h1: Obtain target indicators corresponding to neighboring clients in the federated learning network structure in the same iterative training process, and construct a corresponding comparison view according to the target indicators corresponding to the neighboring clients.

Specifically, in the component comparison view process, each iteration of the training process can be compared, the target indicators corresponding to neighboring clients in the federated learning network structure, and then the client name or client identifier of the neighboring client as the abscissa, The target indicator corresponding to the neighboring client terminal is used as the ordinate to construct a comparison view corresponding to the neighboring client terminal. It should be noted that in the process of training the horizontal federated learning model, due to the nature of the federated learning network, the characteristics of adjacent clients in the federated learning network are similar. Therefore, if the comparison view between two adjacent clients shows There is a big difference between the two clients, you can determine that one of the clients has an abnormal situation during the iterative training process. At this time, you can combine the comparison views between the two clients and the other client to determine the abnormal client end. It is understandable that, except for the first client and the last client in the federated learning network structure, there are two adjacent clients for the other clients, one is the left-adjacent client and the other is the right-adjacent. Client.

Further, in the comparison view in this embodiment, the rectangle in each row represents a client, and the bars in different colors in the rectangle represent the values corresponding to each indicator data of the client, and the same indicator data bar corresponding to different clients The starting point of the shape is the same, so that the similarity between each indicator data can be determined by the ending point of the bar. The top row in the comparison view can be used to represent the federated learning server. Further, it is also possible to obtain comparison views corresponding to different iteration times, and then display the comparison views corresponding to different iteration times in the same interface, so as to view the arrangement of the same client in different iteration times through the comparison views corresponding to the different iteration times Condition. It should be noted that for the comparison view corresponding to the same iteration number, if the comparison view contains all the index data trained in this iteration, the order of the index data corresponding to each client in the comparison view can be determined as needed. If there is recognition accuracy At the time, the clients can be sorted according to the recognition accuracy from large to small to obtain a comparison view, so that the recognition accuracy corresponding to each client in the current iterative training process can be viewed through the comparison view.

Further, when the indicator data of the client corresponding to the horizontal federated learning model is determined, the target indicators of various indicator data are selected, and the similarity between the indicator data of each client and the corresponding target indicator is calculated. At this time, the greater the similarity, the greater the The higher the ranking in the comparison view, the lower the similarity, and the lower the ranking in the comparison view. Specifically, Euclidean distance or cosine distance may be used to calculate the similarity between the index data of each client and the corresponding target index. In the comparison view, the similarity can be represented by a curve. It is understandable that if the calculated similarity is greater than the preset similarity, it means that the corresponding client has undergone major changes during the iterative training process of the horizontal federated learning model; if the calculated similarity is less than or equal to the preset similarity Degree means that the corresponding client is a normal change during the iterative training process of the horizontal federated learning model. The preset similarity can be set according to specific needs, and this embodiment does not limit the preset similarity.

It should be noted that through the comparison view, it can be determined that during the iterative training process, compared with other normal fluctuation clients, there are clients with obvious fluctuations. At this time, you can select the clients with obvious fluctuations and the clients in the comparison view through operation instructions. The nodes corresponding to the clients with normal fluctuations are compared, and the two clients are compared to see the difference between the loss value and recognition accuracy of the two clients and the difference between the running data such as the recognition accuracy, so as to determine the clients with obvious fluctuations. Furthermore, by comparing the views, it is also possible to check whether the gradient change of each client deviates from the normal situation. It can be seen that by comparing the views, we can determine the abnormal clients in the iterative training process of the horizontal federated learning model.

Further, the visual view includes a contribution ranking view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data includes:

Step i: Determine the ranking order of the running data of the client in each iteration of the training.

Step j: Display the ranking training in the form of a box-and-whisker graph to construct a ranking view of contribution degrees corresponding to the horizontal federated learning model.

Further, when the visual view to be constructed is the contribution ranking view, and the running data is obtained, the ranking order of the running data of each client during each iteration of the training process is determined. After determining the ranking order of the running data of each client in each iteration of the training, the ranking order is displayed in the form of a box and whisker diagram to construct a ranking view of the contribution degree corresponding to the horizontal federated learning model. It should be noted that in the contribution degree In the sorting view, you can sort from smallest to largest according to the ranking order, or from largest to smallest according to the ranking order. Specifically, in the contribution ranking view, the clients can be sorted according to the lowest ranking, highest ranking, median ranking, and number of iterations of each client. For example, it is determined that a certain client is Several running data ranks the highest, several running data ranks the lowest, and several running data ranks in the middle. It should be noted that a client does not necessarily participate in all iterations of the horizontal federated learning model. For example, the total number of iterations of the horizontal federated learning model is 100, and a client may only participate in 65 of them. When one of the running data of the client has the highest ranking during a certain iterative training process, it can be determined that the client has the highest ranking once in the current iterative training process. It is understandable that for the same operating data, the corresponding contribution can be determined according to the ranking order. For example, if a certain operating data ranks the highest, it can be determined according to the nature of the operating data that the operating data has the largest contribution to the horizontal federated learning model Or the smallest.

Further, the ranking in the comparison view will also affect the ranking of the contribution ranking view. For example, when selecting the loss value in the comparison view for sorting, each client will get a ranking according to the loss value in each round, so that each client has a ranking in each round, using box-and-whisker plots Indicates the ranking distribution of this client, displayed in the contribution ranking view. At this time, the attribute selected by the user displayed in the contribution ranking view is the contribution ranking when the loss value is lost.

It should be noted that the contribution ranking view uses the box-and-whisker chart design to display the participation of all clients in the iterative training process of the horizontal federated learning model, and is sorted in descending or ascending order. Just like in joint learning, the local data of the client is completely invisible to the federated learning server. Through the ranking of different running data, the contribution of the client to the horizontal federated learning model can be understood. Among them, the loss rate and recognition accuracy may reflect the quality of each client's training data, the amount of training data represents the contribution of the data, and the loss rate indicates that the useless sample data in the sample data to be trained provided by the client accounts for the total waiting time provided by it. The proportion of training sample data. It is understandable that by using different running data to rank clients, it can be determined that the running data of abnormal clients are not all ranked last in each iteration of the training process. Therefore, in the horizontal federated learning model iterative training process , The exception that allows the client to appear only during a few iterations of the training process, but in most of the iterative training process, the client is normal.

Specifically, referring to Fig. 5, Fig. 5 is a schematic diagram of a contribution ranking view in an embodiment of the present application. In Fig. 5, the y (vertical) axis is the client identification of the client, and the x (horizontal) axis is the ranking distribution. Figure 5 shows the ranking distribution of the minimum running data of each client during each iteration of the training process. It can be seen from Figure 5 that the longer the length of the rectangular box corresponding to each client, the more the minimum values exist in the iterative training process of the horizontal federated learning model in its running data.

It is understandable that the existing visual analysis tools such as Turbofan tycoon or Fate-Board can be used to convey the advantages of the federated learning model. Turbofan Visual analysis tools such as tycoon or Fate-Board help to promote the analysis and improvement of the federated learning model by summarizing the logs and performance index data generated by the federated learning process. However, in-depth analysis is lacking. Fine-grained analysis such as analysis of potential client anomalies and contribution evaluation is challenging. For example, in the training process of horizontal federated learning models, the design of privacy protection mechanisms will hinder many basic operations. . If effective analysis is not performed to provide support for subsequent optimization and adjustment, it will affect the effect of the entire horizontal federated learning model training, that is, the accuracy of the recognition data of the horizontal federated learning model obtained by training will be low. In this embodiment, by constructing different visual views, that is, constructing a comparison view, a summary view, a projection view, and a contribution ranking view, etc., through different visual views, analyzes from different dimensions during the iterative training process of the horizontal federated learning model. , The operating conditions of each client, discover the abnormal conditions of each client in time, and then adjust the training process of the horizontal federated learning model in time, thereby further improving the accuracy of the recognition data of the trained horizontal federated learning model.

Further, a third embodiment of the view display method based on the federated learning model of this application is proposed. The difference between the third embodiment of the view display method based on the federated learning model and the first and/or the second embodiment of the view display method based on the federated learning model is that the view display method based on the federated learning model further includes :

Step k, detecting whether an operation instruction to operate the visual view is received.

If the operation instruction is received, the step of determining the content displayed in the visual view according to the training process includes:

Step 1. Determine the content displayed in the visual view according to the operation instruction and the training process.

After the visual view is created, it is detected whether an operation instruction for operating the visual view is received, where the operation instruction is set out by the user according to specific needs. After receiving the operation instruction, control the content displayed in the visual view according to the operation instruction and the training process; when the operation instruction is not received, continue to detect whether an operation instruction to operate the visual view is received. For example, the user can select a node in the projection view by operating instructions, and then the summary view, overview view, comparison view, and contribution ranking view will display the relevant data corresponding to the selected node in the current training process; When the corresponding comparison view is trained for the second iteration, and there are multiple client running data in each comparison view, when a client is selected in one of the comparison views, that is, a customer is clicked in one of the comparison views When the rectangle corresponds to the client, the relevant data corresponding to the client will be displayed in other comparison views. For example, the relevant data corresponding to the client in the other comparison views will be highlighted.

In this embodiment, by controlling the content displayed in the visual view according to the operation instruction and the training process, the display of related content according to the user's requirement is realized, and the intelligence of the display content of the visual view is improved. Further, the embodiment of the present application can determine the overall operating status of each client during the iterative training process of the horizontal federated learning model and the overall operating status of each client through visual views such as summary view, projection view, overview view, comparison view, and contribution ranking view. The correlation of the running data between the terminals, so as to detect the abnormal situation in the iterative training process of the horizontal federated learning model and the contribution of each client to the horizontal federated learning model according to the visual view.

In addition, the present application also provides a view display device based on a federated learning model. Referring to FIG. 6, the view display device based on a federated learning model includes:

The obtaining module 10 is configured to obtain the running data of each client corresponding to the horizontal federated learning model during the iterative training process of the horizontal federated learning model;

The construction module 20 is configured to construct a visual view corresponding to the horizontal federated learning model according to the operating data;

The determining module 30 is configured to determine the training process of the horizontal federated learning model; and determine the content displayed in the visual view according to the training process.

Further, the operating data includes at least one of the following: client identification, the start timestamp of each iteration of training, the number of training samples corresponding to the client, the loss value corresponding to the local model, and the local model trained by each client through local data The start time point and the end time point of, the visual view includes an overview view;

The building module 20 includes:

The coding unit is used to perform visual coding on the operating data to obtain the visualized operating data;

The first construction unit is used to construct an overview view corresponding to the horizontal federated learning model according to the visualized operating data.

Further, the visual view includes a projection view, and the construction module 20 further includes:

The first determining unit is configured to determine index data corresponding to each client terminal according to the operating data;

The second construction unit is configured to construct a projection view corresponding to the horizontal federated learning model according to the index data, wherein each node in the projection view respectively represents a mapping relationship between a client identifier and the number of iterations.

Further, the visual view includes a summary view, and the construction module 20 further includes:

The second determining unit determines statistical data corresponding to the operating data, where the statistical data includes at least one of the following: the number of clients corresponding to the horizontal federated learning model, the number of iterations, and training the horizontal federated learning model The number of changes in the number of samples to be trained, the reduction value corresponding to the loss value corresponding to the horizontal federated learning model, and the increase value of the recognition accuracy of the local model corresponding to each client;

The third construction unit is configured to construct a summary view corresponding to the horizontal federated learning model according to the statistical data.

Further, the visual view includes a comparison view, and the construction module 20 further includes:

The third determining unit is configured to determine the indicator data corresponding to each client according to the operating data, and determine the target indicator corresponding to the last iterative training of the horizontal federated learning model in the indicator data;

The fourth construction unit is used to construct a comparison view corresponding to the horizontal federated learning model according to the target index.

Further, the visual view includes a contribution ranking view, and the construction module 20 further includes:

The fourth determining unit is used to determine the ranking order of the running data of the client in each iteration of the training;

The display unit is configured to display the ranking training in the form of a box-and-whisker graph to construct a ranking view of contribution degrees corresponding to the horizontal federated learning model.

Further, the view display device based on the federated learning model further includes:

The detection module is used to detect whether an operation instruction to operate the visual view is received;

The determination module 30 is further configured to determine the content displayed in the visual view according to the operation instruction and the training process if the operation instruction is received.

The specific implementation of the view display device based on the federated learning model of the present application is basically the same as the foregoing embodiments of the view display method based on the federated learning model, and will not be repeated here.

In addition, this application also provides a view display device based on the federated learning model. As shown in FIG. 7, FIG. 7 is a schematic structural diagram of the hardware operating environment involved in the solution of the embodiment of the present application.

It should be noted that FIG. 7 can be a structural schematic diagram of the hardware operating environment of the display device based on the federated learning model. The view display device based on the federated learning model in the embodiment of the present application may be a terminal device such as a PC and a portable computer.

As shown in FIG. 7, the view display device based on the federated learning model may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art can understand that the structure of the view display device based on the federated learning model shown in FIG. 7 does not constitute a limitation on the view display device based on the federated learning model, and may include more or less components than shown in the figure. Or some parts are combined, or different parts are arranged.

As shown in FIG. 7, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a view display program based on a federated learning model. Among them, the operating system is a program that manages and controls the hardware and software resources of the view display device based on the federated learning model, and supports the running of the view display program based on the federated learning model and other software or programs.

In the view display device based on the federated learning model shown in FIG. 7, the user interface 1003 is mainly used to connect to the terminal device and perform data communication with the terminal device, such as receiving the image to be recognized or the image to be trained sent by the terminal device; the network interface 1004 Mainly used for back-end server to communicate with back-end server; the processor 1001 can be used to call the view display program based on the federated learning model stored in the memory 1005, and execute the steps of the view display method based on the federated learning model as described above .

In addition, an embodiment of the present application also proposes a computer-readable storage medium that stores a view display program based on a federated learning model when the view display program based on a federated learning model is executed by a processor Implement the steps of the view display method based on the federated learning model as described above.

The specific implementation of the computer-readable storage medium of the present application is basically the same as the foregoing embodiments of the view display method based on the federated learning model, and will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or device. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or device that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A view display method based on a federated learning model, wherein the view display method based on a federated learning model includes the following steps:

Acquiring the running data of each client during the iterative training process of the horizontal federated learning model;

Construct a visual view corresponding to the horizontal federated learning model according to the operating data, and determine the training process of the horizontal federated learning model;

The content displayed in the visual view is determined according to the training process.
The view display method based on the federated learning model according to claim 1, wherein the operating data includes at least one of the following: client identification, start timestamp of each iteration of training, the number of training samples corresponding to the client, and local The loss value corresponding to the model, the start time point and the end time point at which each client trains the local model through local data, and the visual view includes an overview view;

The step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data includes:

Performing visual coding on the operating data to obtain visualized operating data;

An overview view corresponding to the horizontal federated learning model is constructed according to the visualized operating data.
The view display method based on the federated learning model according to claim 1, wherein the visual view comprises a projection view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data comprises:

Determine the indicator data corresponding to each client according to the operating data;

Construct a projection view corresponding to the horizontal federated learning model according to the indicator data, wherein each node in the projection view respectively represents a mapping relationship between a client identifier and the number of iterations.
The view display method based on the federated learning model according to claim 1, wherein the visual view comprises a summary view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data comprises:

Determine the statistical data corresponding to the operating data, where the statistical data includes at least one of the following: the number of clients corresponding to the horizontal federated learning model, the number of iterations, and the number of samples to be trained for training the horizontal federated learning model The number of changes in the, the reduction value corresponding to the loss value corresponding to the horizontal federated learning model, and the increase value of the recognition accuracy of the local model corresponding to each client;

Construct a summary view corresponding to the horizontal federated learning model according to the statistical data.
The view display method based on the federated learning model according to claim 4, wherein the step of constructing a summary view corresponding to the horizontal federated learning model according to the statistical data comprises:

According to the statistical data, the number of iterations corresponding to the horizontal federated learning model is taken as the abscissa and the corresponding statistical data is taken as the ordinate to construct a summary view of each statistical data corresponding to the horizontal federated learning model.
The view display method based on a federated learning model according to claim 1, wherein the visual view comprises a comparison view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data comprises:

Determine the indicator data corresponding to each client according to the operating data, and determine the target indicator corresponding to the last iterative training of the horizontal federated learning model in the indicator data;

Construct a comparison view corresponding to the horizontal federated learning model according to the target index.
7. The view display method based on a federated learning model according to claim 6, wherein the step of constructing a comparison view corresponding to the horizontal federated learning model according to the target index comprises:

In the same iterative training process, the target index corresponding to the neighboring client in the federated learning network structure is obtained, and the corresponding comparison view is constructed according to the target index corresponding to the neighboring client.
The view display method based on the federated learning model according to claim 1, wherein the visual view includes a contribution ranking view, and the step of constructing a visual view corresponding to the horizontal federated learning model according to the operating data include:

Determine the ranking order of the client's running data in each iteration of the training;

The ranking training is displayed in the form of a box-and-whisker graph to construct a contribution ranking view corresponding to the horizontal federated learning model.
The view display method based on a federated learning model according to any one of claims 1 to 8, wherein the visual view corresponding to the lateral federated learning model is constructed according to the operating data, and the lateral federated learning is determined After the steps of the model training process, it also includes:

Detecting whether an operation instruction to operate the visual view is received;

If the operation instruction is received, the step of determining the content displayed in the visual view according to the training process includes:

The content displayed in the visual view is determined according to the operation instruction and the training process.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes:

An obtaining module, which is used to obtain the running data of each client corresponding to the horizontal federated learning model during the iterative training process of the horizontal federated learning model;

A building module for building a visual view corresponding to the horizontal federated learning model according to the operating data;

The determining module is used for determining the training process of the horizontal federated learning model; and determining the content displayed in the visual view according to the training process.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes a memory, a processor, and a federated learning model-based device that is stored in the memory and can run on the processor. A view display program, which implements the steps of the view display method based on the federated learning model according to claim 1 when the view display program based on the federated learning model is executed by the processor.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes a memory, a processor, and a federated learning model-based device that is stored in the memory and can run on the processor. A view display program, when the view display program based on the federated learning model is executed by the processor, the steps of the view display method based on the federated learning model as claimed in claim 2 are implemented.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes a memory, a processor, and a federated learning model-based device that is stored in the memory and can run on the processor. A view display program, when the view display program based on the federated learning model is executed by the processor, the steps of the view display method based on the federated learning model according to claim 3 are implemented.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes a memory, a processor, and a federated learning model-based device that is stored in the memory and can run on the processor. A view display program, when the view display program based on the federated learning model is executed by the processor, the steps of the view display method based on the federated learning model according to claim 4 are implemented.
A view display device based on a federated learning model, wherein the view display device based on a federated learning model includes a memory, a processor, and a federated learning model-based device that is stored in the memory and can run on the processor. A view display program, which implements the steps of the view display method based on the federated learning model according to claim 6 when the view display program based on the federated learning model is executed by the processor.
A computer-readable storage medium, wherein a view display program based on a federated learning model is stored on the computer-readable storage medium, and the view display program based on a federated learning model is executed by a processor as described in claim 1. The steps of the view display method based on the federated learning model are described.
A computer-readable storage medium, wherein a view display program based on a federated learning model is stored on the computer-readable storage medium, and the view display program based on a federated learning model is executed by a processor to realize The steps of the view display method based on the federated learning model are described.
A computer-readable storage medium, wherein a view display program based on a federated learning model is stored on the computer-readable storage medium, and when the view display program based on a federated learning model is executed by a processor, the implementation is as described in claim 3. The steps of the view display method based on the federated learning model are described.
A computer-readable storage medium, wherein a view display program based on a federated learning model is stored on the computer-readable storage medium, and when the view display program based on a federated learning model is executed by a processor, the implementation is as described in claim 4 The steps of the view display method based on the federated learning model are described.
A computer-readable storage medium, wherein a view display program based on a federated learning model is stored on the computer-readable storage medium, and when the view display program based on a federated learning model is executed by a processor, the implementation is as described in claim 6 The steps of the view display method based on the federated learning model are described.