CN115828022A - Data identification method, federal training model, device and equipment - Google Patents
Data identification method, federal training model, device and equipment Download PDFInfo
- Publication number
- CN115828022A CN115828022A CN202310145037.2A CN202310145037A CN115828022A CN 115828022 A CN115828022 A CN 115828022A CN 202310145037 A CN202310145037 A CN 202310145037A CN 115828022 A CN115828022 A CN 115828022A
- Authority
- CN
- China
- Prior art keywords
- model
- training
- local
- data
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The embodiment of the specification discloses a data identification method, a federal training model, a device and equipment. The method comprises the following steps: acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training; identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants; training a target program based on the recognition result, and determining a trained target model corresponding to the target program; and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a data identification method, a federal training model, a device and equipment.
Background
Model fusion is an important means for realizing the upgrading and evolution of model capability, and can also be understood as a mode for model training through multi-party collaborative learning. The currently adopted fusion modes include a fusion mode based on an existing model taking ensemble learning as an example, and a fusion mode based on an existing parameter taking federal learning as an example. However, in these fusion modes, the structures of the models that participate in the fusion are generally the same, and the collaborative learning training of multiple parties cannot be realized for models with different structures.
Therefore, how to provide a method capable of performing collaborative learning training on heterogeneous models is an urgent technical problem to be solved.
Disclosure of Invention
The embodiment of the specification provides a data identification method, a federal training model, a device and equipment, which are used for solving the following technical problems: the existing method for identifying data to be processed based on the existing model to obtain an identification result cannot further improve the accuracy of the data identification result.
In order to solve the above technical problem, the embodiments of the present specification are implemented as follows:
the data identification method provided by the embodiment of the specification comprises the following steps:
acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
training a target program based on the recognition result, and determining a trained target model corresponding to the target program;
and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
An embodiment of the present specification further provides a joint training model, including:
the model requiring party acquires the collaborative training model from other participating parties participating in the federal collaborative training; the collaborative training model is a model with optimal performance among other participants; the model requiring party is any party participating in the federal collaborative training;
recognizing local data to be recognized by using the collaborative training model to obtain a recognition result, wherein the local data to be recognized is local data of the model demander;
training a target program based on the recognition result to obtain a trained first model, wherein the target program performs machine learning based on the recognition result;
evaluating the trained first model based on a first local evaluation sample to obtain a first evaluation result; the first local evaluation sample is a sample in a local training sample set of the model demander;
and if the first evaluation result meets a preset condition, determining the trained first model as a trained target model corresponding to the target program.
An embodiment of the present specification further provides a data identification apparatus, where the apparatus includes:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module acquires local data to be identified, the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
the first identification module is used for identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model demand party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
the training module is used for training a target program based on the recognition result and determining a trained target model corresponding to the target program;
and the second identification module is used for inputting the local data to be identified into the trained target model corresponding to the target program to obtain an updated identification result.
An embodiment of the present specification further provides a data identification device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to:
acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any participant participating in federal collaborative training;
identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
training a target program based on the recognition result, and determining a trained target model corresponding to the target program;
and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
One embodiment of the present description can achieve at least the following advantages: in the embodiment of the description, the target program is trained according to the recognition result of the local data to be recognized, the trained target program corresponding to the target program is obtained, and then the local data to be recognized is recognized based on the trained target program corresponding to the target program, so that the updated recognition result is obtained. According to the data identification method, the local data to be identified of the model demander is identified by using the models of other participators, and the local target program is trained based on the identification result, so that the local target program can learn the capabilities of the models of the other participators, and the local target program does not need to be identical to the models of the other participators in structure. In addition, the trained model is evaluated by using the local evaluation sample, so that the recognition effect of the trained model in the local can be ensured, and the problems of poor local data adaptability and the like can be effectively reduced. Therefore, the data identification method enables the identification result to be more accurate.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a schematic diagram of a system architecture provided in an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of a data identification method provided in an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a joint training model provided in an embodiment of the present specification;
FIG. 4 is a schematic flow chart of a Federation training model provided in an embodiment of the present disclosure;
fig. 5 is a schematic diagram of a data identification device according to an embodiment of the present disclosure.
Detailed Description
The existing method is based on the existing model, the data to be processed is identified, and methods such as ensemble learning and federal learning are often adopted.
Ensemble learning is a method for fusing isomorphic or heterogeneous models, a better and more comprehensive 'strong model' is obtained mainly by combining a plurality of 'weak models' with poorer or more single performances, common fusion methods comprise an averaging method, a voting method and a learning method, and typical algorithms comprise a Boosting series and a Bagging series.
Federal learning is a distributed machine learning technology, which performs distributed model training on a multipoint data source, realizes model training in a global scope by exchanging model parameters and intermediate results, and achieves the privacy protection and shared calculation effects of 'data available and invisible'.
In the prior art, aiming at ensemble learning, direct fusion of a local existing model is emphasized, on one hand, access and fusion of foreign model capability cannot be supported, and on the other hand, because each model may depend on different data sets in front, the direct fusion mode cannot ensure the "precedence" of a new model in a local data set, namely, the model is more suitable for use of local data, and the problems of poor local data adaptability and the like are possibly caused; for federal learning, the method emphasizes parameter and intermediate result exchange of multiple models in a distributed environment, requires that data sets depended on before each model are similar in source and structure, and has certain application limitation.
Due to the defects of the prior art, the accuracy of the identification result is not high in the data identification method based on the prior model, and the accuracy of the identification data cannot be rapidly improved.
Based on this, an embodiment of the present specification provides a data identification method, which trains a target program according to an identification result of local data to be identified, to obtain a trained target program corresponding to the target program, and further identifies the local data to be identified according to the trained target program corresponding to the target program, to obtain an updated identification result.
To make the objects, technical solutions and advantages of one or more embodiments of the present disclosure more apparent, the technical solutions of one or more embodiments of the present disclosure will be clearly and completely described below with reference to specific embodiments of the present disclosure and the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present specification, and not all embodiments. All other embodiments that can be derived by a person skilled in the art from the embodiments given herein without making any creative effort fall within the protection scope of one or more embodiments of the present disclosure.
It is to be understood that, although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited by these terms. These terms are only used to distinguish one type of information from another.
The technical solutions provided by the embodiments of the present description are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a system architecture provided in an embodiment of the present disclosure. As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is the medium used to provide communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, 103 interact with a server 105 via a network 104 to receive or send messages or the like. Various client applications may be installed on the terminal devices 101, 102, 103. Such as performing specialized procedures for data recognition, federal model training, etc.
The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various special purpose or general purpose electronic devices including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module.
The server 105 may be a server that provides various services, such as a backend server that provides services for client applications installed on the terminal devices 101, 102, 103. For example, the server may train and run a trained target model corresponding to the target program, and implement model training so as to display the recognition result on the terminal device 101, 102, 103.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module.
Fig. 2 is a schematic flowchart of a data identification method according to an embodiment of the present disclosure. From the viewpoint of the program, the execution subject of the flow may be a program installed in an application server or an application terminal. It is to be appreciated that the method can be performed by any apparatus, device, platform, cluster of devices having computing and processing capabilities. As shown in fig. 2, the data recognition method includes:
step S201: the method comprises the steps of obtaining local data to be identified, wherein the local data to be identified is local data of a model demand party, and the model demand party is any party participating in federal collaborative training.
The local data to be identified can be local data of the model demander, and can also be data to be identified, which needs to be identified by the model demander. In the embodiment of the present specification, the local data to be recognized may be recognized by using the collaborative training model obtained from other participants, so as to obtain a recognition result. The recognition result may represent data including an attribute value tag, and the attribute value may represent a recognition object included in the data to be recognized. For example, the collaborative training model is a model for identifying illegal characters, the local data to be identified is an image or a character including a plurality of characters, and the collaborative training model can mark the illegal characters included in the local data to be identified to obtain an identification result including a label. As another example, a co-trained model is a model for a class of objects, such as identifying whether a military device is included in an image; the local data to be identified is an image containing various objects, such as an image to be issued to a network by a network user; the collaborative training model may mark a military device included in the local data to be identified to obtain an identification result including a tag, for example, if an image to be issued by a user includes an airplane image, the identification result may be an image for identifying the airplane.
Step S203: and identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model demand party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants.
In the embodiment of the specification, the same or similar models can be subjected to federal cooperative training. For example, the models used for recognizing or classifying images can be subjected to federal collaborative training, so that the local model of the model can learn the advantages of other participant models, and the performance of the local model is improved. More specifically, the model participating in the collaborative training may be a model for recognizing characters, symbols, and objects in an image, a model for performing compliance recognition and risk recognition, or a model for recognizing a person, an animal, a vehicle, and a flying device.
For example, in order to ensure the health of network information, it is necessary to classify or check data such as characters and images that need to be distributed on the network, and to prevent unhealthy information from spreading in the network. For another example, in order to acquire a certain type of information in an image, the model in the embodiment of the present specification may be a model that identifies the type of information, for example, in order to acquire information of a vehicle in a road, the model may be a model that identifies the vehicle. It should be understood that the method in the embodiments of the present specification is only an example for explaining the method in the embodiments of the present specification, and the method in the embodiments of the present specification may be applied to various types of models, and is not limited specifically herein.
In the embodiment of the present description, the model demander may use a model with the best performance in the models of the other participants as a collaborative training model, and obtain a trained model based on the collaborative training model, so that the trained model may learn the advantages of the collaborative training model.
Step S205: and training the target program based on the recognition result, and determining a trained target model corresponding to the target program.
In an embodiment of this specification, the training a target program based on the recognition result, and determining a trained target model corresponding to the target program specifically include:
training a target program based on the recognition result to obtain a trained first model, wherein the target program performs machine learning based on the recognition result;
evaluating the trained first model based on a first local evaluation sample to obtain a first evaluation result, wherein the first local evaluation sample is a sample in a local training sample set of the model demander;
and if the first evaluation result meets a preset condition, determining the trained first model as a trained target model corresponding to the target program.
In this embodiment of the present specification, a local target program may be trained by using a recognition result obtained by recognizing local to-be-recognized data of a model demander by using a collaborative training model provided by another participant, so that the local target program performs machine learning based on the recognition result, and further, the target program may learn from the collaborative training model.
In the embodiment of the present specification, the object program may represent a model algorithm used by a model construction phase model demander for constructing a model, or may represent a model to be updated at a model update phase model demander. The data identification method provided in the embodiments of the present description may perform data identification based on a model in a construction phase, or may perform data identification based on a model in an update phase. In practical application, in a model building stage, a model demander can represent a technician for building a model; in the model update phase, the model demander may represent a technician performing the model update, and when the model update may be performed at the model consumer, the model demander may also represent the model consumer.
The local training sample set may include training samples labeled based on expert experience, or labeled samples obtained based on analysis behaviors of a model user. For example, in practical application, the identification data may be preliminarily identified through the network model, and then the identification data including the identification result is sent to a display terminal of a data manager or an auditor, so that the data manager or the auditor may manually check the identification data. When the data management or the auditor audits the information, the data management or auditor can perform analysis behaviors such as browsing, confirming, backing up, modifying and the like on the identification data, and can use the identification data which is in a confirmed state and has the modified data attribute value as training sample data. In practical application, a network model is adopted to identify data to be identified, and an obtained identification result can represent a data attribute value of the data; in the manual verification process, the labeling result of the data management or the auditor on the identification data can also represent the data attribute value of the identification data. In the manual inspection stage, when the data management or the auditor considers that the data attribute value of the audited identification data is incorrect, the data attribute value can be modified to be the correct data attribute value.
In the example of the present specification, the trained first model may be evaluated by using local evaluation data of the model demander, so that the adaptive capacity of the trained first model to the local model demander may be more accurately obtained, and if the trained first model has better local performance, the trained first model may be used as the trained target model corresponding to the target program. In the stage of model federation training, the trained first model can be used as a constructed model; in the model update phase, the trained first model may be used as the updated model.
The method in the embodiment of the present specification may be applied to a model building phase and may also be applied to a model updating phase, where the target program may include a model to be updated in a model demander; the target model may include an updated model corresponding to the model to be updated; alternatively, the target program may include a preset machine learning algorithm; the target model comprises a machine model constructed based on the machine learning algorithm, namely the target model is a machine model constructed based on a target program.
Step S207: and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
And the trained target model corresponding to the target program enables the data effect of the model to be better, so that the local data to be recognized is input into the trained target model corresponding to the target program to obtain an updated recognition result, and the updated recognition result is finally output or displayed.
In order to further ensure that the model can better adapt to local requirements, in this embodiment of the present description, the collaborative training model may be determined from other participants based on local evaluation samples, and optionally, in this embodiment of the present description, the model demander obtains the collaborative training model from other participants participating in federal collaborative training, which may specifically include:
obtaining the latest version of the model owned by each other participant;
evaluating each model of the latest version based on the second local evaluation sample to obtain a second evaluation result; the second local evaluation sample is a part of the local training samples of the model demander;
and determining the model with the best performance in the models of the latest versions as the collaborative training model based on the second evaluation result.
In the example of the present specification, a certain number of samples may be extracted from the local training samples of the model demander as second local evaluation samples, where the second local evaluation samples may be the same as or different from the first evaluation samples, and the model with the optimal evaluation result is determined as the collaborative training model by evaluating the latest version of the model owned by other participants using the second evaluation samples. As an embodiment, the accuracy of the latest version of the model owned by the other participants may be calculated using the second local evaluation sample, wherein the model with the highest accuracy is determined as the co-training model.
In this embodiment of the present description, a target program may also be trained by combining a local training sample, and optionally, the training of the target program based on the recognition result in this embodiment of the present description may specifically include:
obtaining the rest local training samples; the rest local training samples are part or all of the local training samples except for a second local evaluation sample;
and taking the recognition result and the residual local training samples as target training samples to train the target program.
Assuming that the local training sample D0, the evaluation subset DE0 is extracted from the local training sample D0, and the above obtained recognition result is D1, the target training sample Dx may be Dx = D0-DE0+ D1. A partial sample D2 may be selected from D0-DE0, and the partial sample D2 and the recognition result D1 are used as the target training sample, that is, dx = D2+ D1. It is understood that, in practical applications, the number of samples may be selected according to practical requirements, and is not limited specifically herein.
In practical application, a model information list can be set, model information of each participant can be recorded in the list, and any participant can acquire required model information from the list when needing federal cooperative training. Optionally, in this embodiment of the present specification, the obtaining a model of a latest version owned by each of the other participants may specifically include:
obtaining a model information list; the model information list is used for recording model information owned by each participant participating in the federal collaborative training; the model information comprises a model version number and a model address;
determining the model of the latest version owned by each other participant according to the model version number;
and obtaining the model of the latest version owned by each other participant according to the model address of the model of the latest version owned by each other participant.
In this embodiment, each participant may obtain a local model training result by using a respective local training sample, and package the local model training result into an executable container and a version number thereof. The model information list may include the version number of the model and the memory address of the trained model, so that the participating party can obtain the required model.
For the model update phase, the target program in this embodiment of the present specification may include a model to be updated in the model demander; the target model may include an updated model corresponding to the model to be updated; the method may further comprise:
evaluating the model to be updated based on the first local evaluation sample to obtain a third evaluation result;
the first evaluation result satisfies a preset condition, and specifically may include:
the first evaluation result is better than the third evaluation result.
In practical application, the model to be updated and the trained first model may be evaluated by using the first local evaluation sample, and if the evaluation result indicates that the trained first model has better performance than the model to be updated, the model to be updated may be updated to the trained first model, and the trained first model is used to replace the model to be updated. If the trained first model is not better than the model to be updated in performance, the model to be updated can be not updated, and the model can be kept in use. And the collaborative training model can be obtained from other participants again to perform a new round of training.
As another embodiment, in this specification, a local training sample may also be used to train a target program to obtain a model corresponding to the target program, and the model corresponding to the trained target program is compared with a model obtained by training recognition results obtained by using models of other participants, so as to select a better model. Specifically, the method in the embodiment of the present specification may further include:
acquiring a first local training sample, and training the target program to obtain a trained second model; the first local training sample is part or all of a local training sample set of the model demander;
evaluating the trained second model based on the first local evaluation sample to obtain a fourth evaluation result;
the first evaluation result satisfies a preset condition, and specifically may include:
the first evaluation result is better than the fourth evaluation result.
It is understood that, in the present embodiment, the trained second model is a model obtained by training based on the target program.
In the embodiment of the present specification, a local target program may be trained by using a model demander, for example, a basic model algorithm is trained by using local data to construct a model, or a model to be updated is trained to obtain a trained model. The trained second model obtained by training can be evaluated by using the first local evaluation sample, and if the evaluation results of the two models indicate that the trained first model has better performance than the trained second model, the trained first model can be used as the trained target model corresponding to the target program. In practical application, if the performance of the trained first model is not better than that of the trained second model, the trained second model can be used as a trained target model corresponding to the target program.
The evaluation result in the embodiment of the present specification may represent an index result such as accuracy, precision, recall rate, AUC value, and the like of the model. The model quality can be compared aiming at one index result, and the model quality can also be compared by comprehensively considering various indexes.
In this embodiment of the present description, it may also be determined whether the trained model satisfies a condition based on performance of the model itself, where the first evaluation result satisfies a preset condition, and specifically, the determining may include:
the evaluation value represented by the first evaluation result is greater than or equal to a preset threshold value; the evaluation value comprises at least one of accuracy, precision, recall and AUC value.
Wherein whether the model meets the condition may be determined based on one or more evaluation indicators.
To facilitate subsequent model training, the method in this embodiment of the present specification may further include:
storing the version number and the model address of the target model into a model information list; the model information list is used for recording model information owned by each participant participating in the federal collaborative training; the model information includes a model version number and a model address.
In practical application, the trained target model can be packaged into an executable container, and the version number and the address of the model are recorded. The version number may be represented by incremental data or may be represented by a model generation time, and the specific version number is not limited herein.
The method in the embodiment of the specification can be applied to heterogeneous models for carrying out federated collaborative training, and the models of all the participants can be wholly or partially heterogeneous, wherein the collaborative training model can be heterogeneous with a target program.
Due to the fact that the trained target model corresponding to the target program is based on the joint training of the data resources distributed in multiple places, the data resources in multiple places are used for completing federal learning modeling on the premise that data samples in various places do not exist. The local to-be-recognized data of the model demander is recognized by using the models of other participants, and the local target program is trained based on the recognition result, so that the local target program can learn the capabilities of the models of the other participants, the local target program does not need to have the same structure as the models of the other participants, and the isomorphic limitation of the models participating in the traditional federal learning technology is removed. In addition, the trained model is evaluated by using the local evaluation sample, so that the recognition effect of the trained model in the local can be ensured, and the problems of poor local data adaptability and the like can be effectively reduced. Therefore, by adopting the data identification method provided by the embodiment of the specification, the accuracy of data identification can be improved.
An embodiment of the present specification further provides a joint training model, and fig. 3 is a schematic flowchart of a joint training model provided in the embodiment of the present specification. The federal training model comprises the following steps:
step S301: the model requiring party acquires the collaborative training model from other participating parties participating in the federal collaborative training; the collaborative training model is a model with optimal performance among other participants; the model requiring party is any party participating in the federal cooperative training;
s303: and recognizing local data to be recognized by using the collaborative training model to obtain a recognition result, wherein the local data to be recognized is local data of the model demander.
S305: and training a target program based on the recognition result to obtain a trained first model, and performing machine learning on the target program based on the recognition result.
S307: evaluating the trained first model based on a first local evaluation sample to obtain a first evaluation result; the first local evaluation sample is a sample in a local training sample set of the model demander.
S309: and if the first evaluation result meets a preset condition, determining the trained first model as a trained target model corresponding to the target program.
To more clearly illustrate the method for federal training of a model provided in this specification, fig. 4 is a schematic flow chart of a federal training model provided in this specification, and as shown in fig. 4, in this specification, training of the model may be performed in a local environment of a model requiring party, which may specifically include:
step 401: the model demander obtains the model from the other participants.
Where the latest version of the model at the other participants may be obtained. In practical application, the timing model update may be performed based on the method in the embodiment of the present specification, or the model update may be performed when other participants have a new version of the model generated, and the specific time is not specifically limited here.
Step 403: and (4) performing sample extraction from the local training samples to serve as local evaluation samples.
For example, for a model identified by compliance, the evaluation sample may be a sample labeled with a compliance object, such as data labeled with whether compliance is satisfied or not, and data labeled with specific violation content, for example, the compliance content in certain data is labeled as 0, the violation content is labeled as 1, and the evaluation sample may be data containing attribute values of 0, 1, and the like. In practical applications, the specific object content may be represented by an attribute value, and is not limited herein.
Step 405: and evaluating the models of other participants by using the local evaluation sample to obtain an optimal model, and taking the optimal model as a collaborative training model. Assuming that the model 4 in fig. 4 is an optimal model, the model can be used as a collaborative training model for the current training round. The data resource structure and the result type in the model of each participant may be the same, for example, both may be in a key-value form, so that each participant performs model learning.
Step 407: and the model demander extracts partial data from the local original data as local data to be identified.
The original data can be data that needs to be identified by the model demander. For example, the model demander is an organization for performing compliance audit on network data, and the data to be identified may be characters or pictures to be published to the network by a network user. For example, a user wants to issue some texts or pictures to a network through a certain terminal application terminal, a corresponding auditing mechanism can perform compliance auditing on the texts or pictures provided by the user, wherein the auditing mechanism can perform identification by using a compliance identification model, judge whether the texts or pictures contain illegal contents, and also identify the illegal contents contained in the texts or pictures.
Step 409: and identifying the local data to be identified by using the determined collaborative training model to obtain an identification result. Wherein, the identification result may represent data containing attribute value tags. For example, the collaborative training model may be a model for performing compliance recognition by other participants, and the recognition result may be data labeled with violation content, that is, data with a label.
Step 411: and taking the residual samples in the local training samples and the recognition result as target training samples for training the local target program. It may be assumed here that the target program is an old model that needs to be updated at the data demander.
Step 413: and training the local target program based on the target training sample to obtain a trained model, which can also be called a new model.
Step 415: and evaluating the trained model by using a local evaluation sample.
Step 417: if the evaluation indicates that the trained model is better than the previous old model (i.e., the target program), the previous old model may be updated with the trained model.
In practical application, if the trained model has better performance, the recognition result of the local data recognized by other participants is beneficial to improving the performance of the local model, so that the local model can learn from the models of other participants, and the local model can learn the advantages of the models of other participants. For example, on the premise that the models of other parties can correctly identify the data of the local evaluation sample, finer or wider violation contents can be identified, so that when the models of other parties are used for identifying the local data to be identified, identification results including the finer or wider violation contents are obtained, and the identification performance of the local model can be improved by using the identification results.
In practical application, if the evaluation result shows that the performance of the trained model is not better, the model can be subjected to federal training again according to the steps after preset time.
In practical applications, although each model trained by the federal system has the same function, for example, illegal characters are recognized, the recognized data may be different due to different application environments of the models, and each model may also exhibit different advantages as each model is trained or updated in each participant. For example, a model of a party may identify more offending characters. In the example of the specification, the local model can learn the model with advantages to other participants through a collaborative training mode, and the performance of the local model is further improved.
The method provided by the embodiment of the specification can support the fusion of the outlier and the heterogeneous model, and an asynchronous fusion mode is adopted, so that the states of the models are not influenced. And moreover, an indirect capability fusion mode of model-sample-model is adopted, so that the problems of poor local data adaptability and the like possibly caused by direct fusion of the model-model are effectively solved. In addition, the model fusion method provided by the invention cannot cause continuous 'expansion' of the model, thereby supporting the continuous evolution of the model.
The method provided by the embodiment of the specification can be used for carrying out combined training on the data resources distributed in multiple places, and complete federal learning modeling by utilizing the data resources in multiple places on the premise that data samples in various places do not appear in the field. In addition, in the embodiment of the specification, models are interacted in multiple places, the models are not weights in the training process, the local answer set is used for evaluating the capacity of the model in the different places, when the recognition result of the multiple models conflicts, the model with the highest accuracy on the local answer set is taken as the standard, the capacity fusion of the models in the multiple places can be completed as long as the data resource structure and the result type are the same, and the isomorphic limitation on the model participating in the traditional federal learning technology is removed.
Homogeneity requirements on model types and technical systems under traditional federal learning are avoided, and the federal learning can be completed by different types of models as long as data resource structures and result types are the same, for example, key-value structures are the same; meanwhile, local models can be used in local training in various places, model fusion time is set according to needs, and the model fusion time is not influenced by a traditional federal learning mechanism.
The foregoing embodiments of the present specification provide a data identification method, and based on the same idea, the embodiments of the present specification further provide a data identification device. Fig. 5 is a schematic diagram of a data identification device provided in an embodiment of the present specification, where the data identification device includes:
the acquiring module 501 acquires local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
the first identification module 503 is configured to identify the local data to be identified based on a collaborative training model, so as to obtain an identification result, where the collaborative training model is obtained by a model requiring party from other participating parties participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participating parties;
a training module 505, configured to train a target program based on the recognition result, and determine a trained target model corresponding to the target program;
the second recognition module 507 inputs the local data to be recognized into the trained target model corresponding to the target program, and obtains an updated recognition result.
An embodiment of the present specification further provides a data identification device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to:
acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
training a target program based on the recognition result, and determining a trained target model corresponding to the target program;
and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
While particular embodiments of the present specification have been described above, in some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the various embodiments can be referred to each other.
The apparatus, the device, and the method provided in the embodiments of the present specification are corresponding, and therefore, the apparatus and the device also have beneficial technical effects similar to those of the corresponding method, and since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus and device are not described again here.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises that element.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (11)
1. A method of data identification, the method comprising:
acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
training a target program based on the recognition result, and determining a trained target model corresponding to the target program;
and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
2. The data recognition method of claim 1, wherein the training of the target program based on the recognition result and the determination of the trained target model corresponding to the target program specifically comprise:
training a target program based on the recognition result to obtain a trained first model, wherein the target program performs machine learning based on the recognition result;
evaluating the trained first model based on a first local evaluation sample to obtain a first evaluation result, wherein the first local evaluation sample is a sample in a local training sample set of the model demander;
and if the first evaluation result meets a preset condition, determining the trained first model as a trained target model corresponding to the target program.
3. The data recognition method of claim 1, wherein the collaborative training model is obtained by a model demanding party from other participating parties participating in federated collaborative training, and specifically comprises:
obtaining the latest version of the model owned by each other participant;
evaluating each model of the latest version based on a second local evaluation sample to obtain a second evaluation result, wherein the second local evaluation sample is a part of samples in local training samples of the model demander;
and determining the model with the best performance in the models of the latest versions as the collaborative training model based on the second evaluation result.
4. The method according to claim 2, wherein the training the target program based on the recognition result to obtain the trained first model specifically comprises:
obtaining the rest local training samples; the rest local training samples are part or all of the local training samples except for a second local evaluation sample;
and taking the recognition result and the residual local training samples as target training samples to train the target program.
5. The data recognition method of claim 3, wherein the obtaining of the latest version of the model owned by each of the other participants specifically comprises:
obtaining a model information list; the model information list is used for recording model information owned by each participant participating in federated collaborative training, and the model information comprises a model version number and a model address;
determining the latest version of the model owned by each other participant according to the version number of the model;
and obtaining the model of the latest version owned by each other participant according to the model address of the model of the latest version owned by each other participant.
6. The data identification method of claim 2, wherein the target program includes a model to be updated in the model demander; the target model comprises an updated model corresponding to the model to be updated; the method further comprises the following steps:
evaluating the model to be updated based on the first local evaluation sample to obtain a third evaluation result;
the first evaluation result meets a preset condition, and specifically includes:
the first evaluation result is better than the third evaluation result.
7. The data identification method of claim 2, wherein the method further comprises:
acquiring a first local training sample, and training the target program to obtain a trained second model; the first local training sample is part or all of a local training sample set of the model demander;
evaluating the trained second model based on the first local evaluation sample to obtain a fourth evaluation result;
the first evaluation result meets a preset condition, and specifically includes:
the first evaluation result is better than the fourth evaluation result.
8. The data identification method of claim 2, wherein the method further comprises:
storing the version number and the model address of the target model into a model information list; the model information list is used for recording model information owned by each participant participating in the federal collaborative training; the model information includes a model version number and a model address.
9. A federated training model, comprising:
the model requiring party acquires the collaborative training model from other participating parties participating in the federal collaborative training; the collaborative training model is a model with optimal performance among other participants; the model requiring party is any party participating in the federal collaborative training;
recognizing local data to be recognized by using the collaborative training model to obtain a recognition result, wherein the local data to be recognized is local data of the model demander;
training a target program based on the recognition result to obtain a trained first model, wherein the target program performs machine learning based on the recognition result;
evaluating the trained first model based on a first local evaluation sample to obtain a first evaluation result; the first local evaluation sample is a sample in a local training sample set of the model demander;
and if the first evaluation result meets a preset condition, determining the trained first model as a trained target model corresponding to the target program.
10. A data recognition apparatus, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module acquires local data to be identified, the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
the first identification module is used for identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model demand party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
the training module is used for training a target program based on the recognition result and determining a trained target model corresponding to the target program;
and the second identification module is used for inputting the local data to be identified into the trained target model corresponding to the target program to obtain an updated identification result.
11. A data recognition device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring local data to be identified, wherein the local data to be identified is local data of a model demander, and the model demander is any party participating in federal collaborative training;
identifying the local data to be identified based on a collaborative training model to obtain an identification result, wherein the collaborative training model is obtained by a model requiring party from other participants participating in federal collaborative training, and the collaborative training model is a model with optimal performance among the other participants;
training a target program based on the recognition result, and determining a trained target model corresponding to the target program;
and inputting the local data to be recognized into the trained target model corresponding to the target program to obtain an updated recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310145037.2A CN115828022B (en) | 2023-02-21 | 2023-02-21 | Data identification method, federal training model, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310145037.2A CN115828022B (en) | 2023-02-21 | 2023-02-21 | Data identification method, federal training model, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115828022A true CN115828022A (en) | 2023-03-21 |
| CN115828022B CN115828022B (en) | 2023-04-25 |
Family
ID=85522014
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310145037.2A Active CN115828022B (en) | 2023-02-21 | 2023-02-21 | Data identification method, federal training model, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115828022B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116662764A (en) * | 2023-07-28 | 2023-08-29 | 中国电子科技集团公司第十五研究所 | Data identification method for error identification correction, model training method, device and equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408209A (en) * | 2021-06-28 | 2021-09-17 | 淮安集略科技有限公司 | Cross-sample federal classification modeling method and device, storage medium and electronic equipment |
| CN113537512A (en) * | 2021-07-15 | 2021-10-22 | 青岛海尔工业智能研究院有限公司 | Model training method, device, system, equipment and medium based on federal learning |
| CN113762328A (en) * | 2021-06-16 | 2021-12-07 | 京东科技控股股份有限公司 | Model training method, device, equipment and storage medium based on federal learning |
| CN113902122A (en) * | 2021-08-26 | 2022-01-07 | 杭州城市大脑有限公司 | Federal model collaborative training method and device, computer equipment and storage medium |
| CN114332873A (en) * | 2021-12-22 | 2022-04-12 | 北京三快在线科技有限公司 | Training method and device for recognition model |
| CN114676853A (en) * | 2021-04-15 | 2022-06-28 | 腾讯云计算（北京）有限责任公司 | Data processing method, device, equipment and medium |
| US20220366320A1 (en) * | 2021-07-13 | 2022-11-17 | Beijing Baidu Netcom Science Technology Co., Ltd. | Federated learning method, computing device and storage medium |
| CN115358411A (en) * | 2022-08-11 | 2022-11-18 | 联仁健康医疗大数据科技股份有限公司 | Data processing method, device, equipment and medium |
-
2023
- 2023-02-21 CN CN202310145037.2A patent/CN115828022B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114676853A (en) * | 2021-04-15 | 2022-06-28 | 腾讯云计算（北京）有限责任公司 | Data processing method, device, equipment and medium |
| CN113762328A (en) * | 2021-06-16 | 2021-12-07 | 京东科技控股股份有限公司 | Model training method, device, equipment and storage medium based on federal learning |
| CN113408209A (en) * | 2021-06-28 | 2021-09-17 | 淮安集略科技有限公司 | Cross-sample federal classification modeling method and device, storage medium and electronic equipment |
| US20220366320A1 (en) * | 2021-07-13 | 2022-11-17 | Beijing Baidu Netcom Science Technology Co., Ltd. | Federated learning method, computing device and storage medium |
| CN113537512A (en) * | 2021-07-15 | 2021-10-22 | 青岛海尔工业智能研究院有限公司 | Model training method, device, system, equipment and medium based on federal learning |
| CN113902122A (en) * | 2021-08-26 | 2022-01-07 | 杭州城市大脑有限公司 | Federal model collaborative training method and device, computer equipment and storage medium |
| CN114332873A (en) * | 2021-12-22 | 2022-04-12 | 北京三快在线科技有限公司 | Training method and device for recognition model |
| CN115358411A (en) * | 2022-08-11 | 2022-11-18 | 联仁健康医疗大数据科技股份有限公司 | Data processing method, device, equipment and medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116662764A (en) * | 2023-07-28 | 2023-08-29 | 中国电子科技集团公司第十五研究所 | Data identification method for error identification correction, model training method, device and equipment |
| CN116662764B (en) * | 2023-07-28 | 2023-09-29 | 中国电子科技集团公司第十五研究所 | Data identification method for error identification correction, model training method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115828022B (en) | 2023-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427939B (en) | Model generation method and device | |
| JP6765885B2 (en) | Methods and systems for intelligent cloud planning and decommissioning | |
| CN108805091B (en) | Method and apparatus for generating a model | |
| CN108960316B (en) | Method and apparatus for generating a model | |
| CN109976997B (en) | Test method and device | |
| Petersen et al. | Choosing component origins for software intensive systems: In-house, COTS, OSS or outsourcing?—A case survey | |
| KR20200135892A (en) | Method, apparatus and computer program for providing personalized educational curriculum and contents through user learning ability | |
| CN109688428B (en) | Video comment generation method and device | |
| AU2021286415A1 (en) | Course skill matching system and method thereof | |
| US11188517B2 (en) | Annotation assessment and ground truth construction | |
| US11294884B2 (en) | Annotation assessment and adjudication | |
| CN110070076B (en) | Method and device for selecting training samples | |
| US20210311853A1 (en) | Automated breakpoint creation | |
| CN111815169A (en) | Business approval parameter configuration method and device | |
| Al Nawaiseh et al. | A new software quality model for academic information systems ‘case study e-learning system,’ | |
| CN115828022B (en) | Data identification method, federal training model, device and equipment | |
| CN111159241A (en) | Click conversion estimation method and device | |
| CN116601644A (en) | Providing interpretable machine learning model results using distributed ledgers | |
| WO2022038440A1 (en) | Distributed dataset annotation system and method of use | |
| US11378942B2 (en) | Progressive guidance of digital twin model assembly | |
| Lenarduzzi et al. | Towards a methodology for participant selection in software engineering experiments: A vision of the future | |
| CN109840072B (en) | Information processing method and device | |
| Correia et al. | Brazilian data scientists: revealing their challenges and practices on machine learning model development | |
| CN116523263A (en) | Intelligent data processing method, system and storage medium | |
| Michalik et al. | Towards a solution for change impact analysis of software product line products |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |