WO2024032031A1 - 一种数据分析方法及装置 - Google Patents

一种数据分析方法及装置 Download PDF

Info

Publication number
WO2024032031A1
WO2024032031A1 PCT/CN2023/090526 CN2023090526W WO2024032031A1 WO 2024032031 A1 WO2024032031 A1 WO 2024032031A1 CN 2023090526 W CN2023090526 W CN 2023090526W WO 2024032031 A1 WO2024032031 A1 WO 2024032031A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
mtlf
accuracy
evaluation
client
Prior art date
Application number
PCT/CN2023/090526
Other languages
English (en)
French (fr)
Inventor
李卓明
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024032031A1 publication Critical patent/WO2024032031A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design

Definitions

  • the embodiments of the present application relate to the field of communication technology, and in particular, to a data analysis method and device.
  • Federated learning is a type of distributed machine learning that enables data sharing and joint modeling while ensuring data privacy and security.
  • the core idea is to conduct distributed model training among multiple data sources with local databases. Without the need to exchange data samples, only the intermediate parameters of the model need to be exchanged to achieve joint model training.
  • selecting clients to participate in federated learning is a current concern.
  • This application provides a data analysis method and device to determine the clients participating in federated learning during the federated learning process.
  • the first aspect provides a data analysis method.
  • the execution subject of the method is the server MTLF, or a device (circuit, chip or others) configured in the server MTLF.
  • the method includes: The server MTLF sends the first model to each of the N candidate client MTLFs; optionally, the first model is an initial model, an intermediate model, or a final model in the federated learning process.
  • the server MTLF receives first accuracy evaluation information from the N candidate client MTLFs respectively.
  • the first accuracy evaluation information represents the accuracy of the first model determined by the candidate client MTLF using local data, so
  • the N is a positive integer greater than 1; optionally, the local data of the candidate client MTLF is the data collected by the candidate client MTLF in its own service area.
  • the client The local data collected by MTLF in its own service area will not be sent to other client MTLF. Local data can also be called local data set.
  • the server MTLF determines the client MTLF that participates in federated learning among the N candidate client MTLFs based on the N pieces of first accuracy evaluation information.
  • the server MTLF can choose the client MTLF with similar feedback model evaluation information to participate in federated learning.
  • the distribution of their local data is also similar. Selecting client MTLF with similar local data distribution for federated learning can improve the prediction accuracy of the model trained by federated learning.
  • the server MTLF sends a first model evaluation request message to the N candidate client MTLFs respectively, and the first model evaluation request message includes the first model; the server MTLF receives messages from the N candidate clients respectively.
  • the N pieces of first accuracy evaluation information are all the MTLF feedbacks of the N candidate clients.
  • N evaluation values of the first model accuracy, the difference between the evaluation values fed back by any two client MTLFs participating in federated learning is less than or equal to the first threshold.
  • the evaluation value with the largest value and the evaluation value with the smallest value are determined; the difference between the evaluation value with the largest value and the evaluation value with the smallest value is determined , less than or equal to the first threshold, the N candidate clients all participate in federated learning; or, the difference between the largest evaluation value and the smallest evaluation value is greater than the first threshold, determine The average value of the N evaluation values; determine the absolute value of the difference between each evaluation value in the N evaluation values and the average value, and the absolute value of the difference from the evaluation value to the average value is called the The distance between the candidate client corresponding to the evaluation value and the centroid; in the federated learning group composed of the N candidate clients, remove the candidate client MTLF with the largest distance from the centroid to form a new federated learning group; continue in In the new federated learning group, the maximum evaluation value and the minimum evaluation value are determined, and the relationship between the difference between the two evaluation values and the first threshold is determined.
  • the first accuracy evaluation information is the evaluation level of the first model accuracy fed back by the candidate client MTLF, and the evaluation level fed back by the MTLF client participating in federated learning meets the target evaluation level.
  • the N evaluation levels an evaluation level whose difference from the target evaluation level is less than or equal to the third threshold is determined; and the difference from the target evaluation level is less than or equal to the third threshold.
  • the candidate client MTLF corresponding to the three-threshold evaluation level is used as the client MTLF participating in federated learning.
  • the accuracy evaluation information fed back by the selected client MTLF participating in federated learning can be close.
  • the accuracy of the first model fed back by any two client MTLFs is The difference in degree evaluation values is less than or equal to the first threshold, thereby ensuring that the local data distribution of the selected client MTLF participating in federated learning tends to be consistent, and improving the prediction accuracy of the federated learning model.
  • the method further includes: the server MTLF sends the first model to the analytical reasoning function AnLF; the server MTL receives second accuracy evaluation information from the AnLF, and the second accuracy evaluation information indicates that the AnLF uses local The data determines the accuracy of the first model.
  • the AnLF's local data is data collected by the AnLF within its own service area.
  • the server MTLF sends a second model evaluation request message to the AnLF, and the second model evaluation request message includes the first model; the server MTLF receives the second model from the AnLF. and an evaluation request response message, where the second model evaluation request response message includes the second accuracy evaluation information.
  • determining the client MTLFs participating in federated learning among the N candidate client MTLFs based on the N pieces of first accuracy evaluation information includes: based on the N pieces of the first accuracy evaluation information and The second accuracy evaluation information determines the client MTLF that participates in federated learning among the N candidate clients.
  • the second accuracy evaluation information is the reference value of the first model accuracy fed back by the AnLF
  • the first accuracy evaluation information is the reference value fed back by the candidate client MTLF.
  • the difference between the reference value and the evaluation value is less than or equal to the fourth threshold; the candidate client MTLF corresponding to the evaluation value whose difference between the reference value is less than or equal to the fourth threshold is used as a candidate client participating in federated learning.
  • ClientMTLF the second accuracy evaluation information is the reference level of the accuracy of the first model fed back by the AnLF, and the first accuracy evaluation information is the accuracy of the first model fed back by the candidate client MTLF.
  • the evaluation level of the degree, the difference between the evaluation level fed back by the client MTLF participating in federated learning and the reference level is less than or equal to the fifth threshold.
  • determining the client MTLF participating in federated learning among the N candidate clients including: among the N evaluation levels, determining the evaluation level whose difference with the reference level is less than or equal to the fifth threshold;
  • the candidate client MTLF corresponding to the evaluation level whose difference between the reference levels is less than or equal to the fifth threshold is regarded as the client MTLF participating in federated learning.
  • the first accuracy evaluation information of the first model determined by the client MTLF and the first model determined by AnLF should not differ much. If the difference between the two is too large, the local data characteristics of the client MTLF are greatly different from the local data characteristics of AnLF; through the above design, among the N candidate client MTLFs, the local data characteristics of the AnLF are eliminated.
  • Candidate client MTLF with large differences ensures that the local data of client MTLF participating in federated learning has similar characteristics to the local data of AnLF, thereby improving the inference accuracy of the federated learning model.
  • the evaluation value, or reference value includes at least one of the following: accuracy rate, error rate, precision rate, recall rate, mean absolute error, mean absolute percentage error, or mean square error.
  • it also includes: receiving a model request message from the AnLF, the model request message at least including the target accuracy that the model needs to meet; and the first model fed back by the client MTLF participating in the federated learning.
  • the third accuracy evaluation information meets the requirements of the target accuracy, federated learning ends; or, the third accuracy evaluation information of the first model fed back by the client MTLF participating in federated learning does not meet the target accuracy.
  • the second model is determined according to the model parameters fed back by the client MTLF participating in the federated learning.
  • AnLF can notify the server MTLF of the target accuracy that the trained model needs to achieve through the model request message; the server MTLF stops federated learning when the model obtained by federated learning training meets the target accuracy, thereby making the federated learning training The resulting model meets the preset target accuracy requirements.
  • a data analysis method is provided.
  • the execution subject of the method is the client MTLF or AnLF, or a device configured in the client MTLF, or a device configured in AnLF, etc., including: receiving model training logic from the server
  • the model evaluation request message of the function MTLF includes a first model; optionally, the first model is an initial model, an intermediate model, or a final model in the federated learning process.
  • determine the accuracy evaluation information of the first model Using local data, determine the accuracy evaluation information of the first model; and send a model evaluation request response message to the server MTLF, where the model request response message includes the accuracy evaluation information of the first model.
  • the accuracy evaluation information may be first accuracy evaluation information, and the first accuracy evaluation information represents the accuracy of the first model determined by the candidate client MTLF using local data.
  • the accuracy evaluation information is second accuracy evaluation information, and the second accuracy evaluation information represents the accuracy of the first model determined by the analysis and reasoning function AnLF using local data.
  • the accuracy evaluation information of the first model is an evaluation value of the accuracy of the first model
  • using local data to determine the accuracy evaluation information of the first model includes: according to The local data and the first model are used to determine the output of the first model; based on the output of the first model, an evaluation value of the accuracy of the first model is determined.
  • the accuracy evaluation information of the first model is an evaluation level of the accuracy of the first model
  • using local data to determine the accuracy evaluation information of the first model includes: according to Determine the output of the first model based on the local data and the first model; determine the evaluation value of the accuracy of the first model based on the output of the first model; determine the accuracy of the first model based on the output of the first model The evaluation value determines the evaluation level of the accuracy of the first model.
  • the method further includes: sending a model request message to the server MTLF, where the model request message at least includes a model Target accuracy needs to be met.
  • the method further includes: receiving an analysis request message from the user,
  • the analysis request message includes at least one of the following: analysis identification, analysis filtering information, or target accuracy that the analysis needs to meet; it is determined that the local model of the analysis reasoning function AnLF cannot meet the requirements of the analysis request message, and the network warehouse function NRF The MTLF that meets the requirements of the analysis request message cannot be queried.
  • AnLF can match the analysis identifier and model filtering information and meet the target accuracy model, it is determined not to perform federated learning; otherwise, it is determined to perform federated learning and train a user that meets the target accuracy through the federated learning process. model for reasoning about the current task.
  • the evaluation value includes at least one of the following: accuracy rate, error rate, precision rate, recall rate, mean absolute error, mean absolute percentage error, or mean square error.
  • a data analysis method is provided.
  • the execution body of the method is a server MTLF, or a device configured in the server MTLF, including: receiving a model request message from the analysis and reasoning function AnLF, where the model request message includes At least one of the following: analysis identification, model filtering information, or the target accuracy that the model needs to meet.
  • the analysis identification is used to identify the analysis task
  • the model filtering information is used to indicate the conditions that the training data needs to meet during the federated learning process.
  • the target accuracy that the model needs to meet is used to indicate the accuracy that the model that infers the current analysis task needs to meet; according to the model request message, it is determined whether to perform federated learning or not.
  • the server MTLF can determine whether to perform subsequent federated learning based on the model request message, which can avoid unnecessary federated learning, save network resources, and meet the accuracy requirements of the analysis service model.
  • determining whether to perform federated learning based on the model request message includes: determining characteristics of the training data in the federated learning process based on the analysis identification and/or the model filtering information; the training If the characteristics of the data are not related to the geographical location, it is determined to perform federated learning; or, if the characteristics of the training data are related to the geographical location, it is determined not to perform federated learning.
  • determining whether to perform federated learning according to the model request message includes: sending a first model evaluation request message to a client MTLF participating in federated learning, where the first model evaluation request message is used to request The client MTLF reports the accuracy evaluation information of the local model; receives a first model evaluation request response message from the client MTLF, the first model evaluation request response message includes the local model reported by the client MTLF accuracy evaluation message; when the accuracy evaluation information of the local model fed back by the client MTLF meets the target accuracy, it is determined not to perform federated learning; or, the accuracy evaluation information of the local model fed back by the client MTLF When the stated target accuracy is not met, it is determined to perform federated learning.
  • the server MTLF can determine whether to perform subsequent federated learning based on the local model or the model obtained in each round of federated learning, which can avoid unnecessary federated learning and save network resources.
  • the method when it is determined not to perform federated learning, the method further includes: sending a model request response message to the AnLF, where the model request response message includes a federated learning failure reason or an analysis aggregation indication, and the analysis aggregation indication is Yu instructs the AnLF to determine the result of the current analysis task using analysis aggregation.
  • a data analysis method is provided.
  • the execution subject of the method is AnLF, or a device configured in the AnLF.
  • the method includes: receiving an analysis request message from a user, the analysis request message including an analysis identifier and the The accuracy that the analysis hopes to achieve; when the local model of the analysis and reasoning function AnLF cannot meet the requirements of the analysis request message, and the model training logic function MTLF that meets the requirements of the analysis request message cannot be queried in the network warehouse function NRF, the server will MTLF sends a model request message.
  • the model request message includes at least one of the following: analysis identification, model filtering information, or target accuracy that the model needs to meet.
  • the target accuracy that the model needs to meet is the target accuracy that is expected to be achieved according to the analysis.
  • the analysis accuracy is determined, and the model filtering information is used to indicate the conditions that the training data needs to meet during the model training process.
  • the method further includes: receiving a model request response message from the server MTLF.
  • the model request response message includes a federated learning failure reason or an analysis aggregation indication.
  • the analysis aggregation indication is used to instruct the AnLF to utilize analysis aggregation. way to determine the results of the current analysis task.
  • the fifth aspect provides a communication device.
  • the device may be a server MTLF, or a device (such as a chip, etc.) configured in the server MTLF, or a device that can be used in conjunction with the server MTLF.
  • the device has the ability to implement the above first aspect or Functionality of the third aspect method. This function can be implemented by hardware, or it can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more units corresponding to the above functions, such as a transceiver unit and a processing unit.
  • a communication device including units or means (means) for executing each step in the first aspect or the third aspect.
  • a seventh aspect provides a communication device, including a processor and a memory; the memory is used to store computer instructions, and when the device is running, the processor executes the computer instructions stored in the memory, so that the device executes the above-mentioned first aspect Or the method in the third aspect.
  • An eighth aspect provides a communication device, including a processor coupled to a memory.
  • the processor is configured to call a program stored in the memory to execute the method of the first aspect or the third aspect.
  • the memory may be located in the device. Within or outside the device. And there can be one or more processors.
  • a ninth aspect provides a communication device, including a processor and an interface circuit.
  • the processor is configured to communicate with other devices through the interface circuit and execute the method of the first or third aspect.
  • the processor may be one or more indivual.
  • a tenth aspect provides a communication device, which may be a client MTLF, or a device (such as a chip) configured in the client MTLF, or a device that can be used in conjunction with the client MTLF, or the device may be an AnLF, Or a device (such as a chip) configured in the AnLF, or a device that can be used in conjunction with the AnLF, and the device has the function of realizing the above second aspect or the fourth aspect.
  • This function can be implemented by hardware, or it can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more units corresponding to the above functions, such as a transceiver unit and a processing unit.
  • a communication device including units or means for executing each step in the above-mentioned second aspect or fourth aspect.
  • a communication device including a processor and a memory; the memory is used to store computer instructions, and when the device is running, the processor executes the computer instructions stored in the memory, so that the device executes the above-mentioned Methods in the second or fourth aspect.
  • a communication device including a processor coupled to a memory.
  • the processor is configured to call a program stored in the memory to execute the method of the second aspect or the fourth aspect.
  • the memory may be located in the Within the device, it can also be outside the device. And there can be one or more processors.
  • a fourteenth aspect provides a communication device, including a processor and an interface circuit.
  • the processor is configured to communicate with other devices through the interface circuit and execute the method of the second or fourth aspect.
  • the processor may be one or Multiple.
  • a chip system including: a processor for executing the method of any one of the above-mentioned first to fourth aspects.
  • a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When the instruction is run on a communication device, any one of the above-mentioned first to fourth aspects is achieved. The method is executed.
  • a computer program product includes a computer program or instructions.
  • the method of any one of the above-mentioned first to fourth aspects is performed. be executed.
  • An eighteenth aspect provides a communication system, which system includes the device of any one of the foregoing fifth to ninth aspects, and the device of any one of the tenth to fourteenth aspects.
  • a data analysis method includes: the server MTLF sends the first model to N candidate client MTLFs respectively; any candidate client MTLF among the N candidate client MTLFs uses local data, Determine first accuracy evaluation information of the first model, where the first accuracy evaluation information represents the accuracy of the first model determined by the candidate client MTLF using local data, and N is a positive number greater than 1. Integer; the candidate client MTLF sends the first accuracy evaluation information to the server MTLF; the server MTLF determines the clients participating in federated learning among the N candidate client MTLFs based on the N pieces of the first accuracy evaluation information. terminal MTLF.
  • the server MTLF may send a first model evaluation request message to N candidate client MTLFs respectively, and the first model evaluation request message includes the first model; the server MTLF may respectively receive messages from the N candidate client MTLFs.
  • the first accuracy evaluation information is an evaluation value of the first model accuracy fed back by the candidate client MTLF.
  • the candidate client MTLF determines the output of the first model based on local data and the first model; the candidate client determines the evaluation value of the accuracy of the first model based on the output of the first model.
  • the difference between the evaluation values fed back by any two candidate client MTLFs determined by the server MTLF among the client MTLFs participating in federated learning is less than or equal to the first threshold.
  • the server MTLF determines the evaluation value with the largest value and the evaluation value with the smallest value; the difference between the evaluation value with the largest value and the evaluation value with the smallest value is less than or equal to the First threshold, the N candidate clients all participate in federated learning; or, the difference between the evaluation value with the largest value and the evaluation value with the smallest value is greater than the first threshold, determine the value of the N evaluation values average value; determine the absolute value of the difference between each evaluation value in the N evaluation values and the average value; in the federated learning group composed of the N candidate clients, eliminate those that differ from the average value
  • the candidate client MTLF corresponding to the evaluation value with the largest absolute value forms a new federated learning group; continue in the new federated learning group, determine the evaluation value with the largest value and the evaluation value with the smallest value, and determine two The relationship between the difference between the evaluation values and the first threshold.
  • the first accuracy evaluation information is an evaluation level of the first model accuracy fed back by the candidate client MTLF.
  • the client MTLF determines the first model based on local data and the first model. The output of a model; the client MTLF determines the evaluation value of the accuracy of the first model based on the output of the first model; the client MTLF determines the first model based on the evaluation value of the accuracy of the first model Evaluation level of accuracy.
  • the evaluation level of the client MTLF feedback determined by the server MTLF that participates in federated learning satisfies the target evaluation level.
  • the server MTLF determines an evaluation level whose difference from the target evaluation level is less than or equal to a third threshold; the server MTLF determines that the difference from the target evaluation level is less than or equal to the third threshold.
  • the candidate client MTLF corresponding to the evaluation level of the third threshold is used as the client MTLF participating in federated learning.
  • the server MTLF sends the first model to the analysis and reasoning function AnLF;
  • AnLF uses local data to determine the second accuracy evaluation information of the first model, and the second accuracy evaluation information represents The AnLF uses local data to determine the accuracy of the first model.
  • AnLF sends the second accuracy evaluation information to the server MTLF.
  • the server MTLF sends a second model evaluation request message to the AnLF, and the second model evaluation request message includes the first model.
  • the server MTLF receives a second model evaluation request response message from the AnLF, where the second model evaluation request response message includes the second accuracy evaluation information.
  • the server MTLF determines the client MTLFs participating in federated learning among the N candidate client MTLFs based on the N first accuracy evaluation information, including: the server MTLF determines the client MTLFs participating in federated learning based on the N pieces of the first accuracy evaluation information.
  • the degree evaluation information and the second accuracy evaluation information determine the client MTLF that participates in federated learning among the N candidate clients.
  • the second accuracy evaluation information is the reference value of the first model accuracy fed back by the AnLF
  • the first accuracy evaluation information is the reference value fed back by the candidate client MTLF.
  • the evaluation value of the first model accuracy, the difference between the evaluation value fed back by any one of the client MTLFs participating in federated learning determined by the server MTLF and the reference value is less than or equal to the fourth threshold.
  • the server MTLF determines that the difference between the reference value and the reference value is less than or equal to the fourth threshold; the server MTLF determines the evaluation value whose difference between the reference value and the reference value is less than or equal to the fourth threshold.
  • the candidate client MTLF corresponding to the evaluation value is used as the client MTLF participating in federated learning.
  • the second accuracy evaluation information is the reference level of the first model accuracy fed back by the AnLF
  • the first accuracy evaluation information is the reference level of the first model accuracy fed back by the candidate client MTLF.
  • the difference between the evaluation level fed back by the client MTLF participating in federated learning determined by the server MTLF and the reference level is less than or equal to the fifth threshold.
  • the server MTLF determines the evaluation level whose difference from the reference level is less than or equal to the fifth threshold; the server MTLF determines the evaluation level whose difference from the reference level is less than or equal to the fifth threshold.
  • the candidate client MTLF corresponding to the evaluation level is used as the client MTLF participating in federated learning.
  • anLF sends a model request message to the server MTLF, the model request message at least includes the target accuracy that the model needs to meet; the server MTLF sends the first feedback of the client MTLF participating in the federated learning.
  • the server MTLF determines the second model based on the model parameters fed back by the client MTLF participating in federated learning, and uses the second model to continue federated learning.
  • the AnLF before sending the model request message to the server MTLF, the AnLF further includes: the AnLF receives an analysis request message from the user, and the analysis request message includes at least one of the following: analysis identification, analysis filtering information, or the target accuracy that the analysis needs to meet; AnLF determines that AnLF’s local model cannot meet The requirements of the analysis request message, and the MTLF that meets the requirements of the analysis request message cannot be queried in the network warehouse function NRF.
  • a data analysis method including: AnLF receives an analysis request message from a user, and the analysis request message includes an analysis identifier and the accuracy expected to be achieved by the analysis; AnLF's local model cannot satisfy the analysis request message, and when the model training logic function MTLF that meets the requirements of the analysis request message cannot be queried in the network warehouse function NRF, AnLF sends a model request message to the server MTLF, and the model request message includes at least one of the following: Analysis identification, model filtering information, or the target accuracy that the model needs to meet. The target accuracy that the model needs to meet is determined based on the analysis accuracy that the analysis hopes to achieve. The model filtering information is used to indicate that during model training Conditions that the training data needs to meet during the process. The server MTLF determines whether to perform federated learning based on the model request message.
  • the server MTLF determines whether to perform federated learning based on the model request message, including: determining characteristics of the training data in the federated learning process based on the analysis identifier and/or the model filtering information; If the characteristics of the training data are not related to the geographical location, it is determined to perform federated learning; or, if the characteristics of the training data are related to the geographical location, it is determined not to perform federated learning.
  • the server MTLF determines whether to perform federated learning based on the model request message, including: sending a first model evaluation request message to the client MTLF participating in federated learning, where the first model evaluation request message is Requesting the client MTLF to report the accuracy evaluation information of the local model; receiving a first model evaluation request response message from the client MTLF, where the first model evaluation request response message includes the accuracy evaluation information reported by the client MTLF.
  • the accuracy evaluation message of the local model; the accuracy evaluation information of the local model fed back by the client MTLF meets the target accuracy, and it is determined not to perform federated learning; or the accuracy evaluation of the local model fed back by the client MTLF If the information does not meet the stated target accuracy, it is determined to perform federated learning.
  • the server MTLF when the server MTLF determines not to perform federated learning, it further includes: the server MTLF sends a model request response message to the AnLF, and the model request response message includes the federated learning failure reason or an analysis aggregation indication, and the The analysis aggregation instruction is used to instruct the AnLF to use analysis aggregation to determine the result of the current analysis task.
  • the server MTLF when it determines to perform federated learning, it also includes: sending a second model evaluation request message to the client MTLF participating in federated learning, where the second model evaluation request message includes the results obtained from this round of model training.
  • model the second model evaluation request message is used to request the client MTLF to report the accuracy evaluation information of the model obtained by the current round of training; receiving the second model evaluation request response message from the client MTLF, so
  • the second model evaluation request response message includes the accuracy evaluation information of the model obtained by the current round of model training reported by the client MTLF; the accuracy evaluation of the model obtained by the current round of model training reported by the client MTLF.
  • the information meets the stated target accuracy, ending federated learning.
  • FIG. 1 is a schematic diagram of the network architecture provided by this application.
  • FIG. 2 is a schematic diagram of another network architecture provided by this application.
  • FIG. 3 is a flow chart of the data analysis method provided by this application.
  • Figure 4 is another flow chart of the data analysis method provided by this application.
  • Figure 5 is another flow chart of the data analysis method provided by this application.
  • FIG. 6 is a schematic diagram of the device provided by this application.
  • FIG. 7 is another schematic diagram of the device provided by this application.
  • FIG 8 is a schematic diagram of the system provided by this application.
  • FIG. 1 is a schematic architectural diagram of a communication system 1000 applied in an embodiment of the present application.
  • the communication system 1000 includes a core network, which includes one or more of the following entities:
  • Access and mobility management function network element mainly used for the attachment, mobility management and tracking area update process of terminals in mobile networks.
  • Access and mobility management function network elements process non-access stratum (NAS) messages, complete registration management, connection management and reachability management, allocate tracking area list (track area list, TA list) and mobility management etc., and transparently routes session management (SM) messages to the session management network element.
  • NAS non-access stratum
  • the access and mobility management function network element may be the access and mobility management function (AMF).
  • Session management network element Mainly used for session management in mobile networks, such as session establishment, modification, and release. Specific functions include assigning Internet Protocol (IP) addresses to terminals and selecting user plane network elements that provide packet forwarding functions.
  • IP Internet Protocol
  • the session management network element may be a session management function (SMF).
  • Policy control network element includes user subscription data management functions, policy control functions, billing policy control functions, quality of service (QoS) control, etc.
  • the policy control network element can be a policy control function (PCF).
  • PCF policy control function
  • PCF may also be divided into multiple entities according to levels or functions, such as global PCF and PCF within slices, or session management PCF (session, management PCF, SM-PCF) and access management PCF (access management PCF). management PCF, AM-PCF).
  • Network slice selection network element mainly used to select appropriate network slices for terminal services.
  • the network slice selection network element may be a network slice selection function (NSSF) network element.
  • NSSF network slice selection function
  • Unified data management network element responsible for managing terminal contract information.
  • the unified data management network element can be unified data management (UDM).
  • the data analysis network element collects network data from various network functions (NF), such as AMF, SMF, PCF, etc.
  • the data analysis network element can collect network data indirectly from the application function (AF) through the network exposure function (NEF), or directly from the AF; the data analysis network element can also collect network data from the operation management and maintenance ( Operation, administration, and maintenance (OAM) system collects network data.
  • the data analysis network element can analyze and predict based on the collected network data.
  • the data analysis network element collects relevant network data, uses machine learning technology to train and fit the collected network data to a model, and then outputs analysis services based on the model.
  • the data analysis network element can be the network data analytics function (NWDAF) or the management data analytics system (MDAS).
  • User plane network element Mainly responsible for processing user messages, such as forwarding, accounting, legal interception, etc.
  • the user plane network element can also be called a protocol data unit (PDU) session anchor (PDU session anchor, PSA).
  • PDU session anchor PDU session anchor
  • PSA protocol data unit
  • the user plane network element can be the user plane function (UPF).
  • UPF can communicate directly with NWDAF through a service-like interface, or it can communicate with NWDAF through other means, such as through SMF or a private interface or internal interface with NWDAF.
  • Application function network element mainly supports cooperation with the 3rd generation partnership project (3rd generation partnership project, 3GPP) core network interacts to provide services, such as affecting data routing decisions, policy control functions, or providing some third-party services to the network side.
  • 3rd generation partnership project 3rd generation partnership project, 3GPP
  • the application function network element can be AF.
  • Network opening function network element mainly used to support the opening of capabilities and events, such as for securely opening services and capabilities provided by 3GPP network functions to the outside.
  • the network development function network element is also the network exposure function (NEF).
  • Network storage function network element It is mainly used to store network function entities and description information of the services they provide, and supports service discovery and network element entity discovery.
  • the network storage function network element can be a network repository function (NRF).
  • Operation management and maintenance network elements mainly used to manage resource configuration, performance statistics, fault alarms, etc. of network equipment.
  • the operation, management and maintenance network elements can be OAM, etc.
  • the communication system 1000 may also include the following equipment or network elements:
  • Terminal It is a device with wireless sending and receiving functions.
  • the terminal can also be called terminal equipment, user equipment (UE), mobile station, mobile terminal, etc.
  • Terminals can be widely used in various scenarios, such as device-to-device (D2D), vehicle to everything (V2X) communication, machine-type communication (MTC), Internet of Things ( internet of things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wear, smart transportation, smart city, etc.
  • Terminals can be mobile phones, tablets, computers with wireless transceiver functions, wearable devices, vehicles, drones, helicopters, airplanes, ships, robots, robotic arms, smart home devices, etc.
  • the embodiments of this application do not limit the specific technology and specific equipment form used by the terminal.
  • the terminal is used as an example for description below.
  • Access network (AN) equipment used for wireless side access of terminals, which can be base station, evolved base station (evolved NodeB, eNodeB), transmission reception point (TRP), The next generation base station (next generation NodeB, gNB) in the fifth generation (5th generation, 5G) mobile communication system, the next generation base station in the sixth generation (6th generation, 6G) mobile communication system, and the base station in the future mobile communication system Or an access node in a wireless fidelity (WiFi) system, etc.; it can also be a module or unit that completes some functions of the base station, for example, it can be a centralized unit (CU) or a distributed unit (distributed unit, DU).
  • CU centralized unit
  • DU distributed unit
  • the CU here completes the functions of the base station’s radio resource control (RRC) protocol and packet data convergence protocol (PDCP), and can also complete the service data adaptation protocol (SDAP) function;
  • DU completes the functions of the radio link control (RLC) layer and medium access control (MAC) layer of the base station, and can also complete part of the physical (PHY) layer or all of the physical layer.
  • RRC radio resource control
  • PDCP packet data convergence protocol
  • SDAP service data adaptation protocol
  • DU completes the functions of the radio link control (RLC) layer and medium access control (MAC) layer of the base station, and can also complete part of the physical (PHY) layer or all of the physical layer.
  • RRC radio resource control
  • PDCP packet data convergence protocol
  • SDAP service data adaptation protocol
  • DU completes the functions of the radio link control (RLC) layer and medium access control (MAC) layer of the base station, and can also complete part of the physical (PHY) layer or all of the physical layer.
  • PHY physical layer
  • a data network can be a service network that provides data business services to users.
  • the DN can be an IP multi-media service network or the Internet, etc.
  • the terminal device can establish a protocol data unit (PDU) session from the terminal device to the DN to access the DN.
  • PDU protocol data unit
  • the data analysis network element trains a model and outputs analysis results according to the model.
  • NWDAF can be divided into model training logic functions There are two parts (model training logical function, MTLF) and analytical reasoning logical function (analytics logical function, AnLF).
  • NWDAF can have only the MTLF function, only the AnLF function, or both the MTLF function and the AnLF function.
  • MTLF can be an independent network element or a functional unit in NWDAF.
  • AnLF can be an independent network element or a functional unit in NWDAF.
  • MTLF is used to train the model based on the collected data.
  • AnLF is used to use models for inference and provide analysis services to each network element.
  • the Internet of Vehicles server can request AnLF to predict the network performance of a certain location in the next 10 minutes.
  • the network performance includes data such as the maximum sending and receiving speed of the terminal at the location.
  • AnLF can collect data related to terminals at the location, and obtain the network performance prediction model from MTLF.
  • the collected terminal data can be used as input to the network performance prediction model.
  • the output of the model is the predicted time in the location in the next 10 minutes.
  • the terminal transmits and receives data at the maximum speed it can achieve, and then provides it to the Internet of Vehicles server.
  • NWDAF In real networks, multiple NWDAFs are generally deployed. Each NWDAF usually has its own service area.
  • the service area of NWDAF can be an area covered by one or more tracking areas (tracking area, TA).
  • TA consists of broadcasting the same tracking area.
  • the code consists of one or more cells.
  • MTLF in NWDAF When collecting raw data for model training, it is difficult to collect all raw data from distributed data sources in different regions. To solve this problem, MWDAF uses federated learning (FL) technology to train the model. In the process of machine learning, there is no need to centrally transfer the original training data to a certain MWDAF, but only the cooperation between MTLFs in multiple NWDAFs. Each participant can use other data to conduct joint modeling through federation.
  • FL federated learning
  • the MTLF that organizes federated learning is called federated learning server MTLF (FL Server MTLF), which can be referred to as server MTLF;
  • the MTLF that participates in federated learning is called federated learning client MTLF (FL Client MTLF), which can be referred to as client MTLF.
  • the consumer of the analysis service can send an analysis request subscription message or an analysis information request message to AnLF to request AnLF to perform a certain analysis service.
  • AC can be a network element, application function or OAM, etc., without limitation.
  • the MTLF with the server role can be queried in NRF, which is called the server MTLF.
  • AnLF can send a model request message to the queried server MTLF to request the server MTLF to organize federated learning, jointly train the model, and complete the analysis task.
  • the server MTLF when it receives the model request message of AnLF, it can determine multiple client MTLFs participating in federated learning by querying the NRF. Each client MTLF corresponds to a different data source, and the data source can be composed of local data collected by the client MTLF.
  • the server MTLF sends a request message to the determined client MTLF. This request message is used to request the client MTLF to join federated learning.
  • the request message may include analysis identification, filtering information, federated learning group identification, etc.; the client MTLF determines to join the federation.
  • data for model training is collected from various data sources based on analysis identification and filtering information, which can be called training data; the client MTLF sends a response message to the server MTLF to join the federated learning group.
  • the server MTLF organizes each client MTLF participating in federated learning to perform federated learning together. Specifically, the server MTLF sends the initial model to the client MTLF. Each client MTLF uses the locally collected training data to train the initial model and update the model parameters; each client MTLF sends the model parameters obtained by local training to the server MTLF.
  • Server MTLF aggregates each model parameter sent by each client MTLF.
  • AnLF can analyze the analysis task based on the model obtained by the training, and return the analysis results to AC. Or, if the model obtained by the above aggregation cannot meet the target accuracy, or the number of federated learning If the preset number of times is not reached, the server MTLF can use the model obtained in the previous round of aggregation as the initial model for the next round of model training, send it to the client MTLF, and organize the client MTLF to conduct the next round of model training.
  • the server MTLF directly organizes the client MTLF to perform federated learning when receiving the model request message from AnLF.
  • the distribution characteristics of a large amount of data in the network are different.
  • Using data with different distribution characteristics for federated learning may result in a model obtained by federated learning that is not very accurate.
  • a model needs to be trained to predict network load. Assume that the server MTLF determines that there are 3 client MTLFs participating in federated learning.
  • the areas to which the three client MTLFs belong are commercial areas, office areas and residential networks respectively.
  • the network load is heavy in the evening and morning, and the network load is very light during the day on weekdays; for office areas, the network load is heavy during the day on weekdays, and the load is light on weekends; for commercial areas, the network load is heavy on weekends, and the network load is light on weekdays. lighter.
  • the service scope is respectively residential area, commercial area, and office area
  • the three client MTLF will conduct model training based on the locally collected data, and the parameters of the updated model will be sent to the server MTLF, and the server will train the three clients. Aggregation of model parameters sent by end-MTLF may result in lower prediction accuracy of the aggregated model. How to determine the client MTLF participating in federated learning is a technical issue to be solved in this application.
  • This application provides a data analysis method.
  • the server MTLF sends the first model to the determined candidate client MTLF participating in federated learning; when the candidate client MTLF participating in federated learning receives the first model, it uses the local
  • the data determines the accuracy evaluation information of the first model, and the determined accuracy evaluation information is fed back to the server MTLF.
  • the server MTLF determines the client MTLFs that participate in federated learning among the candidate client MTLFs based on the model accuracy evaluation information fed back by the candidate client MTLFs. For example, client MTLF that selects feedback model evaluation information with similar information participates in federated learning.
  • their local data distribution is also similar. Selecting client MTLF with similar local data distribution for federated learning can improve the prediction accuracy of the model trained by federated learning.
  • the process of providing a data analysis method includes at least:
  • Step 300 The server MTLF sends the first model to N candidate client MTLFs respectively, where N is an integer greater than 1.
  • the server MTLF sends a first model evaluation request (model evaluation request) message to each of the N candidate client MTLFs, and the first model evaluation request message includes the first model.
  • model evaluation request model evaluation request
  • Step 301 N candidate client MTLFs respectively send first accuracy evaluation information to the server MTLF.
  • the server can obtain N pieces of first accuracy evaluation information based on the first accuracy evaluation information respectively sent by the N candidate client MTLFs. For example, when any candidate client MTLF among N candidate client MTLFs receives the first model, it can use local data (local data) to determine the accuracy evaluation information of the first model.
  • the accuracy evaluation information is called First accuracy assessment information.
  • the local data of the candidate client is the data collected by the candidate client MTLF in its own service area. During the process of federated learning, the local data collected by the client MTLF will not be sent to other client MTLFs. Local data may also be called a local data set.
  • the client MTLF uses the local training data set to determine the accuracy evaluation information of the first model.
  • the training data set includes input data and label data.
  • the client MTLF can input the input data into the first model; and determine the prediction information of the first model based on the output of the first model.
  • the output of the first model may be prediction information, or the output of the first output may be further processed to obtain prediction information.
  • the above processing includes but is not limited to: transforming the output of the first model in dimensions such as time domain, frequency domain or spatial domain to obtain prediction information, etc.
  • the output of the first model is used as the prediction information of the first model as an example. Compare the prediction information of the first model with the label data in the training data set to determine the first accuracy evaluation information of the first model in the training phase.
  • the accuracy of the first model The degree evaluation information may be an evaluation value.
  • the evaluation value includes at least one of the following: accuracy rate, error rate, precision rate, recall rate, mean absolute error, mean absolute percentage error, or mean square error, etc.
  • the evaluation value of the first model may be the accuracy rate, error rate, precision or recall rate of the model, etc.
  • the evaluation value of the first model may be the mean absolute error, mean absolute percentage error, mean square error, etc. of the model.
  • the accuracy evaluation information of the first model may be an evaluation level.
  • the candidate client MTLF can determine the evaluation level corresponding to the evaluation value of the first model according to the preset correspondence between the evaluation value range and the evaluation level, and the candidate client MTLF feeds back the evaluation level of the first model to the server MTLF.
  • the evaluation level includes a low accuracy level, a medium accuracy level, and a high accuracy level, and each evaluation level corresponds to a different evaluation value range.
  • the candidate client MTLF may determine that the evaluation level corresponding to the evaluation value of the first model is low, medium, or high based on the corresponding relationship between the evaluation level and the evaluation value range.
  • the N candidate client MTLFs may each send a first model evaluation request response (model evaluation request response) message to the server MTLF, where the first model evaluation request response message includes the first accurate value determined by the candidate client MTLF. degree assessment information.
  • Step 302 The server MTLF determines the client MTLF that participates in federated learning among the N candidate client MTLFs based on the N first accuracy evaluation information.
  • the N pieces of first accuracy evaluation information are N evaluation values of the accuracy of the first model fed back by N candidate client MTLFs. Any two clients in the client MTLFs participating in federated learning The difference between the evaluation values fed back by the terminal MTLF is less than or equal to the first threshold.
  • the first threshold is preset, or specified by the protocol, or pre-configured or pre-notified to the server MTLF, etc., without limitation.
  • the clustering method can be used to determine the client MTLF that participates in federated learning among N candidate clients.
  • the implementation method is as follows:
  • N evaluation values determine the maximum evaluation value and the minimum evaluation value; if the difference between the maximum evaluation value and the minimum evaluation value is less than or equal to the first threshold, it is considered that the N candidate clients all meet the conditions. , can participate in federated learning; otherwise, perform step 3 below.
  • the new federated learning group determine the maximum evaluation value and the minimum evaluation value, determine the relationship between the difference between the maximum evaluation value and the minimum evaluation value and the first threshold, and repeat step 3. , until the difference between the evaluation values fed back by any two client MTLFs participating in federated learning is less than or equal to the first threshold. or,
  • the maximum distance method can be used to determine the client MTLF that participates in federated learning among N candidate clients.
  • the implementation method is as follows:
  • the N candidate client MTLFs all meet the conditions and can participate in federated learning; otherwise, perform step 3.
  • the centroid algorithm is used to determine the client MTLF that participates in federated learning among N candidate clients.
  • the implementation method is as follows:
  • N evaluation values determine the largest evaluation value and the smallest evaluation value; if the difference between the largest evaluation value and the smallest evaluation value is less than or equal to the first threshold, then the N All candidate client MTLFs meet the conditions and can participate in federated learning, ending the process; otherwise, continue to step 3.
  • step 4 Determine the distance between each candidate client MTLF and the centroid, and remove the client MTLF with the largest distance from the centroid from the federated learning group to form a new federated learning group.
  • step 2. determine the largest evaluation value and the smallest evaluation value, and determine the relationship between the difference between the two evaluation values and the first threshold, and repeat step 3. and 4, until the difference between the evaluation values fed back by any two client MTLFs among the client MTLFs participating in federated learning is less than or equal to the first threshold.
  • the N first accuracy evaluation information is N evaluation levels of the first model accuracy fed back by N candidate client MTLFs, and the evaluation levels fed back by the MTLF clients participating in federated learning Meet target assessment levels.
  • the server MTLF may determine, among the N evaluation levels, the evaluation levels whose difference from the target evaluation level is less than or equal to the third threshold; and evaluate the evaluation levels whose difference from the target evaluation level is less than or equal to the third threshold.
  • the candidate client MTLF corresponding to the level is used as the client MTLF participating in federated learning.
  • the third threshold is preset, or specified in the protocol, or pre-notified or configured to the server MTLF, and is not limited.
  • the target evaluation level is preset, or specified in the protocol, or notified or configured to the server MTLF in advance, without limitation.
  • the target evaluation level may be AnLF pre-notification server MTLF.
  • the server MTLF may select an evaluation level whose difference from the target evaluation level is less than or equal to the third threshold among the N evaluation levels fed back by the N candidate clients. For example, the value of N is 3, and the evaluation levels of the three candidate client MTLF feedbacks are respectively evaluation level 7, evaluation level 9 and evaluation level 5.
  • the server MTLF can choose between the above three evaluation levels and the target evaluation level 8. The difference is less than or equal to the third threshold (for example, the third threshold is 1).
  • the selected evaluation levels are evaluation level 7 and evaluation level 9, then the candidate client MTLF corresponding to evaluation level 7 and evaluation level 9 , participate in federated learning, evaluate the candidate client MTLF corresponding to level 5, eliminate it from the federated learning group, and no longer participate in federated learning.
  • the server MTLF can also send the first model to AnLF and receive second accuracy evaluation information from AnLF.
  • the second accuracy evaluation information represents the accuracy of the first model determined by the AnLF using local data.
  • Evaluate information For example, the server MTLF may send a second model evaluation request message to AnLF, the second model evaluation request message including the first model, and receive a second model evaluation request response message from AnLF, the second model evaluation request response message including The second accuracy assessment information is included.
  • AnLF receives the first model
  • the collected local data can be used to determine the accuracy of the first model, and the accuracy of the first model is called second accuracy evaluation information.
  • the local data of the AnLF is the data collected by the AnLF within its own service area.
  • AnLF can determine the second accuracy evaluation information of the first model based on the local data that has been collected historically, by comparing the inference results of the first model with the observed label data (i.e., real data from the network) , to determine the accuracy evaluation information of the first model.
  • the first model is used to predict terminal performance 10 minutes in the future.
  • AnLF can use the terminal data collected locally 10 minutes ago as input to the first model.
  • the output of the first model can be considered as the inference result of the first model.
  • the inference result of the first model can be the predicted inference result. Terminal performance after 10 minutes.
  • AnLF compares the predicted terminal performance after 10 minutes inferred by the first model with the real performance of the terminal after 10 minutes collected in the network, and determines the second accuracy evaluation information.
  • the second accuracy evaluation The information may be an evaluation value of the accuracy of the first model, or an evaluation level of the accuracy of the first model.
  • evaluation values and evaluation levels please refer to the previous description.
  • the server MTLF can determine the clients participating in federated learning among the N candidate client MTLFs based on the N first accuracy evaluation information fed back by the N candidate client MTLF and the second accuracy evaluation information fed back by AnLF.
  • MTLF For example, the N first accuracy evaluation information fed back by MTLF of N candidate clients are N evaluation values, and the second accuracy evaluation information fed back by AnLF is the evaluation value, and the evaluation value is called a reference value.
  • the fourth threshold may be a preset , or specified in the protocol, or notified or configured to the server MTLF in advance, there is no restriction.
  • the server MTLF may determine, among the N evaluation values, the evaluation value whose difference from the reference value is less than or equal to the fourth threshold; and select the candidate corresponding to the evaluation value whose difference between the reference value is less than or equal to the fourth threshold.
  • Client MTLF as the client MTLF participating in federated learning.
  • the N first accuracy evaluation information fed back by N candidate clients are N evaluation levels
  • the second accuracy evaluation information fed back by AnLF is the evaluation level, which can be called a reference level.
  • the difference between the evaluation level fed back by the client MTLF participating in federated learning and the reference level is less than or equal to the fifth threshold.
  • the fifth threshold is preset or specified by the protocol, Or notify or configure the server MTLF in advance. For example, among the N evaluation levels, the server MTLF determines the evaluation level whose difference with the reference level is less than or equal to the fifth threshold; and assigns the candidate customers corresponding to the evaluation level whose difference with the reference level is less than or equal to the fifth threshold.
  • the three evaluation levels of MTLF feedback from three clients are evaluation levels 8, 9, and 5 respectively.
  • the evaluation level for AnLF feedback is reference level 8.
  • the difference between the evaluation level 8 and the evaluation level 9 and the reference level 8 is less than or equal to the fifth threshold (the fifth threshold may be 1).
  • the difference between evaluation level 5 and reference level 8 is greater than the fifth threshold.
  • the server MTLF can select the candidate client MTLF corresponding to evaluation level 8 and evaluation level 9 to participate in federated learning, and remove the candidate client corresponding to evaluation level 5 from the federated learning group.
  • the first accuracy evaluation information of the first model determined by the client MTLF and the first accuracy evaluation information determined by AnLF should not differ much. If the difference between the two is too large, the local data characteristics of the client MTLF are greatly different from the local data characteristics of AnLF; through the above design, among the N candidate client MTLFs, the local data characteristics of the AnLF are eliminated.
  • Candidate client MTLF with large differences ensures that the local data of client MTLF participating in federated learning has similar characteristics to the local data of AnLF, thereby improving the inference accuracy of the federated learning model.
  • the above-mentioned first model may be an initial model, an intermediate model, or a final model in the federated learning process.
  • the initial model refers to the initial public model determined by the server MTLF for model training during the federated learning process, or described as the initial public model sent by the server MTLF to the client MTLF during the first round of model training;
  • the intermediate model refers to the public model formed after the parameter aggregation and update of the previous round of federated learning sent by the server MTLF to the client MTLF during the intermediate round of model training. It can also be considered as the initial model of this round of federated learning training. ;
  • the final model refers to the model obtained through federated learning after the last round of model training process. This final model can be considered as the model fed back to AnLF by the server MTLF.
  • a round of model training process includes: the server MTLF can determine the client MTLF that participates in federated learning among the candidate client MTLFs; the server MTLF sends the initial model to each client MTLF respectively; Each client MTLF uses local data to train the initial model and updates the parameters of the initial model; each client MTLF sends the updated initial model parameters to the server MTLF; the server MTLF evaluates the parameters of the initial model fed back by each client MTLF. Aggregate to determine the model obtained in this round of training. The model obtained in this round of training can be used as the initial model for the next round of model training. The client MTLF participating in federated learning in this round is used as the candidate client MTLF for the next round of model training.
  • the server MTLF can determine N candidate clients participating in federated learning by querying the NRF.
  • the server MTLF sends a model evaluation request message to the N candidate clients respectively, and the model evaluation request message includes the initial model.
  • Each candidate client uses local data as a verification set to verify the accuracy of the initial model, determine the accuracy evaluation information of the initial model, and feed it back to the server MTLF.
  • the server MTLF determines the client MTLF that participates in federated learning based on the accuracy evaluation information of the initial model fed back by the N candidate client MTLFs.
  • the server MTLF determines that among the above N candidate client MTLFs, there are M client MTLFs that can participate in federated learning, and the M is a positive integer less than or equal to N; then in the first round of model training process In , the server MTLF sends the initial model to M client MTLF, and the M client MTLF trains the initial model, etc.
  • the server MTLF aggregates the model parameters fed back by the M client MTLFs, determines the model obtained in the first round of training, and the first round of training is completed.
  • the M clients MTLF that participated in federated learning in the first round were used as candidate clients for the second round of model training.
  • the server MTLF sends the model obtained in the first round of training as the initial model of the second round of training to M candidate clients.
  • M candidate clients use local data to verify the accuracy evaluation information of the initial model and feed it back to the server MTLF.
  • the server MTLF determines the X client MTLFs that participate in federated learning based on the accuracy evaluation information fed back by the M candidate clients, where X is an integer less than or equal to M.
  • the server MTLF sends the initial model in the second round of training to X clients.
  • the X clients use local data to train the initial model, update the model parameters, and send the updated model parameters to the server MTLF.
  • the server MTLF aggregates the model parameters fed back by X clients to obtain the model for the second round of training, etc.
  • the subsequent process of other rounds of model training is similar to the above and will not be repeated.
  • the number of rounds of direct federated learning reaches the preset value, or the accuracy evaluation information fed back by the client MTLF participating in federated learning meets the target accuracy, then the federation will be stopped. learning process.
  • the server MTLF sends a model evaluation request message to each candidate client MTLF, and based on the accuracy evaluation information of the model fed back by each client MTLF, it is determined that in this round During the federated learning process, the client MTLF participates in federated learning.
  • the above process can also be described as: after the end of each round of learning, the server MTLF sends a model evaluation request message to the client MTLF, and the model evaluation request message carries the model obtained in this round of training.
  • Client MTLF can use local data to verify the accuracy evaluation information of the model obtained through local training and feed it back to server MTLF.
  • the server MTLF determines whether to continue federated learning based on the accuracy evaluation information of the model fed back by each client MTLF. For example, if the accuracy evaluation information of the model fed back by each client's MTLF meets the target accuracy requirements, federated learning will be stopped, otherwise the next round of federated learning will be continued. That is to say, in this application, after a round of model training is completed, the server MTLF sends a model evaluation request message to the client MTLF.
  • the model evaluation request message carries the model obtained in this round of training. Type, the client MTLF uses local data to determine the accuracy evaluation information of this round of training model, and feeds it back to the server MTLF.
  • the server MTLF can perform the following operations: First, determine whether federated learning needs to continue. Second, if it is determined to continue to perform the next round of federated learning, the client MTLF etc. that will participate in the next round of federated learning can be determined based on the accuracy evaluation information fed back by each client.
  • the above target accuracy may be preset, or specified in the protocol, or may be notified or configured to the server MTLF in advance, etc., without limitation.
  • AnLF sends a model request message to the server MTLF, and the model request message includes the above target accuracy.
  • the server MTLF When the server MTLF receives the above model request message, it can perform model training in the federated learning process according to the above method; when the server MTLF completes training the model, it sends a model request response message to AnLF, and the model request response message includes the federated learning Obtained model.
  • AnLF before AnLF sends a model request message to the server MTLF, it also includes: AnLF receives an analysis request from the user, and the analysis request includes at least one of the following: analysis identification, analysis filtering information, or the goal that the analysis needs to meet. Accuracy.
  • the analysis identification can be used to identify analysis tasks, and the analysis filtering information is used to indicate the conditions that the training data needs to meet during the federated learning process, or it can be described as analysis filtering information to determine the model filtering information, and the model filtering information is used to refer to the conditions in the federated learning process. Conditions that training data needs to meet during the federated learning process, etc.
  • the target accuracy that needs to be met by the analysis can be used to determine the target accuracy that the model needs to meet.
  • the target accuracy can be the target accuracy value, or the target accuracy level, etc., without limitation.
  • AnLF determines that the local model of AnLF cannot meet the requirements of the analysis request message and the MTLF that meets the requirements of the analysis request message cannot be queried in NRF, it can query the MTLF with the server role in NRF.
  • AnLF will The server MTLF sends a model request message and executes the aforementioned method.
  • AnlF receives the federated learning model fed back by the server, it can perform the above analysis tasks based on the model and feed back the analysis results to the user.
  • the server MTLF when the server MTLF receives the accuracy evaluation information fed back by the MTLF of each candidate client, it can determine whether the accuracy evaluation information fed back by the MTLF of each candidate client meets the target accuracy. requirements; optionally, the accuracy evaluation information fed back by the candidate client can be called the third accuracy evaluation information.
  • the third accuracy evaluation information is the same as or different from the aforementioned first accuracy evaluation information, without limitation. If satisfied, the federated learning ends; otherwise, the second model will be determined based on the model update parameters fed back by the MTLF of each candidate client, which can be called the update parameters of the first model. This second model may be an intermediate model in the federated learning process.
  • the server MTLF sends the second model to the candidate client MTLF to perform the next round of federated learning.
  • the candidate client feeds back the first accuracy evaluation information to the server MTLF, and the AnLF feeds back the second accuracy evaluation information to the server MLTF.
  • the server MTLF determines the clients participating in the federated learning based on the first accuracy evaluation information and the second accuracy evaluation information.
  • the end-MTLF solution as shown in Figure 4, provides a data analysis method process, which at least includes:
  • Step 400 Each MTLF registers its own information with the NRF.
  • each MTLF can register the server role or client role it supports in the federated learning process into the NRF.
  • each MTLF can also register the analysis ID (analytics ID), analysis filter information (analytics filter information) corresponding to the model it can provide, and the accuracy evaluation information of the model it can provide into the NRF.
  • Step 401 The consumer AC of the analysis service sends an analysis request subscription message or an analysis information request message to AnLF, which carries at least one of the following: analysis identification (analytics ID), analysis filtering information, or the desired analysis accuracy level (preferred level). of accuracy of analytics).
  • AnLF which carries at least one of the following: analysis identification (analytics ID), analysis filtering information, or the desired analysis accuracy level (preferred level). of accuracy of analytics).
  • the consumer AC of the analysis service can be a network element, application function or OAM, etc., without any restrictions.
  • the analysis identifier is used to identify the analysis task; the analysis filter information can indicate the object of analysis or the scope of the analysis output, etc.
  • the analysis filtering information may be a network slice for the Internet of Vehicles service specified by the network slice identifier.
  • the desired analysis accuracy level may be data such as accuracy, or may be an accuracy level.
  • the accuracy level may be high, medium, or low.
  • the Internet of Vehicles server requests from AnLF the service quality prediction of the network slice of the Internet of Vehicles service at a certain location in the next 10 minutes.
  • the analysis identifier can indicate the service quality prediction service, and the analysis filtering information indicates prediction for the network slice of the Internet of Vehicles service.
  • the desired level of analytical accuracy is high.
  • Step 402 AnLF determines at least one of the following based on the message received in step 401: analysis identification, model filter information (model filer information), or the target accuracy that the model needs to achieve.
  • AnLF can determine the model filtering information based on the analyzed filtering information in step 401, and the model filtering information can indicate the conditions that the training data needs to meet during the federated learning process.
  • the above step 401 may not carry analysis filtering information, and AnLF may determine the model filtering information through other methods without limitation.
  • data that meets the model filtering information conditions can be used for model training.
  • the parameter types included in the model filtering information are the same as the parameter types included in the analysis filtering information.
  • the parameter values of each type of parameter included in the model filtering information and the parameter values of each type of parameter included in the analysis filtering information may also be the same.
  • the model filtering information and the analysis filtering information may include the same content.
  • the model filtering information may be a specific network area, or a network slice identifier, such as a single-network slice selection assistance information (S-NSSAI), a specified specific network slice, or an application identifier. Specific application services specified by (application ID), etc.
  • S-NSSAI single-network slice selection assistance information
  • application ID Specific application services specified by (application ID), etc.
  • AnLF can determine to provide the above analysis request.
  • the conditions that the model of the inference service needs to meet are that the analysis flag is a service quality prediction service, and the model filtering information is a slice of the Internet of Vehicles service.
  • AnLF may determine the target accuracy of the model based on the analysis accuracy level in step 401.
  • the target accuracy of the model may be the target accuracy value of the model, or the target accuracy level of the model, etc.
  • the above-mentioned desired analysis accuracy level can be obtained by data such as accuracy rate, and the desired analysis accuracy can be converted into the target accuracy value that the model hopes to achieve, etc.
  • the target accuracy value of the model can be the accuracy rate, error rate, precision, or recall rate; or when the model is a model that computes regression, the target accuracy value of the model can be the mean absolute error. , average absolute percentage error, or mean square error, etc.
  • the target accuracy value of the model can be converted into the target accuracy level of the model. For example, based on the corresponding relationship between the target accuracy value and the target accuracy level, the target accuracy level corresponding to the target accuracy value is determined. That is to say, the target accuracy that the model in this application needs to achieve can be the target accuracy value or the target accuracy level.
  • the target accuracy that the model needs to achieve reflects the accuracy level required for the model trained by federated learning.
  • the accuracy of the model can be verified by inputting verification data into a model, comparing the output of the model with the accurate output corresponding to the verification data, and determining the accuracy of the model.
  • Step 403 AnLF determines whether AnLF's local model meets the requirements for analysis identification, model filtering information, and target accuracy in the aforementioned step 402. If AnLF's local model can meet the requirements, AnLF uses the local model to perform model inference, and the inference will be The results are fed back to the consumer AC of the analysis as a result of the above analysis task. If AnLF's local model cannot meet the requirements, AnLF queries NRF for an MTLF that meets the above analysis identification, model filtering information, and target accuracy requirements; if an MTLF that meets the requirements can be queried, NRF feeds back the MTLF that meets the requirements to AnLF.
  • AnLF's access address AnLF sends a model request message to MTLF based on the MTLF's access address, and MTLF can return a model request response message to AnLF.
  • the model request response message includes the MTLF response message.
  • the fed-back model that meets the requirements is fed back;
  • AnLF uses the fed-back model that meets the requirements to perform model inference, and the inference results are fed back to the AC as the results of the above analysis tasks.
  • AnLF can query NRF for the MTLF registered as the server role in step 400, and return the access address of the MTLF registered as the server role to AnLF.
  • AnLF executes subsequent step 404 and sends a model request message to MTLF registered as a server role.
  • the above process can be considered as the process of AnLF discovering MTLF.
  • the analysis identified by AnLF is service quality prediction
  • the model filtering information is the network prediction of the Internet of Vehicles service
  • the target accuracy is high.
  • AnLF can determine whether the local model can meet the above conditions; if so, the local model is used to perform model inference, and the inference result is used as the result of the above analysis.
  • AnLF can query NRF for MTLFs that meet the above conditions.
  • an MTLF registers an analysis identifier with the NRF as service quality prediction, the analysis filtering information is network prediction for Internet of Vehicles services, and the accuracy evaluation information of the model is high, then the MTLF can be considered
  • NRF returns the access address of the MTLF that meets the conditions to AnLF
  • AnLF uses the model provided by the MTLF that meets the conditions to perform model inference.
  • AnLF queries NRF in step 400, registers the MTLF whose role is the server, and returns the access address of the MTLF to AnLF.
  • Step 404 AnLF sends a model request message to the server MTLF.
  • the model request message includes analysis identification, model filtering information, and target accuracy that the model needs to meet.
  • Step 405 The server MTLF sends a second model evaluation request message to AnLF, where the second model evaluation request message carries the initial model.
  • Step 406 AnLF uses local data to determine the second accuracy evaluation information of the initial model; AnLF sends a second model evaluation request response message to the server, and the second model evaluation request response message includes the second accuracy evaluation of the initial model. information.
  • the second accuracy evaluation information may be the accuracy evaluation value of the initial model.
  • AnLF inputs locally collected data into the initial model, compares the output of the initial model with the labels of the local data, and determines the accuracy evaluation value of the initial model.
  • the second accuracy evaluation information may be the accuracy evaluation level of the initial model, and AnLF may further determine the accuracy evaluation level of the initial model based on the accuracy evaluation value of the initial model.
  • Step 407 AnLF determines N candidate client MTLFs participating in federated learning through NRF, where N is an integer greater than 1; AnLF sends a first model evaluation request message to the N candidate client MTLFs respectively. The first model evaluation request The initial model is included in the message.
  • Step 408 Each candidate client MTLF among the N candidate client MTLFs determines the first accuracy evaluation information of the initial model using locally collected data. This process is similar to the process of AnLF determining the second accuracy evaluation information, please refer to the previous description.
  • Each candidate client MTLF among the N candidate client MTLFs sends a first model evaluation request response message to the AnLF, where the first model evaluation request response message includes first accuracy evaluation information of the initial model.
  • Step 409 The server MTLF determines the client MTLFs participating in federated learning among the N candidate client MTLFs based on the N first accuracy evaluation information and the second accuracy evaluation information.
  • the process can be described as: the server MTLF modifies the federated learning group including N client MTLFs, and uses client MTLFs consistent with the data distribution of AnLF to participate in federated learning.
  • the server MTLF may select the first accuracy evaluation information that is consistent with the second accuracy evaluation level or close to the second accuracy evaluation value among N first accuracy evaluation information, and compare it with the second accuracy evaluation value. Same level
  • the client MTLF corresponding to the first accuracy evaluation information that is consistent or close to the second accuracy evaluation value is used as the client MTLF participating in the next round of federated learning.
  • the specific process please refer to the description in Figure 3 mentioned above.
  • Step 4010 The client MTLF determined by the server MTLF organization uses local data to perform federated learning.
  • N candidate clients use locally collected data to perform model training on the initial model provided by the server MTLF, and update the parameters of the initial model.
  • the N candidate client MTLFs send the updated parameters of the initial model to the server MTLF, and the server MTLF aggregates the updated parameters of the initial models fed back by the N candidate client MTLFs to determine the first intermediate model.
  • the server MTLF determines the client MTLF that participates in the second round of federated learning among the N candidate client MTLFs.
  • the server MTLF sends the first intermediate model to the determined client MTLF participating in the second round of federated learning.
  • Each client MTLF performs model training on the first intermediate model based on local collected data. Update the parameters of the first intermediate model.
  • the server MTLF determines the second intermediate model based on the updated parameters of the first intermediate model fed back by each client MTLF.
  • the server MTLF selects the client MTLF that participates in the third round of federated learning among the client MTLFs that participate in the second round of federated learning.
  • the server MTLF sends the second intermediate model to the client MTLF participating in the third round of federated learning, and the cycle is executed until the accuracy evaluation information of a model determined by the server MTLF can meet the target accuracy. degree, stop federated learning.
  • the server MTLF can eliminate the client MTLF whose local data distribution is different from AnLF's local data distribution in the federated learning group, so that the model obtained by federated learning has good performance when used for network analysis. Higher accuracy.
  • Step 4011 In the above federated learning process, after multiple rounds of federated learning updates, if the model accuracy evaluation information fed back by each client MTLF can meet the target accuracy level, the server MTLF stops the next round of federated learning. The server MTLF sends a model request response message to AnLF, and the model request response message carries the final model obtained by federated learning.
  • Step 4012 When AnLF obtains the final model, it uses the final model for analysis, and sends an analysis subscription notification message or an analysis information response message to the requester AC of the analysis service, which carries the analysis results.
  • AnLF when AnLF receives the final model, it can collect data locally based on the model filtering information, and use the collected local data as input into the final model.
  • the output of the final model is the analysis result.
  • the server MTLF can filter out the client MTLF participating in the next round of federated learning based on the accuracy evaluation information of the model fed back by AnLF and the accuracy evaluation information of the model fed back by the client MTLF participating in the previous round of federated learning. This prevents the client MTLF from having its inference accuracy fail to meet the requirements when applying the federated learning model because the data characteristics of the local data of the client are different from the data characteristics of AnLF's local data.
  • the method includes: the server MTLF receives a model request message from AnLF.
  • the model request message includes at least one of the following: analysis identification, model filtering information, or target accuracy that the model needs to meet.
  • the analysis identifier is used to identify the analysis task
  • the model filtering information is used to indicate the conditions that the training data needs to meet during the federated learning process
  • the target accuracy that the model needs to meet is used to indicate the training data for inference during the federated learning process.
  • the model of the current analysis task needs to meet the accuracy; the server MTLF requests a message based on the model and determines whether to perform federated learning or not.
  • the server MTLF when it determines not to perform federated learning, it sends a model request response message to AnLF.
  • the model request response message includes the reason for the failure of federated learning, and/or an analysis aggregation indication, and the analysis aggregation indication is used to indicate
  • the AnLF uses analysis aggregation to determine the results of the current analysis task.
  • the server MTLF determines to perform federated learning, at the end of each round of federated learning, the server MTLF A second model evaluation request message can be sent to the client MTLF participating in this round of federated learning.
  • the second model evaluation request message carries the model obtained in this round of training.
  • the second model evaluation request message is used to request the client MTLF to report this Accuracy evaluation information of the model obtained by the round of training, and receiving a second model evaluation request response message from the client MTLF.
  • the second model evaluation request response message includes the accuracy of the model obtained by the current round of training reported by the client MTLF. degree assessment information.
  • the target accuracy may be preset, or preconfigured or notified to the server MTLF, or obtained by the server MTLF in the model request message sent by AnLF, etc., without limitation.
  • the process of providing a data analysis method includes at least:
  • Step 500 Each MTLF registers its own information with the NRF.
  • Step 501 The consumer AC of the analysis service sends an analysis request subscription message or analysis information request message to AnLF, which carries at least one of the following: analysis identification, analysis filtering information, or the desired analysis accuracy level.
  • Step 502 AnLF sends a model request message to the server MTLF.
  • the model request message includes at least one of the following: analysis identification, model filtering information, or target accuracy that the model needs to achieve.
  • the model filtering information is determined based on the analysis filtering information, and the target accuracy that the model needs to achieve is determined based on the desired level of analysis accuracy.
  • AnLF can first determine whether AnLF's local model can meet the above requirements for analysis identification, model filtering information, and target accuracy; if it cannot, then query NRF to see if there is an MTLF that meets the above requirements; if in NRF If the MTLF that meets the above requirements cannot be queried, AnLF searches the NRF for the MTLF that can provide the server role, and sends a model request message to the MTLF that can provide the server role. That is to say, if AnLF can match the analysis identifier and model filtering information and meet the target accuracy model, it is determined not to perform federated learning; otherwise, it is determined to perform federated learning and train a user that meets the target accuracy through the federated learning process. model for reasoning about the current task.
  • Step 503 When receiving the model request message of AnLF, the server MTLF may determine whether federated learning needs to be performed. If federated learning is performed, continue to subsequent step 504.
  • the server MTLF can determine whether to perform federated learning based on the analysis identifier and/or model filtering information carried in the model request message. For example, the server MTLF can determine the characteristics of the training data in the federated learning process based on the analysis identifier and/or model filtering information; if the characteristics of the training data are not related to geographical location, for example, the training data is related to the application business, the server MTLF determines Suitable for federated learning; or, if the characteristics of the training data are related to geographical location, the server MTLF determines that it is not suitable for federated learning.
  • Federated learning can be determined not to be suitable when the analysis identifier and/or model filter information is any of the following:
  • the analysis identification is data network performance analysis (DN performance analytics);
  • the analysis flag is redundant transmission experience related analytics (redundant transmission experience related analytics);
  • the analysis identification is session management congestion control Experience
  • the analysis identifier is dispersion analytics, and the model filtering information is the specified network slice identifier.
  • the analysis is identified as observed service experience related network data analytics (observed service experience related network data analytics);
  • the analysis is identified as slice load level related network data analytics.
  • the analysis identifier and/or model filtering information is any of the following, it can be considered that the network data for training the model is related to the geographical location, then the local data distribution of the client MTLF is different, and it is determined that it is not suitable for federated learning:
  • the analysis identifier is WLAN performance analysis (performance analytics);
  • the analysis identifier is dispersion analytics, and the model filtering information contains the specified network location;
  • the analysis identification is quality of service sustainability analytics (QoS sustainability analytics);
  • the analysis identification is user data congestion analysis (user data congestion analytics);
  • the analysis identifier is user mobility analytics (UE mobility analytics);
  • the analysis identifier is user communication analytics (UE communication analytics);
  • the analysis identification is abnormal behavior related network data analytics
  • the model filtering information is unexpected UE location (unexpected UE location), unexpected long-live/large rate traffic (unexpected long-live/large rate flows), unexpected radio link failures or Ping-ponging across neighboring cells.
  • the analysis identification is network performance analytics, and the model filtering information is the network area of interest or specific gNB;
  • the analysis identifier is NF load analytics, and the model filtering information is the specified NF or the NF of a specific area.
  • the server MTLF can query the MTLF registered as the client role through NRF.
  • the MTLF obtained by the query can be called the client MTLF, and the client MTLF obtained by the query can form a federated learning group.
  • the server MTLF may send a first model evaluation request message to the client MTLF in the federated learning group, where the first model evaluation request message is used to request the client MTLF to report accuracy evaluation information of the local model.
  • the first model evaluation request message may include analysis identification and model filtering information.
  • the client MTLF may locally determine a unique model based on the analysis identifier and model filtering information carried in the first model evaluation request message.
  • Client MTLF can use local training data as a verification set to verify the accuracy evaluation information of the local model.
  • client MTLF can use local training data that meets the model filtering information conditions as the validation set.
  • the client MTLF can use the local training data set of the network slice of the Internet of Vehicles service as the verification set to determine the accuracy of the local model.
  • the client MTLF may send a first model evaluation request response message to the server MTLF, where the message carries the determined accuracy evaluation information of the local model.
  • the accuracy evaluation information may be an accuracy evaluation value, an accuracy evaluation level, etc. For details, please refer to the foregoing description.
  • the server MTLF can determine whether federated learning needs to be performed based on the accuracy evaluation information of the local model fed back by each client MTLF. If the accuracy evaluation information of the local model fed back by the MTLF of each client meets the target accuracy in step 502, it is determined that federated learning does not need to be performed.
  • the server MTLF may send a model request response message to AnLF.
  • the model request response message carries the reason for the federated learning failure and/or an analysis aggregation indication.
  • the analysis aggregation indication is used to instruct AnLF to use the analysis aggregation method for analysis.
  • the analysis aggregation instructs AnLF to directly forward the analysis request subscription message or analysis information request message in step 501 to the NWDAF of each client MTLF, and each NWDAF provides inference prediction results in their respective service areas.
  • the specific inference process is performed by the AnLF in each NWDAF.
  • AnLF just summarizes the inference prediction results of each NWDAF. For example, when the Internet of Vehicles server requests network switching service quality prediction for Internet of Vehicles services at various locations on a path, AnLF can aggregate and summarize the inference results of each NWDAF in the service area that can cover the path and provide it as the final analysis result. To the Internet of Vehicles server.
  • the above is based on each client
  • the feedback of the accuracy evaluation information of the local model to determine whether to perform federated learning can be considered as a process of determining whether the distribution of local data of each client is consistent or similar based on the accuracy evaluation information.
  • the process of determining whether to perform federated learning is focused on describing the accuracy evaluation information of the local model fed back by the client MTLF.
  • the first design described above can be used to determine whether federated learning needs to be performed, or the second design described above can be used to determine whether federated learning needs to be performed. Alternatively, the first and second designs above can be combined while determining whether federated learning needs to be performed. For example, you can first adopt the first design mentioned above to determine whether to perform federated learning. If it is determined that federated learning is not to be performed, a model request response message is sent to AnLF. The model request response message carries the reason for the failure of federated learning. The reason may be that the network data trained on the current model is related to the geographical location, and federation is not appropriate. study.
  • the second design mentioned above can be used to determine whether to perform federated learning. If federated learning is performed, proceed to the next steps of this application. If it is determined that federated learning is not to be performed, a model request response message is sent to AnLF, and the model request response message carries analysis aggregation instructions, etc.
  • Step 504 The server MTLF uses the client MLF to perform federated learning.
  • the federated learning process can be multiple rounds.
  • the server MTLF sends the initial model of this round to the client MTLF.
  • the client MTLF uses the locally collected data to train the initial model, updates the parameters of the initial model, and updates the parameters of the initial model.
  • the server MTLF updates the initial model based on the updated parameters of the initial model fed back by each client, and determines the model for this round of training.
  • the model for this round of training can be used as the initial model for the next round of model training.
  • the server MTLF can send a second model evaluation request message to each client MTLF.
  • the second model evaluation request message carries the model of this round of training and is used to request the client.
  • MTLF reports the accuracy evaluation information of the model for this round of model training.
  • the client MTLF receives the model of this round of training, it verifies the accuracy of the model based on locally collected data, and reports the accuracy evaluation information of this round of training model to the server MTLF.
  • the client MTLF may send a second model evaluation request response message to the server MTLF, where the second model evaluation request response message carries the accuracy evaluation information of the current round of training model.
  • the accuracy evaluation information can be the accuracy evaluation value of the current round of training model, or the accuracy evaluation level of the current round of training model, without limitation.
  • the server MTLF When the server MTLF receives the accuracy evaluation information of this round of training model reported by each client, it can determine whether the accuracy evaluation information of this round of model reported by each client meets the target accuracy in the above step 502. If satisfied, federated learning is stopped; AnLF sends a model request response message to the server MTLF, and the model request response message carries the model obtained by federated learning. Alternatively, if the accuracy evaluation information of this round of models reported by each client does not meet the target accuracy in step 502, the next round of model training is continued.
  • Step 505 The server MTLF sends a model request response message to AnLF.
  • the model request response message carries the model obtained by federated learning, the reason for the failure of federated learning, or instructs AnLF to perform aggregation analysis, etc.
  • Step 506 AnLF performs network analysis and sends an analysis subscription notification message or an analysis information response message to the AC of the analysis service, which carries the analysis results.
  • the model request response message in step 505 above carries a model obtained by federated learning
  • AnLF can perform model inference based on the model obtained by federated learning, and feed the inference results back to AnLF as an analysis result.
  • the model request response message in step 505 above carries instructions for AnLF to perform aggregation analysis
  • AnLF performs aggregation analysis and feeds back the result of the aggregation analysis to the AC as the analysis result.
  • the server MTLF can determine whether to use the local model or the model obtained in each round of federated learning. Performing subsequent federated learning can avoid unnecessary federated learning, save network resources, and at the same time meet the accuracy requirements of the analysis service model.
  • the server MTLF, client MTLF, AnLF, etc. include hardware structures and/or software modules corresponding to each function.
  • the units and method steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software driving the hardware depends on the specific application scenarios and design constraints of the technical solution.
  • Figures 6 and 7 are schematic structural diagrams of possible communication devices provided by embodiments of the present application. These communication devices can be used to implement the functions of the server MTLF, client MTLF or AnLF in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments.
  • the communication device 600 includes a processing unit 610 and a transceiver unit 620 .
  • the communication device 600 is used to implement the functions of the server MTLF, client MTLF or AnLF in the method embodiments shown in FIG. 3, FIG. 4, or FIG. 5.
  • the transceiver unit 620 is used to send the first model to the N candidate client model training logic functions MTLF respectively; receive them respectively.
  • First accuracy evaluation information from the N candidate client MTLFs the first accuracy evaluation information represents the accuracy of the first model determined by the candidate client MTLFs using local data, the N is a positive integer greater than 1;
  • the processing unit 610 is configured to determine the client MTLF that participates in federated learning among the N candidate client MTLFs according to the N pieces of the first accuracy evaluation information.
  • the transceiver unit 620 is used to receive a model evaluation request message from the server model training logic function MTLF, the model The evaluation request message includes the first model; the processing unit 610 is configured to use local data to determine the accuracy evaluation information of the first model, where the accuracy evaluation information is the first accuracy evaluation information and the second accuracy evaluation Information; the transceiver unit 620 is also configured to send a model evaluation request response message to the server MTLF, where the model request response message includes the accuracy evaluation information of the first model.
  • the transceiver unit 620 is used to receive a model request message from the analysis and reasoning function AnLF, where the model request message includes at least one of the following : Analysis identification, model filtering information, or the target accuracy that the model needs to meet.
  • the analysis identification is used to identify the analysis task.
  • the model filtering information is used to indicate the conditions that the training data needs to meet in the federated learning process.
  • the model needs The met target accuracy is used to indicate the accuracy that the model for inferring the current analysis task needs to meet; the processing unit 610 is used to determine whether to execute federated learning according to the model request message.
  • the transceiver unit 620 is used to receive an analysis request message from the user, where the analysis request message includes an analysis identifier and what the analysis hopes to achieve. Accuracy; the processing unit 610 is used to determine that the local model of the analysis and reasoning function AnLF cannot meet the requirements of the analysis request message, and the model training logic function MTLF that meets the requirements of the analysis request message cannot be queried in the network warehouse function NRF; sending and receiving Unit 620 is also configured to send a model request message to the server MTLF.
  • the model request message includes at least one of the following: analysis identification, model filtering information, or target accuracy that the model needs to meet.
  • the target accuracy that the model needs to meet is accurate. The degree is determined based on the analysis accuracy that the analysis hopes to achieve, and the model filtering information is used to indicate the conditions that the training data needs to meet during the model training process.
  • the communication device 700 includes a processor 710 and an interface circuit 720 .
  • the processor 710 and the interface circuit 720 are coupled to each other.
  • the interface circuit 720 may be a transceiver or an input-output interface.
  • the communication device 700 may also include a memory 730 for storing instructions executed by the processor 710 or input data required for the processor 710 to run the instructions or data generated after the processor 710 executes the instructions.
  • the processor 710 is used to implement the functions of the above-mentioned processing unit 610, and the interface circuit 720 is used to implement the functions of the above-mentioned transceiver unit 620.
  • the server MTLF module implements the functions of the server MTLF in the above method embodiment.
  • the server MTLF module receives information from other modules in the server MTLF (such as radio frequency modules or antennas), which is sent by the client MTLF or AnLF to the server MTLF; or, the server MTLF module sends information to other modules in the server MTLF (such as Radio frequency module or antenna) sends information, which is sent by the server MTLF to the client MTLF or AnLF.
  • the client MTLF module implements the functions of client MTLF in the above method embodiment.
  • the client MTLF module receives information from other modules (such as radio frequency modules or antennas) in the client MTLF.
  • the information is sent by the server MTLF to the client MTLF; or, the client MTLF module sends information to other modules in the client MTLF. (such as a radio frequency module or antenna) sends information, which is sent by the client MTLF to the server MTLF.
  • the AnLF module When the above communication device is a module applied to AnLF, the AnLF module implements the functions of AnLF in the above method embodiment.
  • the AnLF module receives information from other modules in AnLF (such as radio frequency modules or antennas), which is sent to AnLF by the server MTLF; or, the AnLF module sends information to other modules in AnLF (such as radio frequency modules or antennas), This information is sent by AnLF to the server MTLF.
  • this application also provides a communication system 800, which includes the aforementioned device 810 corresponding to server MTLF and the device 820 corresponding to client MTLF.
  • the communication system also includes: AnLF corresponding device 830.
  • processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), or application-specific integrated circuit (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.
  • CPU central processing unit
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • a general-purpose processor can be a microprocessor or any conventional processor.
  • the method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions.
  • Software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory, flash memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory In memory, register, hard disk, mobile hard disk, CD-ROM or any other form of storage medium well known in the art.
  • An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and storage media may be located in an ASIC. Additionally, the ASIC can be located in the base station or terminal. Of course, the processor and the storage medium may also exist as discrete components in the base station or terminal.
  • the computer program product includes one or more computer programs or instructions.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, a network device, a user equipment, or other programmable device.
  • the computer program or instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another.
  • the computer program or instructions may be transmitted from a website, computer, A server or data center transmits via wired or wireless means to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media.
  • the available media may be magnetic media, such as floppy disks, hard disks, and tapes; optical media, such as digital video optical disks; or semiconductor media, such as solid-state hard drives.
  • the computer-readable storage medium may be volatile or nonvolatile storage media, or may include both volatile and nonvolatile types of storage media.
  • “at least one” refers to one or more, and “plurality” refers to two or more.
  • “And/or” describes the relationship between associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural.
  • the character “/” generally indicates that the related objects before and after are an “or” relationship; in the formula of this application, the character “/” indicates that the related objects before and after are a kind of "division” Relationship.
  • “Including at least one of A, B and C” may mean: including A; including B; including C; including A and B; including A and C; including B and C; including A, B and C.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Biophysics (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Geometry (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种数据分析方法及装置，该方法包括：服务器MTLF向N个候选客户端MTLF分别发送第一模型；分别接收来自所述N个候选客户端MTLF的第一准确度评估信息，所述第一准确度评估信息表示所述候选客户端MTLF使用本地的数据确定的所述第一模型的准确度，所述N为大于1的正整数；服务器MTLF根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF。例如，服务器MTLF可选择第一准确度评估信息相似或趋于一致的客户端MTLF参与联邦学习，保证联邦学习得到的模型的预测准确度。

Description

一种数据分析方法及装置
相关申请的交叉引用
本申请要求在2022年08月09日提交中国专利局、申请号为202210951168.5、申请名称为“一种数据分析方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及通信技术领域，尤其涉及一种数据分析方法及装置。
背景技术
联邦学习(federated learning，FL)是一种分布式机器学习，其在保障数据隐私安全的基础上，实现数据共享，共同建模。其核心思想是通过在多个拥有本地数据库的数据源之间进行分布式模型训练，在不需要交换数据样本的前提下，仅需要交换模型中间参数，实现模型联合训练。在联邦学习中，选择参与联邦学习的客户端，是当前关注的问题。
发明内容
本申请提供一种数据分析方法及装置，以确定联邦学习过程中参与联邦学习的客户端。
第一方面，提供一种数据分析方法，该方法的执行主体为服务器MTLF，或者配置于服务器MTLF中的装置(电路、芯片或其它等)，以服务器MTLF为执行主体为例，该方法包括：服务器MTLF向N个候选客户端MTLF分别发送第一模型；可选的，所述第一模型为联邦学习过程中的初始模型、中间模型、或最终模型。服务器MTLF分别接收来自所述N个候选客户端MTLF的第一准确度评估信息，所述第一准确度评估信息表示候选客户端MTLF使用本地的数据确定的所述第一模型的准确度，所述N为大于1的正整数；可选的，所述候选客户端MTLF本地的数据(local data)为所述候选客户端MTLF在自身服务区内收集的数据，在联邦学习过程中，客户端MTLF在自身服务区域收集的本地的数据不会发送给其它客户端MTLF。本地的数据也可称为本地的数据集(local data set)。服务器MTLF根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF。
通过上述设计，服务器MTLF可选择反馈的模型评估信息相近的客户端MTLF参与联邦学习。对于模型的准确度评估信息相近的客户端MTLF，其本地的数据的分布也是相近的。选择本地的数据分布相近的客户端MTLF进行联邦学习，可提高联邦学习训练的模型的预测准确度。
在一种设计中，服务器MTLF分别向所述N个候选客户端MTLF发送第一模型评估请求消息，所述第一模型评估请求消息中包括所述第一模型；服务器MTLF分别接收来自所述N个候选客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述第一准确度评估信息。
在一种设计中，N个所述第一准确度评估信息为所述N个候选客户端MTLF反馈的所 述第一模型准确度的N个评估值，所述参与联邦学习的客户端MTLF中任两个客户端MTLF反馈的评估值间的差值小于或等于第一阈值。例如，在一种实现方式中，在所述N个评估值中，确定取值最大的评估值与取值最小的评估值；所述取值最大的评估值与取值最小的评估值之差，小于或等于所述第一阈值，所述N个候选客户端均参与联邦学习；或者，所述取值最大的评估值与取值最小的评估值之差，大于所述第一阈值，确定所述N个评估值的平均值；确定所述N个评估值中每个评估值与所述平均值的差的绝对值，所述评估值到所述平均值的差的绝对值称为所述评估值对应的候选客户端到质心的距离；在所述N个候选客户端组成的联邦学习组中，剔除到所述质心距离最大的候选户端MTLF，组成新的联邦学习组；继续在所述新的联邦学习组中，确定最大的评估值与最小评估值，且确定两个评估值之差与第一阈值的大小关系。或者，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级，所述参与联邦学习的客户端MTLF反馈的评估等级满足目标评估等级。例如，在一种实现方式中，在N个评估等级中，确定与目标评估等级间的差值小于或等于第三阈值的评估等级；将与所述目标评估等级间的差值小于或等于第三阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
通过上述方法，可使得在N个候选客户端MTLF中，所选择的参与联邦学习的客户端MTLF所反馈的准确度评估信息相接近，比如，任两个客户端MTLF反馈的第一模型的准确度评估值的差值小于或等于第一阈值，从而保证所选择的参与联邦学习的客户端MTLF的本地的数据分布趋于一致，提高联邦学习的模型的预测准确度。
在一种设计中，还包括：服务器MTLF向分析推理功能AnLF发送第一模型；服务器MTL接收来自所述AnLF的第二准确度评估信息，所述第二准确度评估信息表示所述AnLF使用本地的数据确定的所述第一模型的准确度。可选的，所述AnLF的本地的数据为所述AnLF在自身服务区域内收集的数据。例如，在一种实现方式中，服务器MTLF向所述AnLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括所述第一模型；服务器MTLF接收来自所述AnLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述第二准确度评估信息。
在一种设计中，根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF，包括：根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF。
在一种设计中，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考值，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估值，所述参与联邦学习的客户端MTLF中任一个客户端MTLF反馈的评估值与所述参考值间的差值小于或等于第四阈值。所述根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF，包括：在N个评估值中，确定与所述参考值间的差值小于或等于第四阈值的评估值；将与所述参考值间的差值小于或等于第四阈值的评估值对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。或者，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考等级，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级，所述参与联邦学习的客户端MTLF所反馈的评估等级与所述参考等级间的等级间的差值小于或等于第五阈值。根据N个所述第一准确度评估信息和所述第二准确度评估 信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF，包括：在N个评估等级中，确定与所述参考等级间的差值小于或等于第五阈值的评估等级；将与所述参考等级间的差值小于或等于第五阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
通过上述设计，如果客户端MTLF的本地的数据的特征与AnLF本地的数据的特征趋于一致或相似，则客户端MTLF确定的第一模型的第一准确度评估信息和AnLF确定的第一模型的第二准确度评估信息应该相差不大。若两者相差太大，则客户端MTLF的本地的数据特征与AnLF的本地的数据特征有较大差异；通过上述设计，在N个候选客户端MTLF中，剔除掉与AnLF的本地的数据特征差异较大的候选客户端MTLF，从而保证参与联邦学习的客户端MTLF的本地的数据与AnLF的本地的数据的特征相近，提高联邦学习的模型的推理准确度。
在一种设计中，所述评估值，或参考值包括以下至少一项：正确率、错误率、精度率、召回率、平均绝对误差、平均绝对百分比误差、或均方误差。
在一种设计中，还包括：接收来自所述AnLF的模型请求消息，所述模型请求消息中至少包括模型需要满足的目标准确度；参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息满足所述目标准确度的要求时，结束联邦学习；或者，参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息不满足所述目标准确度的要求，根据所述参与联邦学习的客户端MTLF反馈的模型参数，确定第二模型。
通过上述设计，AnLF可通过模型请求消息将训练的模型需要达到的目标准确度通知服务器MTLF；服务器MTLF在联邦学习训练得到的模型满足该目标准确度时，则停止联邦学习，从而使得联邦学习训练得到的模型满足预设的目标准确度的要求。
第二方面，提供一种数据分析方法，该方法的执行主体为客户端MTLF或AnLF，或者配置于客户端MTLF中的装置，或者配置于AnLF中的装置等，包括：接收来自服务器模型训练逻辑功能MTLF的模型评估请求消息，所述模型评估请求消息中包括第一模型；可选的，所述第一模型为联邦学习过程中的初始模型、中间模型、或最终模型。利用本地的数据，确定所述第一模型的准确度评估信息；向所述服务器MTLF发送模型评估请求响应消息，所述模型请求响应消息中包括所述第一模型的准确度评估信息。
在一种设计中，所述准确度评估信息可以为第一准确度评估信息，所述第一准确度评估信息表示候选客户端MTLF使用本地的数据确定的所述第一模型的准确度。或者，所述准确度评估信息为第二准确度评估信息，所述第二准确度评估信息表示分析推理功能AnLF使用本地的数据确定的所述第一模型的准确度。
在一种设计中，所述第一模型的准确度评估信息为所述第一模型准确度的评估值，所述利用本地的数据，确定所述第一模型的准确度评估信息，包括：根据所述本地的数据和所述第一模型，确定所述第一模型的输出；根据所述第一模型的输出，确定所述第一模型准确度的评估值。
在一种设计中，所述第一模型的准确度评估信息为所述第一模型准确度的评估等级，所述利用本地的数据，确定所述第一模型的准确度评估信息，包括：根据所述本地的数据和所述第一模型，确定所述第一模型的输出；根据所述第一模型的输出，确定所述第一模型准确度的评估值；根据所述第一模型准确度的评估值，确定所述第一模型准确度的评估等级。
在一种设计中，当该第二方面的执行主体为AnLF或配置于AnLF中的装置时，所述方法还包括：向所述服务器MTLF发送模型请求消息，所述模型请求消息中至少包括模型需要满足的目标准确度。
在一种设计中，当该第二方面的执行主体为AnLF或配置于AnLF中的装置时，在所述向服务器MTLF发送模型请求消息之前，还包括：接收来自用户的分析请求消息，所述分析请求消息中包括以下至少一项：分析标识、分析过滤信息、或该分析需要满足的目标准确度；确定分析推理功能AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的MTLF。
通过上述设计，AnLF如果能够匹配到分析标识和模型过滤信息，并且满足目标准确度的模型，则确定不执行联邦学习；否则，确定执行联邦学习，通过联邦学习过程训练出满足目标准确度的用于推理当前任务的模型。
在一种设计中，所述评估值，包括以下至少一项：正确率、错误率、精度率、召回率、平均绝对误差、平均绝对百分比误差、或均方误差。
第三方面，提供一种数据分析方法，该方法的执行体主体为服务器MTLF，或者配置于服务器MTLF中的装置，包括：接收来自分析推理功能AnLF的模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述分析标识用于标识分析任务，所述模型过滤信息用于指示联邦学习过程中训练数据需要满足的条件，所述模型需要满足的目标准确度用于指示推理当前分析任务的模型需要满足的准确度；根据所述模型请求消息，确定执行或不执行联邦学习。
通过上述设计，服务器MTLF可根据模型请求消息，确定是否进行后续联邦学习，可以避免不必要的联邦学习，节约了网络资源，同时满足了分析服务的模型的准确度要求。
在一种设计中，根据所述模型请求消息，确定执行或不执行联邦学习，包括：根据所述分析标识和/或所述模型过滤信息，确定联邦学习过程中训练数据的特征；所述训练数据的特征与地理位置无关，确定执行联邦学习；或者，所述训练数据的特征与地理位置相关，确定不执行联邦学习。
在一种设计中，根据所述模型请求消息，确定执行或不执行联邦学习，包括：向参与联邦学习的客户端MTLF发送第一模型评估请求消息，所述第一模型评估请求消息用于请求所述客户端MTLF上报本地模型的准确度评估信息；接收来自所述客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述客户端MTLF上报的本地模型的准确度评估消息；所述客户端MTLF反馈的本地模型的准确度评估信息满足所述目标准确度时，确定不执行联邦学习；或者，所述客户端MTLF反馈的本地模型的准确度评估信息不满足所述目标准确度时，确定执行联邦学习。
通过上述设计，服务器MTLF可根据本地模型或每轮联邦学习得到的模型，确定是否进行后续联邦学习，可以避免不必要的联邦学习，节约了网络资源。
在一种设计中，在确定不执行联邦学习时，还包括：向所述AnLF发送模型请求响应消息，所述模型请求响应消息中包括联邦学习失败原因或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式确定当前分析任务的结果。
在一种设计中，在确定执行联邦学习时，还包括：向参与联邦学习的客户端MTLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括本轮模型训练得到的模型，所述第二模型评估请求消息用于请求所述客户端MTLF上报所述本轮训练得到的模型的准 确度评估信息；接收来自所述客户端MTLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述客户端MTLF上报的本轮模型训练得到的模型的准确度评估信息；所述客户端MTLF上报的所述本轮模型训练得到的模型的准确度评估信息满足所述目标准确度，结束联邦学习。
第四方面，提供一种数据分析方法，该方法的执行主体为AnLF，或配置于AnLF中的装置，该方法包括：接收来自用户的分析请求消息，所述分析请求消息中包括分析标识和该分析希望达到的准确度；分析推理功能AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的模型训练逻辑功能MTLF时，向服务器MTLF发送模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述模型需要满足的目标准确度是根据所述分析希望达到的分析准确度确定的，所述模型过滤信息用于指示在模型训练过程中训练数据需要满足的条件。
在一种设计中，还包括：接收来自服务器MTLF的模型请求响应消息，所述模型请求响应消息中包括联邦学习失败原因或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式，确定当前分析任务的结果。
第五方面，提供一种通信装置，该装置可以是服务器MTLF，或者配置于服务器MTLF中的装置(例如芯片等)，或者能够和服务器MTLF匹配使用的装置，该装置具有实现上述第一方面或第三方面方法的功能。该功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的单元，比如，收发单元和处理单元。
第六方面，提供一种通信装置，包括用于执行上述第一方面或第三方面中各个步骤的单元或手段(means)。
第七方面，提供一种通信装置，包括处理器和存储器；该存储器用于存储计算机指令，当该装置运行时，该处理器执行该存储器存储的计算机指令，以使该装置执行上述第一方面或第三方面中的方法。
第八方面，提供一种通信装置，包括与存储器耦合的处理器，该处理器用于调用所述存储器中存储的程序，以执行上述第一方面或第三方面的方法，该存储器可以位于该装置之内，也可以处于该装置之外。且上述处理器可以为一个或多个。
第九方面，提供一种通信装置，包括处理器和接口电路，所述处理器用于通过接口电路与其它装置通信，并执行上述第一方面或第三方面的方法，处理器可以为一个或多个。
第十方面，提供一种通信装置，该装置可以是客户端MTLF，或者配置于客户端MTLF中的装置(如芯片)，或者能够和客户端MTLF匹配使用的装置，或者该装置可以是AnLF，或者配置于AnLF中的装置(如芯片)，或者能够和AnLF匹配使用的装置，该装置具有实现上述第二方面或第四方面的功能。该功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的单元，比如，收发单元和处理单元。
第十一方面，提供一种通信装置，包括用于执行上述第二方面或第四方面中的各个步骤的单元或手段(means)。
第十二方面，提供一种通信装置，包括处理器和存储器；该存储器用于存储计算机指令，当该装置运行时，该处理器执行该存储器存储的计算机指令，以使该装置执行上述第 二方面或第四方面中的方法。
第十三方面，提供一种通信装置，包括与存储器耦合的处理器，该处理器用于调用所述存储器中存储的程序，以执行上述第二方面或第四方面的方法，该存储器可以位于该装置之内，也可以处于该装置之外。且上述处理器可以为一个或多个。
第十四方面，提供一种通信装置，包括处理器和接口电路，所述处理器用于通过接口电路与其它装置通信，并执行上述第二方面或第四方面的方法，处理器可以为一个或多个。
第十五方面，提供一种芯片系统，包括：处理器，用于执行上述第一方面至第四方面中任一方面的方法。
第十六方面，提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在通信装置上运行时，使得上述第一方面至第四方面中的任一方面的方法被执行。
第十七方面，提供一种计算机程序产品，该计算机程序产品包括计算机程序或指令，当计算机程序或指令被通信装置运行时，使得上述第一方面至第四方面中的任一种方面的方法被执行。
第十八方面，提供一种通信系统，该系统包括前述第五方面至第九方面中任一方面的装置，和第十方面至第十四方面中任一方面的装置。
第十九方面，提供一种数据分析方法，该方法包括：服务器MTLF向N个候选客户端MTLF分别发送第一模型；N个候选客户端MTLF中的任一个候选客户端MTLF利用本地的数据，确定所述第一模型的第一准确度评估信息，所述第一准确度评估信息表示候选客户端MTLF使用本地的数据确定的所述第一模型的准确度，所述N为大于1的正整数；候选客户端MTLF向所述服务器MTLF发送所述第一准确度评估信息；服务器MTLF根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF。
在一种设计中，所述服务器MTLF可分别向N个候选客户端MTLF发送第一模型评估请求消息，所述第一模型评估请求消息中包括第一模型；服务器MTLF分别接收来自所述N个候选客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述第一准确度评估信息。
在一种设计中，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估值。例如，候选客户端MTLF根据本地的数据和所述第一模型，确定所述第一模型的输出；候选客户端根据所述第一模型的输出，确定所述第一模型准确度的评估值。服务器MTLF确定的参与联邦学习的客户端MTLF中任两个候选客户端MTLF反馈的评估值间的差值小于或等于第一阈值。例如，服务器MTLF在所述N个评估值中，确定取值最大的评估值与取值最小的评估值；所述取值最大的评估值与取值最小的评估值之差小于或等于所述第一阈值，所述N个候选客户端均参与联邦学习；或者，所述取值最大的评估值与取值最小的评估值之差大于所述第一阈值，确定所述N个评估值的平均值；确定所述N个评估值中每个评估值与所述平均值的差的绝对值；在所述N个候选客户端组成的联邦学习组中，剔除与所述平均值的差的绝对值最大的评估值对应的候选户端MTLF，组成新的联邦学习组；继续在所述新的联邦学习组中，确定取值最大的评估值与取值最小的评估值，且确定两个评估值之差与所述第一阈值的大小关系。
在一种设计中，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级。例如，客户端MTLF根据本地的数据和所述第一模型，确定所述第 一模型的输出；客户端MTLF根据所述第一模型的输出，确定所述第一模型准确度的评估值；客户端MTLF根据所述第一模型准确度的评估值，确定所述第一模型准确度的评估等级。服务器MTLF确定的所述参与联邦学习的客户端MTLF反馈的评估等级满足目标评估等级。例如，服务器MTLF在所述N个评估等级中，确定与所述目标评估等级间的差值小于或等于第三阈值的评估等级；服务器MTLF将与所述目标评估等级间的差值小于或等于第三阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
在一种设计中，还包括：服务器MTLF向分析推理功能AnLF发送第一模型；AnLF利用本地的数据，确定所述第一模型的第二准确度评估信息，所述第二准确度评估信息表示所述AnLF使用本地的数据确定的所述第一模型的准确度。AnLF向所述服务器MTLF发送所述第二准确度评估信息。
在一种设计中，服务器MTLF向所述AnLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括所述第一模型。服务器MTLF接收来自所述AnLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述第二准确度评估信息。
在一种设计中，服务器MTLF根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF，包括：服务器MTLF根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF。
在一种设计中，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考值，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估值，服务器MTLF确定的参与联邦学习的客户端MTLF中任一个客户端MTLF反馈的评估值与所述参考值间的差值小于或等于第四阈值。例如，服务器MTLF在N个评估值中，确定与所述参考值间的差值小于或等于第四阈值的评估值；服务器MTLF将与所述参考值间的差值小于或等于第四阈值的评估值对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
在一种设计中，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考等级，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级，服务器MTLF确定的参与联邦学习的客户端MTLF所反馈的评估等级与所述参考等级间的等级间的差值小于或等于第五阈值。例如，服务器MTLF在N个评估等级中，确定与所述参考等级间的差值小于或等于第五阈值的评估等级；服务器MTLF将与所述参考等级间的差值小于或等于第五阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
在一种设计中，还包括：AnLF向服务器MTLF发送模型请求消息，所述模型请求消息中至少包括模型需要满足的目标准确度；服务器MTLF在参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息满足所述目标准确度的要求时，结束联邦学习；或者，参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息不满足所述目标准确度的要求，服务器MTLF根据所述参与联邦学习的客户端MTLF反馈的模型参数，确定第二模型，利用第二模型继续联邦学习。
在一种设计中，所述AnLF在向服务器MTLF发送模型请求消息之前，还包括：所述AnLF接收来自用户的分析请求消息，所述分析请求消息中包括以下至少一项：分析标识、分析过滤信息、或该分析需要满足的目标准确度；AnLF确定AnLF的本地模型不能满足 所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的MTLF。
第二十方面，提供一种数据分析方法，包括：AnLF接收来自用户的分析请求消息，所述分析请求消息中包括分析标识和该分析希望达到的准确度；AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的模型训练逻辑功能MTLF时，AnLF向服务器MTLF发送模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述模型需要满足的目标准确度是根据所述分析希望达到的分析准确度确定的，所述模型过滤信息用于指示在模型训练过程中训练数据需要满足的条件。服务器MTLF根据所述模型请求消息，确定执行或不执行联邦学习。
在一种设计中，服务器MTLF根据所述模型请求消息，确定执行或不执行联邦学习，包括：根据所述分析标识和/或所述模型过滤信息，确定联邦学习过程中训练数据的特征；所述训练数据的特征与地理位置无关，确定执行联邦学习；或者，所述训练数据的特征与地理位置相关，确定不执行联邦学习。
在一种设计中，服务器MTLF根据所述模型请求消息，确定执行或不执行联邦学习，包括：向参与联邦学习的客户端MTLF发送第一模型评估请求消息，所述第一模型评估请求消息用于请求所述客户端MTLF上报本地模型的准确度评估信息；接收来自所述客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述客户端MTLF上报的本地模型的准确度评估消息；所述客户端MTLF反馈的本地模型的准确度评估信息满足所述目标准确度，确定不执行联邦学习；或者，所述客户端MTLF反馈的本地模型的准确度评估信息不满足所述目标准确度，确定执行联邦学习。
在一种设计中，服务器MTLF在确定不执行联邦学习时，还包括：服务器MTLF向所述AnLF发送模型请求响应消息，所述模型请求响应消息中包括联邦学习失败原因或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式确定当前分析任务的结果。
在一种设计中，服务器MTLF在确定执行联邦学习时，还包括：向参与联邦学习的客户端MTLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括本轮模型训练得到的模型，所述第二模型评估请求消息用于请求所述客户端MTLF上报所述本轮训练得到的模型的准确度评估信息；接收来自所述客户端MTLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述客户端MTLF上报的本轮模型训练得到的模型的准确度评估信息；所述客户端MTLF上报的所述本轮模型训练得到的模型的准确度评估信息满足所述目标准确度，结束联邦学习。
附图说明
图1为本申请提供的网络架构示意图；
图2为本申请提供的另一种网络架构示意图；
图3为本申请提供的数据分析方法的一流程图；
图4为本申请提供的数据分析方法的另一流程图；
图5为本申请提供的数据分析方法的又一流程图；
图6为本申请提供的装置的一示意图；
图7为本申请提供的装置的另一示意图；
图8为本申请提供的系统的示意图。
具体实施方式
图1是本申请的实施例应用的通信系统1000的架构示意图。如图1所示，该通信系统1000包括核心网，核心网中包括以下一项或多项实体：
接入和移动管理功能网元：主要用于移动网络中的终端的附着、移动性管理和跟踪区更新流程。接入和移动管理功能网元处理非接入层(non access stratum，NAS)消息、完成注册管理、连接管理以及可达性管理、分配跟踪区域列表(track area list，TA list)以及移动性管理等，并且透明路由会话管理(session management，SM)消息到会话管理网元。在第5代(5th generation，5G)通信系统中，接入和移动管理功能网元可以是接入与移动性管理功能(access and mobility management function，AMF)。
会话管理网元：主要用于移动网络中的会话管理，如会话建立、修改、释放。具体功能如为终端分配互联网协议(internet protocol，IP)地址、选择提供报文转发功能的用户面网元等。在5G通信系统中，会话管理网元可以是会话管理功能(session management function，SMF)。
策略控制网元：包含用户签约数据管理功能、策略控制功能、计费策略控制功能、服务质量(quality of service，QoS)控制等。在5G通信系统中，策略控制网元可以是策略控制功能(policy control function，PCF)。需要指出，实际网络中PCF还可能按照层次或按功能分为多个实体，例如全局PCF和切片内的PCF，或者会话管理PCF(session,management PCF，SM-PCF)和接入管理PCF(access management PCF，AM-PCF)。
网络切片选择网元：主要用于为终端的业务选择合适的网络切片。在5G通信系统中，网络切片选择网元可以是网络切片选择功能(network slice selection function，NSSF)网元。
统一数据管理网元：负责管理终端的签约信息。在5G通信系统中，统一数据管理网元可以是统一数据管理(unified data management，UDM)。
数据分析网元：数据分析网元从各个网络功能(network function，NF)，例如AMF、SMF、PCF等，收集网络数据。数据分析网元可以通过网络开发功能(network exposure function，NEF)间接从应用功能(application function，AF)收集网络数据，或直接从AF收集网络数据；数据分析网元还可以从运行管理和维护(operation,administration,and maintenance，OAM)系统收集网络数据。数据分析网元可以根据收集的上述网络数据进行分析和预测。数据分析网元收集相关网络数据，使用机器学习技术，将收集的网络数据进行训练拟合出一个模型，然后根据模型输出分析服务。在5G通信系统中，数据分析网元可以是网络数据分析功能(network data analytics function，NWDAF)，或者管理数据分析系统(management data analytics system，MDAS)。
用户面网元：主要负责对用户报文进行处理，如转发、计费、合法监听等。用户面网元也可以称为协议数据单元(protocol data unit，PDU)会话锚点(PDU session anchor，PSA)。在5G通信系统中，用户面网元可以是用户面功能(user plane function，UPF)。UPF可以通过类似服务化的接口直接和NWDAF通信，也可以通过其他途径，例如通过SMF或者和NWDAF之间的私有接口或内部接口，和NWDAF通信。
应用功能网元：主要支持与第三代合作伙伴计划((3rd generation partnership project， 3GPP)核心网交互来提供服务，例如影响数据路由决策、策略控制功能或者向网络侧提供第三方的一些服务。在5G通信系统中，应用功能网元可以是AF。
网络开放功能网元：主要用于支持能力和事件的开放，如用于安全地向外部开放由3GPP网络功能提供的业务和能力等。在5G通信系统中，网络开发功能网元也以是网络开放功能(network exposure function，NEF)。
网络存储功能网元：主要用于保存网络功能实体以及其提供服务的描述信息，支持服务发现和网元实体发现等。在5G通信系统中，网络存储功能网元可以是网络存储器功能(network repository function，NRF)。
操作管理维护网元：主要用于对网络设备的资源配置、性能统计、故障告警等进行管理。在5G通信系统中，操作管理维护网元可以是OAM等。
除了上述核心网涉及的实体外，通信系统1000还可以包括下列设备或网元：
终端：是一种具有无线收发功能的设备。终端也可以称为终端设备、用户设备(user equipment，UE)、移动台、移动终端等。终端可以广泛应用于各种场景，例如，设备到设备(device-to-device，D2D)、车物(vehicle to everything，V2X)通信、机器类通信(machine-type communication，MTC)、物联网(internet of things，IOT)、虚拟现实、增强现实、工业控制、自动驾驶、远程医疗、智能电网、智能家具、智能办公、智能穿戴、智能交通、智慧城市等。终端可以是手机、平板电脑、带无线收发功能的电脑、可穿戴设备、车辆、无人机、直升机、飞机、轮船、机器人、机械臂、智能家居设备等。本申请的实施例对终端所采用的具体技术和具体设备形态不做限定。为了便于描述，下文以终端作为例子进行描述。
接入网(access network，AN)设备：用于负责终端的无线侧接入，可以是基站(base station)、演进型基站(evolved NodeB，eNodeB)、发送接收点(transmission reception point，TRP)、第五代(5th generation，5G)移动通信系统中的下一代基站(next generation NodeB，gNB)、第六代(6th generation，6G)移动通信系统中的下一代基站、未来移动通信系统中的基站或无线保真(wireless fidelity，WiFi)系统中的接入节点等；也可以是完成基站部分功能的模块或单元，例如，可以是集中式单元(central unit，CU)，也可以是分布式单元(distributed unit，DU)。这里的CU完成基站的无线资源控制(radio resource control，RRC)协议和分组数据汇聚层协议(packet data convergence protocol，PDCP)的功能，还可以完成业务数据适配协议(service data adaptation protocol，SDAP)的功能；DU完成基站的无线链路控制(radio link control，RLC)层和介质访问控制(medium access control，MAC)层的功能，还可以完成部分物理(physical，PHY)层或全部物理层的功能，有关上述各个协议层的具体描述，可以参考3GPP的相关技术规范。接入网设备可以是宏基站，也可以是微基站或室内站，还可以是中继节点或施主节点等。本申请的实施例对接入网设备所采用的具体技术和具体设备形态不做限定。为了便于描述，下文以基站作为接入网设备的例子进行描述。
数据网络(data network，DN)可以是为用户提供数据业务服务的服务网络。例如，DN可以是IP多媒体业务(IP multi-media service)网络或互联网(internet)等。其中，终端设备可以建立从终端设备到DN的协议数据单元(protocol data unit，PDU)会话，来访问DN。
在本申请中，为了提供分析服务，数据分析网元训练模型，并根据模型输出分析结果。以数据分析网元为5G通信系统中的NWDAF为例，NWDAF可以分为模型训练逻辑功能 (model training logical function，MTLF)和分析推理逻辑功能(analytics logical function，AnLF)两个部分。其中，NWDAF可以只具有MTLF功能，或者只具有AnLF功能，或者同时具有MTLF功能和AnLF功能。MTLF可以是独立的网元，也可以是NWDAF中的功能单元。同理，AnLF可以是独立的网元，也可以是NWDAF中的功能单元。其中，MTLF用于根据收集的数据训练模型，模型训练完成后，向一个或多个AnLF分发训练好的模型。AnLF用于使用模型进行推理，向各个网元提供分析服务。例如，车联网服务器可以向AnLF请求预测未来10分钟内某个地点的网络性能，该网络性能包括该地点的终端的最大收发速度等数据。AnLF可以收集该地点的终端的相关数据，且从MTLF获取网络性能预测模型，将收集的终端数据能作为输入，输入到网络性能预测模型中，该模型的输出为预测的未来10分钟该地点中终端收发数据所能达到的最大速度，然后提供给车联网服务器。
现实网络中一般会部署多个NWDAF，每个NWDAF通常都有自身的服务区域，NWDAF的服务区域可以是一个或多个跟踪区(tracking area，TA)覆盖的区域，TA由广播同一个跟踪区编码的一个或多个小区组成。NWDAF中的MTLF在收集原始数据进行模型训练时，很难从不同区域的分布式数据源收集所有原始数据。为了解决这个问题，MWDAF采用联邦学习(federated learning，FL)技术来训练模型。在进行机器学习的过程中，不需要将原始的训练数据集中传输到某个MWDAF，而只需要多个NWDAF中的MTLF之间合作，各个参与方可借助其他的数据进行联合建模，通过联邦学习进行数据联合训练，建立共享的机器学习模型。其中，组织联邦学习的MTLF称为联邦学习服务器MTLF(FL Server MTLF)，可简称为服务器MTLF；参与联邦学习的MTLF称为联邦学习客户端MTLF(FL Client MTLF)，可简称为Client MTLF。
如图2所示，分析服务的消费者(analytics consumer，AC)可以向AnLF发送分析请求订阅消息或分析信息请求消息等，用于请求AnLF执行某项分析服务。AC可以为网络网元、应用功能或OAM等，不作限制。在AnLF的本地模型不能完成该项分析任务，且在NRF中查询不到能执行该项分析任务的MTLF时，可以在NRF中查询具有服务器角色的MTLF，称为服务器MTLF。AnLF可以向查询到的服务器MTLF发送模型请求消息，以请求服务器MTLF组织联邦学习，联合训练模型，完成该项分析任务。具体的，服务器MTLF在接收到AnLF的模型请求消息时，可通过查询NRF，确定参与联邦学习的多个客户端MTLF。每个客户端MTLF对应不同的数据源(data source)，数据源可以是客户端MTLF收集的本地的数据组成的。服务器MTLF向确定的客户端MTLF发送请求消息，该请求消息用于请求客户端MTLF加入联邦学习，该请求消息中可包括分析标识、过滤信息和联邦学习组标识等；客户端MTLF在确定加入联邦学习组后，根据分析标识和过滤信息从各个数据源收集用于模型训练的数据，可称为训练数据；客户端MTLF向服务器MTLF发送加入联邦学习组的响应消息。服务器MTLF在接收到加入联邦学习组的响应消息后，组织参与联邦学习的各个客户端MTLF一起进行联邦学习。具体的，服务器MTLF将初始的模型发送给客户端MTLF。各个客户端MTLF使用本地收集的训练数据对初始模型进行训练，更新模型参数；各个客户端MTLF将本地训练得到的模型参数发送给服务器MTLF。服务器MTLF将各个客户端MTLF发送的各个模型参数进行聚合。若该聚合得到的模型的准确度满足目标准确度，或者联合学习的次数达到预设次数，则停止联邦学习，将训练得到的模型发送给AnLF。AnLF可根据该训练得到的模型，对分析任务进行分析，且将分析结果返回AC。或者，若上述聚合得到的模型不能满足目标准确度，或者联邦学习的次数 未达到预设次数，则服务器MTLF可将上一轮聚合得到的模型，作为下一轮模型训练的初始模型，发送给客户端MTLF，组织客户端MTLF进行下一轮模型训练。
在一种设计中，服务器MTLF在接收到AnLF的模型请求消息时，直接组织客户端MTLF进行联邦学习。但在现实中，网络中的大量数据的分布特征是不同的，采用不同分布特征的数据进行联邦学习，可能导致联邦学习得到的模型的准确度并不高。例如，对于网络负荷预测的分析任务，要训练一个模型预测网络负荷。假设服务器MTLF确定有3个客户端MTLF参与联邦学习。该3个客户端MTLF所属的区域，分别为商业区、办公区和居民网。对于居民区在晚上和早晨网络负荷较重，工作日白天网络负荷很轻；对于办公区在工作日白天网络负荷很重，周末负荷很轻；对于商业区周末网络负荷很重，工作日网络负荷较轻。如果服务范围分别为居民区、商业区、和办公区的3个客户端MTLF，均根据本地收集的数据进行模型训练，且将更新的模型的参数，发送给服务器MTLF，由服务器对3个客户端MTLF发送的模型参数进行聚合，可能导致聚合得到的模型的预测准确度较像低。如何确定参与联邦学习的客户端MTLF，是本申请待解决的技术问题。
本申请提供一种数据分析方法，在该方法中：服务器MTLF向确定的参与联邦学习的候选客户端MTLF发送第一模型；参与联邦学习的候选客户端MTLF在接收到第一模型时，使用本地的数据确定第一模型的准确度评估信息，且将确定的准确度评估信息反馈给服务器MTLF。服务器MTLF根据候选客户端MTLF反馈的模型准确度评估信息，确定候选客户端MTLF中参与联邦学习的客户端MTLF。例如，选择反馈的模型评估信息相近的客户端MTLF参与联邦学习。可选的，对于模型的评估信息相近的客户端MTLF，其本地的数据的分布也是相近的。选择本地的数据分布相近的客户端MTLF进行联邦学习，可提高联邦学习训练的模型的预测准确度。如图3所示，提供一种数据分析方法的流程，至少包括：
步骤300：服务器MTLF向N个候选客户端MTLF分别发送第一模型，N为大于1的整数。
例如，服务器MTLF向N个候选客户端MTLF分别发送第一模型评估请求(model evaluation request)消息，所述第一模型评估请求消息中包括所述第一模型。
步骤301：N个候选客户端MTLF分别向服务器MTLF发送第一准确度评估信息。
服务器根据N个候选客户端MTLF分别发送的第一准确度评估信息，可以获得N个第一准确度评估信息。例如，N个候选客户端MTLF中的任一个候选客户端MTLF在接收到第一模型时，可使用本地的数据(local data)确定第一模型的准确度评估信息，该准确度评估信息称为第一准确度评估信息。所述候选客户端的本地的数据是候选客户端MTLF在自身服务区域内收集的数据，在联邦学习的过程中，客户端MTLF收集的本地的数据不会发送给其它客户端MTLF。本地的数据也可称为本地的数据集(local data set)。例如，客户端MTLF使用本地的训练数据集，确定第一模型的准确度评估信息。训练数据集中包括输入数据和标签数据，客户端MTLF可将输入数据，输入到第一模型中；根据第一模型的输出，确定第一模型的预测信息。比如，第一模型的输出可以即为预测信息，或者，可对第一输出的输出进行进一步处理，得到预测信息。上述处理包括但不限于：对第一模型的输出在时域、频域或空域等维度进行变换，得到预测信息等。在后续描述中，将第一模型的输出作为第一模型的预测信息为例描述。将第一模型的预测信息与训练数据集中的标签数据进行比较，确定第一模型在训练阶段的第一准确度评估信息。所述第一模型的准确 度评估信息可以为评估值。所述评估值包括以下至少一项：正确率、错误率、精度率、召回率、平均绝对误差、平均绝对百分比误差、或均方误差等。例如，当第一模型为计算分类的模型时，该第一模型的评估值可为模型的正确率、错误率、精度或召回率等。或者，当第一模型为计算回归的模型时，第一模型的评估值可为模型的平均绝对误差、平均绝对百分比误差或均方误差等。或者，所述第一模型的准确度评估信息可以为评估等级。进一步的，候选客户端MTLF可根据预设的评估值的范围与评估等级的对应关系，确定第一模型的评估值对应的评估等级，候选客户端MTLF向服务器MTLF反馈第一模型的评估等级。例如，所述评估等级包括低准确等级、中准确等级和高准确等级，每个评估等级对应不同的评估值范围。候选客户端MTLF在确定第一模型的评估值时，可根据评估等级与评估值范围的对应关系，确定第一模型的评估值对应的评估等级为低、中、或高。可选的，N个候选客户端MTLF可分别向服务器MTLF发送第一模型评估请求响应(model evaluation request response)消息，所述第一模型评估请求响应消息包括该候选客户端MTLF确定的第一准确度评估信息。
步骤302:服务器MTLF根据N个第一准确度评估信息，确定N个候选客户端MTLF中参与联邦学习的客户端MTLF。
在一种设计中，N个所述第一准确度评估信息为N个候选客户端MTLF反馈的第一模型准确度的N个评估值，所述参与联邦学习的客户端MTLF中任两个客户端MTLF反馈的评估值间的差值小于或等于第一阈值，所述第一阈值为预设的，或协议规定的，或预配置或预通知服务器MTLF的等，不作限制。
例如，可采用聚类法，确定N个候选客户中参与联邦学习的客户端MTLF，实现方式如下：
1、获取N个候选客户端反馈的第一模型准确度的N个评估值；
2、在N个评估值中，确定最大的评估值和最小的评估值；若最大的评估值与最小的评估值之差小于或等于第一阈值，则认为该N个候选客户端都满足条件，均可参与联邦学习；否则，执行下述步骤3。
3、确定N个评估值中，最大的评估值与最小的评估值的平均值。在N个评估值，确定大于上述平均值的评估值的数量为第一数据，小于上述平均值的评估值的数量为第二数量；若第一数量大于第二数量，则将上述最小评估值对应的候选客户端MTLF，从联邦学习组中剔除，组成新的联邦学习组；或者，若第一数量小于或等于第二数量，则将上述最大评估值对应的候选客户端MTLF，从联邦学习组中剔除，组成新的联邦学习组。继续执行上述步骤2，在新的联邦学习组中，确定最大的评估值和最小的评估值，确定最大的评估值与最小的评估值的差与第一阈值的大小关系，并重复执行步骤3，直至所述参与联邦学习的客户端MTLF中任两个客户端MTLF反馈的评估值间的差值小于或等于第一阈值。或者，
可采用最大距离法，确定N个候选客户端中参与联邦学习的客户端MTLF，实现方式如下：
1、在N个候选客户端反馈的第一模型准确度的N个评估值中，确定任两个评估值的差的绝对值，该将任两个评估值的差的绝对值作为两者的距离；
2、如果N个评估值中，任两个评估值的距离都小于或等于第一阈值，则该N个候选客户端MTLF均满足条件，均可参与联邦学习；否则，执行步骤3。
3、确定每一个客户端MTLF的评估值与其它任一个客户端MTLF的评估值的和，累加本客户端MTLF与其它任一个客户端MTLF评估值的和作为本客户端MTLF的距离之和，计算每一个客户端MTLF的距离之和，将距离之和最大的客户端MTLF从联邦学习组中剔除，组成新的联邦学习组。继续上述执行步骤2，在新的联邦学习组中，确定任两个客户端MTLF对应的评估值与第一阈值的大小关系的判断，并重复执行步骤3，直至所述参与联邦学习的客户端MTLF中任两个客户端MTLF反馈的评估值间的差值小于或等于第一阈值。或者，
采用质心算法，确定N个候选客户端中参与联邦学习的客户端MTLF，实现方式如下：
1、获取N个候选客户端反馈的第一模型的准确度的N个评估值；
2、在N个评估值中，确定取值最大的评估值和取值最小的评估值；如果取值最大的评估值与取值最小的评估值之差小于或等于第一阈值，则该N个候选客户端MTLF均满足条件，均可参与联邦学习，结束流程；否则继续执行步骤3。
3、确定该N个评估值的平均值；确定每一个评估值与上述平均值的差的绝对值，两者差的绝对值称为该评估值对应的候选客户端MTLF到质心的距离；
4、确定每个候选客户端MTLF到质心的距离，将到质心的距离最大的客户端MTLF，在联邦学习组中剔除，组成新的联邦学习组。继续执行上述步骤2，在新的联邦学习组中，确定取值最大的评估值与取值最小的评估值，且确定两个评估值之差与第一阈值的大小关系，并重复执行步骤3和4，直至所述参与联邦学习的客户端MTLF中任两个客户端MTLF反馈的评估值间的差值小于或等于第一阈值。
在另一种设计中，N个所述第一准确度评估信息为N个候选客户端MTLF反馈的第一模型准确度的N个评估等级，所述参与联邦学习的客户端MTLF反馈的评估等级满足目标评估等级。例如，服务器MTLF可在所述N个评估等级中，确定与目标评估等级间的差值小于或等于第三阈值的评估等级；将与目标评估等级间的差值小于或等于第三阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。该第三阈值为预设的，或者协议规定的，或预通知或配置给服务器MTLF的，不作限制。所述目标评估等级为预设的，或者协议规定的，或者预先通知或配置给服务器MTLF的，不作限制。例如，所述目标评估等级可以是AnLF预通知服务器MTLF的。
例如，预先划分10个评估等级，评估等级的索引为1至10，其中评估等级的索引的值越大，则代表其对应模型的准确度越高。服务器MTLF可以在N个候选客户端反馈的N个评估等级中，选择与目标评估等级间，评估等级的差值小于或等于第三阈值的评估等级。比如，N的取值为3，3个候选客户端MTLF反馈的评估等级分别评估等级7、评估等级9和评估等级5，服务器MTLF可以在上述3个评估等级中，选择与目标评估等级8间的差值小于或等于第三阈值(例如，第三阈值为1)的评估等级，选择出的评估等级为评估等级7和评估等级9，则评估等级7和评估等级9对应的候选客户端MTLF，参与联邦学习，评估等级5对应的候选客户端MTLF，剔除出联邦学习组，不再参与联邦学习。
可选的，服务器MTLF还可以向AnLF发送第一模型，接收来自AnLF的第二准确度评估信息，所述第二准确度评估信息表示所述AnLF使用本地的数据确定的第一模型的准确度评估信息。例如，服务器MTLF可以向AnLF发送第二模型评估请求消息，该第二模型评估请求消息中包括第一模型，接收来自AnLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述第二准确度评估信息。AnLF在接收到第一模型时， 可利用收集的本地的数据，确定第一模型的准确度，该第一模型的准确度称为第二准确度评估信息。所述AnLF的本地数据为所述AnLF在自身服务区域内收集的数据。例如，AnLF可以根据历史上已经收集的本地的数据，确定第一模型的第二准确度评估信息，通过将第一模型的推理结果与观察到的标签数据(即来自网络的真实数据)进行比较，来确定第一模型的准确度评估信息。比如，第一模型用于预测未来10分钟后的终端性能。则AnLF可将在10分钟前在本地收集的终端数据作为输入，输入到第一模型，该第一模型的输出可认为是第一模型的推理结果，第一模型的推理结果可以为预测的在10分钟后的终端性能。AnLF将第一模型推理出的预测的终端在10分钟后的终端性能，与在网络中采集的终端在10分钟后的真实性能作比较，确定第二准确度评估信息，该第二准确度评估信息可为第一模型准确度的评估值，或者第一模型准确度的评估等级。关于评估值和评估等级可参见前述说明。在该设计中，服务器MTLF可根据N个候选客户端MTLF反馈的N个第一准确度评估信息和AnLF反馈的第二准确度评估信息，确定N个候选客户端MTLF中参与联邦学习的客户端MTLF。例如，N个候选客户端MTLF反馈的N个第一准确度评估信息为N个评估值，AnLF反馈的第二准确度评估信息为评估值，将该评估值称为参考值。则在N个候选客户端MTLF中，参与联邦学习的客户端MTLF中任一个客户端MTLF所反馈的评估值与参考值间的差值小于或等于第四阈值，该第四阈值可以为预设的，或者协议规定的，或者预先通知或配置给服务器MTLF的，不作限制。比如，服务器MTLF可在N个评估值中，确定与参考值间的差值小于或等于第四阈值的评估值；将与参考值间的差值小于或等于第四阈值的评估值对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。或者，N个候选客户端反馈的N个第一准确度评估信息为N个评估等级，AnLF反馈的第二准确度评估信息为评估等级，可称为参考等级。在N个候选客户端MTLF中，参与联邦学习的客户端MTLF所反馈的评估等级与参考等级间的差值小于或等于第五阈值，所述第五阈值为预设的，或者协议规定的，或者预先通知或配置给服务器MTLF的。例如，服务器MTLF在N个评估等级中，确定与参考等级间的差值小于或等于第五阈值的评估等级；将与参考等级间的差值小于或等于第五阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF等。举例来说，3个客户端MTLF反馈的3个评估等级分别为评估等级8、9和5。AnLF反馈的评估等级为参考等级8。评估等级8和评估等级9，与参考等级8间的差值，小于或等于第五阈值(该第五阈值可以为1)。评估等级5与参考等级8间的差值，大于第五阈值。则服务器MTLF可以选择评估等级8和评估等级9对应的候选客户端MTLF参与联邦学习，将评估等级5对应的候选客户端剔除联邦学习组。
在上述设计中，如果客户端MTLF的本地的数据的特征与AnLF本地的数据的特征趋于一致或相似，则客户端MTLF确定的第一模型的第一准确度评估信息和AnLF确定的第一模型的第二准确度评估信息应该相差不大。若两者相差太大，则客户端MTLF的本地的数据特征与AnLF的本地的数据特征有较大差异；通过上述设计，在N个候选客户端MTLF中，剔除掉与AnLF的本地的数据特征差异较大的候选客户端MTLF，从而保证参与联邦学习的客户端MTLF的本地的数据与AnLF的本地的数据的特征相近，提高联邦学习的模型的推理准确度。
可选的，上述第一模型可以为联邦学习过程中的初始模型、中间模型、或最终模型。所述初始模型指联邦学习过程中，服务器MTLF确定的用于模型训练的初始公共模型，或者描述为，在首轮模型训练过程中，服务器MTLF向客户端MTLF发送的初始公共模型； 所述中间模型指在中间轮模型训练过程中，服务器MTLF向客户端MTLF发送的通过上一轮联邦学习的参数汇聚更新后形成的公共模型，它也可以认为是本轮联邦学习训练的初始模型；所述最终模型指在最后一轮模型训练过程后，通过联邦学习获得的模型，该最终模型可认为是服务器MTLF反馈给AnLF的模型。
在一种可能的实现方式中，一轮模型训练的过程，包括：服务器MTLF可在候选客户端MTLF中，确定参与联邦学习的客户端MTLF；服务器MTLF将初始模型分别发送给各个客户端MTLF；各个客户端MTLF利用本地的数据对初始模型进行训练，更新初始模型的参数；各个客户端MTLF将更新的初始模型参数，发送给服务器MTLF；服务器MTLF对各个客户端MTLF反馈的初始模型的参数进行聚合，确定本轮训练获得的模型。本轮训练获得的模型可作为下一轮模型训练的初始模型。本轮参与联邦学习的客户端MTLF作为下一轮模型训练的候选客户端MTLF。例如，在第一轮模型训练过程中，服务器MTLF可通过在NRF中查询，确定参与联邦学习的N个候选客户端。服务器MTLF向N个候选客户端分别发送模型评估请求消息，该模型评估请求消息中包括初始模型。各个候选客户端利用本地的数据作为验证集，验证初始模型的准确性，确定初始模型的准确度评估信息，且反馈给服务器MTLF。服务器MTLF根据该N个候选客户端MTLF反馈的初始模型的准确度评估信息，确定参与联邦学习的客户端MTLF。比如，通过上述评估过程，服务器MTLF确定上述N个候选客户端MTLF中，有M个客户端MTLF可参与联邦学习，所述M为小于或等于N的正整数；则在第一轮模型训练过程中，服务器MTLF向M个客户端MTLF发送初始模型，由M个客户端MTLF对初始模型进行训练等。服务器MTLF对M个客户端MTLF反馈的模型参数进行聚合，确定该第一轮训练获得的模型，第一轮训练的过程完成。在第二轮模型训练过程中，第一轮参与联邦学习的M个客户端MTLF，作为第二轮模型训练的候选客户端。服务器MTLF将第一轮训练得到的模型作为第二轮训练的初始模型发送给M个候选客户端。M个候选客户端利用本地的数据，验证初始模型的准确度评估信息，且反馈给服务器MTLF。服务器MTLF根据M个候选客户端反馈的准确度评估信息，确定参与联邦学习的X个客户端MTLF，所述X为小于或等于M的整数。后续，服务器MTLF向X个客户端发送第二轮训练中的初始模型，X个客户端利用本地的数据，对初始模型进行训练，更新模型的参数，且将更新的模型参数发送给服务器MTLF。服务器MTLF对X个客户端反馈的模型参数进行聚合，得到第二轮训练的模型等。后续其它轮模型训练的过程，与前述相似，不再赘述，直接联邦学习的轮数达到预设值，或者，参与联邦学习的客户端MTLF反馈的准确度评估信息满足目标准确度，则停止联邦学习过程。应当指出，在前述描述中，是以在每轮联邦学习之前，服务器MTLF向各个候选客户端MTLF发送模型评估请求消息，且根据各个客户端MTLF反馈的模型的准确度评估信息，确定在本轮联邦学习过程中，参与联邦学习的客户端MTLF。或者，上述过程也可以描述为，在每轮学习的结束之后，服务器MTLF向客户端MTLF发送模型评估请求消息，该模型评估请求消息中携带有本轮训练获得的模型。客户端MTLF可以利用本地的数据，验证本地训练获得的模型的准确度评估信息，且反馈给服务器MTLF。由服务器MTLF根据各个客户端MTLF反馈的模型的准确度评估信息，确定是否继续联邦学习。例如，如果各个客户端MTLF反馈的模型的准确性评估信息都满足目标准确度的要求，则停止联邦学习，否则继续下一轮联邦学习。也就是说，在本申请中，在一轮模型训练结束后，服务器MTLF向客户端MTLF发送模型评估请求消息，该模型评估请求消息中携带有本轮训练获取的模 型，客户端MTLF利用本地的数据，确定本轮训练模型的准确度评估信息，且反馈给服务器MTLF。服务器MTLF根据各个客户端MTLF反馈的准确度评估信息，可以执行以下操作：第一，判断是否需要继续执行联邦学习。第二，如果确定继续执行下一轮联邦学习，则可以根据各个客户端反馈的准确度评估信息，确定参与下一轮联邦学习的客户端MTLF等。上述目标准确度可以为预设的，或者协议规定的，或者预先通知或配置给服务器MTLF的等，不作限制。例如，在一种实现方式中，AnLF向服务器MTLF发送模型请求消息，该模型请求消息中包括上述目标准确度。服务器MTLF在接收到上述模型请求消息时，可按照上述方法进行联邦学习过程中的模型训练；服务器MTLF在训练完模型时，向AnLF发送模型请求响应消息，所述模型请求响应消息中包括联邦学习获得的模型。可选的，在AnLF向服务器MTLF发送模型请求消息之前，还包括：AnLF接收来自用户的分析请求，该分析请求中至少包括以下一项：分析标识、分析过滤信息、或该分析需要满足的目标准确度。所述分析标识可用于标识分析任务，分析过滤信息用于指示在联邦学习过程中训练数据需要满足的条件，或者描述为分析过滤信息用于确定模型过滤信息，该模型过滤信息用于指于在联邦学习过程中训练数据需要满足的条件等。所述分析需要满足的目标准确度，可用于确定模型需要满足的目标准确度，该目标准确度可以为目标准确值，或者目标准确等级等，不作限制。AnLF在确定所述AnLF的本地模型不能满足分析请求消息的要求，且NRF中不能查询到满足该分析请求消息要求的MTLF时，可以在NRF中查询具有服务器角色的MTLF，作为服务器MTLF，AnLF向该服务器MTLF发送模型请求消息，执行前述方法。在AnlF在接收到服务器反馈的联邦学习的模型时，可根据该模型执行上述分析任务，且将分析结果反馈给用户。更详细的过程，可参见图4中的说明。
在一种设计中，在每轮联邦学习过程中，服务器MTLF在接收到各个候选客户端MTLF反馈的准确度评估信息时，可确定各个候选客户端MTLF反馈的准确度评估信息是否满足目标准确度的要求；可选的，候选客户端反馈的准确度评估信息可称为第三准确度评估信息，该第三准确度评估信息与前述第一准确度评估信息相同，或不同，不作限制。如果满足，则结束联邦学习；否则，将根据各个候选客户端MTLF反馈的模型更新参数，可称为第一模型的更新参数，确定第二模型。该第二模型可以是联邦学习过程中的中间模型。在下一轮模型训练中，服务器MTLF将第二模型发送给候选客户端MTLF，执行下一轮联邦学习。
针对候选客户端向服务器MTLF反馈第一准确度评估信息，AnLF向服务器MLTF反馈第二准确度评估信息，服务器MTLF根据第一准确度评估信息和第二准确度评估信息，确定参与联邦学习的客户端MTLF的方案，如图4所示，提供一种数据分析方法的流程，至少包括：
步骤400：各个MTLF向NRF注册自身的信息。
例如，各个MTLF可将自身在联邦学习过程中支持的服务器角色或客户端角色注册到NRF中。可选的，各个MTLF还可以将自身能够提供的模型对应的分析标识(analytics ID)，分析过滤信息(analytics filter information)以及MTLF自身能够提供的模型的准确度评估信息也注册到NRF中。
步骤401：分析服务的消费者AC向AnLF发送分析请求订阅消息或分析信息请求消息，其中携带以下至少一项：分析标识(analytics ID)、分析过滤信息、或希望达到的分析准确水平(preferred level of accuracy of analytics)。
其中，分析服务的消费者AC可以为网络网元、应用功能或OAM等，不作限制。分析标识用于标识分析任务；分析过滤信息可以指示分析的对象或分析输出的范围等。例如，分析过滤信息可以是由网络切片标识指定的用于车联网服务的网络切片。其中，希望达到的分析准确性水平可以是准确率等数据，或者可以是准确等级，例如，准确等级可以为高、中、低。例如，车联网服务器向AnLF请求未来10分钟内某个地点的车联网服务的网络切片的服务质量预测，分析标识可指示服务质量预测服务，分析过滤信息指示针对车联网服务的网络切片进行预测，希望达到的分析准确水平为高。
步骤402：AnLF根据步骤401中接收的消息，确定以下至少一项：分析标识、模型过滤信息(model filer information)或模型需要达到的目标准确度。
其中，AnLF可根据步骤401的分析过滤信息，确定模型过滤信息，所述模型过滤信息可指示联邦学习过程中训练数据需要满足的条件。或者，上述步骤401中可不携带分析过滤信息，AnLF可通过其它方式，确定模型过滤信息，不作限制。在后续模型训练和联邦学习的过程中，可使用满足模型过滤信息条件的数据进行模型训练。所述模型过滤信息中包括的参数类型和分析过滤信息中包括的参数类型相同。可选的，模型过滤信息中包括的每类型参数的参数值，与分析过滤信息中包括的每类型参数的参数值，也可以相同。也就是说，模型过滤信息与分析过滤信息两者所包括的内容可相同。例如，所述模型过滤信息可以为特定网络区域，或者网络切片标识，例如单个网络切片选择辅助信息(single-network slice selection assistance information，S-NSSAI)、指定的特定的网络切片、或者由应用标识(application ID)指定的特定应用业务等。举例来说，车联网服务器向AnLF发送的分析请求消息中携带的分析标识为服务质量预测服务，分析过滤信息指示针对用于车联网服务的网络切片进行预测，则AnLF可以确定对上述分析请求提供推理服务的模型需要满足的条件为分析标识为服务质量预测服务，模型过滤信息为车联网服务切片。
AnLF可根据步骤401中的分析准确水平，确定模型的目标准确度。所述模型的目标准确度可以为模型的目标准确值、或模型的目标准确等级等。前已述，上述希望达到的分析准确性水平可以准确率等数据，可将希望达到的分析的准确度等转换为模型希望达到的目标准确值等。例如，当模型为计算分类的模型时，模型的目标准确值可以为正确率、错误率、精度或召回率；或者，当模型为计算回归的模型时，模型的目标准确值可以为平均绝对误差、平均绝对百分比误差、或均方误差等。进一步，可将模型的目标准确值，转换为模型的目标准确等级。例如，根据目标准确值与目标准确等级的对应关系，确定目标准确值对应的目标准确等级。也就是说，本申请中的模型需要达到的目标准确度可以为目标准确值，也可以为目标准确等级。该模型需要达到的目标准确度反映了对联邦学习训练的模型的准确水平的要求。例如，可采用以下方式验证模型的准确度，将验证数据输入到一个模型中，将该模型的输出与该验证数据对应的准确输出相对比，确定该模型的准确度等。
步骤403：AnLF判断AnLF本地的模型是否满足前述步骤402中的分析标识、模型过滤信息和目标准确度的要求，如果AnLF的本地模型能满足要求，则AnLF使用本地的模型进行模型推理，将推理结果作为上述分析任务的结果反馈给分析的消费者AC。如果AnLF的本地模型不能满足要求，则AnLF在NRF中查询能满足上述分析标识、模型过滤信息和目标准确度要求的MTLF；如果能查询到满足要求的MTLF，则NRF向AnLF反馈满足要求的MTLF的访问地址；AnLF根据该MTLF的访问地址，向MTLF发送模型请求消息，MTLF可以向AnLF返回模型请求响应消息，该模型请求响应消息中包括MTLF反 馈的满足要求的模型；AnLF利用该反馈的满足要求的模型进行模型推理，将推理结果将为上述分析任务的结果反馈给AC。或者，如果在NRF也不能查询到满足要求的MTLF，则AnLF可以在NRF中，查询在前述步骤400中注册为服务器角色的MTLF，且将注册为服务器角色的MTLF的访问地址返回AnLF。AnLF执行后续步骤404，向注册为服务器角色的MTLF发送模型请求消息。上述过程中，可认为是AnLF发现MTLF的过程。
举例来说，AnLF根据AC发送的请求，所确定的分析标识为服务质量预测，模型过滤信息为车联网服务的网络预测，目标准确度为高。则AnLF可确定本地的模型是否能满足上述条件；如果满足，则利用本地的模型进行模型推理，将推理结果作为上述分析的结果。或者，如果本地的模型不能满足上述条件，则AnLF可在NRF中查询满足上述条件的MTLF。比如，如果在前述步骤400中，有一个MTLF向NRF注册的分析标识为服务质量预测、分析过滤信息为车联网服务的网络预测，该模型的准确度评估信息为高，则该MTLF可认为是满足条件的MTLF，NRF将满足条件的MTLF的访问地址返回AnLF，由AnLF利用该满足条件的MTLF提供的模型进行模型推理。或者，如果在NRF中也不能查询到满足条件的MTLF，则AnLF在NRF中查询在步骤400中，注册角色为服务器的MTLF，且将该MTLF的访问地址返回AnLF。
步骤404：AnLF向服务器MTLF发送模型请求消息，该模型请求消息中包括分析标识、模型过滤信息和模型需要满足的目标准确度等。
步骤405：服务器MTLF向AnLF发送第二模型评估请求消息，该第二模型评估请求消息中携带有初始模型。
步骤406：AnLF利用本地的数据，确定初始模型的第二准确度评估信息；AnLF向服务器发送第二模型评估请求响应消息，该第二模型评估请求响应消息中包括初始模型的第二准确度评估信息。
该第二准确度评估信息可以为初始模型的准确度评估值。例如，AnLF将本地收集的数据输入到初始模型中，将初始模型的输出与该本地的数据的标签作对比，确定初始模型的准确度评估值。或者，该第二准确度评估信息可以为初始模型的准确度评估等级，AnLF可根据初始模型的准确度评估值，进一步确定初始模型的准确度评估等级。
步骤407：AnLF通过NRF确定N个参与联邦学习的候选客户端MTLF，所述N为大于1的整数；AnLF分别向N个候选客户端MTLF发送第一模型评估请求消息，该第一模型评估请求消息中包括初始模型。
步骤408：N个候选客户端MTLF中的每个候选客户端MTLF，使用本地收集的数据，确定初始模型的第一准确度评估信息。该过程与AnLF确定第二准确度评估信息的过程相似，可参见前述说明。N个候选客户端MTLF中的每个候选客户端MTLF向AnLF发送第一模型评估请求响应消息，该第一模型评估请求响应消息中包括初始模型的第一准确度评估信息。
步骤409：服务器MTLF根据N个第一准确度评估信息和第二准确度评估信息，确定N个候选客户端MTLF中，参与联邦学习的客户端MTLF。该过程可描述为：服务器MTLF修改包括N个客户端MTLF的联邦学习组，并使用与AnLF的数据分布一致的客户端MTLF参与联邦学习。
例如，服务器MTLF可以在N个第一准确度评估信息中，选择与第二准确度评估等级一致或与第二准确度评估值相接近的第一准确度评估信息，将与第二准确度评估等级相一 致或与第二准确度评估值相接近的第一准确度评估信息对应的客户端MTLF，作为参与下一轮联邦学习的客户端MTLF。具体的过程，可参见前述图3中的说明。
步骤4010：服务器MTLF组织确定的客户端MTLF，使用本地的数据进行联邦学习。
在第一轮联邦学习过程中，N个候选客户端利用本地收集的数据，对服务器MTLF提供的初始模型进行模型训练，更新初始模型的参数。N个候选客户端MTLF将初始模型的更新参数发送给服务器MTLF，服务器MTLF对N个候选客户端MTLF反馈的初始模型的更新参数进行聚合，确定第一中间模型。按照前述步骤405至步骤409中的说明，服务器MTLF在N个候选客户端MTLF中，确定参与第二轮联邦学习的客户端MTLF。在第二轮联邦学习的过程中，服务器MTLF向确定的参与第二轮联邦学习的客户端MTLF发送第一中间模型，各个客户端MTLF根据本地的收集数据，对第一中间模型进行模型训练，更新第一中间模型的参数。服务器MTLF根据各个客户端MTLF反馈的第一中间模型的更新参数中，确定第二中间模型。按照前述步骤405至步骤409中的说明，服务器MTLF在参与第二轮联邦学习的客户端MTLF中，选择参与第三轮联邦学习的客户端MTLF。在第三轮联邦学习的过程中，服务器MTLF将第二中间模型发送给参与第三轮联邦学习的客户端MTLF，循环执行，直至服务器MTLF确定的某个模型的准确度评估信息能满足目标准确度为止，则停止联邦学习。通过上述多轮的筛选过程，服务器MTLF能够将本地的数据分布与AnLF的本地的数据分布不同的客户端MTLF在联邦学习组中剔除掉，从而使得联邦学习得到的模型在用于网络分析时具有更高的准确性。
步骤4011：在上述联邦学习过程中，经过多轮联邦学习更新后，如果各个客户端MTLF反馈的模型准确度评估信息能满足目标准确度水平，则服务器MTLF停止进行下一轮联邦学习。服务器MTLF向AnLF发送模型请求响应消息，该模型请求响应消息中携带有联邦学习得到的最终模型。
步骤4012：AnLF在获得该最终模型时，利用最终模型进行分析，且向分析服务的请求者AC发送分析订阅通知消息，或者分析信息响应消息，该消息中携带有分析结果。
例如，AnLF在接收到该最终模型时，可根据模型过滤信息在本地收集数据，将收集的本地的数据作为输入，输入到最终模型中，该最终模型的输出为分析结果。
通过上述方法，服务器MTLF可根据AnLF反馈的模型的准确度评估信息和参与上一轮联邦学习的客户端MTLF反馈的模型的准确度评估信息，筛选出参与下一轮联邦学习的客户端MTLF，从而避免客户端MTLF因为本地的数据的数据特征与AnLF的本地的数据的数据特征不同，而导致AnLF在应用联邦学习的模型时，其推理准确度不能满足要求。
本申请还提供一种数据分析方法，该方法包括：服务器MTLF接收来自AnLF的模型请求消息，该模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述分析标识用于标识分析任务，模型过滤信息用于指示联邦学习过程中训练数据需要满足的条件，所述模型需要满足的目标准确度用于指示在联邦学习过程中训练的用于推理当前分析任务的模型需要满足的准确度；服务器MTLF根据该模型请求消息，确定执行或不执行联邦学习。可选的，在服务器MTLF确定不执行联邦学习时，向AnLF发送模型请求响应消息，该模型请求响应消息中包括联邦学习失败的原因，和/或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式确定当前分析任务的结果。在服务器MTLF确定执行联邦学习时，在每轮联邦学习结束时，服务器MTLF 可以向参与本轮联邦学习的客户端MTLF发送第二模型评估请求消息，该第二模型评估请求消息中携带有本轮训练获得的模型，第二模型评估请求消息用于请求客户端MTLF上报本轮训练获得的模型的准确度评估信息，接收来自客户端MTLF的第二模型评估请求响应消息，该第二模型评估请求响应消息中包括所述客户端MTLF上报的本轮训练获得的模型的准确度评估信息。在客户端MTLF反馈的模型的准确度评估信息满足目标准确度时，则停止联邦学习。可选的，所述目标准确度可以是预设的，或者预先配置或者通知服务器MTLF的，或者，服务器MTLF在AnLF发送的模型请求消息中获取的等，不作限制。
如图5所示，提供一种数据分析方法的流程，至少包括：
步骤500：各个MTLF向NRF注册自身的信息。
步骤501：分析服务的消费者AC向AnLF发送分析请求订阅消息或分析信息请求消息，其中携带以下至少一项：分析标识、分析过滤信息、或希望达到的分析准确水平。
步骤502：AnLF向服务器MTLF发送模型请求消息，该模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要达到的目标准确度。模型过滤信息是根据分析过滤信息确定的，模型需要达到的目标准确度是根据希望达到的分析准确水平所确定的。
可选的，AnLF可首先确定AnLF的本地模型能否满足上述分析标识、模型过滤信息和目标准确度的要求；如果不能满足，再在NRF中查询是否存在满足上要求的MTLF；如果在NRF中也查询不到满足上述要求的MTLF，则AnLF在NRF中查询能提供服务器角色的MTLF，且向该能提供服务器角色的MTLF发送模型请求消息。也就是说，AnLF如果能够匹配到分析标识和模型过滤信息，并且满足目标准确度的模型，则确定不执行联邦学习；否则，确定执行联邦学习，通过联邦学习过程训练出满足目标准确度的用于推理当前任务的模型。
步骤503：服务器MTLF在接收到AnLF的模型请求消息时，可确定是否需要执行联邦学习。若执行联邦学习，则继续执行后续步骤504。
在一种设计中，服务器MTLF可以根据模型请求消息中携带的分析标识和/或模型过滤信息等，确定是否执行联邦学习。例如，服务器MTLF可根据分析标识和/或模型过滤信息，确定联邦学习过程中训练数据的特征；若该训练数据的特征与地理位置无关时，例如该训练数据与应用业务相关，则服务器MTLF确定适合进行联邦学习；或者，若该训练数据的特征与地理位置相关时，则服务器MTLF确定不适合进行联邦学习。
当分析标识和/或模型过滤信息为以下任一项时，可确定不适合进行联邦学习：
(1)分析标识为数据网络性能分析(DN performance analytics)；
(2)分析标识为冗余传输经验相关分析(redundant transmission experience related analytics)；
(3)分析标识为会话管理拥塞控制体验(session management congestion control Experience)；
(4)分析标识为分散分析(dispersion analytics)，模型过滤信息为指定网络切片标识。
(5)分析标识为观察到的服务体验相关的网络数据分析(observed service experience related network data analytics)；
(6)分析标识为切片负载级别相关的网络数据分析(slice load level related network data analytics)。
当分析标识和/或模型过滤信息为以下任一项时，可认为训练模型的网络数据与地理位置相关，那么客户端MTLF的本地的数据分布不同，确定不适合进行联邦学习：
(1)分析标识为WLAN性能分析(performance analytics)；
(2)分析标识为分散分析(dispersion analytics)，模型过滤信息为包含指定网络位置；
(3)分析标识为服务质量可持续性分析(QoS sustainability analytics)；
(4)分析标识为用户数据拥塞分析(user data congestion analytics)；
(5)分析标识为用户移动性分析(UE mobility analytics)；
(6)分析标识为用户通信分析(UE communication analytics)；
(7)分析标识为预设的UE行为参数(expected UE behavioural parameters)；
(8)分析标识为异常行为相关网络数据分析(abnormal behaviour related network data analytics)，模型过滤信息为意外的UE位置(unexpected UE location)、意外的长寿命/大速率流量(unexpected long-live/large rate flows)，意外的无线电链路故障(unexpected radio link failures)或跨相邻小区乒乓效应(Ping-ponging across neighbouring cells)。
(9)分析标识为网络性能分析(network performance analytics)，模型过滤信息为感兴趣的网络区域或特定gNB；
(10)分析标识为NF负载分析(NF load analytics)，模型过滤信息为指定NF或特定区域的NF。
在另一种设计中，服务器MTLF可通过NRF查询注册为客户端角色的MTLF，该查询得到的MTLF可称为客户端MTLF，查询得到的客户端MTLF可组成联邦学习组。服务器MTLF可向联邦学习组中的客户端MTLF发送第一模型评估请求消息，该第一模型评估请求消息用于请求客户端MTLF上报本地模型的准确度评估信息。可选的，第一模型评估请求消息中可包括分析标识和模型过滤信息。客户端MTLF可根据第一模型评估请求消息中携带的分析标识和模型过滤信息，在客户端MTLF本地确定唯一的模型。客户端MTLF可使用本地的训练数据作为验证集，验证本地模型的准确度评估信息。可选的，客户端MTLF可使用满足模型过滤信息条件的本地训练数据作为验证集。例如，当分析标识为服务质量预设，模型过滤信息为车联网服务的网络切片时，客户端MTLF可使用车联网服务的网络切片的本地的训练数据集作为验证集，确定本地模型的准确度评估信息。客户端MTLF可以向服务器MTLF发送第一模型评估请求响应消息，该消息中携带有确定的本地模型的准确度评估信息。该准确度评估信息可以为准确度评估值，或准确度评估等级等，具体可参见前述说明。服务器MTLF可根据各个客户端MTLF反馈的本地模型的准确度评估信息，确定是否需要执行联邦学习。若各个客户端MTLF反馈的本地模型的准确度评估信息，都满足上述步骤502中的目标准确度，则确定不需要执行联邦学习。服务器MTLF可以向AnLF发送模型请求响应消息，该模型请求响应消息中携带联邦学习失败的原因，和/或分析聚合指示，该分析聚合指示用于指示AnLF使用分析聚合方式进行分析。例如，所述分析聚合指示AnLF直接向各个客户端MTLF的NWDAF转发步骤501中的分析请求订阅消息或分析信息请求消息，各个NWDAF在各自的服务区域内给出推理预测结果。应该指出，具体由各个NWDAF中的AnLF执行具体的推理过程。AnLF将各个NWDAF的推理预测结果进行汇总即可。例如，车联网服务器请求一条路径上各个位置为车联网服务的网络切换服务质量预测时，AnLF可将服务区域内能够覆盖该路径上的各个NWDAF的推理结果进行聚合汇总，作为最终的分析结果提供给车联网服务器。上述根据各个客户端 反馈的本地模型的准确度评估信息，确定是否执行联邦学习的过程，可认为根据准确度评估信息，确定各个客户端的本地的数据的分布是否一致或相似的过程。在图5的流程中，着重描述根据客户端MTLF反馈的本地模型的准确度评估信息，确定是否执行联邦学习的过程。
可采用上述第一种设计，确定是否需要执行联邦学习，或者可采用上述第二种设计，确定是否需要执行联邦学习。或者，可结合上述第一种设计和第二种设计，同时确定是否需要执行联邦学习。比如，可先采用上述第一种设计，确定是否执行联邦学习。如果确定不执行联邦学习，则向AnLF发送模型请求响应消息，该模型请求响应消息中携带有联邦学习失败的原因，该原因可以为对当前模型训练的网络数据与地理位置相关，不适当进行联邦学习。或者，确定执行联邦学习，可再采用上述第二种设计，判断是否执行联邦学习。如果执行联邦学习，则继续执行本申请的后续步骤。如果确定不执行联邦学习，则向AnLF发送模型请求响应消息，该模型请求响应消息中携带有分析聚合指示等。
步骤504：服务器MTLF利用客户端MLF进行联邦学习。
该联邦学习的过程可以是多轮的。在每轮联邦学习的过程中，服务器MTLF向客户端MTLF发送本轮的初始模型，客户端MTLF使用本地收集的数据，对初始模型进行训练，更新初始模型的参数，且将初始模型的更新参数发送给服务器MTLF。服务器MTLF根据各个客户端反馈的初始模型的更新参数，对初始模型进行更新，确定本轮训练的模型，该本轮训练的模型可作为下一轮模型训练的初始模型。可选的，在每轮模型训练的结束时，服务器MTLF可向各个客户端MTLF发送第二模型评估请求消息，该第二模型评估请求消息中携带有本轮训练的模型，用于请求客户端MTLF上报本轮模型训练的模型的准确度评估信息。客户端MTLF在接收到本轮训练的模型时，根据本地收集的数据验证该模型的准确性，且向服务器MTLF上报本轮训练模型的准确度评估信息。例如，客户端MTLF可向服务器MTLF发送第二模型评估请求响应消息，该第二模型评估请求响应消息中携带有本轮训练模型的准确度评估信息。该准确度评估信息可以为本轮训练模型的准确度评估值，或者本轮训练模型的准确度评估等级，不作限制。服务器MTLF在接收到各个客户端上报本轮训练模型的准确度评估信息时，可判断各个客户端上报的本轮模型的准确度评估信息是否满足上述步骤502中的目标准确度。如果满足，则停止联邦学习；AnLF向服务器MTLF发送模型请求响应消息，该模型请求响应消息中携带有联邦学习得到的模型。或者，如果各个客户端上报的本轮模型的准确度评估信息不满足上述步骤502中的目标准确度，则继续下一轮模型训练。
步骤505：服务器MTLF向AnLF发送模型请求响应消息，该模型请求响应消息中携带有联邦学习得到的模型、联邦学习失败的原因、或指示AnLF进行聚合分析等。
步骤506：AnLF进行网络分析，向分析服务的消息者AC发送分析订阅通知消息，或分析信息响应消息，该消息中携带有分析结果。
例如，若上述步骤505中的模型请求响应消息中携带有联邦学习得到的模型，则AnLF可根据该联邦学习得到的模型进行模型推理，将推理结果作为分析结果，反馈给AnLF。或者，若上述步骤505中的模型请求响应消息中携带有指示AnLF进行聚合分析，则AnLF进行聚合分析，将聚合分析的结果作为分析结果，反馈给AC，具体聚合分析的过程可参见前述说明。
通过上述设计，服务器MTLF可根据本地模型或每轮联邦学习得到的模型，确定是否 进行后续联邦学习，可以避免不必要的联邦学习，节约了网络资源，同时满足了分析服务的模型的准确度要求。
可以理解的是，为了实现上述实施例的功能，服务器MTLF、客户端MTLF和AnLF等包括了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本申请中所公开的实施例描述的各示例的单元及方法步骤，本申请能够以硬件或硬件和计算机软件相结合的形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用场景和设计约束条件。
图6和图7为本申请的实施例提供的可能的通信装置的结构示意图。这些通信装置可以用于实现上述方法实施例中服务器MTLF、客户端MTLF或AnLF的功能，因此也能实现上述方法实施例所具备的有益效果。
如图6所示，通信装置600包括处理单元610和收发单元620。通信装置600用于实现上述图3、图4、或图5中所示的方法实施例中服务器MTLF、客户端MTLF或AnLF的功能。
当通信装置600用于实现图3或图4所示的方法实施例中的服务器MTLF的功能时：收发单元620用于向N个候选客户端模型训练逻辑功能MTLF分别发送第一模型；分别接收来自所述N个候选客户端MTLF的第一准确度评估信息，所述第一准确度评估信息表示所述候选客户端MTLF使用本地的数据确定的所述第一模型的准确度，所述N为大于1的正整数；处理单元610用于根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF。
当通信装置600用于实现图3或图4所示的方法实施例中客户端MTLF或AnLF的功能时：收发单元620用于接收来自服务器模型训练逻辑功能MTLF的模型评估请求消息，所述模型评估请求消息中包括第一模型；处理单元610用于利用本地的数据，确定所述第一模型的准确度评估信息，所述准确度评估信息为第一准确度评估信息和第二准确度评估信息；收发单元620还用于向所述服务器MTLF发送模型评估请求响应消息，所述模型请求响应消息中包括所述第一模型的准确度评估信息。
当通信装置600用于实现图5所示的方法实施例中的服务器MTLF的功能时：收发单元620用于接收来自分析推理功能AnLF的模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述分析标识用于标识分析任务，所述模型过滤信息用于指示联邦学习过程中训练数据需要满足的条件，所述模型需要满足的目标准确度用于指示推理当前分析任务的模型需要满足的准确度；处理单元610用于根据所述模型请求消息，确定执行或不执行联邦学习。
当通信装置600用于实现图5所示的方法实施例中的AnLF的功能时：收发单元620用于接收来自用户的分析请求消息，所述分析请求消息中包括分析标识和该分析希望达到的准确度；处理单元610用于确定分析推理功能AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的模型训练逻辑功能MTLF；收发单元620，还用于向服务器MTLF发送模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述模型需要满足的目标准确度是根据所述分析希望达到的分析准确度确定的，所述模型过滤信息用于指示在模型训练过程中训练数据需要满足的条件。
有关处理单元610和收发单元620更详细的描述可以直接参考图3、图4或图5所示 的方法实施例中相关描述直接得到，这里不加赘述。
如图7所示，通信装置700包括处理器710和接口电路720。处理器710和接口电路720之间相互耦合。可以理解的是，接口电路720可以为收发器或输入输出接口。可选的，通信装置700还可以包括存储器730，用于存储处理器710执行的指令或存储处理器710运行指令所需要的输入数据或存储处理器710运行指令后产生的数据。
当通信装置700用于实现图3、图4或图5所示的方法时，处理器710用于实现上述处理单元610的功能，接口电路720用于实现上述收发单元620的功能。
当上述通信装置为应用于服务器MTLF的模块时，该服务器MTLF模块实现上述方法实施例中服务器MTLF的功能。该服务器MTLF模块从服务器MTLF中的其它模块(如射频模块或天线)接收信息，该信息是客户端MTLF或AnLF发送给服务器MTLF的；或者，该服务器MTLF模块向服务器MTLF中的其它模块(如射频模块或天线)发送信息，该信息是服务器MTLF发送给客户端MTLF或AnLF的。
当上述通信装置为应用于客户端MTLF的模块时，该客户端MTLF模块实现上述方法实施例中客户端MTLF的功能。该客户端MTLF模块从客户端MTLF中的其它模块(如射频模块或天线)接收信息，该信息是服务器MTLF发送给客户端MTLF的；或者，该客户端MTLF模块向客户端MTLF中的其它模块(如射频模块或天线)发送信息，该信息是客户端MTLF发送给服务器MTLF。
当上述通信装置为应用于AnLF的模块时，该AnLF模块实现上述方法实施例中AnLF的功能。该AnLF模块从AnLF中的其它模块(如射频模块或天线)接收信息，该信息是服务器MTLF发送给AnLF的；或者，该AnLF模块向AnLF中的其它模块(如射频模块或天线)发送信息，该信息是AnLF发送给服务器MTLF。
如图8所示，本申请还提供一种通信系统800，该系统中包括前述服务器MTLF对应的装置810和客户端MTLF对应的装置820。可选的，该通信系统还包括：AnLF对应的装置830。
可以理解的是，本申请的实施例中的处理器可以是中央处理单元(central processing unit，CPU)，还可以是其它通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array，FPGA)或者其它可编程逻辑器件、晶体管逻辑器件，硬件部件或者其任意组合。通用处理器可以是微处理器，也可以是任何常规的处理器。
本申请的实施例中的方法步骤可以通过硬件的方式来实现，也可以由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器、闪存、只读存储器、可编程只读存储器、可擦除可编程只读存储器、电可擦除可编程只读存储器、寄存器、硬盘、移动硬盘、CD-ROM或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于基站或终端中。当然，处理器和存储介质也可以作为分立组件存在于基站或终端中。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机程序或指令。在计算机上加载和执行所述计算机程序或指令时， 全部或部分地执行本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、网络设备、用户设备或者其它可编程装置。所述计算机程序或指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机程序或指令可以从一个网站站点、计算机、服务器或数据中心通过有线或无线方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是集成一个或多个可用介质的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，例如，软盘、硬盘、磁带；也可以是光介质，例如，数字视频光盘；还可以是半导体介质，例如，固态硬盘。该计算机可读存储介质可以是易失性或非易失性存储介质，或可包括易失性和非易失性两种类型的存储介质。
在本申请的各个实施例中，如果没有特殊说明以及逻辑冲突，不同的实施例之间的术语和/或描述具有一致性、且可以相互引用，不同的实施例中的技术特征根据其内在的逻辑关系可以组合形成新的实施例。
本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A,B可以是单数或者复数。在本申请的文字描述中，字符“/”，一般表示前后关联对象是一种“或”的关系；在本申请的公式中，字符“/”，表示前后关联对象是一种“相除”的关系。“包括A，B和C中的至少一个”可以表示：包括A；包括B；包括C；包括A和B；包括A和C；包括B和C；包括A、B和C。
可以理解的是，在本申请的实施例中涉及的各种数字编号仅为描述方便进行的区分，并不用来限制本申请的实施例的范围。上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定。

Claims (41)

  1. 一种数据分析方法，其特征在于，包括：
    向N个候选客户端模型训练逻辑功能MTLF分别发送第一模型；
    分别接收来自所述N个候选客户端MTLF的第一准确度评估信息，所述第一准确度评估信息表示候选客户端MTLF使用本地的数据确定的所述第一模型的准确度，所述N为大于1的正整数；
    根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF。
  2. 如权利要求1所述的方法，其特征在于，所述向N个候选客户端MTLF分别发送第一模型，包括：向所述N个候选客户端MTLF分别发送第一模型评估请求消息，所述第一模型评估请求消息中包括所述第一模型。
  3. 如权利要求1或2所述的方法，其特征在于，所述分别接收来自所述N个候选客户端MTLF的第一准确度评估信息，包括：分别接收来自所述N个候选客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述第一准确度评估信息。
  4. 如权利要求1至3中任一项所述的方法，其特征在于，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估值，所述参与联邦学习的客户端MTLF中任两个候选客户端MTLF反馈的评估值间的差值小于或等于第一阈值。
  5. 如权利要求4所述的方法，其特征在于，根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF，包括：
    在所述N个评估值中，确定取值最大的评估值与取值最小的评估值；
    所述取值最大的评估值与取值最小的评估值之差小于或等于所述第一阈值，所述N个候选客户端均参与联邦学习；或者，
    所述取值最大的评估值与取值最小的评估值之差大于所述第一阈值，确定所述N个评估值的平均值；确定所述N个评估值中每个评估值与所述平均值的差的绝对值；在所述N个候选客户端组成的联邦学习组中，剔除与所述平均值的差的绝对值最大的评估值对应的候选户端MTLF，组成新的联邦学习组；继续在所述新的联邦学习组中，确定取值最大的评估值与取值最小的评估值，且确定两个评估值之差与所述第一阈值的大小关系。
  6. 如权利要求1至3中任一项所述的方法，其特征在于，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级，所述参与联邦学习的客户端MTLF反馈的评估等级满足目标评估等级。
  7. 如权利要求6所述的方法，其特征在于，根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF，包括：
    在所述N个评估等级中，确定与所述目标评估等级间的差值小于或等于第三阈值的评估等级；
    将与所述目标评估等级间的差值小于或等于第三阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
  8. 如权利要求1至3中任一项所述的方法，其特征在于，还包括：
    向分析推理功能AnLF发送第一模型；
    接收来自所述AnLF的第二准确度评估信息，所述第二准确度评估信息表示所述AnLF使用本地的数据确定的所述第一模型的准确度。
  9. 如权利要求8所述的方法，其特征在于，所述向AnLF发送第一模型，包括：向所述AnLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括所述第一模型。
  10. 如权利要求8或9所述的方法，其特征在于，所述接收来自所述AnLF的第二准确度评估信息，包括：接收来自所述AnLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述第二准确度评估信息。
  11. 如权利要求8至10中任一项所述的方法，其特征在于，根据N个所述第一准确度评估信息，确定所述N个候选客户端MTLF中参与联邦学习的客户端MTLF，包括：
    根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF。
  12. 如权利要求8至11中任一项所述的方法，其特征在于，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考值，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估值，所述参与联邦学习的客户端MTLF中任一个客户端MTLF反馈的评估值与所述参考值间的差值小于或等于第四阈值。
  13. 如权利要求12所述的方法，其特征在于，根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF，包括：
    在N个评估值中，确定与所述参考值间的差值小于或等于第四阈值的评估值；
    将与所述参考值间的差值小于或等于第四阈值的评估值对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
  14. 如权利要求8至11中任一项所述的方法，其特征在于，所述第二准确度评估信息为所述AnLF反馈的所述第一模型准确度的参考等级，所述第一准确度评估信息为所述候选客户端MTLF反馈的所述第一模型准确度的评估等级，所述参与联邦学习的客户端MTLF所反馈的评估等级与所述参考等级间的等级间的差值小于或等于第五阈值。
  15. 如权利要求11或14所述的方法，其特征在于，根据N个所述第一准确度评估信息和所述第二准确度评估信息，确定所述N个候选客户端中参与联邦学习的客户端MTLF，包括：
    在N个评估等级中，确定与所述参考等级间的差值小于或等于第五阈值的评估等级；
    将与所述参考等级间的差值小于或等于第五阈值的评估等级对应的候选客户端MTLF，作为参与联邦学习的客户端MTLF。
  16. 如权利要求4、5、12或13所述的方法，其特征在于，所述评估值，或参考值包括以下至少一项：正确率、错误率、精度率、召回率、平均绝对误差、平均绝对百分比误差、或均方误差。
  17. 如权利要求1至16中任一项所述的方法，其特征在于，所述第一模型为联邦学习过程中的初始模型、中间模型、或最终模型。
  18. 如权利要求1至17中任一项所述的方法，其特征在于，还包括：
    接收来自所述AnLF的模型请求消息，所述模型请求消息中至少包括模型需要满足的目标准确度；
    参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息满足所述目标准确度的要求时，结束联邦学习；或者，
    参与联邦学习的客户端MTLF反馈的所述第一模型的第三准确度评估信息不满足所述目标准确度的要求，根据所述参与联邦学习的客户端MTLF反馈的模型参数，确定第二模型。
  19. 一种数据分析方法，其特征在于，包括：
    接收来自服务器模型训练逻辑功能MTLF的第一模型；
    利用本地的数据，确定所述第一模型的准确度评估信息；
    向所述服务器MTLF发送所述第一模型的准确度评估信息。
  20. 如权利要求19所述的方法，其特征在于，所述准确度评估信息为第一准确度评估信息，所述第一准确度评估信息表示候选客户端MTLF使用本地的数据确定的所述第一模型的准确度。
  21. 如权利要求19所述的方法，其特征在于，所述准确度评估信息为第二准确度评估信息，所述第二准确度评估信息表示分析推理功能AnLF使用本地的数据确定的所述第一模型的准确度。
  22. 如权利要求19至21中任一项所述的方法，其特征在于，所述接收来自服务器模型训练逻辑功能MTLF的第一模型，包括：接收来自服务器模型训练逻辑功能MTLF的模型评估请求消息，所述模型评估请求消息中包括第一模型；
    所述向所述服务器MTLF发送所述第一模型的准确度评估信息，包括：向所述服务器MTLF发送模型评估请求响应消息，所述模型请求响应消息中包括所述第一模型的准确度评估信息。
  23. 如权利要求19至22中任一项所述的方法，其特征在于，所述第一模型的准确度评估信息为所述第一模型准确度的评估值，所述利用本地的数据，确定所述第一模型的准确度评估信息，包括：
    根据所述本地的数据和所述第一模型，确定所述第一模型的输出；
    根据所述第一模型的输出，确定所述第一模型准确度的评估值。
  24. 如权利要求19至22中任一项所述的方法，其特征在于，所述第一模型的准确度评估信息为所述第一模型准确度的评估等级，所述利用本地的数据，确定所述第一模型的准确度评估信息，包括：
    根据所述本地的数据和所述第一模型，确定所述第一模型的输出；
    根据所述第一模型的输出，确定所述第一模型准确度的评估值；
    根据所述第一模型准确度的评估值，确定所述第一模型准确度的评估等级。
  25. 如权利要求19至24中任一项所述的方法，其特征在于，还包括：
    向所述服务器MTLF发送模型请求消息，所述模型请求消息中至少包括模型需要满足的目标准确度。
  26. 如权利要求25所述的方法，其特征在于，在所述向服务器MTLF发送模型请求消息之前，还包括：
    接收来自用户的分析请求消息，所述分析请求消息中包括以下至少一项：分析标识、分析过滤信息、或该分析需要满足的目标准确度；
    确定分析推理功能AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的MTLF。
  27. 如权利要求23至26中任一项所述的方法，其特征在于，所述评估值，包括以下至 少一项：正确率、错误率、精度率、召回率、平均绝对误差、平均绝对百分比误差、或均方误差。
  28. 如权利要求19至27中任一项所述的方法，其特征在于，所述第一模型为联邦学习过程中的初始模型、中间模型、或最终模型。
  29. 一种数据分析方法，其特征在于，包括：
    接收来自分析推理功能AnLF的模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述分析标识用于标识分析任务，所述模型过滤信息用于指示联邦学习过程中训练数据需要满足的条件，所述模型需要满足的目标准确度用于指示推理当前分析任务的模型需要满足的准确度；
    根据所述模型请求消息，确定执行或不执行联邦学习。
  30. 如权利要求29所述的方法，其特征在于，根据所述模型请求消息，确定执行或不执行联邦学习，包括：
    根据所述分析标识和/或所述模型过滤信息，确定联邦学习过程中训练数据的特征；
    所述训练数据的特征与地理位置无关，确定执行联邦学习；或者，
    所述训练数据的特征与地理位置相关，确定不执行联邦学习。
  31. 如权利要求29所述的方法，其特征在于，根据所述模型请求消息，确定执行或不执行联邦学习，包括：
    向参与联邦学习的客户端MTLF发送第一模型评估请求消息，所述第一模型评估请求消息用于请求所述客户端MTLF上报本地模型的准确度评估信息；
    接收来自所述客户端MTLF的第一模型评估请求响应消息，所述第一模型评估请求响应消息中包括所述客户端MTLF上报的本地模型的准确度评估消息；
    所述客户端MTLF反馈的本地模型的准确度评估信息满足所述目标准确度，确定不执行联邦学习；或者，
    所述客户端MTLF反馈的本地模型的准确度评估信息不满足所述目标准确度，确定执行联邦学习。
  32. 如权利要求29至31中任一项所述的方法，其特征在于，在确定不执行联邦学习时，还包括：
    向所述AnLF发送模型请求响应消息，所述模型请求响应消息中包括联邦学习失败原因或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式确定当前分析任务的结果。
  33. 如权利要求29至31中任一项所述的方法，其特征在于，在确定执行联邦学习时，还包括：
    向参与联邦学习的客户端MTLF发送第二模型评估请求消息，所述第二模型评估请求消息中包括本轮模型训练得到的模型，所述第二模型评估请求消息用于请求所述客户端MTLF上报所述本轮训练得到的模型的准确度评估信息；
    接收来自所述客户端MTLF的第二模型评估请求响应消息，所述第二模型评估请求响应消息中包括所述客户端MTLF上报的本轮模型训练得到的模型的准确度评估信息；
    所述客户端MTLF上报的所述本轮模型训练得到的模型的准确度评估信息满足所述目标准确度，结束联邦学习。
  34. 一种数据分析方法，其特征在于，包括：
    接收来自用户的分析请求消息，所述分析请求消息中包括分析标识和该分析希望达到的准确度；
    分析推理功能AnLF的本地模型不能满足所述分析请求消息的要求，且网络仓库功能NRF中不能查询到满足所述分析请求消息要求的模型训练逻辑功能MTLF时，向服务器MTLF发送模型请求消息，所述模型请求消息中包括以下至少一项：分析标识、模型过滤信息、或模型需要满足的目标准确度，所述模型需要满足的目标准确度是根据所述分析希望达到的分析准确度确定的，所述模型过滤信息用于指示在模型训练过程中训练数据需要满足的条件。
  35. 如权利要求34所述的方法，其特征在于，还包括：
    接收来自服务器MTLF的模型请求响应消息，所述模型请求响应消息中包括联邦学习失败原因或分析聚合指示，所述分析聚合指示用于指示所述AnLF利用分析聚合的方式，确定当前分析任务的结果。
  36. 一种通信装置，其特征在于，包括处理器和接口电路，所述接口电路用于接收来自所述通信装置之外的其它通信装置的信号并传输至所述处理器或将来自所述处理器的信号发送给所述通信装置之外的其它通信装置，所述处理器通过逻辑电路或执行代码指令用于实现如权利要求1至18中任一项所述的方法，或者如权利要求29至33中的任一项所述的方法。
  37. 一种通信装置，其特征在于，包括用于执行如权利要求1至18中的任一项所述方法的单元，或者如权利要求29至33中的任一项所述方法的单元。
  38. 一种通信装置，其特征在于，包括处理器和接口电路，所述接口电路用于接收来自所述通信装置之外的其它通信装置的信号并传输至所述处理器或将来自所述处理器的信号发送给所述通信装置之外的其它通信装置，所述处理器通过逻辑电路或执行代码指令用于实现如权利要求19至28中的任一项所述的方法，或者如权利要求34或35所述的方法。
  39. 一种通信装置，其特征在于，包括用于执行如权利要求19至28中的任一项所述方法的单元，或者如权利要求34或35所述方法的单元。
  40. 一种通信系统，其特征在于，包括如权利要求36的通信装置和权利要求38的通信装置，或者包括如权利要求37的通信装置和权利要求39的通信装置。
  41. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有计算机程序或指令，当所述计算机程序或指令被通信装置执行时，实现如权利要求1至18中任一项所述的方法，或者如权利要求19至28中的任一项所述的方法，或者如权利要求29至33中的任一项所述的方法，或者如权利要求34或35所述的方法。
PCT/CN2023/090526 2022-08-09 2023-04-25 一种数据分析方法及装置 WO2024032031A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210951168.5A CN117675596A (zh) 2022-08-09 2022-08-09 一种数据分析方法及装置
CN202210951168.5 2022-08-09

Publications (1)

Publication Number Publication Date
WO2024032031A1 true WO2024032031A1 (zh) 2024-02-15

Family

ID=89850596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/090526 WO2024032031A1 (zh) 2022-08-09 2023-04-25 一种数据分析方法及装置

Country Status (2)

Country Link
CN (1) CN117675596A (zh)
WO (1) WO2024032031A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112181666A (zh) * 2020-10-26 2021-01-05 华侨大学 一种基于边缘智能的设备评估和联邦学习重要性聚合方法、系统、设备和可读存储介质
CN113191484A (zh) * 2021-04-25 2021-07-30 清华大学 基于深度强化学习的联邦学习客户端智能选取方法及系统
CN114021464A (zh) * 2021-11-09 2022-02-08 京东科技信息技术有限公司 数据处理方法、装置和存储介质
CN114385376A (zh) * 2021-12-09 2022-04-22 北京理工大学 一种异构数据下边缘侧联邦学习的客户端选择方法
CN114564746A (zh) * 2022-02-28 2022-05-31 浙江大学 基于客户端权重评价的联邦学习方法和系统
US20220172844A1 (en) * 2019-09-26 2022-06-02 Fujifilm Corporation Machine learning system and method, integration server, information processing apparatus, program, and inference model creation method
CN114611716A (zh) * 2022-03-01 2022-06-10 亚信科技（中国）有限公司 联邦学习系统构建方法、装置、电子设备及可读存储介质
US20220230062A1 (en) * 2019-03-01 2022-07-21 Telefonaktiebolaget Lm Ericsson (Publ) Dynamic network configuration

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220230062A1 (en) * 2019-03-01 2022-07-21 Telefonaktiebolaget Lm Ericsson (Publ) Dynamic network configuration
US20220172844A1 (en) * 2019-09-26 2022-06-02 Fujifilm Corporation Machine learning system and method, integration server, information processing apparatus, program, and inference model creation method
CN112181666A (zh) * 2020-10-26 2021-01-05 华侨大学 一种基于边缘智能的设备评估和联邦学习重要性聚合方法、系统、设备和可读存储介质
CN113191484A (zh) * 2021-04-25 2021-07-30 清华大学 基于深度强化学习的联邦学习客户端智能选取方法及系统
CN114021464A (zh) * 2021-11-09 2022-02-08 京东科技信息技术有限公司 数据处理方法、装置和存储介质
CN114385376A (zh) * 2021-12-09 2022-04-22 北京理工大学 一种异构数据下边缘侧联邦学习的客户端选择方法
CN114564746A (zh) * 2022-02-28 2022-05-31 浙江大学 基于客户端权重评价的联邦学习方法和系统
CN114611716A (zh) * 2022-03-01 2022-06-10 亚信科技（中国）有限公司 联邦学习系统构建方法、装置、电子设备及可读存储介质

Also Published As

Publication number Publication date
CN117675596A (zh) 2024-03-08

Similar Documents

Publication Publication Date Title
US11418413B2 (en) Sharable storage method and system for network data analytics
US20230090022A1 (en) Method and device for selecting service in wireless communication system
US11758416B2 (en) System and method of network policy optimization
US11743823B2 (en) Efficient MICO mode management method utilizing network analysis information in 5G mobile network system
US20230083982A1 (en) Communication method, apparatus, and system
US11202240B2 (en) Systems and methods for managing and monitoring communication sessions
WO2021135187A1 (zh) 一种切片控制方法及装置
US20220103644A1 (en) Method and apparatus for renewing subscription for network data analysis in wireless communication system
US20220408293A1 (en) Method and device for providing network analysis information for rfsp index selection in mobile communication network
Starks et al. Mobile distributed complex event processing—Ubi Sumus? Quo vadimus?
CN104584647A (zh) 无线局域网中基于服务差异化进行动态关联排序的系统和方法
KR20210143563A (ko) 이동통신 네트워크에서 단말에 Deterministic Communication을 지원하는 방법 및 장치
CN116325686A (zh) 一种通信方法和装置
WO2024032031A1 (zh) 一种数据分析方法及装置
WO2022116061A1 (zh) 一种通信方法和装置
KR20220001797A (ko) 무선 통신 네트워크에서 네트워크 분석 정보 제공 방법 및 장치
KR20210144535A (ko) 무선 통신 네트워크에서 네트워크 데이터 수집 및 분석을 위한 이동성 지원 방법 및 장치
WO2024099016A1 (zh) 一种通信方法及装置
WO2023078183A1 (zh) 一种数据收集方法及通信装置
WO2023185807A1 (zh) 一种通信方法及装置
WO2023213134A1 (zh) 一种数据报告的方法、装置及系统
WO2024113288A1 (zh) 通信方法和通信装置
WO2024104339A1 (zh) 数据传输方法、装置、相关设备及存储介质
WO2024016954A1 (zh) 一种授权方法和通信装置
US20230262487A1 (en) Apparatus and method for selecting training ue in a mobile communication system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23851261

Country of ref document: EP

Kind code of ref document: A1