WO2025251996A1 - 通信的方法和通信装置 - Google Patents

通信的方法和通信装置

Info

Publication number
WO2025251996A1
WO2025251996A1 PCT/CN2025/098029 CN2025098029W WO2025251996A1 WO 2025251996 A1 WO2025251996 A1 WO 2025251996A1 CN 2025098029 W CN2025098029 W CN 2025098029W WO 2025251996 A1 WO2025251996 A1 WO 2025251996A1
Authority
WO
WIPO (PCT)
Prior art keywords
mean
index
model
local
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/098029
Other languages
English (en)
French (fr)
Inventor
张征明
孙琰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of WO2025251996A1 publication Critical patent/WO2025251996A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B17/00—Monitoring; Testing
    • H04B17/30—Monitoring; Testing of propagation channels
    • H04B17/391—Modelling the propagation channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/10—Scheduling measurement reports ; Arrangements for measurement reports

Definitions

  • This application relates to the field of communications, and more specifically, to a method and apparatus for communication.
  • AI artificial intelligence
  • ML machine learning
  • a centralized training approach is typically used to train AI models, where the AI model and training dataset are concentrated on a single computing node, which then completes the training.
  • centralized training has several problems. For example, training data for centralized training usually comes from user devices, requiring users to upload data. Uploading this data can be costly, and the collection and use of this data may also infringe on user privacy. Distributed training can solve these problems.
  • Sub-computing nodes can train AI models based on their local training data and collaborate with the central computing node by uploading the AI model's parameters to complete the training.
  • This application provides a communication method and communication device to improve the accuracy of batch normalization in the distributed training process, thereby improving the performance of distributed training.
  • a communication method is provided, which can be executed by a communication device or a module applied to the communication device (e.g., a chip or circuit), wherein the communication device can be a sub-node in the method embodiment.
  • the sub-node and the central node can collaborate to complete the training of the AI model.
  • the child node can be a device on the terminal device side or a device on the network device side.
  • the terminal device side can include at least one of a terminal device or an AI entity on the terminal device side.
  • the AI entity on the terminal device side can be the terminal device itself or an AI entity serving the terminal device, such as a server, like an over-the-top (OTT) server or a cloud server.
  • the network device side can include at least one of a network device or an AI entity on the network device side.
  • the AI entity on the network device side can be the network device itself or an AI entity serving the network device, such as a radio access network (RAN) intelligent controller (RIC), operation administration and maintenance (OAM), or a server, such as an OTT server or a cloud server.
  • RAN radio access network
  • RIC radio access network intelligent controller
  • OAM operation administration and maintenance
  • the method includes: receiving first information, the first information including a first index and information indicating a first mean, the first mean being used to calculate a first variance, the first variance being the variance of the input of a first network layer in an AI model; and sending second information, the second information including a second index and information indicating the first variance, the first index and the second index having a correlation relationship.
  • the child node calculates the variance of the input of the network layer in the local AI model based on the first mean from the central node, i.e., the first variance, instead of calculating the first variance based on the mean of the input of the network layer in the local AI model obtained from the local dataset.
  • This helps to avoid the influence of the non-independent and identically distributed local data of each child node, and obtains a more accurate first variance.
  • This is intended to enable the central node to aggregate a more accurate global variance based on the more accurate first variance, thereby improving the accuracy of batch normalization processing in the distributed training process and improving the performance of distributed training.
  • the correlation between local variance and global mean can be determined by indexing, avoiding child nodes from calculating local variance based on unrelated global mean. For example, it avoids child nodes from calculating the variance of the input of the first network layer based on the local mean corresponding to other network layers, or from calculating the variance of the input of the first network layer in this round based on the global mean of other rounds, thus helping to ensure the accuracy of variance.
  • the first network layer can be any Batch Normalization (BN) layer in the AI model.
  • BN Batch Normalization
  • the input to the first network layer can also be replaced with features or feature channels of the first network layer.
  • the method may also include receiving model parameters of the AI model.
  • the child node before the start of the current training round, can calculate the variance of the input of the first network layer based on the model parameters of the received AI model. This is beneficial to ensure that each child node uses a unified AI model to calculate the variance of the input of the first network layer. In other words, it is beneficial to ensure that each child node uses a unified AI model when calculating the variance of the input of the first network layer of its local model, thereby improving the accuracy of the aggregated global variance.
  • the first index and the first mean are related.
  • the first index and the first mean are associated with the same round and/or the same network layer.
  • a round can be understood as starting from the central node sending the model parameters of the global model to each child node and ending with the central node aggregating and generating a new global model.
  • the first index and the first mean are carried in the same signaling.
  • the index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured association relationship.
  • the first index indicates the identifier of the first network layer.
  • the first index indicates the training rounds associated with the first mean.
  • the first index can indicate the identifier of the first network layer, which helps to avoid child nodes calculating the variance of the input of the first network layer based on the local mean corresponding to other network layers.
  • the first index can also indicate the training epochs related to the first mean, which helps to avoid child nodes calculating the variance of the input of the network layer in the current epoch based on the global mean of other epochs, thereby helping to ensure the accuracy of the variance.
  • the method further includes: sending third information, which is used to request model parameters of the AI model.
  • sending a third message includes sending a third message if the round indicated by the first index is different from the round at the child node.
  • the index value of the first index includes the number of the training epochs associated with the first mean. If the epoch number indicated by the first index is greater than the epoch number at the child node, third information is sent.
  • the child node can request the model parameters of the AI model from the central node, which helps to ensure that each child node calculates its local variance based on a unified AI model, thereby further improving the accuracy of the global variance obtained by aggregation.
  • the method further includes: receiving fourth information, the fourth information including a third index and information indicating a second variance, at least one of the first index and the second index being associated with the third index, and the first mean and the second variance being used for training the AI model.
  • the second variance can be obtained by aggregating the second variances indicated by the fourth information from multiple child nodes.
  • the second variance can be the average of the second variances indicated by the fourth information from multiple child nodes.
  • the second variance can be a weighted average of the second variances indicated by the fourth information from multiple child nodes.
  • the method before receiving the first information, further includes: sending fifth information, the fifth information including a fourth index and information indicating a second mean, the second mean being the mean of the inputs of a first network layer in the AI model, and at least one of the first index and the second index being associated with the fourth index.
  • the first mean can be obtained by aggregating the second means indicated by the fifth information from multiple child nodes.
  • the first mean can be the average of the second means indicated by the fifth information from multiple child nodes.
  • the first mean can be a weighted average of the second means indicated by the fifth information from multiple child nodes.
  • a communication method is provided, which can be executed by a communication device or a module (e.g., a chip or circuit) applied to the communication device, wherein the communication device can be a central node in the method embodiment.
  • the central node can collaborate with child nodes to complete the training of an AI model.
  • the central node can be a device on the terminal device side or a device on the network device side.
  • the terminal device side can include at least one of a terminal device or an AI entity on the terminal device side.
  • the AI entity on the terminal device side can be the terminal device itself or an AI entity serving the terminal device, such as a server, like an OTT server or a cloud server.
  • the network device side can include at least one of a network device or an AI entity on the network device side.
  • the AI entity on the network device side can be the network device itself or an AI entity serving the network device, such as a RIC, OAM, or a server, like an OTT server or a cloud server.
  • the method includes: sending first information to multiple devices, the first information including a first index and information indicating a first mean, the first mean being used to calculate a first variance, the first variance being the variance of the input of a first network layer in the AI model of each of the multiple devices; and receiving second information from each of the multiple devices, the second information including a second index and information indicating the first variance, the first index and the second index having a correlation relationship.
  • the multiple devices corresponding to the destination of the first information and the multiple devices corresponding to the source of the second information may be the same or different.
  • the multiple devices corresponding to the destination of the first information and the multiple devices corresponding to the source of the second information are at least partially the same.
  • the multiple devices corresponding to the destination of the first information may include the multiple devices corresponding to the source of the second information.
  • the central node can distribute a first mean to the child nodes, enabling the child nodes to calculate the variance (i.e., the first variance) of the input to the network layer in their local AI model based on the first mean, instead of calculating the first variance based on the mean of the input to the network layer in their local AI model obtained from the child nodes' local datasets.
  • This helps avoid the influence of the non-independent and identically distributed local data of each child node, resulting in a more accurate first variance.
  • This allows the central node to aggregate a more accurate global variance based on the more accurate first variance, thereby improving the accuracy of batch normalization processing during distributed training and enhancing the performance of distributed training.
  • the first index and the first mean are related.
  • the index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured association relationship.
  • the first index indicates the identifier of the first network layer.
  • the first index is related to the first mean in the training rounds.
  • the method further includes: receiving third information from a first device among multiple devices, the third information being used to request model parameters of the AI model.
  • the method further includes: sending fourth information to multiple devices, the fourth information including a third index and information indicating a second variance, at least one of the first index and the second index being associated with the third index, and the first mean and the second variance being used for training the AI model.
  • the multiple devices corresponding to the destination of the first information and the multiple devices corresponding to the destination of the fourth information may be the same or different.
  • the multiple devices corresponding to the source of the second information and the multiple devices corresponding to the destination of the fourth information may be the same or different.
  • the method further includes: determining a second variance based on a first variance indicated by second information from each of the plurality of devices.
  • the second variance could be the average of the second variances indicated by the fourth information from each of the multiple devices.
  • the second variance satisfies:
  • M represent the second variance
  • M represent the number of devices
  • represent the number of training samples on the j-th device.
  • M represent the first variance indicated by the second information from the j-th device, where M is a positive integer and
  • the method before sending the first information to the multiple devices, further includes: receiving fifth information from each of the multiple devices, the fifth information including a fourth index and information indicating a second mean, the second mean being the mean of the inputs of a first network layer in the AI model of each of the multiple devices, and at least one of the first index and the second index being associated with the fourth index.
  • the multiple devices corresponding to the destination of the first information and the multiple devices corresponding to the source of the fifth information may be the same or different.
  • the multiple devices corresponding to the source of the second information and the multiple devices corresponding to the source of the fifth information may be the same or different.
  • the method further includes: determining first information based on fifth information of each of the multiple devices, wherein the first mean is based on a second mean.
  • a communication method is provided, which can be executed by a communication device or a module applied to the communication device (e.g., a chip or circuit), wherein the communication device can be a sub-node in the method embodiment.
  • the sub-node and the central node can collaborate to complete the training of the AI model.
  • the child node can be a device on the terminal device side or a device on the network device side.
  • the terminal device side can include at least one of a terminal device or an AI entity on the terminal device side.
  • the AI entity on the terminal device side can be the terminal device itself or an AI entity serving the terminal device, such as a server, like an over-the-top (OTT) server or a cloud server.
  • the network device side can include at least one of a network device or an AI entity on the network device side.
  • the AI entity on the network device side can be the network device itself or an AI entity serving the network device, such as a radio access network (RAN) intelligent controller (RIC), operation administration and maintenance (OAM), or a server, such as an OTT server or a cloud server.
  • RAN radio access network
  • RIC radio access network intelligent controller
  • OAM operation administration and maintenance
  • the method includes sending a seventh message, the seventh message including information indicating a third mean and a fourth mean, the third mean being the mean of the inputs to the first network layer in the AI model, and the fourth mean being the mean of the squares of the inputs to the first network layer.
  • the child node sends the mean of the input of the first network layer of the AI model and the mean of the square of the input to the central node, so that the central node can calculate the global variance based on the mean of the input of the first network layer of the AI model on the child node and the mean of the square of the input.
  • This helps to avoid the influence of the non-independent and identically distributed local data of each child node, so that the central node can aggregate to obtain a more accurate global variance, thereby improving the accuracy of batch normalization processing in the distributed training process and improving the performance of distributed training.
  • the number of interactions between child nodes and the central node is reduced, which helps to reduce interaction overhead.
  • the method may also include receiving model parameters of the AI model.
  • the child node before the start of the current training round, can calculate the mean of the input of the first network layer and the mean of the square of the input based on the model parameters of the received AI model. This helps to ensure that each child node uses a unified AI model to calculate the third and fourth means, thereby improving the accuracy of the global mean and global variance determined based on the third and fourth means.
  • the method further includes: receiving eighth information, the eighth information including information indicating the fifth mean and the third difference, the fifth mean and the third difference being related to the first network layer, the fifth mean and the third difference being used for training the AI model.
  • the fifth mean can be obtained based on the third mean indicated by the seventh information from multiple child nodes.
  • the fifth mean is the average of the third means indicated by the seventh information from multiple child nodes.
  • the fifth mean can be a weighted average of the third means indicated by the seventh information from multiple child nodes.
  • the third difference is based on the third mean and the fourth mean.
  • the seventh information also includes the fifth index
  • the eighth information also includes the sixth index, with the fifth index and the sixth index having a correlation relationship.
  • At least one of the third mean and the fourth mean is related to the fifth index.
  • the correlation between the mean and variance can be determined by indexing, avoiding the central node from calculating the global variance based on unrelated means. For example, it avoids the central node from calculating the global mean and global variance of the first network layer based on the mean of the inputs of other network layers and the mean of the squares of the inputs, or it avoids the central node from calculating the global mean and global variance of the first network layer in the current round based on the mean of the inputs of the first network layer in other rounds and the mean of the squares of the inputs, thereby helping to ensure the accuracy of the mean and variance.
  • the index values of the fifth index and the sixth index are the same, or the fifth index and the sixth index have a pre-configured or pre-defined association relationship.
  • the fifth index indicates the identifier of the first network layer.
  • the fifth index indicates the training rounds associated with the third mean and/or the fourth mean.
  • a communication method is provided, which can be executed by a communication device or a module (e.g., a chip or circuit) applied to the communication device, wherein the communication device can be a central node in the method embodiment.
  • a communication device or a module (e.g., a chip or circuit) applied to the communication device, wherein the communication device can be a central node in the method embodiment.
  • the central node can be a device on the terminal device side or a device on the network device side.
  • the terminal device side can include at least one of a terminal device or an AI entity on the terminal device side.
  • the AI entity on the terminal device side can be the terminal device itself or an AI entity serving the terminal device, such as a server, like an OTT server or a cloud server.
  • the network device side can include at least one of a network device or an AI entity on the network device side.
  • the AI entity on the network device side can be the network device itself or an AI entity serving the network device, such as a RIC, OAM, or a server, like an OTT server or a cloud server.
  • the method includes: receiving seventh information from each of a plurality of devices, the seventh information including information indicating a third mean and a fourth mean, the third mean being the input of a first network layer in an artificial intelligence (AI) model of each of the plurality of devices, and the fourth mean being the mean of the squares of the inputs of the first network layer.
  • the seventh information including information indicating a third mean and a fourth mean
  • the third mean being the input of a first network layer in an artificial intelligence (AI) model of each of the plurality of devices
  • the fourth mean being the mean of the squares of the inputs of the first network layer.
  • AI artificial intelligence
  • the central node can calculate the global variance based on the mean of the input of the first network layer of the AI model on the child node and the mean of the square of the input. This helps to avoid the influence of the non-independent and identically distributed local data of each child node, so that the central node can aggregate to obtain a more accurate global variance, thereby improving the accuracy of batch normalization processing in the distributed training process and improving the performance of distributed training.
  • the number of interactions between child nodes and the central node is reduced, which helps to reduce interaction overhead.
  • the method further includes: sending an eighth message to multiple devices, the eighth message including information indicating a fifth mean and a third difference, the fifth mean and the third difference being related to the first network layer, and the fifth mean and the third difference being used for training the AI model.
  • the multiple devices corresponding to the source end of the seventh information and the multiple devices corresponding to the destination end of the eighth information can be the same or different.
  • the method further includes: determining a third difference based on a third mean and a fourth mean indicated by seventh information from each of the multiple devices.
  • the third difference satisfies:
  • the seventh information also includes the fifth index
  • the eighth information also includes the sixth index, with the fifth index and the sixth index having a correlation relationship.
  • At least one of the third and fourth means is related to the fifth index.
  • the index values of the fifth index and the sixth index are the same, or the fifth index and the sixth index have a pre-configured or pre-defined association relationship.
  • the fifth index indicates the identifier of the first network layer.
  • the fifth index indicates the training rounds associated with the third and/or fourth means.
  • a communication device can be a terminal device, or a device, module, circuit, or chip configured within a terminal device, or a device compatible with a terminal device.
  • the communication device may include modules corresponding to each of the methods/operations/steps/actions described in the first or third aspect. These modules can be hardware circuits, software, or a combination of hardware circuits and software.
  • the communication device may include a processing module and a communication module.
  • the sending module is used to perform the sending action in the method described in the first or third aspect above
  • the processing module is used to perform the processing action in the method described in the first or third aspect above
  • the receiving module is used to perform the receiving action in the method described in the first or third aspect above.
  • a communication device can be a network device, or a device, module, circuit, or chip configured within a network device, or a device compatible with a network device.
  • the communication device may include modules corresponding to the methods/operations/steps/actions described in the second or fourth aspect. These modules may be hardware circuits, software, or a combination of hardware circuits and software.
  • the communication device may include a processing module and a communication module.
  • the receiving module is used to perform the receiving action in the method described in the second or fourth aspect above
  • the processing module is used to perform the processing action in the method described in the second or fourth aspect above
  • the sending module is used to perform the sending action in the method described in the second or fourth aspect above.
  • a seventh aspect provides a communication apparatus comprising one or more processors coupled to one or more storage media, the one or more storage media storing instructions which, when executed by the one or more processors, cause a method as in the first aspect or any possible implementation thereof to be implemented, cause a method as in the second aspect or any possible implementation thereof to be implemented, cause a method as in the third aspect or any possible implementation thereof to be implemented, or cause a method as in the fourth aspect or any possible implementation thereof to be implemented.
  • a communication apparatus comprising one or more processors, the one or more processors being configured to process data and/or information such that a method as in the first aspect or any possible implementation thereof is implemented, a method as in the second aspect or any possible implementation thereof is implemented, a method as in the third aspect or any possible implementation thereof is implemented, or a method as in the fourth aspect or any possible implementation thereof is implemented.
  • the communication device may further include a communication interface for receiving data and/or information and transmitting the received data and/or information to the processor.
  • the communication interface may also be used to output data and/or information processed by the processor.
  • a chip including a processor, the processor being configured to execute a program or instructions to cause the method as in the first aspect or any possible implementation thereof to be implemented, the method as in the second aspect or any possible implementation thereof to be implemented, the method as in the third aspect or any possible implementation thereof to be implemented, or the method as in the fourth aspect or any possible implementation thereof to be implemented.
  • the chip may further include a memory for storing programs or instructions.
  • the chip may further include the transceiver.
  • the chip is an application-specific integrated circuit (ASIC) or a system-on-chip (SoC).
  • ASIC application-specific integrated circuit
  • SoC system-on-chip
  • a computer-readable storage medium including instructions that, when executed by a processor, cause a method as in the first aspect or any possible implementation of the first aspect to be implemented, cause a method as in the second aspect or any possible implementation of the second aspect to be implemented, cause a method as in the third aspect or any possible implementation of the third aspect to be implemented, or cause a method as in the fourth aspect or any possible implementation of the fourth aspect to be implemented.
  • a computer program product comprising computer program code or instructions, which, when executed, cause the method of the first aspect or any possible implementation thereof to be implemented, cause the method of the second aspect or any possible implementation thereof to be implemented, cause the method of the third aspect or any possible implementation thereof to be implemented, or cause the method of the fourth aspect or any possible implementation thereof to be implemented.
  • a communication system comprising one or more of the following means: a communication device for performing the method of the first aspect or any possible implementation thereof, a communication device for performing the method of the second aspect or any possible implementation thereof, a communication device for performing the method of the third aspect or any possible implementation thereof, or a communication device for performing the method of the fourth aspect or any possible implementation thereof.
  • the communication system may include the communication device provided in the fifth aspect, and/or the communication device provided in the sixth aspect.
  • Figure 1 is a schematic diagram of a communication system applicable to an embodiment of this application.
  • FIG. 2 is a schematic diagram of another communication system applicable to an embodiment of this application.
  • FIG. 3 is a schematic diagram of another communication system applicable to an embodiment of this application.
  • Figure 4 is a schematic diagram of the application framework of the communication system applicable to the embodiments of this application;
  • Figure 5 is a schematic flowchart of a communication method provided in an embodiment of this application.
  • Figure 6 is a schematic diagram of information indicating the global mean provided in an embodiment of this application.
  • Figure 7 is a schematic diagram of an association relationship format according to an embodiment of this application.
  • FIG. 8 is a schematic flowchart of another communication method provided in an embodiment of this application.
  • Figure 9 is a schematic diagram of an example of information #4 in an embodiment of this application.
  • Figure 10 is a schematic diagram of an example of information #1 in an embodiment of this application.
  • Figure 11 is a schematic diagram of an example of information #2 in an embodiment of this application.
  • Figure 12 is a schematic flowchart of another communication method provided in an embodiment of this application.
  • FIG. 13 is a schematic flowchart of another communication method provided in an embodiment of this application.
  • Figure 14 is a schematic block diagram of a communication device provided in an embodiment of this application.
  • Figure 15 is a schematic block diagram of another communication device provided in an embodiment of this application.
  • the technical solutions provided in this application can be applied to various communication systems, such as: 5th generation (5G) or new radio (NR) systems, long term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, wireless local area network (WLAN) systems, satellite communication systems, future communication systems such as future communication network mobile communication systems, or integrated systems of multiple systems.
  • 5G 5th generation
  • NR new radio
  • LTE long term evolution
  • LTE LTE frequency division duplex
  • TDD LTE time division duplex
  • WLAN wireless local area network
  • satellite communication systems satellite communication systems
  • future communication systems such as future communication network mobile communication systems, or integrated systems of multiple systems.
  • D2D device-to-device
  • V2X vehicle-to-everything
  • M2M machine-to-machine
  • MTC machine-type communication
  • IoT Internet of Things
  • a device can send signals to or receive signals from another device. These signals can include information, signaling, or data.
  • the device can also be replaced by an entity, network entity, network element, communication equipment, communication module, node, communication node, etc.
  • This disclosure uses a device as an example.
  • a communication system can include at least one terminal device and at least one network device.
  • the network device can send downlink signals to the terminal device, the terminal device can send uplink signals to the network device, the network device can send signals to another network device, and the terminal device can send sidelink signals to another terminal device.
  • the terminal device in this disclosure can be replaced by a first device, and the network device can be replaced by a second device, both performing the corresponding communication methods described in this disclosure.
  • the terminal device may also be referred to as user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user device.
  • UE user equipment
  • Terminal devices can be devices that provide voice/data, such as handheld devices with wireless connectivity, in-vehicle devices, etc.
  • terminals include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in telemedicine, wireless terminals in smart grids, wireless terminals in transportation safety, and wireless terminals in smart cities.
  • the embodiments of this application do not limit the scope to wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, wearable devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMNs).
  • SIP session initiation protocol
  • WLL wireless local loop
  • PDAs personal digital assistants
  • handheld devices with wireless communication capabilities computing devices or other processing devices connected to a wireless modem
  • wearable devices terminal devices in 5G networks
  • PLMNs public land mobile networks
  • the terminal device can also be a wearable device.
  • Wearable devices also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes.
  • Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories.
  • Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction.
  • wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.
  • the device for implementing the functions of the terminal device can be the terminal device itself, or it can be any device capable of supporting the terminal device in implementing those functions, such as a chip system.
  • This device can be installed in or used in conjunction with the terminal device.
  • the chip system can be composed of chips or may include chips and other discrete components.
  • This embodiment only uses the terminal device as an example to illustrate the device for implementing the functions of the terminal device, and does not constitute a limitation on the solution of this embodiment.
  • the network device in this application embodiment may include a device for communicating with a terminal device.
  • the network device may include an access network device or a wireless access network device, such as a base station.
  • the wireless access network device in this application embodiment may refer to a RAN node (or device) that connects the terminal device to the wireless network.
  • a base station can broadly encompass various names listed below, or be replaced by names such as: NodeB, evolved NodeB (eNB), next-generation NodeB (gNB), relay station, access point, transmitting and receiving point (TRP), transmitting point (TP), master station, auxiliary station, motor slide retainer (MSR) node, home base station, network controller, access node, wireless node, access point (AP), transmission node, transceiver node, baseband unit (BBU), remote radio unit (RRU), active antenna unit (AAU), remote radio head (RRH), central unit (CU), distributed unit (DU), radio unit (RU), positioning node, etc.
  • a base station can be a macro base station, micro base station, relay node, donor node, or a combination thereof.
  • a base station can also refer to a communication module, modem, or chip installed within the aforementioned equipment or apparatus.
  • a base station can also be a mobile switching center and equipment performing base station functions in D2D, V2X, and M2M communications, network-side equipment in future communication networks, or equipment performing base station functions in future communication systems.
  • a base station can support networks using the same or different access technologies.
  • a RAN node can also be a server, wearable device, vehicle, or in-vehicle equipment.
  • access network equipment in V2X technology can be a roadside unit (RSU).
  • RSU roadside unit
  • Base stations can be fixed or mobile.
  • a helicopter or drone can be configured to act as a mobile base station, and one or more cells can move depending on the location of the mobile base station.
  • a helicopter or drone can be configured as a device to communicate with another base station.
  • the network devices mentioned in the embodiments of this application may be devices including CU, DU, or CU and DU, or devices with control plane CU nodes (central unit-control plane (CU-CP)) and user plane CU nodes (central unit-user plane (CU-UP)) and DU nodes.
  • the network devices may include gNB-CU-CP, gNB-CU-UP, and gNB-DU.
  • RAN nodes collaborate to assist terminals in achieving wireless access, with different RAN nodes each implementing some of the base station's functions.
  • RAN nodes can be CUs, DUs, CU-CPs, CU-UPs, or RUs.
  • CUs and DUs can be configured separately or included in the same network element, such as a BBU.
  • RUs can be included in radio frequency equipment or radio frequency units, such as RRUs, AAUs, or RRHs.
  • RAN nodes can support one or more types of fronthaul interfaces, and different fronthaul interfaces correspond to DU and RU with different functions.
  • the DU is configured to implement one or more baseband functions
  • the RU is configured to implement one or more radio frequency functions.
  • some baseband functions for downlink and/or uplink such as, for downlink, precoding, digital beamforming (BF), or one or more of inverse fast fourier transform (IFFT)/cyclic prefix addition (CP), are moved from DU to RU; and for uplink, digital beamforming (BF), or one or more of fast fourier transform (FFT)/cyclic prefix removal (CP) are moved from DU to RU.
  • IFFT inverse fast fourier transform
  • CP cyclic prefix removal
  • the interface can be an enhanced common public radio interface (eCPRI).
  • eCPRI enhanced common public radio interface
  • the segmentation between DU and RU differs, corresponding to different categories (Cat) of eCPRI, such as eCPRI Cat A, B, C, D, E, and F.
  • Cat categories of eCPRI
  • the DU is configured to implement one or more functions before and after the layer mapping (i.e., coding, rate matching, scrambling, modulation, layer mapping).
  • Other functions after the layer mapping e.g., resource element (RE) mapping, digital beamforming (BF), or one or more of inverse fast Fourier transform (IFFT)/adding a cyclic prefix (CP)
  • the de-RE mapping is used as the dividing line.
  • the DU is configured to implement one or more functions preceding de-mapping (i.e., decoding, rate matching de-matching, descrambling, demodulation, inverse discrete Fourier transform (IDFT), channel equalization, and one or more functions in de-RE mapping).
  • Other functions following de-mapping e.g., one or more functions in digital BF or fast Fourier transform (FFT)/de-CP
  • FFT fast Fourier transform
  • the processing unit in the BBU used to implement baseband functions is called the baseband high (BBH) unit
  • the processing unit in the RRU/AAU/RRH used to implement baseband functions is called the baseband low (BBL) unit.
  • CU or CU-CP and CU-UP
  • DU or RU
  • RU may have different names, but those skilled in the art will understand their meaning.
  • ORAN open RAN
  • CU can also be called O-CU (open CU)
  • DU can also be called O-DU
  • CU-CP can also be called O-CU-CP
  • CU-UP can also be called O-CU-UP
  • RU can also be called O-RU.
  • Any of the units among CU (or CU-CP, CU-UP), DU, and RU in this application can be implemented through software modules, hardware modules, or a combination of software modules and hardware modules.
  • the apparatus for implementing the functions of a network device can be a network device itself; it can also be an apparatus capable of supporting the network device in implementing those functions, such as a chip system, hardware circuit, software module, or a hardware circuit plus a software module.
  • This apparatus can be installed in the network device or used in conjunction with the network device.
  • the example of a network device being used to implement the functions of a network device is provided only and does not constitute a limitation on the solutions described in this embodiment.
  • Network devices and/or terminal devices can be deployed on land, including indoors, outdoors, handheld, and/or vehicle-mounted; they can also be deployed on water (such as ships); and they can also be deployed in the air (such as airplanes, balloons, and/or satellites).
  • the embodiments of this application do not limit the scenarios in which the network devices and terminal devices are located.
  • terminal devices and network devices can be hardware devices, software functions running on dedicated hardware, software functions running on general-purpose hardware, such as virtualization functions instantiated on a platform (e.g., a cloud platform), or entities that include dedicated or general-purpose hardware devices and software functions.
  • a platform e.g., a cloud platform
  • This application does not limit the specific form of terminal devices and network devices.
  • wireless communication networks such as mobile communication networks
  • the services supported by the networks are becoming increasingly diverse, thus requiring increasingly diverse demands.
  • networks need to support ultra-high speeds, ultra-low latency, and/or massive connectivity.
  • This characteristic makes network planning, network configuration, and/or resource scheduling increasingly complex.
  • network functions become more powerful, such as supporting higher spectrum levels, supporting higher-order multiple-input multiple-output (MIMO) technologies, supporting beamforming, and/or supporting beam management
  • MIMO multiple-input multiple-output
  • AI nodes also known as AI entities
  • AI entities may be introduced into the network.
  • the AI entity can be deployed in one or more of the following locations within the communication system: access network devices, terminal devices, or core network devices, or the AI entity can be deployed independently, for example, in a location other than any of the aforementioned devices, such as the host or cloud server of an OTT system.
  • the AI entity can communicate with other devices in the communication system, which can be one or more of the following: network devices, terminal devices, or network elements of the core network.
  • the AI entity can include an AI entity on the network device side, an AI entity on the terminal device side, or an AI entity on the core network side.
  • this application does not limit the number of AI entities. For example, when there are multiple AI entities, they can be divided based on function, such as different AI entities being responsible for different functions.
  • AI entities can be independent devices, or they can be integrated into the same device to achieve different functions. Alternatively, they can be network components in hardware devices, software functions running on dedicated hardware, or virtualization functions instantiated on a platform (e.g., a cloud platform). This application does not limit the specific form of the aforementioned AI entities.
  • AI entities can be AI network elements or AI modules. AI entities are used to implement corresponding AI functions. AI modules deployed in different network elements can be the same or different. Depending on the different parameter configurations, the AI model within an AI entity can achieve different functions.
  • the AI model within an AI entity can be configured based on one or more of the following parameters: structural parameters (e.g., at least one of the following: number of neural network layers, neural network width, inter-layer connections, neuron weights, neuron activation function, or biases in the activation function), input parameters (e.g., the type and/or dimension of the input parameters), or output parameters (e.g., the type and/or dimension of the output parameters).
  • the biases in the activation function can also be referred to as the biases of the neural network.
  • An AI entity can have one or more models.
  • a model can infer an output that includes one or more parameters.
  • the learning, training, or inference processes of different models can be deployed on different entities or devices, or they can be deployed on the same entity or device.
  • Figure 1 is a schematic diagram of a communication system applicable to the communication method of this application embodiment.
  • the communication system 100 may include at least one network device, such as network device 110 shown in Figure 1; the communication system 100 may also include at least one terminal device, such as terminal device 120 and terminal device 130 shown in Figure 1.
  • Network device 110 and terminal devices can communicate via a wireless link.
  • the communication devices in this communication system for example, network device 110 and terminal device 120, can communicate via multi-antenna technology.
  • FIG 2 is a schematic diagram of another communication system applicable to the communication method of this application embodiment.
  • the communication system 200 shown in Figure 2 further includes an AI network element 140.
  • the AI network element 140 is used to perform AI-related operations, such as building training datasets or training AI models.
  • network device 110 can send data related to the training of the AI model to AI network element 140, which then constructs a training dataset and trains the AI model.
  • the data related to the training of the AI model may include data reported by the terminal device.
  • AI network element 140 can send the results of operations related to the AI model to network device 110, which then forwards them to the terminal device.
  • the results of operations related to the AI model may include at least one of the following: a trained AI model, model evaluation results, or test results.
  • a portion of the trained AI model may be deployed on network device 110, and another portion on the terminal device.
  • the trained AI model may be deployed on network device 110.
  • the trained AI model may be deployed on the terminal device.
  • Figure 2 is only used as an example of the AI network element 140 being directly connected to the network device 110.
  • the AI network element 140 can also be connected to a terminal device.
  • the AI network element 140 can be connected to both the network device 110 and a terminal device simultaneously.
  • the AI network element 140 can also be connected to the network device 110 through a third-party network element. This application embodiment does not limit the connection relationship between the AI network element and other network elements.
  • the AI network element 140 can also be configured as a module in network devices and/or terminal devices, for example, in network device 110 or terminal device as shown in Figure 1.
  • One or more AI modules can be deployed in network device 110.
  • One or more AI modules can be deployed in terminal device.
  • Figures 1 and 2 are simplified schematic diagrams for ease of understanding.
  • the communication system may also include other devices, such as wireless relay devices and/or wireless backhaul devices, which are not shown in Figures 1 and 2.
  • the communication system may include multiple network devices or multiple terminal devices. The embodiments of this application do not limit the number of network devices and terminal devices included in the communication system.
  • FIG. 3 is a schematic diagram of a possible application framework of a communication system according to an embodiment of this application.
  • network elements in the communication system are connected through interfaces (e.g., NG, Xn) or air interfaces.
  • These network element nodes such as core network equipment, access network nodes (RAN nodes), terminal equipment, or one or more devices in operation administration and maintenance (OAM), are equipped with one or more AI modules (only one is shown in Figure 3 for clarity).
  • the access network node can be a single RAN node or can include multiple RAN nodes, for example, including CU and DU.
  • CU and/or DU can also be equipped with one or more AI modules.
  • CU can also be split into CU-CP and CU-UP.
  • One or more AI models are provided in CU-CP and/or CU-UP.
  • CU and DU are connected through an F1 interface.
  • CU and CU are connected through an Xn interface.
  • the network device can be a network device equipped with one or more AI modules.
  • the network device can be one or more devices in the core network, access network (RAN) node, or OAM as shown in Figure 3.
  • the AI module can be the RIC shown in Figure 4, such as a near real-time RIC or a non-real-time RIC.
  • the near real-time RIC is set in the RAN node (e.g., in CU, DU), while the non-real-time RIC is set in the OAM, cloud server, core network device, or other network device.
  • the RIC can obtain subsets from multiple terminal devices from the RAN node (e.g., CU, CU-CP, CU-UP, DU, and/or RU), reassemble them into a training dataset #2, and train based on the training dataset #2.
  • the near real-time RIC and the non-real-time RIC can also be set up separately as a network element; the network device can be a near real-time RIC or a non-real-time RIC.
  • Figure 4 illustrates a possible application framework in a communication system.
  • the communication system includes a RAN intelligent controller (RIC).
  • the RIC can be the AI module shown in Figure 3, used to implement AI-related functions.
  • RICs include near-real-time RICs (near-RT RICs) and non-real-time RICs (non-RT RICs).
  • Non-real-time RICs primarily process non-real-time information, such as data that is not sensitive to latency, with latency in the order of seconds.
  • Real-time RICs primarily process near-real-time information, such as data that is relatively sensitive to latency, with latency in the order of tens of milliseconds.
  • NRT RICs are used for model training and inference. For example, they are used to train AI models and then use those models for inference.
  • NRT RICs can obtain network-side and/or terminal-side information from RAN nodes (e.g., CUs, CU-CPs, CU-UPs, DUs, and/or RUs) and/or terminals. This information can be used as training data or inference data.
  • the NRT RIC can deliver inference results to RAN nodes and/or terminals.
  • inference results can be exchanged between CUs and DUs, and/or between DUs and RUs.
  • the NRT RIC delivers inference results to a DU, which then forwards them to an RU.
  • Non-real-time RICs are also used for model training and inference. For example, they can be used to train AI models and then use those models for inference.
  • Non-real-time RICs can obtain network-side and/or terminal-side information from RAN nodes (e.g., CUs, CU-CPs, CU-UPs, DUs, and/or RUs) and/or terminals. This information can be used as training data or inference data, and the inference results can be delivered to RAN nodes and/or terminals.
  • RAN nodes e.g., CUs, CU-CPs, CU-UPs, DUs, and/or RUs
  • This information can be used as training data or inference data, and the inference results can be delivered to RAN nodes and/or terminals.
  • inference results can be exchanged between CUs and DUs, and/or between DUs and RUs; for example, a non-real-time RIC delivers inference results
  • Near real-time RICs and non-real-time RICs can also be configured as separate network elements.
  • near real-time RICs and non-real-time RICs can also be part of other devices.
  • near real-time RICs can be set in RAN nodes (e.g., CU, DU), while non-real-time RICs can be set in OAM, cloud servers, core network devices, or other network devices.
  • Neural networks are a specific implementation of AI or machine learning (ML). According to the general approximation theorem, neural networks can theoretically approximate any continuous function, thus enabling them to learn arbitrary mappings.
  • the AI model disclosed herein can be a deep neural network (DNN).
  • DNN deep neural network
  • Traditional communication systems require extensive expert knowledge to design communication modules, while deep learning communication systems based on deep neural networks (DNN) can automatically discover hidden pattern structures from large datasets, establish mapping relationships between data, and achieve performance superior to traditional modeling methods.
  • a neural network can be composed of neurons, each of which performs a weighted summation of its input values, and the result is then passed through a non-linear function to produce the output.
  • DNNs typically have a multi-layered structure, with each layer containing multiple neurons. The input layer processes the received values through neurons and then passes them to the hidden layers. Similarly, the hidden layers then pass the calculation results to the final output layer, producing the final output of the DNN.
  • DNNs typically have more than one hidden layer, and these hidden layers often directly affect the ability to extract information and fit functions. Increasing the number of hidden layers or widening the width of each layer can improve the function fitting ability of a DNN.
  • the weights in each neuron are the parameters of the DNN network model.
  • the model parameters are optimized through the training process, enabling the DNN network to extract data features and express mapping relationships.
  • DNNs generally use supervised or unsupervised learning strategies to optimize model parameters.
  • DNNs can include feedforward neural networks (FNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs), etc.
  • FNNs feedforward neural networks
  • CNNs convolutional neural networks
  • RNNs recurrent neural networks
  • CNNs are neural networks specifically designed to process data with a grid-like structure. For example, time-series data (discrete sampling along the time axis) and image data (two-dimensional discrete sampling) can both be considered grid-like data.
  • CNNs do not use all the input information at once for computation; instead, they use a fixed-size window to extract a portion of the information for convolution operations, which significantly reduces the computational cost of model parameters.
  • each window can use different convolution kernels, allowing CNNs to better extract features from the input data.
  • RNNs are a type of distributed neural network (DNN) that utilizes feedback time-series information. Their input includes the current input value and their own output value from the previous time step. RNNs are well-suited for acquiring temporally correlated sequence features, and are particularly applicable to applications such as speech recognition and channel coding/decoding.
  • FNN networks The characteristic of FNN networks is that neurons in adjacent layers are completely connected to each other, which makes FNNs typically require a large amount of storage space and result in high computational complexity.
  • the FNN, CNN, and RNN mentioned above are all constructed based on neurons.
  • each neuron performs a weighted summation operation on its input values, and the result of the weighted summation is used to generate the output through a nonlinear function.
  • the weights of the weighted summation operation of neurons in a neural network and the nonlinear function are called the parameters of the neural network.
  • the parameters of all neurons in a neural network constitute the parameters of that neural network.
  • Centralized training also known as centralized training, refers to a training method where the machine learning model and training data are concentrated on a single computing node, which then completes all training steps.
  • Distributed training also known as distributed learning, refers to algorithms and systems that utilize multiple computing nodes for machine learning or deep learning.
  • training typically involves multiple sub-components and a central compute node.
  • the user's raw data resides locally.
  • Sub-components collaborate with the central compute node to complete training by uploading neural network parameters. This avoids direct sharing of raw data, thus protecting the privacy and security of user data.
  • Federated learning refers to training an algorithm on multiple distributed edge devices or computing nodes that have local data samples, and allows for differences in the data distribution among the computing nodes.
  • Federated learning is an important framework in distributed learning.
  • one or more child computing nodes train neural networks on their local data, while the central computing node is responsible for aggregating the neural network parameters to build a global model.
  • Federated learning consists of two parts: a central computation node and child computation nodes.
  • the central computation node can also be simply referred to as the central node.
  • the child computation nodes can also be simply referred to as child nodes.
  • the local datasets of the M child nodes are represented as ⁇ D1 , D2 , ..., DM ⁇ , where M is a positive integer.
  • the local dataset of child node k is denoted as... Where xi is the input to the model, i.e., the i-th sample, yi is the label corresponding to the input xi , and Nk is the amount of data in Dk .
  • the central node and child nodes each have their own neural network models f( ⁇ , wc ) and f( ⁇ , wk ), respectively.
  • wc represents the parameters of the central node's neural network model, also known as the global model
  • wk represents the parameters of the child node's neural network model.
  • the child node's neural network model can also be called the local model.
  • the local dataset is used for training the local model.
  • the central node and child nodes collaborate to solve the following optimization problem:
  • L ⁇ sub>k ⁇ /sub> (w; D ⁇ sub> k ⁇ /sub> ) represents the proportion of the local dataset of child node k to the total dataset.
  • L ⁇ sub>k ⁇ /sub>(w;D ⁇ sub>k ⁇ /sub>) represents the local loss value of the loss function for child node k.
  • L(w) represents the loss value.
  • w represents the parameters of the neural network model.
  • the training process of federated learning is as follows: First, in the initial stage of training, the central node sends the parameters w ⁇ sub> c ⁇ /sub> of the global model to all child nodes. Then, each child node's model undergoes multiple rounds of training using its local dataset D ⁇ sub>k ⁇ /sub> ; this step is called local training.
  • the specific process of local training includes the following steps.
  • Child node k first randomly selects a batch of training data from the local dataset Dk.
  • is the amount of data in Bk , i.e., the number of training data in this batch
  • is a positive integer
  • t is a positive integer.
  • Child node k calculates the output for input x i And calculate the loss value according to the loss function l(.,.). in, This represents the parameters of the local model at the t-th iteration.
  • Steps (1) to (4) above are repeated until all samples in the local dataset have been used, that is, the local model has been updated a certain number of times. At that time, the local training for this round is completed, among which This indicates that x is rounded down. Then, the child nodes send the trained model parameters to the central node. The central node aggregates the received model parameters to obtain new model parameters w ⁇ sub>c ⁇ /sub> . After aggregation, the central node and child nodes share the aggregated global model parameters w ⁇ sub>c ⁇ /sub> , and then repeat the above steps until the model converges.
  • Batch normalization (BN) layers normalize the features output by hidden layers, mitigating overfitting and the vanishing gradient problem, and providing regularization, which helps improve the training performance of AI models.
  • Batch normalization makes the outputs of the intermediate layers more stable.
  • a corresponding batch normalization layer can be placed after one or more network layers.
  • a batch normalization layer can be placed after the second layer and another after the fourth layer. The output of the second layer becomes the input of the corresponding batch normalization layer, and vice versa.
  • x ⁇ sub>l ⁇ /sub> represents the input of the l-th layer of the neural network model
  • h ⁇ sub> l ⁇ /sub>(x ⁇ sub> l ⁇ /sub>) represents the output of the l-th layer of the neural network model.
  • the output of the l-th layer of this neural network model is the input of the batch normalization layer corresponding to the l-th layer.
  • ⁇ sub> B ⁇ /sub> represents the mean of h ⁇ sub>l ⁇ /sub> (x ⁇ sub> l ⁇ /sub> ).
  • ⁇ is a constant to ensure the denominator is not zero, typically taken as 10 ⁇ sup> -6 ⁇ /sup>.
  • ⁇ sub>l ⁇ /sub> and ⁇ sub> l ⁇ /sub> are the trainable parameters of this batch of normalized layers. This represents the output of the normalized layer.
  • B represents a batch of training data in the dataset. This dataset is used to train the neural network model. In the above formula, ⁇ and B... It can be determined using the following formula.
  • represents the number of training data sets. This represents the input of the i-th sample in the l-th layer.
  • the local data of child nodes are not independent and identically distributed, meaning that the data distribution of each child node is different. This leads to differences in the mean and variance of the batch normalization layer statistics of the local model, ultimately resulting in the performance of distributed training being worse than that of centralized training.
  • child nodes can upload the mean and variance of the batch normalization layer inputs of their local model, as well as the trained model parameters.
  • the central node aggregates the received mean, variance, and model parameters, and sends the aggregated mean, variance, and new global model parameters to the child nodes. The above steps are repeated until the model converges.
  • the aggregated variance obtained by the above method is not the same as the variance in centralized training.
  • the mean and variance of the input to the batch normalization layer of the local model can be calculated using the following formula.
  • B ⁇ sub>m ⁇ /sub> represent a batch of training data on child node m, i.e., the m-th batch of data.
  • B ⁇ sub>m ⁇ /sub> represent the number of data in this batch of training data.
  • the central node can aggregate the mean and variance of the feedback from its M child nodes using the following formula:
  • ⁇ ⁇ sub>l,B ⁇ /sub> represents the mean of the input to the batch normalization layer corresponding to the l-th layer of the aggregated neural network model. This represents the variance of the input to the batch normalization layer corresponding to the l-th layer of the aggregated neural network model.
  • the variance of the input to the batch normalization layer corresponding to the l-th layer of the neural network model can be calculated using the following formula.
  • B B1 ⁇ B2 ⁇ ... ⁇ BM , ⁇ 'l ,B represents the mean of the input of the batch normalization layer corresponding to the l-th layer of the training data B.
  • Bm be the variance of the input to the batch normalization layer corresponding to the l-th layer of the m-th batch of training data. This indicates that the input is In this case, the output of the l-th layer of the neural network model is the input of the batch normalization layer corresponding to the l-th layer.
  • the local models of child nodes can use Batch Normalization (BN) layers to resist overfitting and achieve better performance.
  • BN Batch Normalization
  • each sub-computing node is not independent and identically distributed, and there are differences in the mean and variance of the BN layer statistics of each sub-computing node, which affects the accuracy of batch normalization and thus affects the performance of distributed training.
  • child nodes can upload the mean and variance corresponding to the BN layer and the model parameters of the trained local model after local training.
  • the central node aggregates the received mean, variance, and model parameters, and sends the aggregated mean, variance, and model parameters of the new global model to each child node.
  • the child nodes train the new global model locally based on the aggregated mean and variance, and repeat the above process until the model converges.
  • the variance obtained by this aggregation method is different from the variance of centralized training. Specifically, comparing the above formulas (6), (8), (9), and (10), it can be seen that there is a difference in the variance calculated under the two methods.
  • the aggregated variance obtained by aggregating the variance of child nodes in distributed training is different from the variance of centralized training, which affects the accuracy of batch normalization processing in the distributed training process, thereby affecting the performance of distributed training and making it difficult to achieve the performance of centralized training.
  • this application provides a communication method and communication device, which is beneficial to improving the accuracy of batch normalization processing in the distributed training process, thereby improving the performance of distributed training.
  • the indication includes direct indication (also known as explicit indication) and implicit indication.
  • Direct indication information A refers to information A being included; implicit indication information A refers to information A being indicated through the correspondence between information A and information B, and through direct indication information B.
  • the correspondence between information A and information B can be predefined, pre-stored, pre-burned, or pre-configured.
  • information C is used to determine information D, including both situations where information D is determined solely based on information C and situations where it is determined based on information C and other information. Furthermore, information C can also be used to determine information D indirectly, for example, where information D is determined based on information E, and information E is determined based on information C.
  • network element A sends information A to network element B can be understood as network element B being the destination of information A or an intermediate network element in the transmission path between the destination and network element B, which may include sending information directly or indirectly to network element B.
  • Network element B receives information A from network element A can be understood as network element A being the source of information A or an intermediate network element in the transmission path between the source and network element A, which may include receiving information directly or indirectly from network element A.
  • Information may undergo necessary processing between the source and destination, such as format changes, but the destination can understand the valid information from the source. Similar expressions in this application can be understood in a similar way and will not be elaborated further here.
  • the central node can collaborate with M child nodes to complete the training of the AI model.
  • M is an integer greater than 1.
  • this application embodiment mainly uses the communication between one of the M child nodes (such as child node #1) and the central node as an example.
  • Child node #1 can be any of the M child nodes.
  • the behavior of other child nodes can refer to the behavior of child node #1.
  • the central node can be an AI entity on the network device side
  • the child nodes can be AI entities on the terminal device side.
  • the network device side includes network devices, or other devices that communicate with network devices, such as devices controlled by or serving network devices.
  • the AI entity can be the network device itself, or an AI entity that communicates with the network device.
  • the central node can be a RIC, OAM, or a server, such as an OTT server or a cloud server. Near real-time RICs are set up in RAN nodes, such as in CU/DU.
  • the terminal device side includes the terminal device itself, or other devices that communicate with the terminal device, such as devices controlled by or serving the terminal device.
  • the AI entity can be the terminal device itself, or an AI entity that communicates with the terminal device.
  • a child node can be a server, such as an OTT server or a cloud server.
  • the central node can be an AI entity on the terminal device side
  • the child nodes can be AI entities on the terminal device side.
  • the central node can be a server
  • the multiple child nodes can be multiple terminal devices communicating with the server.
  • the AI model after the AI model is trained, it can be deployed on the terminal device side, or on the network device side, or on both the terminal device side and the network device side.
  • the output of the AI model can include discrete or continuous numerical values.
  • the AI model when the AI model is used to perform a classification task, it can output discrete numerical values.
  • the AI model when the AI model is used to perform a regression task, it can output continuous numerical values.
  • the central node can also train the AI model based on the training dataset on the central node.
  • the central node can also be considered a child node.
  • the central node also handles operations related to model training on the child nodes.
  • the aforementioned M child nodes can be considered to include the central node.
  • Child nodes can be called child compute nodes.
  • M child nodes can also be replaced by M devices.
  • the central node can also be called the central computing node.
  • the AI model on the central node can be called the global model.
  • the AI model on the child nodes can be called the local model on the child node.
  • the AI model on the central node and the AI models on the M child nodes have the same structure.
  • Figure 5 is a schematic flowchart of a communication method provided in this application.
  • the method 500 shown in Figure 5 can be applied to a distributed training scenario.
  • the central node sends the aggregated global mean corresponding to the BN layer to the child nodes, and the child nodes calculate the variance of the BN layer input based on the global mean corresponding to the BN layer.
  • the central node can aggregate the variances of the BN layer inputs fed back by the child nodes to obtain the global variance corresponding to the BN layer.
  • method 500 may include the following steps.
  • the central node sends information #1 (an example of the first information) to multiple child nodes (child nodes #1 and #2 in Figure 5).
  • Information #1 includes index #1 (an example of the first index) and information indicating the global mean #1 (an example of the first mean).
  • the global mean #1 can be used to calculate the local variance #1 (an example of the first variance).
  • the local variance #1 of a child node is the variance of the input to the network layer #1 (an example of the first network layer) of the AI model on the child node.
  • the global mean #1 can be used to calculate the variance of the input to the network layer #1 of the AI model on these multiple child nodes.
  • index #1 is related to the global mean #1.
  • the global mean #1 can be determined and used to calculate the local difference #1.
  • the multiple child nodes in step 510 are some or all of the aforementioned M child nodes.
  • the multiple child nodes may include child node #1 (an example of the first device). Taking child node #1 as an example, child node #1 receives information #1 from the central node and can calculate the local variance #1 of child node #1 based on the global mean #1, that is, the variance of the input of the network layer #1 of the AI model on child node #1.
  • the global mean #1 is the global mean corresponding to network layer #1, or in other words, the global mean related to network layer #1.
  • This global mean #1 can be obtained by aggregating the local means #1 of one or more child nodes.
  • the local mean #1 of a child node is the mean of the input to network layer #1 of the AI model on that child node. The method for determining the global mean #1 will be described later.
  • Multiple child nodes can also be replaced by multiple devices.
  • Multiple child nodes send message #2 (an example of the second message) to the central node.
  • Message #2 includes index #2 (an example of the second index) and information indicating the local local difference #1.
  • Index #2 and index #1 are related.
  • the information #2 from the multiple child nodes includes information indicating the local difference #1 of the multiple child nodes.
  • the multiple child nodes in step 520 are some or all of the aforementioned M child nodes.
  • the multiple child nodes in step 520 can be the same as or different from the multiple child nodes in step 510.
  • the child nodes in step 520 are at least partially the same as the child nodes in step 510.
  • the child nodes in step 510 may include the child nodes in step 520.
  • These multiple child nodes may include child node #1.
  • child node #1 Taking child node #1 as an example, child node #1 sends information #2 to the central node.
  • the local difference #1 indicated by information #2 from child node #1 is the local difference #1 of child node #1.
  • index #2 is related to the local local difference #1. Based on the correlation between index #2 and index #1, it can be determined that the local local difference #1 is related to the global mean #1, or in other words, the local local difference #1 is determined based on the global mean #1. Furthermore, the local local difference #1 can be determined based on the global mean #1 and the input of network layer #1.
  • the AI model may contain multiple batch normalization (BN) layers.
  • the correlation between the local local error and the global mean can be determined through indexing, preventing child nodes from calculating the local local error based on irrelevant global means. For example, it prevents child nodes from calculating the local local error of the current BN layer's input based on the global mean corresponding to other BN layers, or from calculating the local local error of the current round based on the global mean of other rounds, thereby helping to ensure the accuracy of the local local error.
  • method 500 may also include step 530.
  • the central node sends information #3 (an example of the fourth information) to multiple child nodes.
  • Information #3 includes index #3 (an example of the third index) and information indicating the global variance #1 (an example of the second variance). Index #1 and index #3 are correlated, and/or, index #2 and index #3 are correlated.
  • the global mean #1 and global variance #1 can be used for training AI models on the child nodes.
  • the central node can determine the global variance #1 based on the local variance #1 indicated by information #2 from one or more child nodes.
  • the local variance #1 indicated by information #2 from multiple child nodes is the local variance #1 of those multiple child nodes.
  • the central node can aggregate the local variances #1 of one or more child nodes to obtain the global variance #1, and then distribute the global variance #1 to multiple child nodes via information #3.
  • the global variance can also be called the aggregated variance or the aggregated variance.
  • the multiple child nodes in step 530 are some or all of the aforementioned M child nodes.
  • the multiple child nodes in step 530 can be the same as or different from the multiple child nodes in step 510.
  • the multiple child nodes in step 530 can be the same as or different from the multiple child nodes in step 520.
  • Child nodes can include child node #1.
  • child node #1 after receiving information #3 from the central node, child node #1 can train the local model based on the global mean #1 and the global variance #1.
  • Global variance #1 is the global variance corresponding to network layer #1, or in other words, the global variance related to network layer #1.
  • the method for determining global variance #1 can be found in the description below.
  • index #3 is correlated with global variance #1.
  • the correlation between global variance #1, global mean #1, and local variance #1 can be determined based on the correlation between index #3 and index #1, and the correlation between index #1 and index #2.
  • the correlation between global variance #1, global mean #1, and local variance #1 can be determined based on the correlation between index #3 and index #2, and the correlation between index #1 and index #2.
  • the correlation between global variance #1, global mean #1, and local variance #1 can be determined based on the correlation between index #3, index #1, and index #2.
  • method 500 may also include step 540.
  • Multiple child nodes send information #4 (an example of the fifth information) to the central node.
  • Information #4 includes index #4 (an example of the fourth index) and information indicating the local mean #1 (an example of the second mean).
  • the local mean #1 of the child node is the mean of the inputs to network layer #1 of the AI model on the child node.
  • Index #4 is associated with at least one of the following: index #1, index #2, or index #3.
  • the information #4 from multiple child nodes includes information indicating the local mean #1 of those multiple child nodes.
  • the multiple child nodes in step 540 are some or all of the aforementioned M child nodes.
  • the multiple child nodes in step 540 can be the same as or different from the multiple child nodes in step 510.
  • the multiple child nodes in step 540 can be the same as or different from the multiple child nodes in step 520.
  • the multiple child nodes in step 540 can be the same as or different from the multiple child nodes in step 530.
  • Child node #1 can include child node #1.
  • child node #1 sends information #4 to the central node.
  • the local mean #1 indicated by information #4 sent by child node #1 is the local mean #1 of child node #1, that is, the mean of the input of the network layer #1 of the AI model on child node #1.
  • the central node can determine the global mean based on the local mean #1 indicated by information #4 from one or more child nodes.
  • the local mean #1 indicated by information #4 from multiple child nodes is the local mean #1 of multiple child nodes.
  • the central node can aggregate the local mean #1 of one or more child nodes to obtain the global mean #1.
  • the global mean #1 is distributed to multiple child nodes via information #1.
  • the global mean can also be called the aggregated mean or the aggregated mean.
  • index #4 is related to the local mean #1.
  • Index #4 is related to at least one of index #1, index #2 or index #3. Based on this relationship, it can be determined that the local mean #1 is related to at least one of the global mean #1, local variance #1 or global variance #1.
  • step numbers in the embodiments of this application are for ease of description only and do not constitute a limitation on the execution order of the steps in the embodiments of this application.
  • step 540 may be executed before step 510.
  • the AI model may include one or more Batch Normalization (BN) layers.
  • Network layer #1 can be any one of these BN layers.
  • this embodiment only uses one BN layer as an example.
  • the processing methods of other BN layers can refer to those related to network layer #1.
  • the input of one network layer can be used as the output of another network layer.
  • the input of network layer #1 can be understood as the output of the layer preceding it.
  • the input of network layer #1 can be replaced with the output of network layer #2, which can be the layer preceding a batch normalization (BN) layer.
  • BN batch normalization
  • the input to network layer #1 can include one or more features, or one or more feature channels.
  • the input to network layer #1 can also be replaced with the features of network layer #1, or the feature channels of network layer #1.
  • the mean of the input to network layer #1 is the mean of the features of network layer #1.
  • the variance of the input to network layer #1 is the variance of the features of network layer #1.
  • the mean of the input to network layer #1 of the AI model on the child node is determined based on a batch of training data on that child node.
  • the amount of training data Bm on child node m is
  • inputs to network layer #1 The average of these
  • child node m can calculate its local mean #1 based on formula (5) mentioned earlier, and the BN layer corresponding to layer l can be considered as network layer #1.
  • the central node can aggregate the local mean #1 of one or more child nodes to obtain the global mean #1.
  • the one or more child nodes can be some or all of the M child nodes, that is, the central node can determine the global mean #1 based on the local mean #1 of some or all child nodes.
  • the one or more child nodes belong to the multiple child nodes in step 510.
  • the central node determines the global mean #1 based on the local mean #1 of the M child nodes.
  • the central node can receive information #4 from M child nodes and obtain the local mean #1 of those M child nodes.
  • the central node aggregates the local mean #1 of those M child nodes to obtain the global mean #1.
  • the central node determines the global mean #1 based on the local mean #1 of some of the M child nodes.
  • the central node only receives information #4 from a subset of its M child nodes.
  • the central node can aggregate the received local averages #1 of the child nodes to obtain the global average #1.
  • child node #1 might not have sent message #4. Or, child node #1 might have sent message #4 to the central node, but the central node did not receive the local mean #1 reported by child node #1. In this case, the central node can choose not to use child node #1's local mean #1 when calculating the global mean #1.
  • the central node can decide on its own to determine the global mean #1 based on the local mean #1 of some of its M child nodes.
  • the central node receives the local mean #1 reported by the child node #1, the central node can decide not to use the local mean #1 of the child node #1 to calculate the global mean #1.
  • the central node can aggregate the local average of its child nodes using various aggregation methods.
  • a central node aggregating the local mean #1 of one or more child nodes may include: the central node calculating the average of the local mean #1 of the multiple child nodes and using it as the global mean #1.
  • the global mean #1 can satisfy the following formula:
  • ⁇ l,B represents the global mean of the BN layer (i.e., network layer #1) corresponding to the l-th layer of the AI model, i.e., the global mean #1.
  • This represents the mean of the inputs to the BN layer corresponding to the l-th layer of the AI model, i.e., the local mean #1.
  • the central node aggregating the local mean #1 of the one or more child nodes may include: the central node calculating a weighted average of the local means #1 of the multiple child nodes and using it as the global mean #1. For example, in the calculation of the weighted average, the weight corresponding to the local mean #1 of each child node may be positively correlated with the amount of batch training data of each child node. Taking the one or more child nodes as all of M child nodes as an example, the central node may calculate the global mean #1 based on the formula (7) above.
  • the global mean #1 can be used to calculate the local difference #1 of multiple child nodes.
  • Child node #1 after receiving the global mean #1, child node #1 does not calculate the local difference #1 based on the local mean #1, but rather calculates the local difference #1 based on the global mean #1.
  • the local difference #1 can satisfy the following formula:
  • ⁇ ⁇ sub>l,B ⁇ /sub> represents the variance of the input to the BN layer (i.e., network layer #1) corresponding to the l-th layer of the AI model of child node m, i.e., the local variance #1 of child node m.
  • ⁇ sub>l,B ⁇ /sub> represents the global mean corresponding to the BN layer corresponding to the l-th layer, i.e., the global mean #1. This represents the input of the i-th sample in the l-th layer. This indicates that the input is In the case of child node m, the output of the l-th layer of the AI model is the input of the BN layer corresponding to the l-th layer.
  • the central node can aggregate the local variances #1 of one or more child nodes to obtain the global variance #1.
  • the one or more child nodes can be some or all of the M child nodes, meaning the central node can determine the global variance #1 based on the local variance #1 of some or all child nodes. These one or more child nodes belong to the multiple child nodes in step 520. A detailed description of these one or more child nodes can be found in the descriptions of the one or more child nodes in the process of calculating the global mean #1 described above. It should be understood that the one or more child nodes to which the local variance #1 used to calculate the global variance #1 belongs and the one or more child nodes to which the local mean #1 used to calculate the global mean #1 belongs can be the same or different.
  • the central node aggregating the local variance #1 of one or more child nodes may include: the central node calculating the average of the local variance #1 of the multiple child nodes and using it as the global variance #1.
  • the central node aggregating the local variances #1 of one or more child nodes may include: the central node calculating a weighted average of the local variances #1 of the multiple child nodes, and using this average as the global variance #1. For example, in the calculation of the weighted average, the weights corresponding to the local variances #1 of each child node may be positively correlated with the amount of batch training data of each child node. Taking the multiple child nodes as all of M child nodes as an example, the central node may calculate the global variance #1 based on the formula (8) above.
  • the child node calculates the local variance #1 related to the global mean #1 based on the global mean #1 from the central node. This helps to avoid the influence of the non-independent and identically distributed local data of each child node, and obtains a more accurate local variance #1. This is intended to enable the central node to aggregate a more accurate global variance #1 based on the more accurate local variance #1, thereby improving the accuracy of batch normalization processing in the distributed training process and improving the performance of distributed training.
  • index #1 The correlation between index #1 and global mean #1 is explained below.
  • a round can be understood as the process from the central node sending the global model parameters to each child node to the central node aggregating and generating a new global model.
  • Different rounds can be distinguished by different round identifiers.
  • the round identifier can be the current round number.
  • the current round number can reflect the number of rounds already completed, i.e., the current round number.
  • the current round number can be equal to the current round number.
  • the round number and the round number can be positively correlated; for example, the larger the round number, the larger the round number.
  • the current round number can also be independent of the current round number, as long as it distinguishes different rounds. For ease of description, this embodiment mainly uses the example of the current round number reflecting the current round number for illustration.
  • the central node each time it sends the model parameters of the global model to the child nodes, or each time it updates the global model, the central node enters a new round.
  • the central node can update the round identifier of the central node, for example, by incrementing the round number at the central node, or by incrementing the number of the training rounds related to the global model.
  • the child node each time it receives the model parameters of the global model, or each time it updates its local model based on the received model parameters of the global model, the child node enters a new round.
  • the child node can update the round identifier of its local round, for example, by incrementing the round number at the child node, or by incrementing the number of the training rounds related to the local model.
  • each time the central node sends the model parameters of the global model to the child node it can also send the updated round identifier to the child node, and the child node can update its local round identifier based on the received round identifier.
  • the round identifier can also be the number of times the global model has been updated, or the number of interactions between the central node and its child nodes regarding the model parameters of the global model.
  • index #1 and global mean #1 can be understood as both index #1 and global mean #1 being related to the same round and/or the same network layer.
  • the central node calculates the global mean *a* based on the local means *a* of multiple child nodes.
  • the central node sends an index *a* and the global mean *a* (related to the first round) to the child nodes.
  • the child nodes calculate their local variance *a* based on the global mean *a* and report it to the central node.
  • the central node calculates the global variance *a* based on the local variances *a* of multiple child nodes and sends it to the child nodes.
  • the child nodes train their local models based on the global variance *a* and the global mean *a*. After local training is complete, the model parameters of the trained local model are reported to the central node.
  • the central node aggregates the model parameters of the local models reported by multiple child nodes to generate a new global model.
  • the central node then sends the parameters of the new global model to the child nodes, thus entering the second round.
  • the central node calculates the global mean *b* based on the local means *b* of multiple child nodes.
  • the central node then sends the index *b* and the global mean *b* to the child nodes.
  • the index *b* is relevant to the second round.
  • Each child node calculates its local variance *b* based on the global mean *b* and reports it to the central node.
  • the central node calculates the global variance *b* based on the local variances *b* of multiple child nodes and sends it to the child nodes.
  • index *a* is the index related to the global mean *a*
  • index *b* is the index related to the global mean *b*.
  • Index *a* can be considered an index unrelated to the global mean *b*.
  • index #1 The correlation between index #1 and global mean #1 can be demonstrated in several ways.
  • index #1 and global mean #1 can be carried in the same signaling. That is, the first information can be carried in the same signaling.
  • the index that carries the same signaling as the global mean #1 is the index #1 associated with the global mean #1.
  • the indices related to the multiple global mean values can be indices that have a preset positional relationship with the positions of the multiple global mean values.
  • index #1 could be an index that is following and adjacent to the global mean #1.
  • index #1 and global mean #1 can be carried in two separate signaling messages. That is, the first information can be carried in two signaling messages. These two signaling messages are related. After receiving these two signaling messages, the child node calculates its local difference #1 based on the global mean #1.
  • index #1 The information indicated by index #1 will be explained below based on two examples (Example #1 and Example #2).
  • index #1 could indicate the training epoch associated with the global mean #1.
  • the training epochs related to the global mean #1 are the epochs at the central node.
  • index #1 can indicate the identifier of the training round associated with the global mean #1.
  • index #1 could include the training round number associated with the global mean #1.
  • child node #1 After receiving information #1, child node #1 can determine in which round the global mean #1 was calculated based on index #1, and then calculate the local difference #1 based on the global mean #1 in that round.
  • index #1 can indicate the identifier of the AI model corresponding to the global mean #1.
  • Index #1 can indicate the training rounds associated with the global mean #1 by indicating the identifier of the AI model.
  • the identifier of an AI model can be its version number.
  • the current AI model can be assigned a different identifier than the AI models in previous rounds, such as setting a new version number.
  • the version number of the AI model in the first round could be V1.1
  • the version number of the AI model in the second round could be V1.2
  • each time the central node aggregates and generates a new global model it can assign a different identifier to the new global model than the previous global models, such as setting a new version number.
  • the initial global model version number could be V1.0
  • the version number of the global model after the first update could be V1.1, and so on.
  • the central node can distribute the model parameters and identifier of the new global model to each child node.
  • the child nodes update their local models based on the model parameters of the new global model, and correspondingly, the identifier of the new global model serves as the identifier of the updated local model.
  • Child node #1 can determine which version of the AI model corresponds to the global mean #1 based on the identifier of the AI model corresponding to the global mean #1, that is, determine in which round the global mean #1 was calculated.
  • index #1 can indicate network layer #1. That is, index #1 can indicate the network layer corresponding to the global mean #1.
  • index #1 may include an identifier for network layer #1.
  • the identifier for a network layer may be a layer number or an index of the network layer.
  • the layer number may be an integer.
  • Child node #1 can determine which network layer the global mean #1 is associated with based on index #1, thereby determining that the global mean #1 is used to calculate the variance of the input of that network layer in the AI model of child node #1.
  • network layer #1 can also be indicated in other ways.
  • the position of the information indicating the global mean #1 in information #1 can be used to indicate network layer #1.
  • the AI model may include multiple batch normalization (BN) layers, and information #1 may include information indicating the global mean corresponding to these multiple BN layers.
  • the order of the fields indicating the global mean corresponding to these multiple BN layers is consistent with the order of these multiple BN layers in the AI model.
  • the fields indicating the global mean corresponding to the first BN layer to the fields indicating the global mean corresponding to the last BN layer are arranged in information #1 in either a front-to-back or back-to-front order.
  • Figure 6 shows an example of the information in information #1 that indicates the global mean.
  • an AI model includes three batch normalization (BN) layers, arranged from front to back in the AI model as BN layer L1, BN layer L2, and BN layer L3, with corresponding global means of global mean a, global mean a', and global mean a”, respectively.
  • Information #1 can include information indicating the global means corresponding to these three BN layers.
  • the global mean a, global mean a', and global mean a” are arranged in information #1 according to the order of the corresponding BN layers in the AI model.
  • the leftmost global mean is the global mean a corresponding to BN layer L1
  • the rightmost global mean is the global mean a corresponding to BN layer L3.
  • Example 1 and Example 2 can also be used together. That is, index #1 can indicate network layer #1 and the training round associated with the global mean #1.
  • index #1 can indicate the identifier of network layer #1 and the identifier of the training epoch associated with the global mean #1.
  • index #1 may include an identifier of network layer #1 and an identifier of the training epoch associated with the global mean #1.
  • index #1 may include two fields (also referred to as information elements) carrying the identifier of network layer #1 and the identifier of the training epoch associated with the global mean #1, respectively.
  • index #1 may include the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with the global mean #1.
  • index #1 may include a field (also referred to as a information element) that carries the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with the global mean #1.
  • an index value of 01 for index #1 can indicate that network layer #1 is the first BN layer, and the training epoch number associated with the global mean #1 is 1; an index value of 02 for index #1 can indicate that network layer #1 is the second BN layer, and the training epoch number associated with the global mean #1 is 1; an index value of 03 for index #1 can indicate that network layer #1 is the first BN layer, and the training epoch number associated with the global mean #1 is 2.
  • index #1 can indicate the identifier of network layer #1 and the identifier of the AI model corresponding to global mean #1.
  • the correlation between index #2 and local difference #1 can be understood as the correlation between index #2 and local difference #1 and the same round and/or the same network layer.
  • the central node calculates the global mean *a* based on the local means *a* of multiple child nodes.
  • the central node sends the global mean *a* to the child nodes, which then calculate their local variance *a* based on the global mean *a* and report the index *a* and local variance *a* to the central node.
  • the index *a* is relevant to the first round.
  • the central node calculates the global variance *a* based on the local variances *a* of multiple child nodes and sends it to the child nodes.
  • the child nodes train their local models based on the global variance *a* and the global mean *a*. After local training is complete, the child nodes report the model parameters of their trained local models to the central node.
  • the central node aggregates the model parameters of the local models reported by multiple child nodes to generate a new global model.
  • the central node then sends the parameters of the new global model to the child nodes, thus entering the second round.
  • the central node calculates the global mean *b* based on the local means *b* of multiple child nodes.
  • the central node sends the global mean *b* to the child nodes, which then calculate their local variance *b* based on the global mean *b* and report the index *b* and local variance *b* to the central node.
  • the index *b* is relevant to the second round.
  • the central node calculates the global variance *b* based on the local variances *b* of multiple child nodes and sends it to the child nodes.
  • index *a* is the index related to local variance *a*
  • index *b* is the index related to local variance *b*.
  • Index *a* can be considered an index unrelated to local variance *b*.
  • index #2 and local difference #1 can be carried in the same signaling. That is, the second information can be carried in the same signaling.
  • index #2 and local difference #1 can be carried in two separate signaling messages. That is, the second information can be carried in two related signaling messages.
  • index #1 For a detailed description, please refer to the correlation between index #1 and global mean #1 mentioned earlier; it will not be repeated here.
  • Example #3 The information indicated by index #2 will be explained below based on two examples (Example #3 and Example #4).
  • index #2 could indicate the training rounds associated with local difference #1.
  • the training rounds related to the local difference #1 are the rounds of the child nodes.
  • index #2 can indicate the identifier of the training rounds related to the local difference.
  • index #2 could include the number of the training rounds associated with local difference #1.
  • the central node After receiving information #2, the central node can determine in which round the local variance #1 was calculated based on index #2, and then calculate the global variance #1 based on the local variance #1 in that round.
  • index #2 can indicate the identifier of the AI model corresponding to local difference #1.
  • Index #2 can indicate the training rounds associated with local difference #1 by indicating the identifier of the AI model corresponding to local difference #1.
  • the local difference #1 corresponds to the AI model, that is, the local difference #1 is calculated based on this AI model.
  • the identifier of an AI model can be its version number.
  • the central node determines which version of the AI model corresponds to the local difference #1 based on the version number of the AI model corresponding to the local difference #1, that is, in which round the local difference #1 was calculated.
  • index #2 can indicate network layer #1. That is, index #2 can indicate the network layer corresponding to local variance #1.
  • index #2 may include the identifier of network layer #1.
  • the central node can determine which network layer its local variance #1 is associated with based on index #2, and thus determine that the local variance #1 is used to calculate the global variance corresponding to that network layer.
  • network layer #1 can also be indicated in other ways.
  • the location of the information in information #2 that indicates the local difference #1 can be used to indicate network layer #1.
  • index #2 can indicate network layer #1 and the training rounds associated with local difference #1.
  • index #2 may indicate the identifier of network layer #1 and the identifier of the training epoch associated with local difference #1.
  • index #2 may include the identifier of network layer #1 and the identifier of the training epoch associated with local geolocation #1.
  • index #2 may include two fields (also called information elements) carrying the identifier of network layer #1 and the identifier of the training epoch associated with local geolocation #1, respectively.
  • index #2 may include the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with local locality #1.
  • index #2 may include a field (also called a information element) that carries the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with local locality #1.
  • index #2 can indicate the identifier of network layer #1 and the identifier of the AI model corresponding to local difference #1.
  • index #2 For a detailed description of index #2, please refer to index #1 above; it will not be repeated here.
  • index #3 and global variance #1 can be found in the previous descriptions of the correlation between index #1 and global mean #1, and the correlation between index #2 and local variance #1.
  • the description of index #3 can be found in the descriptions of index #1 and index #2, and will not be repeated here.
  • index #4 and local mean #1 can be found in the previous descriptions of the correlation between index #1 and global mean #1, and the correlation between index #2 and local difference #1.
  • the description of index #4 can be found in the descriptions of index #1 and index #2, and will not be repeated here.
  • index #1 The relationship between index #1 and index #2 is explained below.
  • index #1 and index #2 have the same index value.
  • index #1 and index #2 are related, meaning that index #1 and index #2 have the same index value.
  • index #1 and index #2 have a predefined association relationship.
  • association between index #1 and index #2 can be predefined.
  • index #1 and index #2 have a pre-configured association.
  • Method 500 may also include: the central node sending information #5 to multiple child nodes, where information #5 indicates the association between two indexes.
  • a relationship between two indexes can be established if the index values of the two indexes satisfy a preset condition #1.
  • This preset condition #1 can be predefined, or it can be indicated by information #5.
  • a correlation between two indexes can be established if the difference between their index values is 1. That is, if the difference between the index values of two indexes is 1, then the two indexes are related. In this case, index #1 and index #2 are related, meaning the difference between their index values is 1. For instance, after a child node receives information 'a' and its associated index 'a', when sending information 'b' related to 'a', it can increment the index value of index 'a' by 1 to obtain the index value of index 'b', and then send information 'b' and index 'b'.
  • the central node When the central node receives information 'b' and index 'b', it can determine that index 'a' and index 'b' are related based on the fact that the difference between index 'b' and index 'a' is 1, thus determining that information 'b' is related to information 'a'.
  • Figure 7 illustrates a format diagram of an association relationship according to an embodiment of this application.
  • this format can be distributed from the central node to the child nodes.
  • the format can be predefined.
  • Child nodes can determine whether the mean and variance are related based on this format. For example, as shown in Figure 7, a child node can fill in the training epoch number associated with the current local model at the "Empoch Index Number t of the Model" position. Upon receiving the global mean #1 and index #1, the child node can fill in the training epoch number associated with the global mean #1 indicated by index #1 and the layer number of the network layer #1 corresponding to the global mean #1 at the "Empoch Index Number t' of the Mean" and "Layer Number l of the Mean” positions, respectively.
  • the training epoch number associated with the global mean #1 is related to the epoch number of the current local model, such as whether they are the same.
  • a child node receives global variance #1 and index #3, it can fill in the "round index number t of variance” and the "layer number l' of variance” positions with the training round number related to global variance #1 indicated by index #3 and the layer number of network layer #1 corresponding to global variance #1, respectively.
  • round index number t of variance and “round index number t' of mean”, as well as “layer number l' of variance” and “layer number l of mean”, it can be determined whether the received global variance #1 and global mean #1 are related. For example, if they are exactly the same, it means that the two are related; otherwise, they are not related.
  • index #3 is associated with at least one of the following: index #1 or index #2.
  • index #4 is associated with at least one of the following: index #1, index #2, or index #3.
  • index #1 and index #2 For a description of the relationship between the two indexes, please refer to index #1 and index #2; it will not be repeated here.
  • method 500 may also include: child node #1 receiving model parameters from the AI model of the central node.
  • each child node After receiving the model parameters of the AI model from the central node, each child node can update its local model based on the model parameters of the AI model.
  • the updated model parameters of the local model are the model parameters of the AI model issued by the central node.
  • the updated local model is the AI model issued by the central node.
  • model parameters of the AI model can be the model parameters of the AI model used to calculate the local mean in step 540.
  • each child node can calculate the average input of network layer #1 of the AI model issued by the central node based on its local training data and report it to the central node.
  • each child node can calculate the average input of network layer #1 based on the latest global model issued by the central node before starting local training, instead of calculating the average input of network layer #1 after the local model parameters have changed after local training has started.
  • each sub-node before the start of this round of training, can calculate the mean of the input of the BN layer based on the model parameters of the received AI model. This is beneficial to ensure that each sub-node uses a unified AI model to calculate the mean of the input of the BN layer. In other words, it is beneficial to ensure that each sub-node uses a unified AI model when calculating the mean of the input of the BN layer of its local model, thereby improving the accuracy of the global mean and global variance.
  • model parameters of the AI model can be the model parameters of the AI model used to calculate the local difference in step 510, or the model parameters of the AI model can be the model parameters of the AI model corresponding to the global mean #1.
  • each child node can calculate the variance of the input to network layer #1 of the AI model issued by the central node based on its local training data and the global mean #1, and report it to the central node.
  • each child node can calculate the variance of the input to network layer #1 based on its local training data and the latest global model issued by the central node before starting local training, instead of calculating the variance of the input to network layer #1 after the parameters of the local model have changed after local training has started.
  • each sub-node before the start of this round of training, can calculate the variance of the input of the BN layer based on the model parameters of the received AI model. This helps to ensure that each sub-node uses a unified AI model to calculate the variance of the input of the BN layer. In other words, it helps to ensure that each sub-node uses a unified AI model when calculating the variance of the input of the BN layer of its local model, thereby improving the accuracy of the global variance.
  • the central node and/or child nodes can determine whether the AI model on the child node is consistent with the global model on the central node.
  • the following explanation uses Scheme #1 and Scheme #2 as examples.
  • the child nodes can determine whether the local model is consistent with the global model on the central node.
  • child node #1 can receive information from the central node, which can be used to determine whether child node #1 needs to obtain the model parameters of the AI model from the central node.
  • the AI model can be the latest global model, and this information can be used to determine whether the AI model on child node #1 is the latest global model.
  • index #1 can be used to determine whether the AI model on child node #1 is the AI model corresponding to the global mean #1, or whether it is the latest global model.
  • the AI model corresponding to the global mean #1 is usually the latest global model.
  • index #1 can be used to indicate the training rounds associated with the global mean #1.
  • Child node #1 can determine whether the local model is the latest global model based on whether the round indicated by index #1 is the same as the round of child node #1.
  • index #1 could include the training epoch number associated with the global mean #1. If the epoch number indicated by index #1 is greater than the epoch number of child node #1, it means that child node #1 has not received the new global model from the central node. Thus, child node #1 can determine whether its local model is the latest global model based on the epoch number indicated by index #1 and the epoch number of child node #1.
  • index #1 can be used to indicate the AI model corresponding to the global mean #1.
  • Child node #1 can determine whether the local model is the latest global model based on whether the version of the AI model indicated by index #1 is the same as the version of the local model.
  • index #1 can include the version number of the AI model corresponding to the global mean #1. If the version number of the local model is different from the version number of the AI model indicated by index #1, it means that the child node #1 has not received the new global model issued by the central node. Therefore, the child node #1 can determine whether the local model is the latest global model based on the version number of the AI model indicated by index #1 and the version number of the local model.
  • the central node can also send other information so that child node #1 can determine whether the local model is the latest global model.
  • method 500 may also include: child node #1 sending message #6 (an example of third message) to the central node.
  • message #6 is used to request model parameters of the AI model.
  • Child node #1 determines that its local model is not the AI model corresponding to the global mean #1, child node #1 can request the model parameters of that AI model from the central node, and then update its local model based on those parameters. In this way, child node #1 can calculate its local variance #1 based on the AI model corresponding to the global mean #1.
  • child node #1 determines that its local model is not the latest global model, child node #1 can request the model parameters of the latest global model from the central node, and then update its local model based on those parameters. In this way, child node #1 can calculate its local variance #1 based on the latest global model.
  • index #1 which includes the training round numbers related to the global mean #1, as an illustration.
  • child node #1 sends information #6 to the central node.
  • Child node #1 After sending message #6, child node #1 receives the model parameters of the AI model sent by the central node. Child node #1 updates its local model based on these model parameters, that is, it updates the model parameters of the local model to the model parameters of the AI model sent by the central node, and then calculates the local difference #1 based on the updated local model.
  • child node #1 receives information #1 from the central node.
  • the index #1 in this information #1 indicates round number 2, meaning the global mean #1 indicated by this information #1 is the global mean #1 in the second round.
  • Child node #1's round number is 1, meaning the local model on the current child node #1 is based on the model parameters of the global model received in the first round.
  • Child node #1 can determine that this global mean #1 is used to calculate the local variance #1 in the second round, i.e., the local variance #1 is calculated based on the global model in the second round, and the current local model is not the global model in the second round.
  • Child node #1 can send information #6 to the central node to request the model parameters of the global model in the second round.
  • the central node can send the model parameters of the global model in the second round to child node #1.
  • Child node #1 updates its local model based on the model parameters of the global model in the second round and updates the round number of the training associated with the local model to 2.
  • Child node #1 calculates the local variance #1 based on the updated local model and the global mean #1.
  • child node #1 can calculate local difference #1 based on the current local model.
  • the central node can determine whether the local model is consistent with the global model on the central node.
  • child node #1 can send information to the central node, which can be used to determine whether the model parameters of the AI model need to be sent to child node #1.
  • the AI model can be the latest global model, and this information can be used to determine whether the AI model on child node #1 is the latest global model.
  • index #4 can be used to determine whether the AI model on child node #1 is the AI model corresponding to the global mean #1, or whether it is the latest global model.
  • index #4 can be used to indicate the training rounds associated with the local mean #1.
  • the central node can determine whether the local model on child node #1 is the latest global model based on whether the round indicated by index #4 is the same as the round of the central node.
  • index #4 could include the training epoch number associated with the local mean #1. If the epoch number indicated by index #4 is less than the epoch number of the central node, it means that child node #1 has not received the new global model from the central node. Therefore, the central node can determine whether the local model on child node #1 is the latest global model based on the epoch number indicated by index #4 and the epoch number of the central node.
  • index #4 can be used to indicate the AI model corresponding to the local mean #1.
  • the central node can determine whether the local model of child node #1 is the latest global model based on whether the version of the AI model indicated by index #4 is the same as the version of the global model.
  • index #4 could include the version number of the AI model corresponding to the local mean #1. If the version number of the global model is the same as the version number of the AI model indicated by index #4, it means that child node #1 has not received the new global model from the central node. Therefore, the central node can determine whether the local model on child node #1 is the latest global model based on the version number of the AI model indicated by index #4 and the version number of the global model.
  • Child node #1 can also send other information so that the central node can determine whether the local model on child node #1 is the latest global model.
  • index #2 can be used to determine whether the AI model on child node #1 is the AI model corresponding to the global mean #1, or whether it is the latest global model.
  • index #2 can be used to indicate the training rounds associated with local difference #1.
  • index #2 can be used to indicate the AI model corresponding to local difference #1.
  • index #4 For specific judgment methods, please refer to the relevant description of index #4 in the previous text. Replace index #4 with index #2 and replace local mean #1 with local difference #1.
  • method 500 may include: the central node sending the model parameters of the AI model to child node #1 again.
  • the central node sent the model parameters of the AI model to the child node #1 twice, and the model parameters of the AI model were the same in both times.
  • the central node can send the model parameters of the latest global model to the child node, allowing child node #1 to update its local model based on the latest global model parameters.
  • index #4 which includes the round number related to the local mean #1.
  • the central node sends the model parameters of the AI model to the child node #1 again.
  • the central node sent the same AI model parameters to the child node #1 before and after receiving information #4 from the child node #1.
  • Child node #1 After sending message #4, child node #1 receives the model parameters of the AI model sent by the central node. Child node #1 updates its local model based on these AI model parameters, that is, it updates the local model parameters to match the model parameters of the AI model sent by the central node, and then calculates the local local difference #1 based on the updated local model. In this way, child node #1 can calculate the local local difference #1 based on the latest global model.
  • the central node receives information #4 from child node #1.
  • the index #1 in information #4 indicates round number 2, meaning the local mean #1 indicated by information #4 is the local mean #1 from round two.
  • the training round number associated with the global model is 3, meaning the current global model is the global model obtained from the aggregation in round three.
  • the central node can determine that this local mean #1 was used to calculate the global mean #1 in round two, and that the current global model is not the global model from round two.
  • the central node can then send the model parameters of the global model from round three back to child node #1.
  • Child node #1 updates its local model based on the model parameters of the global model from round three and updates the training round number associated with the local model to 3.
  • Child node #1 calculates the local variance #1 based on the updated local model and the global mean #1.
  • the central node can use the local mean #1 indicated by information #4 to calculate the global mean #1. In other words, the central node can ignore the local mean #1 indicated by information #4.
  • child node #1 can execute step 540 again after receiving the latest global model parameters after sending message #4.
  • the central node can calculate the global mean #1 based on the local mean #1 indicated by the resent message #4.
  • the central node may also calculate the global mean #1 without using the local mean #1 reported again by child node #1.
  • the central node may also use the local mean #1 indicated by the information #4 received before sending the latest global model parameters to the child node #1 again to calculate the global mean #1.
  • index #2 which includes the training round numbers related to the local difference #1, as an illustration.
  • the central node sends the model parameters of the AI model to the child node #1 again.
  • the central node sends the same AI model parameters to the child node #1 before and after receiving information #2 from the child node #1.
  • the central node can use the local variance #1 indicated by information #2 to calculate the global variance #1. In other words, the central node can ignore the local variance #1 indicated by information #2.
  • Child node #1 can update its local model based on the latest global model parameters and then calculate its local variance #1 based on the updated local model. In this way, child node #1 can calculate its local variance #1 based on the latest global model. Child node #1 can then send message #2 again. In this case, the central node can calculate the global variance #1 based on the local variance #1 indicated by the resent message #2. Alternatively, the central node may choose not to use the local variance #1 reported again by child node #1 to calculate the global variance #1.
  • the central node may also use the local variance #1 indicated by the information #2 received before sending the latest global model parameters to the child node #1 again to calculate the global variance #1.
  • Child node #1 can also use other information to enable the central node to determine whether the AI model on child node #1 is the latest global model. Furthermore, the central node can determine whether it needs to reissue the model parameters of the latest global model based on the determination result. The central node can determine whether the local mean #1 and/or local variance #1 reported by child node #1 are based on the latest global model, and based on this, determine whether to use the local mean #1 and/or local variance #1 reported by child node #1 to calculate the global mean #1 and/or global variance #1.
  • Global variance #1 and global mean #1 can be used to train AI models on child nodes.
  • child node #1 can perform one or more forward and backward propagations based on global variance #1 and global mean #1 to update the model parameters of the local model.
  • the specific training process can be found later and will not be described in detail here.
  • Figure 5 uses only two child nodes as an example and does not constitute a limitation on the number of child nodes in the embodiments of this application.
  • Figure 8 shows a schematic flowchart of a communication method according to an embodiment of this application.
  • Method 800 shown in Figure 8 can be considered a specific implementation of method 500.
  • the central node aggregates the parameters of the local models uploaded by each child node to generate a new global model.
  • the central node distributes the parameters of the new global model to each child node.
  • Child nodes can calculate the mean of the features of the BN layer based on the same model and upload the mean to the central node.
  • the central node aggregates the mean of the features of the BN layer uploaded by each child node to generate a global mean of the features of the BN layer and distributes the global mean to each child node.
  • the child nodes calculate the variance of the features of the BN layer based on the global mean and upload the variance to the central node.
  • the central node aggregates the variance of the features of the BN layer uploaded by each child node to generate a global variance of the features of the BN layer.
  • method 800 is mainly illustrated using the central node as a network device and the child nodes as terminal devices.
  • the central node can be a base station.
  • Network-side equipment can be network devices, modules within network devices (such as chips), software containing network device functions (such as control subsystems), or other devices communicating with the network devices, such as AI network elements (which can be servers, such as OTT devices or cloud servers), or a combination of network devices and other devices communicating with them.
  • Terminal-side equipment can be terminal devices, modules within terminal devices (such as chips), software containing terminal device functions (such as control subsystems), or other devices communicating with the terminal devices, such as AI network elements (which can be servers, such as OTT devices or cloud servers), or a combination of terminal devices and other devices communicating with them.
  • the central node can also be replaced with equipment other than those on the network device side, and child nodes can also be replaced with equipment other than those on the terminal device side.
  • Figure 8 uses only one child node as an example for illustration; the behavior of other child nodes can be referenced from this child node.
  • method 800 may include the following steps.
  • the central node sends model configuration information and model training information to the child nodes.
  • the central node can broadcast the same model configuration information and model training information to all child nodes.
  • the model configuration information may include at least one of the following: the type of the initial AI model, the structural information of the initial AI model, or the model parameters of the initial AI model.
  • the type of the initial AI model may include at least one of the following: DNN, CNN, or Transformer model.
  • the structural information of the initial AI model may include at least one of the following: the number of hidden layers in the DNN, the number of neurons in each layer or part of the layers, and the location of the activation function or batch normalization layer.
  • the type and structure information of the initial AI model can be represented by configuration text or by a code script that can be compiled to generate the corresponding initial AI model.
  • the model parameters of the initial AI model can be generated by the central node using a certain strategy.
  • the model parameters of the initial AI model can be randomly generated or pre-trained by the central node.
  • the model parameters of the initial AI model can also be obtained by the central node from other entities. These other entities can be other devices communicating with the central node, such as servers, OTT devices, or cloud servers.
  • model training information may include at least one of the following: the optimizer used by child nodes to perform gradient updates, regularization penalty terms (such as L2 penalty terms), initial learning rate, gradient update batch size, or data augmentation method.
  • regularization penalty terms such as L2 penalty terms
  • initial learning rate initial learning rate
  • gradient update batch size initial learning rate
  • data augmentation method data augmentation method
  • the optimizer can be any of the following: stochastic gradient descent (SGD), root mean square propagation (RMSprop), or adaptive momentum estimation (Adam).
  • SGD stochastic gradient descent
  • RMSprop root mean square propagation
  • Adam adaptive momentum estimation
  • the central node distributes model configuration information, which indicates the model parameters of the initial AI model.
  • the round number at the central node can be set to 1, indicating that the central node is entering the first round of distributed training. That is, the training round number related to the initial AI model is 1.
  • child nodes can also send the amount of local training sample data to the central node.
  • the central node can also send the format of the correlation between the mean and variance of the batch normalization layer to the child nodes.
  • the format for the relationship between the mean and variance of the batch normalization layer can also be predefined.
  • the child node After receiving the model configuration information, the child node can update its local model based on the model parameters of the initial AI model and set the local round number to 1, thus entering the first round of distributed training. That is, the round number related to the local model is 1.
  • the child node performs training of the local model.
  • the child node trains its local model based on its local training dataset to update the model's parameters. This update process is called the iterative process.
  • the local model's parameters may be updated once or multiple times.
  • the child node feeds back the updated model parameters of the local model to the central node.
  • a child node can send the updated local model parameters back to the central node if condition #1 is met.
  • Condition #1 can be understood as the upload condition for model parameters. After updating the model parameters of the local model, child node #1 can determine whether condition #1 is met. If it is, it will upload the updated model parameters of the local model.
  • condition #1 can be a condition related to the number of updates to the local model.
  • condition #1 could be that the number of updates t of the local model is n times a constant T.
  • T is a positive integer.
  • n is a positive integer.
  • T can be predefined, configured by the central node, or determined by the child nodes themselves.
  • n can be predefined, configured by the central node, or determined by the child nodes themselves.
  • condition #1 can be a condition related to the model parameters of the updated local model.
  • condition #1 could be that the difference between the updated local model parameters and the local model parameters before the start of the current training round is greater than or equal to threshold #1.
  • model parameters of the local model before the start of the current round of training are the same as the model parameters of the global model issued by the central node of the current round.
  • Threshold #1 can be predefined, configured by the central node, or determined by the child node itself.
  • condition #1 can be a termination condition for local model training in a round.
  • the termination condition is met, the training of the local model in this round can be terminated. That is, when the training of the local model terminates, the updated model parameters of the local model are fed back to the central node.
  • This termination condition can be predefined, configured by the central node (e.g., indicated by model training information), or determined by the child node itself.
  • the central node can send instruction messages to each child node to notify them to provide updated model parameters for their local model.
  • the child node After receiving this instruction, the child node can feed back the model parameters of the current local model to the central node.
  • the central node aggregates the model parameters fed back by the child nodes to update the global model.
  • the central node After receiving model parameters from one or more child nodes, the central node performs an aggregation operation to generate a new global model. For example, the aggregation operation can calculate the average of the model parameters from multiple child nodes. Alternatively, it can calculate a weighted average of the model parameters from multiple child nodes.
  • the one or more child nodes can be determined in several ways. For example, the receiving process can continue for a period of time before ending, and the model parameters received from the child nodes during this period can be used to aggregate and obtain the global model. Alternatively, the central node can end after receiving a certain number of model parameters from the child nodes. These model parameters can also be used to aggregate and obtain the global model.
  • the central node sends the updated global model parameters to the child nodes.
  • the round number at the central node is incremented by 1, that is, the round number of the training related to the updated global model is incremented by 1.
  • the central node can broadcast the updated global model parameters to its child nodes.
  • the child node updates the local model based on the received model parameters.
  • the local round number is incremented by 1, that is, the training round number related to the local model is incremented by 1.
  • the child node calculates the mean of the input to the local model's BN layer (such as network layer #1) (such as local mean #1).
  • the child node calculates the mean of the inputs to one or more BN layers in the local model.
  • the child node sends information to the central node indicating the mean of the input to the BN layer and the relevant index (such as index #4).
  • An index associated with a mean can be used to indicate the number of the training epoch associated with that mean and the identifier of the BN layer associated with that mean. Based on this index, it can be determined which epoch and which BN layer the mean corresponds to; or, based on this index, it can be determined which epoch and which BN layer the mean was used to calculate the global mean.
  • Figure 9 shows a schematic diagram of the format of Message #4.
  • Figure 9 only uses an AI model with two Batch Normalization (BN) layers as an example.
  • the mean values of the inputs to the two BN layers are mean b and mean b', respectively.
  • information #4 includes two means, namely mean b and mean b'.
  • the index adjacent to a mean after a mean is the index associated with that mean.
  • the index value associated with mean b includes the round number c and the layer number L1
  • the index value associated with mean b' includes the round number c and the layer number L2.
  • the central node obtains the global mean by aggregating the average values fed back by the child nodes.
  • the central node can determine the training round number related to the mean b as c based on information #4, and the layer number related to the mean b as L1. That is, the mean b is used to calculate the global mean corresponding to the BN layer with layer number L1 in round c.
  • the central node can aggregate the average input of each child node's BN layer based on the index to obtain the global average for each BN layer.
  • the central node sends information indicating the global mean (e.g., global mean #1) and related indexes (e.g., index #1) to the child nodes.
  • global mean e.g., global mean #1
  • index #1 e.g., index #1
  • An index associated with a global mean can be used to indicate the training epoch number associated with that global mean and the identifier of the BN layer associated with that global mean. Based on this index, it can be determined which epoch and which BN layer the global mean corresponds to; or, based on this index, it can be determined which epoch and which BN layer's input variance the global mean was used to calculate.
  • Figure 10 shows a schematic diagram of the format of information #1.
  • Figure 10 only uses an AI model with two BN layers as an example.
  • the global means corresponding to these two BN layers are global mean a and global mean a', respectively.
  • information #1 includes two means: the global mean a and the global mean a'.
  • the index adjacent to a given mean is the index associated with that mean.
  • the index value associated with the global mean a includes the round number c' and the layer number L1
  • the index value associated with the global mean b includes the round number c' and the layer number L2.
  • the child node determines whether it needs to obtain the latest global model parameters based on the index.
  • step 812 If yes, proceed to step 812; otherwise, proceed to step 815.
  • a child node can determine whether it needs to obtain the latest global model parameters based on the round number indicated by index #1. Taking Figures 9 and 10 as examples, the child node sends information #4 to the central node, where index #4 indicates the round number c, meaning the child node is in round c, and the training round number related to the child node's local model is c.
  • the central node sends information #1 to the child node, where index #1 indicates the round number c', meaning the central node is in round c', and the training round number related to the global model is c'. If c and c' are the same, the child node can determine that its current local model is the latest global model and proceeds to step 815. If c is less than c', the child node can determine that its current local model is not the latest global model and proceeds to step 812.
  • the child node sends a message (such as message #6) to the central node to request the model parameters of the AI model.
  • Child nodes can send messages to the central node to request the central node to send the latest global model parameters.
  • the central node sends the latest global model parameters to the child nodes.
  • the child node updates the local model based on the received model parameters.
  • the local round number is incremented by 1, that is, the training round number related to the local model is incremented by 1.
  • the child node calculates the variance of the input to the local model's BN layer based on the global mean (e.g., local variance #1).
  • a child node may perform step 815 if the number of the round indicated by the index associated with the global mean is equal to the number of the training round associated with the local model.
  • the child node may execute step 815 after receiving the model parameters from the central node after sending message #6.
  • the child node can determine the correlation between the global mean issued by the central node and the variance of the input of each BN layer in the local model based on index #1, and then calculate the variance of the input of each BN layer in the local model based on the global mean corresponding to each BN layer.
  • the central node can determine the training round number related to the global mean a as c' based on information #1, and the layer number related to the global mean a as L1. That is, the global mean a is used to calculate the variance of the input of the BN layer with layer number L1 in the local model in round c'.
  • steps 811 to 814 above are optional. If method 800 includes steps 811 to 814, it helps ensure that each child node can calculate its local locality based on the latest global model parameters in step 815, that is, it helps ensure that each child node calculates its local locality based on a unified AI model. If method 800 does not include steps 811 to 814, then the child nodes may calculate their local locality based on an outdated model, which is obtained from the global model issued in previous rounds.
  • the child node sends information indicating the variance of the input to the BN layer and related indexes (such as index #2) to the central node.
  • An index associated with a variance can be used to indicate the training epoch number associated with that variance and the identifier of the BN layer associated with that variance. Based on this index, it can be determined which epoch and which BN layer the variance corresponds to; or, based on this index, it can be determined which epoch and which BN layer the variance was used to calculate the global variance.
  • Figure 11 shows a schematic diagram of the format of Message #2.
  • Figure 11 uses an AI model with two Batch Normalization (BN) layers as an example.
  • the variances of the inputs to the two BN layers are variance d and variance d', respectively.
  • information #2 includes two variances, namely variance d and variance d'.
  • the index adjacent to a variance is the index associated with that variance.
  • the index value of the index associated with variance d includes the round number c” and the layer number L1
  • the index value of the index associated with variance d' includes the round number c” and the layer number L2.
  • the central node aggregates the variance fed back by the child nodes based on the index to obtain the global variance (e.g., global variance #1).
  • the central node can determine the training round number related to variance d as c” based on information #2, and the layer number related to variance d as L1. That is, variance d is used to calculate the global variance corresponding to the BN layer with layer number L1 in round c”.
  • the central node can aggregate the variance of the inputs of each BN layer of each child node according to the index to obtain the global variance corresponding to each BN layer.
  • the central node sends information indicating the global variance and related indexes (such as index #3) to the child nodes.
  • An index associated with a global variance can be used to indicate the number of the training epoch associated with that global variance and the identifier of the BN layer associated with that global variance. Based on this index, it can be determined which epoch and which BN layer the global variance corresponds to; or, in other words, it can be determined which BN layer in which epoch the global variance was used as input during training.
  • message #3 can be referenced from message #1, and will not be repeated here.
  • step 802 Return to step 802 and repeat steps 802 to 818 until the global model training is complete, for example, until the global model converges, such as when the model performance meets the requirements.
  • step 802 after receiving the global mean and global variance, the child node can train the local model based on the global mean and global variance.
  • the following is an illustrative example of the training process when the global mean and global variance are received.
  • the training process can be divided into two phases: Phase 1 and Phase 2.
  • Phase 1 the child node can use the global mean and global variance corresponding to each BN layer to perform one or more forward and backward propagation to update the model parameters of the local model.
  • Equation (2) the mean and variance in Equation (2) are replaced with the global mean and global variance respectively to calculate the input of each BN layer, and based on this, one or more forward and backward propagation are performed to update the model parameters of the local model.
  • the mean and variance of the input of each BN layer can be statistically analyzed based on the local model updated in Phase 1.
  • a batch of training data is input into the updated local model to obtain the input of the BN layer based on the batch of training data, and then the mean and variance of the input of the BN layer are calculated.
  • the child node can perform one or more forward and backward propagation based on the statistically obtained mean and variance of the input of each BN layer to update the parameters of the local model.
  • the output of each BN layer can be calculated using formula (2), and one or more forward and backward propagation updates can be performed based on this to update the model parameters of the local model. Steps 803 to 818 are then performed after this training process.
  • Figure 12 is a schematic flowchart of another communication method provided in this application.
  • the method 1200 shown in Figure 12 can be applied to distributed training scenarios.
  • the central node aggregates the mean of the inputs to the BN layer fed back by the child nodes and the mean of the squares of the inputs to the BN layer, and obtains the global variance corresponding to the BN layer based on the aggregation result.
  • method 1200 may include the following steps.
  • message #7 (an example of the seventh message) to the central node.
  • Message #7 includes information about the local mean #2 (an example of the third mean) and the local mean #3 (an example of the fourth mean).
  • the local mean #2 is the input to network layer #1 in the AI model on the child node, and the local mean #3 is the mean of the squares of that input.
  • the description of network layer #1 in method 1200 can be referred to the description of network layer #1 in method 500, and the description of local mean #2 in method 1200 can be referred to the description of local mean #1 in method 500. It will not be repeated in method 1200.
  • Local mean #2 and local mean #3 can be used to calculate global variance #2 (an example of third variance).
  • the information #7 from multiple child nodes includes information indicating the local average #2 and local average #3 of those child nodes.
  • the local average #2 and local average #3 indicated by the information #7 from multiple child nodes are respectively the local average #2 and local average #3 of those child nodes.
  • the multiple child nodes in step 1210 are some or all of the aforementioned M child nodes.
  • Child node #1 can include child node #1.
  • child node #1 sends information #7 to the central node.
  • the local average #2 indicated by information #7 from child node #1 is the local average #2 of child node #1.
  • the local average #3 indicated by information #7 from child node #1 is the local average #3 of child node #1.
  • Multiple child nodes can also be replaced by multiple devices.
  • method 1200 may also include step 1220.
  • the central node determines the global variance #2 based on the local mean #2 and local mean #3 indicated by information #7 from one or more child nodes.
  • Global variance #2 is the global variance corresponding to network layer #1, or in other words, the global variance related to network layer #1.
  • the method for determining global variance #2 can be found in the description below.
  • One or more child nodes in step 1220 are part or all of the aforementioned M child nodes.
  • One or more child nodes in step 1220 belong to multiple child nodes in step 1210.
  • method 1200 may also include step 1230.
  • the central node sends message #8 (an example of the eighth message) to multiple child nodes.
  • message #8 includes information indicating the global mean #2 (an example of the fifth mean) and the global variance #2.
  • Global mean #2 is the global mean corresponding to network layer #1, or in other words, the global mean related to network layer #1. A description of global mean #2 can be found in Global Mean #1 in Method 500.
  • Global mean #2 and global variance #2 can be used to train AI models on child nodes.
  • the multiple child nodes in step 1230 are some or all of the aforementioned M child nodes.
  • the multiple child nodes in step 1230 can be the same as or different from the multiple child nodes in step 1210.
  • Child nodes can include child node #1.
  • child node #1 after receiving information #8 from the central node, child node #1 can train the local model based on the global mean #2 and the global variance #2.
  • the mean of the squares of the inputs to network layer #1 of the AI model on the child node is determined based on a batch of training data on that child node.
  • the data size of a batch of training data Bm on child node m is
  • inputs to network layer #1 The average of the squares of these
  • the central node can aggregate the local mean #2 of one or more child nodes to obtain the global mean #2.
  • the one or more child nodes can be some or all of the M child nodes, that is, the central node can determine the global mean #2 based on the local mean #2 of some or all child nodes.
  • the specific method for determining the global mean #2 can refer to the method for determining the global mean #1 in method 500. Replace the global mean #1 with the global mean #2, replace the local mean #1 with the local mean #2, and replace information #4 with information #7.
  • the central node can aggregate the local mean #2 and local mean #3 of one or more child nodes, and calculate the global variance #2 based on the aggregated results of local mean #2 and local mean #3.
  • the aggregated result of local mean #2 can be the global mean #2.
  • the one or more child nodes can be some or all of the M child nodes.
  • the central node aggregating the local mean #2 of one or more child nodes may include: the central node calculating the average of the local mean #2 of the multiple child nodes and using it as the aggregation result of the local mean #2.
  • the central node aggregating the local means #2 of one or more child nodes may include: the central node calculating a weighted average of the local means #2 of the multiple child nodes and using it as the aggregation result of the local means #2. For example, in the calculation of the weighted average, the weight corresponding to the local means #2 of each child node may be positively correlated with the amount of batch training data of each child node.
  • the central node aggregating the local mean #3 of one or more child nodes may include: the central node calculating the average of the local mean #3 of the multiple child nodes to obtain the aggregated result of the local mean #3.
  • the central node aggregating the local mean #3 of one or more child nodes may include: the central node calculating a weighted average of the local means #3 of the multiple child nodes and using this as the aggregation result of the local means #3. For example, in the calculation of the weighted average, the weight corresponding to the local mean #3 of each child node may be positively correlated with the amount of batch training data of each child node.
  • the global variance #2 can be calculated by combining the aggregated results of local mean #2 and local mean #3.
  • the global variance #2 satisfies the following formula:
  • the mean of the squares of the inputs to this BN layer which is the aggregated result of the local mean #3 indicated by information #7 from one or more child nodes.
  • E[h l (x)] represents the global mean corresponding to this BN layer, which is the global mean #2, which is the aggregated result of the local mean #2 indicated by information #7 from one or more child nodes.
  • B represents the training dataset on multiple child nodes.
  • the central node calculates the global variance #2 based on the mean of the input of the BN layer of the AI model on the child node and the mean of the square of the input. This helps to avoid the influence of the non-independent and identically distributed local data of each child node, so that the central node can aggregate to obtain a more accurate global variance #2, thereby improving the accuracy of batch normalization processing in the distributed training process and improving the performance of distributed training.
  • the number of interactions between child nodes and the central node is reduced, which helps to reduce interaction overhead.
  • information #7 also includes index #5 (an example of the fifth index).
  • Index #5 is associated with at least one of the following: local mean #2 or local mean #3.
  • the local mean #2 and local mean #3 can be determined and used to calculate the global variance #2.
  • index #5 and local mean #2 can be understood as both being related to the same round and/or the same network layer.
  • index #5 The correlation between index #5 and local mean #2 can be demonstrated in several ways.
  • index #5 and local average #2 can be carried in the same signaling. That is, information #7 can be carried in the same signaling.
  • index #5 and local average #2 can be carried in two separate signaling messages. That is, information #7 can be carried in two related signaling messages.
  • index #1 The information indicated by index #1 will be explained below based on two examples (Example #5 and Example #6).
  • index #5 could indicate the training rounds associated with local mean #2 and/or local mean #3.
  • the central node After receiving information #7, the central node can determine in which round the local mean #2 and local mean #3 were calculated based on index #5, and then calculate the global variance #2 based on the local mean #2 and local mean #3 in that round.
  • index #5 may indicate the identifier of the training round associated with local mean #2 and/or local mean #3.
  • index #5 could include the training round number associated with local mean #2 and/or local mean #3.
  • index #5 may indicate the identifier of the AI model corresponding to local mean #2 and/or local mean #3.
  • Index #5 can indicate the training rounds associated with local mean #2 and/or local mean #3 by indicating the identifier of the AI model corresponding to local mean #2 and/or local mean #3.
  • the central node After receiving information #7, the central node can determine which version of the AI model the local mean #2 and local mean #3 correspond to based on index #5, that is, determine which round the local mean #2 and local mean #3 correspond to.
  • index #5 can indicate network layer #1. That is, index #5 can indicate the network layer corresponding to local mean #2 or local mean #3.
  • index #5 may include the identifier of network layer #5.
  • the central node can determine which network layer the local mean #2 and local mean #3 are associated with based on index #5, and thus determine which local mean #2 and local mean #3 are used to calculate the global variance corresponding to that network layer.
  • network layer #1 can also be indicated in other ways.
  • the location of the information in information #7 indicating local mean #2 or local mean #3 can be used to indicate network layer #1.
  • index #5 can indicate network layer #1, and the training round associated with local mean #2 or local mean #3.
  • index #5 may indicate the identifier of network layer #1, and the identifier of the training epoch associated with local mean #2 or local mean #3.
  • index #5 may include an identifier for network layer #1 and an identifier for the training epoch associated with local mean #2 or local mean #3.
  • index #5 may include two fields (or information elements) carrying the identifier for network layer #1 and the identifier for the training epoch associated with local mean #2 or local mean #3, respectively.
  • index #5 may include the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with local mean #2 or local mean #3.
  • index #5 may include a field (or information element) carrying the joint encoding result of the identifier of network layer #1 and the identifier of the training epoch associated with local mean #2 or local mean #3.
  • index #5 can indicate the identifier of network layer #1 and the identifier of the AI model corresponding to local mean #2 or local mean #3.
  • information #8 also includes index #6 (an example of the sixth index).
  • Index #6 is associated with at least one of the following: global mean #2 or global variance #2.
  • index #5 and index #6 are related.
  • the global variance #2 is determined based on the local mean #2 and local mean #3.
  • index #5 The relationship between index #5 and index #6 is explained below.
  • index #5 and index #6 have the same index value.
  • index #5 and index #6 have a predefined association.
  • index #5 and index #6 have a pre-configured association.
  • Method 1200 may also include: the central node sending information #9 to multiple child nodes, where information #9 indicates the association between two indexes.
  • a relationship between two indexes can be established if the index values of the two indexes satisfy a preset condition #2.
  • This preset condition #2 can be predefined, or it can be indicated by information #9.
  • indexes #1 and #2 in method 500 please refer to indexes #1 and #2 in method 500, which will not be repeated here.
  • method 1200 may also include: child node #1 receiving model parameters from the AI model of the central node.
  • model parameters of the AI model can be the model parameters of the AI model used to calculate the local mean #2 and the local mean #3.
  • each child node can calculate the mean and the mean square of the input to network layer #1 of the AI model distributed by the central node based on its local training data, and then report this to the central node.
  • each child node can calculate the mean and the mean square of the input to network layer #1 based on the latest global model distributed by the central node before starting local training, instead of calculating the mean and the mean square of the input to network layer #1 after the parameters of the local model have changed since local training began.
  • each sub-node before the start of this round of training, can calculate the mean of the input to the BN layer and the mean of the square of the input based on the model parameters of the received AI model. This helps to ensure that each sub-node uses a unified AI model to calculate the mean of the input to the BN layer and the mean of the square of the input. In other words, it helps to ensure that each sub-node uses a unified AI model when calculating the mean of the input to the BN layer and the mean of the square of the input of its local model, thereby improving the accuracy of the global mean and global variance.
  • the central node and/or child nodes can determine whether the AI model on the child node is consistent with the global model on the central node.
  • the following explanation uses scheme #3 as an example.
  • the central node can determine whether the local model is consistent with the global model on the central node.
  • child node #1 can send information to the central node, which can be used to determine whether the model parameters of the AI model need to be sent to child node #1.
  • the AI model can be the latest global model, and this information can be used to determine whether the AI model on child node #1 is the latest global model.
  • index #5 can be used to determine whether the AI model on child node #1 is the AI model corresponding to global mean #2 and/or global variance #2, or whether it is the latest global model.
  • index #5 can be used to indicate the training rounds associated with local mean #2 and/or local mean #3.
  • the central node can determine whether the local model on child node #1 is the latest global model based on whether the round indicated by index #5 is the same as the round of the central node.
  • index #5 can be used to indicate the AI model corresponding to local mean #2 and/or local mean #3.
  • the central node can determine whether the local model of child node #1 is the latest global model based on whether the version of the AI model indicated by index #5 is the same as the version of the global model.
  • Child node #1 can also send other information so that the central node can determine whether the local model on child node #1 is the latest global model.
  • method 500 may include: the central node sending the model parameters of the AI model to child node #1 again.
  • the central node sent the model parameters of the AI model to the child node #1 twice, and the model parameters of the AI model were the same in both times.
  • the central node can send the model parameters of the latest global model to the child node, so that child node #1 can update its local model based on the model parameters of the latest global model.
  • index #5 may include the training round number associated with local mean #2 and/or local mean #3. If the round number indicated by index #5 is less than the round number of the central node, the central node sends the model parameters of the AI model to the child node #1 again.
  • the central node sent the same AI model parameters to the child node #1 before and after receiving information #7 from the child node #1.
  • Child node #1 After sending message #7, child node #1 receives the model parameters of the AI model sent by the central node. Child node #1 updates its local model based on these AI model parameters, that is, it updates the local model's model parameters to the model parameters of the AI model sent by the central node.
  • the central node can calculate the global mean #2 and/or global variance #2 without using the local mean #2 and local mean #3 indicated by information #7. In other words, the central node can ignore information #7 from child node #1.
  • child node #1 can calculate the local mean #2 and local mean #3 based on the latest global model parameters and then execute step 1210 again.
  • the central node can calculate the global mean #2 and/or global variance #2 based on the local mean #2 and local mean #3 indicated by the resent message #7.
  • the central node may also calculate the global mean #2 and/or global variance #2 without using the local mean #2 and local mean #3 reported again by child node #1.
  • the central node may also use the local mean #2 and local mean #3 indicated by the information #7 received before sending the latest global model parameters to the child node #1 again.
  • Child node #1 can also use other information to enable the central node to determine whether the AI model on child node #1 is the latest global model. Furthermore, the central node can determine whether it needs to reissue the model parameters of the latest global model based on the determination result. The central node can also determine whether the local mean #2 and local mean #3 reported by child node #1 are based on the latest global model, and based on this, determine whether to use the local mean #2 and local mean #3 reported by child node #1 to calculate the global mean #1 and global variance #1.
  • the child node can be used to determine whether the local model is the latest global model.
  • index #6 can be used to determine whether the AI model on child node #1 is the AI model corresponding to the global mean #2 and/or global variance #2, or whether it is the latest global model.
  • Global variance #2 and global mean #2 can be used to train AI models on child nodes.
  • child node #1 can perform one or more forward and backward propagations based on global variance #2 and global mean #2 to update the model parameters of the local model.
  • the specific training process can be found above and will not be repeated here.
  • Figure 13 shows a schematic flowchart of a communication method according to an embodiment of this application.
  • Method 1300 shown in Figure 13 can be considered a specific implementation of method 1200.
  • the central node aggregates the parameters of the local models uploaded by each child node to generate a new global model.
  • the central node distributes the parameters of the new global model to each child node.
  • the child nodes can calculate the mean of the BN layer features and the mean of the squares of the BN layer features based on the same global model, and upload them to the central node.
  • the central node aggregates the mean of the BN layer features uploaded by each child node to generate a global mean of the BN layer features.
  • the central node calculates the global variance of the BN layer features based on the global mean and the mean of the squares of the BN layer features uploaded by each child node.
  • method 1300 is mainly illustrated using the example of the central node being a network device and the child nodes being terminal devices.
  • the central node can be replaced with a device other than the network device, and the child nodes can also be replaced with devices other than the terminal device. See the previous text for details; they will not be repeated here.
  • Figure 13 uses one child node as an example for illustration; the behavior of other child nodes can be referenced from this child node.
  • method 1300 may include the following steps.
  • the descriptions of steps 1301 to 1306 are consistent with those of steps 801 to 806, and specific descriptions can be found in steps 801 to 806.
  • the central node sends model configuration information and model training information to the child nodes.
  • the child node performs training of the local model.
  • the child node feeds back the updated model parameters of the local model to the central node.
  • the central node aggregates the model parameters fed back by the child nodes to update the global model.
  • the central node sends the updated global model parameters to the child nodes.
  • the child node updates the local model based on the received model parameters.
  • the child node calculates the mean of the input to the local model's BN layer (e.g., network layer #1) (e.g., local mean #2) and the mean of the square of that input (e.g., local mean #3).
  • the local model's BN layer e.g., network layer #1
  • the mean of the square of that input e.g., local mean #3.
  • the child node calculates the mean of the inputs to one or more BN layers in the local model, as well as the mean of the squares of those inputs.
  • the child node sends information to the central node indicating the mean of the input to the BN layer, information indicating the mean of the square of the input, and the relevant index (such as index #5).
  • the central node determines the global mean (e.g., global mean #2) and global variance (global variance #2) by aggregating the mean of the inputs to the BN layer fed back by the index aggregator child nodes and the mean of the square of the inputs.
  • global mean e.g., global mean #2
  • global variance global variance #2
  • the central node can aggregate the mean of the inputs of each BN layer from each child node based on the index to obtain the global mean for each BN layer.
  • the central node can also aggregate the mean of the squares of the inputs of each BN layer from each child node based on the index to obtain the aggregated mean of the squares of the inputs of each BN layer.
  • the central node determines the global variance based on the aggregated result of the global mean for each BN layer and the mean of the squares of the inputs of each BN layer.
  • the central node sends information indicating the global mean and global variance, along with relevant indices (such as index #6), to the child nodes.
  • step 1302 after receiving the global mean and global variance, the child node can train its local model based on the global mean and global variance.
  • the training process please refer to step 802 above.
  • this application also provides corresponding apparatuses, which include modules for executing the methods described above. These modules can be software, hardware, or a combination of both. It is understood that the technical features described in the above method embodiments are also applicable to the following apparatus embodiments.
  • FIG 14 is a schematic diagram of a communication device 1900 provided in an embodiment of this application.
  • the device 1900 includes a transceiver unit 1910 and a processing unit 1920.
  • the transceiver unit 1910 can be used to implement corresponding communication functions.
  • the transceiver unit 1910 can also be referred to as a communication interface or communication unit, etc.
  • the processing unit 1920 can be used to implement corresponding processing or control functions, such as configuring resources.
  • the device 1900 further includes a storage unit that can be used to store instructions and/or data.
  • the processing unit 1920 can read the instructions and/or data in the storage unit to enable the device to perform the operation of the device or network element in the foregoing method embodiments.
  • the device 1900 can be a central node, or a communication device applied to or used in conjunction with a central node to implement a communication method executed on the central node side; or, the device 1900 can be a child node, or a communication device applied to or used in conjunction with a child node to implement a communication method executed on the child node side.
  • the device 1900 When the device 1900 is applied to the central node, it can implement the steps or processes corresponding to those performed by the central node in the above method embodiments. Specifically, the transceiver unit 1910 can be used to perform transceiver-related operations of the central node in the above method embodiments, and the processing unit 1920 can be used to perform processing-related operations of the central node in the above method embodiments.
  • the device 1900 can implement the steps or processes corresponding to the child node executed in the above method embodiment.
  • the transceiver unit 1910 can be used to perform the transceiver-related operations of the child node in the above method embodiment
  • the processing unit 1920 can be used to perform the processing-related operations of the child node in the above method embodiment.
  • the device 1900 here is embodied in the form of a functional unit.
  • the term "unit” here can refer to an ASIC, electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, integrated logic circuitry, and/or other suitable components supporting the described functions.
  • a processor e.g., a shared processor, a proprietary processor, or a group processor, etc.
  • memory for executing one or more software or firmware programs, integrated logic circuitry, and/or other suitable components supporting the described functions.
  • the device 1900 may be specifically a sub-node in the above embodiments, used to execute the various processes and/or steps corresponding to the sub-nodes in the above method embodiments; or, the device 1900 may be specifically a central node in the above embodiments, used to execute the various processes and/or steps corresponding to the central node in the above method embodiments. To avoid repetition, further details are omitted here.
  • the apparatus 1900 of each of the above schemes has the function of implementing the corresponding steps performed by the devices (such as child nodes or central nodes) in the above methods.
  • the function can be implemented in hardware or by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the above functions; for example, a transceiver unit can be replaced by a transceiver (e.g., the transmitting unit in the transceiver unit can be replaced by a transmitter, and the receiving unit in the transceiver unit can be replaced by a receiver), and other units, such as processing units, can be replaced by processors, each executing the transceiver operations and related processing operations in the respective method embodiments.
  • the aforementioned transceiver unit 1910 can also be a transceiver circuit (e.g., it may include a receiving circuit and a transmitting circuit), and the processing unit 1920 can be a processing circuit.
  • the processing circuit may include one or more processors, or circuits in one or more processors used for processing or control functions, etc.
  • the device in Figure 14 can be a network element or device as described in the preceding embodiments, or it can be a chip or a chip system, such as a SoC.
  • the transceiver unit can be an input/output circuit or a communication interface; the processing unit is a processor, microprocessor, or integrated circuit integrated on the chip. No limitations are imposed here.
  • FIG 15 is a schematic diagram of another communication apparatus 2000 provided in an embodiment of this application.
  • the apparatus 2000 includes a processor 2010, which is used to execute computer programs or instructions stored in a memory 2020, or to read data/signaling stored in the memory 2020, to perform the methods in the above method embodiments.
  • a processor 2010 which is used to execute computer programs or instructions stored in a memory 2020, or to read data/signaling stored in the memory 2020, to perform the methods in the above method embodiments.
  • the device 2000 further includes a memory 2020 for storing computer programs or instructions and/or data.
  • the memory 2020 may be integrated with the processor 2010 or may be disposed separately.
  • the device 2000 further includes a transceiver circuit 2030 for receiving and/or transmitting signals.
  • a processor 2010 is used to control the transceiver circuit 2030 to receive and/or transmit signals.
  • the processor 2010 may also be replaced by a processing circuit.
  • the device 2000 can be a network element or device as described in the foregoing embodiments, or it can be a chip or chip system.
  • the transceiver circuit 2030 can be a transceiver.
  • the transceiver circuit 2030 can be an interface circuit or an input/output interface.
  • the device 2000 can be applied to a central node.
  • the device 2000 can be a central node itself, or it can be any device capable of supporting a central node and implementing the functions of the central node in any of the examples described above.
  • the device 2000 is used to implement the operations performed by the central node in the various method embodiments described above.
  • processor 2010 is used to execute computer programs or instructions stored in memory 2020 to implement the relevant operations of the central node in the various method embodiments above.
  • the device 2000 can be applied to child nodes.
  • the device 2000 can be a child node itself, or it can be a device capable of supporting child nodes and implementing the functions of child nodes in any of the examples mentioned above.
  • the device 2000 is used to implement the operations performed by child nodes in the various method embodiments described above.
  • processor 2010 is used to execute computer programs or instructions stored in memory 2020 to implement the relevant operations of the child nodes in the various method embodiments above.
  • processors mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), ASICs, field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor can be a microprocessor or any conventional processor.
  • Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.
  • Volatile memory can be random access memory (RAM).
  • RAM can be used as an external cache.
  • RAM includes the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous linked dynamic random access memory
  • DR RAM direct rambus RAM
  • the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component
  • the memory storage module
  • memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.
  • This application also provides a computer-readable storage medium storing computer instructions for implementing the methods executed by the communication device in the above-described method embodiments.
  • the computer program when executed by the computer, it enables the computer to implement the methods executed by the child nodes in the various embodiments of the above methods.
  • the computer program when executed by the computer, it enables the computer to implement the methods executed by the central node in the various embodiments of the above methods.
  • This application also provides a computer program product comprising instructions which, when executed by a computer, implement the methods performed by a device (such as a child node or a central node) in the above-described method embodiments.
  • This application also provides a communication system, including the aforementioned child nodes and central node.
  • the child nodes and central node can implement the communication methods shown in any of the foregoing examples.
  • system may also include devices that communicate with the aforementioned sub-nodes and/or central node.
  • the disclosed apparatus and methods can be implemented in other ways.
  • the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods.
  • multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of apparatus or units may be electrical, mechanical, or other forms.
  • implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof.
  • software When implemented using software, it can be implemented entirely or partially in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer can be a personal computer, a server, or a network device, etc.
  • the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another.
  • the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.
  • the available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
  • the aforementioned available media include, but are not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本申请提供了，一种通信的方法和通信装置，该方法包括：接收第一信息，第一信息包括第一索引和指示第一均值的信息，第一均值用于第一方差的计算，第一方差为AI模型中的第一网络层的输入的方差；发送第二信息，第二信息包括第二索引和指示第一方差的信息，第一索引和第二索引具有关联关系。本申请实施例的方案，有利于得到更准确的第一方差，以期使得能够基于更准确的第一方差聚合得到更准确的全局方差，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。

Description

通信的方法和通信装置
本申请要求在2024年06月05日提交中国国家知识产权局、申请号为202410725330.0的中国专利申请的优先权，发明名称为“通信的方法和通信装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信领域，并且，更具体地，涉及一种通信的方法和通信装置。
背景技术
在无线通信网络中，例如在移动通信网络中，网络支持的业务越来越多样，因此需要满足的需求越来越多样。为了支持多样化的业务，可以将人工智能(artificialintelligence，AI)技术引入无线通信网络中，从而实现网络智能化。大部分的机器学习(machine learning，ML)，例如，基于深度神经网络的学习算法，需要海量的数据来进行训练。通常采用集中式训练的方式对AI模型进行训练，即将AI模型和训练数据集集中在一个计算节点上，由该计算节点完成AI模型的训练。然而集中式训练存在一些问题。例如，集中式训练的训练数据通常来源于用户设备，即需要用户上传数据。上传这些数据可能需要较大的开销，同时，收集和使用这些数据也可能会侵犯用户的隐私。分布式训练可以解决上述问题。在分布式训练中，通常由多个子计算节点和一个中心计算节点共同完成训练。子计算节点可以基于本地的训练数据训练AI模型，通过上传AI模型的参数与中心计算节点协作完成训练。
然而，如何提高分布式训练的性能是一个亟待解决的问题。
发明内容
本申请提供一种通信的方法和通信装置，以期提升分布式训练过程中的批规范化的准确性，从而有利于提高分布式训练的性能。
第一方面，提供了一种通信的方法，可以由通信装置或应用于通信装置的模块(例如，芯片或电路等)执行，该通信装置可以为方法实施例中的子节点。子节点与中心节点可以协作完成AI模型的训练。
子节点可以为终端设备侧的设备或网络设备侧的设备。其中，终端设备侧可以包括终端设备或终端设备侧的AI实体中的至少一项。终端设备侧的AI实体可以为终端设备本身，也可以为服务于终端设备的AI实体，例如服务器，比如过顶(over the top，OTT)服务器或云端服务器。网络设备侧可以包括网络设备或网络设备侧的AI实体中的至少一项。网络设备侧的AI实体可以为网络设备本身，也可以为服务于网络设备的AI实体，例如无线接入网(radio access network，RAN)智能控制器(RAN intelligent controller，RIC)，操作维护管理(operation administration and maintenance，OAM)，或，服务器，比如OTT服务器或云端服务器。
该方法包括：接收第一信息，第一信息包括第一索引和指示第一均值的信息，第一均值用于第一方差的计算，第一方差为AI模型中的第一网络层的输入的方差；发送第二信息，第二信息包括第二索引和指示第一方差的信息，第一索引和第二索引具有关联关系。
在本申请实施例的方案中，子节点根据来自中心节点的第一均值计算与本地的AI模型中的网络层的输入的方差，即第一方差，而不是根据基于本地数据集统计得到的本地的AI模型中的网络层的输入的均值来计算第一方差，有利于避免各个子节点的本地数据非独立同分布的影响，得到更准确的第一方差，以期使得中心节点能够基于更准确的第一方差聚合得到更准确的全局方差，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。
同时，通过索引可以确定本地方差与全局均值之间的相关性，避免子节点基于不相关的全局均值计算本地方差，例如，避免子节点基于其他网络层对应的本地均值来计算第一网络层的输入的方差，或者，避免子节点基于其他回合的全局均值来计算本回合的第一网络层的输入的方差，从而有利于保证方差的准确性。
示例性地，第一网络层可以为AI模型中的任一BN层。示例性地，第一网络层的输入也可以替换为第一网络层的特征或特征通道。
可选地，该方法还可以包括：接收该AI模型的模型参数。
在本申请实施例的方案中，子节点在本轮训练开始之前，可以基于接收到的AI模型的模型参数计算第一网络层的输入的方差，有利于保证各个子节点使用统一的AI模型来计算第一网络层的输入的方差，或者说，有利于保证各个子节点计算本地模型的第一网络层的输入的方差时各个子节点的本地模型采用统一的AI模型，从而有利于提高聚合得到的全局方差的准确性。
结合第一方面，在第一方面的某些实现方式中，第一索引和第一均值相关。
第一索引和第一均值与同一个回合和/或同一个网络层相关。
一个回合可以理解为，从中心节点向各个子节点发送全局模型的模型参数开始到中心节点聚合生成新的全局模型结束。
示例性地，第一索引和第一均值承载于同一信令中。
结合第一方面，在第一方面的某些实现方式中，第一索引和第二索引的索引值相同，或者，第一索引和第二索引具有预定义或预配置的关联关系。
结合第一方面，在第一方面的某些实现方式中，第一索引指示第一网络层的标识。
结合第一方面，在第一方面的某些实现方式中，第一索引指示与第一均值相关的训练的回合。
在本申请实施例的方案中，第一索引可以指示第一网络层的标识，有利于避免子节点基于其他网络层对应的本地均值来计算第一网络层的输入的方差。第一索引可以指示与第一均值相关的训练的回合，有利于避免子节点基于其他回合的全局均值来计算本回合的网络层的输入的方差，从而有利于保证方差的准确性。
结合第一方面，在第一方面的某些实现方式中，方法还包括：发送第三信息，第三信息用于请求AI模型的模型参数。
可选地，发送第三信息，包括：在第一索引指示的回合与子节点处的回合不同的情况下，发送第三信息。
例如，第一索引的索引值包括与第一均值相关的训练的回合的编号。在第一索引指示的回合的编号大于子节点处的回合的编号的情况下，发送第三信息。
根据本申请实施例的方案，子节点可以向中心节点请求AI模型的模型参数，有利于保证各个子节点均是基于统一的AI模型来计算本地方差，从而有利于进一步提高聚合得到的全局方差的准确性。
结合第一方面，在第一方面的某些实现方式中，发送第二信息后，方法还包括：接收第四信息，第四信息包括第三索引和指示第二方差的信息，第一索引和第二索引中的至少一项与第三索引具有关联关系，第一均值和第二方差用于AI模型的训练。
可选地，第二方差可以是通过聚合来自多个子节点的第四信息所指示的第二方差得到的。例如，第二方差可以是来自多个子节点的第四信息所指示的第二方差的平均值。再如，第二方差可以是来自多个子节点的第四信息所指示的第二方差的加权平均值。
结合第一方面，在第一方面的某些实现方式中，接收第一信息之前，方法还包括：发送第五信息，第五信息包括第四索引和指示第二均值的信息，第二均值为AI模型中的第一网络层的输入的均值，第一索引和第二索引中的至少一项与第四索引具有关联关系。
可选地，第一均值可以是通过聚合来自多个子节点的第五信息所指示的第二均值得到的。例如，第一均值可以是来自多个子节点的第五信息所指示的第二均值的平均值。再如，第一均值可以是来自多个子节点的第五信息所指示的第二均值的加权平均值。
第二方面，提供了一种通信的方法，可以由通信装置或应用于通信装置的模块(例如，芯片或电路等)执行，该通信装置可以为方法实施例中的中心节点。中心节点可以和子节点协作完成AI模型的训练。
中心节点可以为终端设备侧的设备或网络设备侧的设备。其中，终端设备侧可以包括终端设备或终端设备侧的AI实体中的至少一项。终端设备侧的AI实体可以为终端设备本身，也可以为服务于终端设备的AI实体，例如服务器，比如OTT服务器或云端服务器。网络设备侧可以包括网络设备或网络设备侧的AI实体中的至少一项。网络设备侧的AI实体可以为网络设备本身，也可以为服务于网络设备的AI实体，例如RIC，OAM，或，服务器，比如OTT服务器或云端服务器。
该方法包括：向多个设备发送第一信息，第一信息包括第一索引和指示第一均值的信息，第一均值用于第一方差的计算，第一方差为多个设备中的每个设备的AI模型中的第一网络层的输入的方差；接收来自多个设备中的每个设备的第二信息，第二信息包括第二索引和指示第一方差的信息，第一索引和第二索引具有关联关系。
其中，第一信息的目的端所对应的多个设备和第二信息的源端所对应的多个设备可以相同，也可以不同。第一信息的目的端所对应的多个设备和第二信息的源端所对应的多个设备至少部分相同。例如，第一信息的目的端所对应的多个设备可以包括第二信息的源端所对应的多个设备。
在本申请实施例的方案中，中心节点可以向子节点下发第一均值，以使得子节点可以基于第一均值计算与本地的AI模型中的网络层的输入的方差，即第一方差，而不是根据基于子节点的本地数据集统计得到的本地的AI模型中的网络层的输入的均值来计算第一方差，有利于避免各个子节点的本地数据非独立同分布的影响，得到更准确的第一方差，以期使得中心节点能够基于更准确的第一方差聚合得到更准确的全局方差，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。结合第二方面，在第二方面的某些实现方式中，第一索引和第一均值相关。
结合第二方面，在第二方面的某些实现方式中，第一索引和第二索引的索引值相同，或者，第一索引和第二索引具有预定义或预配置的关联关系。
结合第二方面，在第二方面的某些实现方式中，第一索引指示第一网络层的标识。
结合第二方面，在第二方面的某些实现方式中，第一索引与第一均值相关的训练的回合。
结合第二方面，在第二方面的某些实现方式中，方法还包括：接收来自多个设备中的第一设备的第三信息，第三信息用于请求AI模型的模型参数。
结合第二方面，在第二方面的某些实现方式中，方法还包括：向多个设备发送第四信息，第四信息包括第三索引和指示第二方差的信息，第一索引和第二索引中的至少一项与第三索引具有关联关系，第一均值和第二方差用于AI模型的训练。
其中，第一信息的目的端所对应的多个设备和第四信息的目的端所对应的多个设备可以相同，也可以不同。第二信息的源端所对应的多个设备和第四信息的目的端所对应的多个设备可以相同，也可以不同。
结合第二方面，在第二方面的某些实现方式中，方法还包括：根据来自多个设备中的每个设备的第二信息所指示的第一方差确定第二方差。
例如，第二方差可以是来自多个设备中的每个设备的第四信息所指示的第二方差的平均值。
结合第二方面，在第二方面的某些实现方式中，第二方差满足：
其中，表示第二方差，M表示多个设备的数量，|Bj|表示多个设备中的第j个设备上的训练样本的数量，表示来自第j个设备的第二信息所指示的第一方差，M为正整数，|Bj|为正整数。
结合第二方面，在第二方面的某些实现方式中，在向多个设备发送第一信息之前，方法还包括：接收来自多个设备中的每个设备的第五信息，第五信息包括第四索引和指示第二均值的信息，第二均值为多个设备中每个设备的AI模型中的第一网络层的输入的均值，第一索引和第二索引中的至少一项与第四索引具有关联关系。
其中，第一信息的目的端所对应的多个设备和第五信息的源端所对应的多个设备可以相同，也可以不同。第二信息的源端所对应的多个设备和第五信息的源端所对应的多个设备可以相同，也可以不同。第四信息的目的端所对应的多个设备和第五信息的源端所对应的多个设备可以相同，也可以不同。
结合第二方面，在第二方面的某些实现方式中，方法还包括：基于多个设备中的每个设备的第五信息确定第一信息，其中，第一均值基于第二均值。
第三方面，提供了一种通信的方法，可以由通信装置或应用于通信装置的模块(例如，芯片或电路等)执行，该通信装置可以为方法实施例中的子节点。子节点和中心节点可以协作完成AI模型的训练。
子节点可以为终端设备侧的设备或网络设备侧的设备。其中，终端设备侧可以包括终端设备或终端设备侧的AI实体中的至少一项。终端设备侧的AI实体可以为终端设备本身，也可以为服务于终端设备的AI实体，例如服务器，比如过顶(over the top，OTT)服务器或云端服务器。网络设备侧可以包括网络设备或网络设备侧的AI实体中的至少一项。网络设备侧的AI实体可以为网络设备本身，也可以为服务于网络设备的AI实体，例如无线接入网(radio access network，RAN)智能控制器(RAN intelligent controller，RIC)，操作维护管理(operation administration and maintenance，OAM)，或，服务器，比如OTT服务器或云端服务器。
该方法包括：发送第七信息，第七信息包括指示第三均值和第四均值的信息，第三均值为AI模型中的第一网络层的输入的均值，第四均值为第一网络层的输入的平方的均值。
在本申请实施例的方案中，子节点将AI模型的第一网络层的输入的均值以及该输入的平方的均值发送至中心节点，以使得中心节点可以基于子节点上的AI模型的第一网络层的输入的均值和该输入的平方的均值来计算全局方差，有利于避免各个子节点的本地数据非独立同分布的影响，以期使得中心节点能够聚合得到更准确的全局方差，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。
同时，在本申请实施例中，子节点和中心节点交互次数更少，有利于减少交互开销。
可选地，该方法还可以包括：接收该AI模型的模型参数。
在本申请实施例的方案中，子节点在本轮训练开始之前，可以基于接收到的AI模型的模型参数计算第一网络层的输入的均值以及该输入的平方的均值，有利于保证各个子节点使用统一的AI模型来计算第三均值和第四均值，从而有利于提高基于第三均值和第四均值确定的全局均值和全局方差的准确性。
结合第三方面，在第三方面的某些实现方式中，方法还包括：接收第八信息，第八信息包括指示第五均值和第三方差的信息，第五均值和第三方差与第一网络层相关，第五均值和第三方差用于AI模型的训练。
可选地，第五均值可以是基于来自多个子节点的第七信息所指示的第三均值得到的。例如，第五均值为来自多个子节点的第七信息所指示的第三均值的平均值。再如，第五均值为来自多个子节点的第七信息所指示的第三均值的加权平均值。
结合第三方面，在第三方面的某些实现方式中，第三方差基于第三均值和第四均值。
结合第三方面，在第三方面的某些实现方式中，第七信息还包括第五索引，第八信息还包括第六索引，第五索引和第六索引具有关联关系。
结合第三方面，在第三方面的某些实现方式中，第三均值和第四均值中的至少一项和第五索引相关。
在本申请实施例的方案中，通过索引可以确定均值与方差之间的相关性，避免中心节点基于不相关的均值来计算全局方差，例如，避免中心节点基于其他网络层的输入的均值以及输入的平方的均值来计算第一网络层对应的全局均值和全局方差，或者，避免中心节点基于其他回合的第一网络层的输入的均值以及输入的平方的均值来计算本回合的第一网络层对应的全局均值和全局方差，从而有利于保证均值和方差的准确性。
结合第三方面，在第三方面的某些实现方式中，第五索引和第六索引的索引值相同，或者，第五索引和第六索引具有预配置或预定义的关联关系。
结合第三方面，在第三方面的某些实现方式中，第五索引指示第一网络层的标识。
结合第三方面，在第三方面的某些实现方式中，第五索引指示与第三均值和/或第四均值相关的训练的回合。
第四方面，提供了一种通信的方法，可以由通信装置或应用于通信装置的模块(例如，芯片或电路等)执行，该通信装置可以为方法实施例中的中心节点。
中心节点可以为终端设备侧的设备或网络设备侧的设备。其中，终端设备侧可以包括终端设备或终端设备侧的AI实体中的至少一项。终端设备侧的AI实体可以为终端设备本身，也可以为服务于终端设备的AI实体，例如服务器，比如OTT服务器或云端服务器。网络设备侧可以包括网络设备或网络设备侧的AI实体中的至少一项。网络设备侧的AI实体可以为网络设备本身，也可以为服务于网络设备的AI实体，例如RIC，OAM，或，服务器，比如OTT服务器或云端服务器。
该方法包括：接收来自多个设备中的每个设备的第七信息，第七信息包括指示第三均值和第四均值的信息，第三均值为多个设备中的每个设备的人工智能AI模型中的第一网络层的输入，第四均值为第一网络层的输入的平方的均值。
在本申请实施例的方案中，中心节点可以基于子节点上的AI模型的第一网络层的输入的均值和该输入的平方的均值来计算全局方差，有利于避免各个子节点的本地数据非独立同分布的影响，以期使得中心节点能够聚合得到更准确的全局方差，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。
同时，在本申请实施例中，子节点和中心节点交互次数更少，有利于减少交互开销。
结合第四方面，在第四方面的某些实现方式中，方法还包括：向多个设备发送第八信息，第八信息包括指示第五均值和第三方差的信息，第五均值和第三方差与第一网络层相关，第五均值和第三方差用于AI模型的训练。
其中，第七信息的源端所对应的多个设备和第八信息的目的端所对应的多个设备可以相同，也可以不同。
结合第四方面，在第四方面的某些实现方式中，方法还包括：根据来自多个设备中的每个设备的第七信息所指示的第三均值和第四均值确定第三方差。
结合第四方面，在第四方面的某些实现方式中，第三方差满足：
其中，表示第三方差，表示第六均值，第六均值为来自多个设备中的每个设备的第七信息所指示的第四均值的平均值，Ε[hl(x)]表示第五均值，第五均值为来自多个设备中的每个设备的第七信息所指示的第三均值的平均值，B表示多个设备上的训练数据集。
结合第四方面，在第四方面的某些实现方式中，第七信息还包括第五索引，第八信息还包括第六索引，第五索引和第六索引具有关联关系。
结合第四方面，在第四方面的某些实现方式中，第三均值和第四均值中的至少一项和第五索引相关。
结合第四方面，在第四方面的某些实现方式中，第五索引和第六索引的索引值相同，或者，第五索引和第六索引具有预配置或预定义的关联关系。
结合第四方面，在第四方面的某些实现方式中，第五索引指示第一网络层的标识。
结合第四方面，在第四方面的某些实现方式中，第五索引指示与第三均值和/或第四均值相关的训练的回合。
第五方面，提供了一种通信装置，该通信装置可以是终端设备，也可以是被配置设置于终端设备中的装置、模块、电路或芯片等，或者是能够和终端设备匹配使用的装置。一种设计中，该通信装置可以包括执行第一方面或第三方面所描述的方法/操作/步骤/动作所一一对应的模块，该模块可以是硬件电路，也可是软件，也可以是硬件电路结合软件实现。一种设计中，该通信装置可以包括处理模块和通信模块。
其中，发送模块用于执行如上第一方面或第三方面所描述方法中的发送动作，处理模块则用于执行如上第一方面或第三方面所描述方法中的涉及处理的动作，接收模块用于执行如上第一方面或第三方面所描述方法中的涉及接收的动作。
第六方面，提供了一种通信装置，该通信装置可以是网络设备，也可以是被配置设置于网络设备中的装置、模块、电路或芯片等，或者是能够和网络设备匹配使用的装置。一种设计中，该通信装置可以包括执行第二方面或第四方面所描述的方法/操作/步骤/动作所一一对应的模块，该模块可以是硬件电路，也可是软件，也可以是硬件电路结合软件实现。一种设计中，该通信装置可以包括处理模块和通信模块。
其中，接收模块用于执行如上第二方面或第四方面所描述方法中的接收动作，处理模块则用于执行如上第二方面或第四方面所描述方法中的涉及处理的动作，发送模块用于执行如上第二方面或第四方面所描述方法中的发送动作。
第七方面，提供一种通信装置，包括与一个或多个存储介质耦合的一个或多个处理器，该一个或多个存储介质存储有指令，该指令被一个或多个处理器运行时，以使得如第一方面或第一方面的任一可能的实现方式中的方法被实现，使得如第二方面或第二方面的任一可能的实现方式中的方法被实现，使得如第三方面或第三方面的任一可能的实现方式中的方法被实现，或者使得如第四方面或第四方面的任一可能的实现方式中的方法被实现。
第八方面，提供一种通信装置，包括一个或多个处理器，所述一个或多个处理器用于处理数据和/或信息，以使得如第一方面或第一方面的任一可能的实现方式中的方法被实现，使得如第二方面或第二方面的任一可能的实现方式中的方法被实现，使得如第三方面或第三方面的任一可能的实现方式中的方法被实现，或者使得如第四方面或第四方面的任一可能的实现方式中的方法被实现。
可选地，所述通信装置还可以包括通信接口，所述通信接口用于接收数据和/或信息，并将接收到的数据和/或信息传输至所述处理器。可选地，所述通信接口还用于输出经处理器处理之后的数据和/或信息。
第九方面，提供一种芯片，包括处理器，所述处理器用于运行程序或指令，以使得如第一方面或第一方面的任一可能的实现方式中的方法被实现，使得如第二方面或第二方面的任一可能的实现方式中的方法被实现，使得如第三方面或第三方面的任一可能的实现方式中的方法被实现，或者使得如第四方面或第四方面的任一可能的实现方式中的方法被实现。
可选地，所述芯片还可以包括存储器，所述存储器用于存储程序或指令。可选地，所述芯片还可以包括所述收发器。
可选地，所述芯片为专用集成电路(application specific integrated circuit，ASIC)或片上系统(system on chip，SoC)。
第十方面，提供一种计算机可读存储介质，所述计算机可读存储介质包括指令，当该指令被处理器运行时，使得如第一方面或第一方面的任一可能的实现方式中的方法被实现，使得如第二方面或第二方面的任一可能的实现方式中的方法被实现，使得如第三方面或第三方面的任一可能的实现方式中的方法被实现，或者使得如第四方面或第四方面的任一可能的实现方式中的方法被实现。
第十一方面，提供一种计算机程序产品，所述计算机程序产品包括计算机程序代码或指令，当所述计算机程序代码或指令被运行时，使得如第一方面或第一方面的任一可能的实现方式中的方法被实现，使得如第二方面或第二方面的任一可能的实现方式中的方法被实现，使得如第三方面或第三方面的任一可能的实现方式中的方法被实现，或者使得如第四方面或第四方面的任一可能的实现方式中的方法被实现。
第十二方面，提供一种通信系统，该通信系统包括以下装置中的一项或多项的组合：执行第一方面或第一方面的任一可能的实现方式中方法的通信装置，执行第二方面或第二方面的任一可能的实现方式中方法的通信装置，执行第三方面或第三方面的任一可能的实现方式中方法的通信装置，或者，执行第四方面或第四方面的任一可能的实现方式中方法的通信装置。例如，该通信系统可以包括第五方面提供的通信装置，和/或，第六方面提供的通信装置。
附图说明
图1是适用于本申请实施例的一种通信系统的示意图；
图2是适用于本申请实施例的另一种通信系统的示意图；
图3是适用于本申请实施例的又一种通信系统的示意图；
图4是适用于本申请实施例的通信系统的应用框架的示意图；
图5是本申请实施例提供的一种通信的方法的示意性流程图；
图6是本申请实施例提供的指示全局均值的信息的示意图；
图7是本申请实施例的一种关联关系的格式示意图；
图8是本申请实施例提供的另一种通信的方法的示意性流程图；
图9是本申请实施例的一个信息#4的示例的示意图；
图10是本申请实施例的一个信息#1的示例的示意图；
图11是本申请实施例的一个信息#2的示例的示意图；
图12是本申请实施例提供的又一种通信的方法的示意性流程图；
图13是本申请实施例提供的又一种通信的方法的示意性流程图；
图14是本申请实施例提供的一种通信的装置的示意性框图；
图15是本申请实施例提供的另一种通信的装置的示意性框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
本申请提供的技术方案可以应用于各种通信系统，例如：第五代(5th generation，5G)或新无线(new radio，NR)系统、长期演进(long term evolution，LTE)系统、LTE频分双工(frequency division duplex，FDD)系统、LTE时分双工(time division duplex，TDD)系统、无线局域网(wireless local area network，WLAN)系统、卫星通信系统、未来的通信系统，如未来通信网络移动通信系统，或者多种系统的融合系统等。本申请提供的技术方案还可以应用于设备到设备(device to device，D2D)通信，车到万物(vehicle-to-everything，V2X)通信，机器到机器(machine to machine，M2M)通信，机器类型通信(machine type communication，MTC)，以及物联网(internet of things，IoT)通信系统或者其他通信系统。
通信系统中的一个设备可以向另一个设备发送信号或从另一个设备接收信号。其中信号可以包括信息、信令或者数据等。其中，设备也可以被替换为实体、网络实体、网元、通信设备、通信模块、节点、通信节点等等，本公开中以设备为例进行描述。例如，通信系统可以包括至少一个终端设备和至少一个网络设备。在该通信系统中，网络设备可以向终端设备发送下行信号，终端设备可以向网络设备发送上行信号，网络设备可以向另一网络设备发送信号，终端设备可以向另一终端设备发送侧行信号。可以理解的是，本公开中的终端设备可以替换为第一设备，网络设备可以替换为第二设备，二者执行本公开中相应的通信方法。
在本申请实施例中，终端设备也可以称为用户设备(user equipment，UE)、接入终端、用户单元、用户站、移动站、移动台、远方站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置。
终端设备可以是一种提供语音/数据的设备，例如，具有无线连接功能的手持式设备、车载设备等。目前，一些终端的举例为：手机(mobile phone)、平板电脑、笔记本电脑、掌上电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程医疗中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端、蜂窝电话、无绳电话、会话启动协议(session initiation protocol，SIP)电话、无线本地环路(wireless local loop，WLL)站、个人数字助理(personal digital assistant，PDA)、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、可穿戴设备，5G网络中的终端设备或者未来演进的公用陆地移动通信网络(public land mobile network，PLMN)中的终端设备等，本申请实施例对此并不限定。
作为示例而非限定，在本申请实施例中，该终端设备还可以是可穿戴设备。可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。
本申请实施例中，用于实现终端设备的功能的装置可以是终端设备，也可以是能够支持终端设备实现该功能的装置，例如芯片系统，该装置可以被安装在终端设备中或者和终端设备匹配使用。本申请实施例中，芯片系统可以由芯片构成，也可以包括芯片和其他分立器件。在本申请实施例中仅以用于实现终端设备的功能的装置为终端设备为例进行说明，不对本申请实施例的方案构成限定。
本申请实施例中的网络设备可以包括用于与终端设备通信的设备，比如，该网络设备可以包括接入网设备或无线接入网设备，如该接入网设备可以是基站。本申请实施例中的无线接入网设备可以是指将终端设备接入到无线网络的RAN节点(或设备)。基站可以广义的覆盖如下中的各种名称，或与如下名称进行替换，比如：节点B(NodeB)、演进型基站(evolved NodeB，eNB)、下一代基站(next generation NodeB，gNB)、中继站、接入点、传输点(transmitting and receiving point，TRP)、发射点(transmitting point，TP)、主站、辅站、多制式无线(motor slide retainer，MSR)节点、家庭基站、网络控制器、接入节点、无线节点、接入点(access point，AP)、传输节点、收发节点、基带单元(baseband unit，BBU)、射频拉远单元(remote radio unit，RRU)、有源天线单元(active antenna unit，AAU)、射频头(remote radio head，RRH)、中心单元(central unit，CU)、分布式单元(distributed unit，DU)、射电单元(radio unit,RU)、定位节点等。基站可以是宏基站、微基站、中继节点、施主节点或类似物，或其组合。基站还可以指用于设置于前述设备或装置内的通信模块、调制解调器或芯片。基站还可以是移动交换中心以及D2D、V2X、M2M通信中承担基站功能的设备、未来通信网络中的网络侧设备、未来的通信系统中承担基站功能的设备等。基站可以支持相同或不同接入技术的网络。可选的，RAN节点还可以是服务器，可穿戴设备，车辆或车载设备等。例如，V2X技术中的接入网设备可以为路侧单元(road side unit，RSU)。本申请的实施例对网络设备所采用的具体技术和具体设备形态不做限定。
基站可以是固定的，也可以是移动的。例如，直升机或无人机可以被配置成充当移动基站，一个或多个小区可以根据该移动基站的位置移动。在其他示例中，直升机或无人机可以被配置成用作与另一基站通信的设备。
在一些部署中，本申请实施例所提及的网络设备可以为包括CU、或DU、或包括CU和DU的设备、或者控制面CU节点(中央单元控制面(central unit-control plane，CU-CP))和用户面CU节点(中央单元用户面(central unit-user plane，CU-UP))以及DU节点的设备。例如，网络设备可以包括gNB-CU-CP、gNB-CU-UP和gNB-DU。
在一些部署中，由多个RAN节点协作协助终端实现无线接入，不同RAN节点分别实现基站的部分功能。例如，RAN节点可以是CU，DU，CU-CP，CU-UP，或者RU等。CU和DU可以是单独设置，或者也可以包括在同一个网元中，例如BBU中。RU可以包括在射频设备或者射频单元中，例如包括在RRU、AAU或RRH中。
RAN节点可以支持一种或多种类型的前传接口，不同的前传接口分别对应具有不同功能的DU和RU。
若DU和RU之间的前传接口为通用公共无线电接口(common public radio interface，CPRI)，DU被配置用于实现基带功能中的一项或多项，RU被配置用于实现射频功能中的一项或多项。
若DU和RU之间的前传接口为另一种接口，其相对于CPRI，将下行和/或上行的部分基带功能，比如，针对下行，预编码(precoding)，数字波束赋形(beamforming，BF)，或快速傅立叶反变换(inverse fast fourier transform，IFFT)/添加循环前缀(cyclic prefix，CP)中的一项或多项，从DU中移至RU中实现，针对上行，数字波束赋形(beamforming，BF)，或快速傅立叶变换(fast fourier transform，FFT)/去除循环前缀(cyclic prefix，CP)中的一项或多项，从DU中移至RU中实现。
一种可能的实现方式，该接口可以为增强型通用公共无线电接口(enhanced common public radio interface，eCPRI)。在eCPRI架构下，DU和RU之间的切分方式不同，对应不同类型(category，Cat)的eCPRI，比如eCPRI Cat A，B，C，D，E，F。
以eCPRI Cat A为例，对于下行传输，以层映射为切分，DU被配置用于实现层映射及之前的一项或多项功能(即编码、速率匹配，加扰，调制，层映射中的一项或多项)，而层映射之后的其他功能(例如，资源元素(resource element,RE)映射，数字波束赋形(beamforming，BF)，或快速傅立叶反变换(inverse fast Fourier transform，IFFT)/添加循环前缀(cyclic prefix，CP)中的一项或多项)移至RU中实现。对于上行传输，以解RE映射为切分，DU被配置用于实现解映射及之前的一项或多项功能(即解码，解速率匹配，解扰，解调，离散傅里叶逆变换(inverse discrete Fourier transform，IDFT)，信道均衡，解RE映射中的一项或多项功能)，而解映射之后的其他功能(例如，数字BF或快速傅里叶变换(fast Fourier transform，FFT)/去CP中的一项或多项)移至RU中实现。可以理解的是，关于各种类型的eCPRI所对应的DU和RU的功能描述，可以参考eCPRI协议，在此不予赘述。
一种可能的设计中，BBU中用于实现基带功能的处理单元称为基带高层(base band high，BBH)单元，RRU/AAU/RRH中用于实现基带功能的处理单元称为基带低层(base band low，BBL)单元。
在不同系统中，CU(或CU-CP和CU-UP)、DU或RU也可以有不同的名称，但是本领域的技术人员可以理解其含义。例如，在开放式RAN(open RAN，ORAN)系统中，CU也可以称为O-CU(开放式CU)，DU也可以称为O-DU，CU-CP也可以称为O-CU-CP，CU-UP也可以称为O-CU-UP，RU也可以称为O-RU。本申请中的CU(或CU-CP、CU-UP)、DU和RU中的任一单元，可以是通过软件模块、硬件模块、或者软件模块与硬件模块结合来实现。
本申请实施例中，用于实现网络设备的功能的装置可以是网络设备；也可以是能够支持网络设备实现该功能的装置，例如芯片系统、硬件电路、软件模块、或硬件电路加软件模块。该装置可以被安装在网络设备中或者和网络设备匹配使用。在本申请实施例中仅以用于实现网络设备的功能的装置为网络设备为例进行说明，不对本申请实施例的方案构成限定。
网络设备和/或终端设备可以部署在陆地上，包括室内、室外、手持、和/或车载；也可以部署在水面(如轮船等)上；还可以部署在空中(如飞机、气球、和/或卫星)上。本申请实施例中对网络设备和终端设备所处的场景不做限定。
此外，终端设备和网络设备可以是硬件设备，也可以是在专用硬件上运行的软件功能，通用硬件上运行的软件功能，比如，是平台(例如，云平台)上实例化的虚拟化功能，又或者，是包括专用或通用硬件设备和软件功能的实体，本申请对于终端设备和网络设备的具体形态不作限定。
在无线通信网络中，例如在移动通信网络中，网络支持的业务越来越多样，因此需要满足的需求越来越多样。例如，网络需要能够支持超高速率、超低时延、和/或超大连接。该特点使得网络规划、网络配置、和/或资源调度越来越复杂。此外，由于网络的功能越来越强大，例如支持的频谱越来越高、支持高阶多入多出(multiple input multiple output，MIMO)技术、支持波束赋形、和/或支持波束管理等新技术，使得网络节能成为了热门研究课题。这些新需求、新场景和新特性给网络规划、运维和高效运营带来了前所未有的挑战。为了迎接该挑战，可以将人工智能技术引入无线通信网络中，从而实现网络智能化。
为了在无线网络中支持人工智能(artificialintelligence，AI)技术，网络中还可能引入AI节点(也可以称为AI实体)。
可选地，AI实体可以部署于该通信系统中的如下位置中的一项或多项：接入网络设备、终端设备、或核心网设备等，或者，AI实体也可单独部署，例如，部署于上述任一项设备之外的位置，比如，OTT系统的主机或云端服务器中。AI实体可以与通信系统中的其它设备通信，其它设备例如可以为以下中的一项或多项：网络设备，终端设备，或，核心网的网元等。基于该AI实体所服务的对象，AI实体可以包括网络设备侧的AI实体，终端设备侧的AI实体，或核心网侧的AI实体。
可以理解，本申请对于AI实体的数量不予限制。例如，当有多个AI实体时，多个AI实体可以基于功能进行划分，如不同的AI实体负责不同的功能。
还可以理解，AI实体可以是各自独立的设备，也可以集成于同一设备中实现不同的功能，或者可以是硬件设备中的网络元件，也可以是在专用硬件上运行的软件功能，或者是平台(例如，云平台)上实例化的虚拟化功能，本申请对于上述AI实体的具体形态不作限定。
AI实体可以为AI网元或AI模块。AI实体用以实现相应的AI功能。不同网元中部署的AI模块可以相同或不同。AI实体中的AI模型根据不同的参数配置，AI实体可以实现不同的功能。AI实体中的AI模型可以是基于以下一项或多项参数配置的：结构参数(例如神经网络层数、神经网络宽度、层间的连接关系、神经元的权值、神经元的激活函数、或激活函数中的偏置中的至少一项)、输入参数(例如输入参数的类型和/或输入参数的维度)、或输出参数(例如输出参数的类型和/或输出参数的维度)。其中，激活函数中的偏置还可以称为神经网络的偏置。
一个AI实体可以具有一个或多个模型。一个模型可以推理得到一个输出，该输出包括一个参数或者多个参数。不同模型的学习过程、训练过程、或推理过程可以部署在不同的实体或设备中，或者可以部署在相同的实体或设备中。
图1是适用于本申请实施例的通信方法的一种通信系统的示意图。如图1所示，通信系统100可以包括至少一个网络设备，例如图1所示的网络设备110；通信系统100还可以包括至少一个终端设备，例如图1所示的终端设备120和终端设备130。网络设备110与终端设备(如终端设备120和终端设备130)可通过无线链路通信。该通信系统中的各通信设备之间，例如，网络设备110与终端设备120之间，可通过多天线技术通信。
图2是适用于本申请实施例的通信方法的另一种通信系统的示意图。相较于图1所示的通信系统100而言，图2所示的通信系统200还包括AI网元140。AI网元140用于执行AI相关的操作，例如，构建训练数据集或训练AI模型等。
在一种可能的实现方式中，网络设备110可以将与AI模型的训练相关的数据发送给AI网元140，由AI网元140构建训练数据集，并训练AI模型。例如，与AI模型的训练相关的数据可以包括终端设备上报的数据。AI网元140可以将AI模型相关的操作的结果发送至网络设备110，并通过网络设备110转发至终端设备。例如，AI模型相关的操作的结果可以包括以下至少一项：已完成训练的AI模型、模型的评估结果或测试结果等。示例性地，已完成训练的AI模型的一部分可以部署于网络设备110上，另一部分部署于终端设备上。可替换地，已完成训练的AI模型可以部署于网络设备110上。或者，已完成训练的AI模型可以部署于终端设备上。
应理解，图2仅以AI网元140与网络设备110直接相连为例进行说明，在其他场景中，AI网元140也可以与终端设备相连。或者，AI网元140可以同时与网络设备110和终端设备相连。或者，AI网元140还可以通过第三方网元与网络设备110相连。本申请实施例对AI网元与其他网元的连接关系不做限定。
AI网元140也可以作为一个模块设置于网络设备和/或终端设备中，例如，设置于图1所示的网络设备110或终端设备中。网络设备110中可以部署一个或多个AI模块。终端设备中可以部署一个或多个AI模块。
需要说明的是，图1和图2仅为便于理解而示例的简化示意图，例如，通信系统中还可以包括其它设备，如还可以包括无线中继设备和/或无线回传设备等，图1和图2中未予以画出。在实际应用中，该通信系统可以包括多个网络设备，也可以包括多个终端设备。本申请实施例对通信系统中包括的网络设备和终端设备的数量不做限定。
图3为本申请实施例的一种通信系统的可能的应用框架的示意图。如图3所示，通信系统中网元之间通过接口(例如NG，Xn)，或空口相连。这些网元节点，例如核心网设备、接入网节点(RAN节点)、终端设备或操作维护管理(operation administration and maintenance，OAM)中的一个或多个设备中设置有一个或多个AI模块(为清楚起见，图3中仅示出1个)。接入网节点可以作为单独的RAN节点，也可以包括多个RAN节点，例如，包括CU和DU。CU和/或DU也可以设置一个或多个AI模块。可选的，CU还可以被拆分为CU-CP和CU-UP。CU-CP和/或CU-UP中设置有一个或多个AI模型。示例性地，CU和DU之间通过F1接口相连。CU和CU之间通过Xn接口相连。
网络设备可以为设置有一个或多个AI模块的网络设备。网络设备可以为图3所示的核心网设备、接入网节点(RAN节点)或OAM中的一个或多个设备。比如，AI模块可以为图4所示的RIC，如近实时RIC或非实时RIC等。例如，近实时RIC设置在RAN节点中(例如，CU、DU中)，而非实时RIC设置在OAM中、云服务器中、核心网设备、或者其他网络设备中。RIC可以通过从RAN节点(例如CU、CU-CP、CU-UP、DU和/或RU)获得来自多个终端设备的子集，重组为训练数据集#2，并基于训练数据集#2进行训练。示例性地，近实时RIC，非实时RIC也可以分别作为一个网元单独设置，网络设备可以为近实时RIC或非实时RIC。
图4为通信系统中的一种可能的应用框架示意图。如图4所示，通信系统中包括RAN智能控制器(RAN intelligent controller，RIC)。例如，RIC可以是图3中示出的AI模块，用于实现AI相关的功能。RIC包括近实时RIC(near-real time RIC，near-RT RIC)和非实时RIC(non-real time RIC，Non-RT RIC)。其中，非实时RIC主要处理非实时的信息，比如，对时延不敏感的数据，该数据的时延可以为秒级。实时RIC主要处理近实时的信息，比如，对时延相对敏感的数据，该数据的时延为数十毫秒级。
近实时RIC用于进行模型训练和推理。例如，用于训练AI模型，利用该AI模型进行推理。近实时RIC可以从RAN节点(例如CU、CU-CP、CU-UP、DU和/或RU)和/或终端获得网络侧和/或终端侧的信息。该信息可以作为训练数据或者推理数据。可选的，近实时RIC可以将推理结果递交给RAN节点和/或终端。可选的，CU和DU之间，和/或DU和RU之间可以交互推理结果。例如近实时RIC将推理结果递交给DU，DU将其发给RU。
非实时RIC也用于进行模型训练和推理。例如，用于训练AI模型，利用该模型进行推理。非实时RIC可以从RAN节点(例如CU、CU-CP、CU-UP、DU和/或RU)和/或终端获得网络侧和/或终端侧的信息。该信息可以作为训练数据或者推理数据，推理结果可以被递交给RAN节点和/或终端。可选的，CU和DU之间，和/或DU和RU之间可以交互推理结果，例如非实时RIC将推理结果递交给DU，由DU将其发给RU。
近实时RIC，非实时RIC也可以分别作为一个网元单独设置。可选的，近实时RIC、非实时RIC也可以作为其他设备的一部分，例如，近实时RIC设置在RAN节点中(例如，CU，DU中)，而非实时RIC设置在OAM中、云服务器中、核心网设备、或者其他网络设备中。
为了便于理解本申请实施例的方案，下面对本申请实施例可能涉及的术语进行解释。
(1)神经网络(neural network，NN)：
神经网络是AI或机器学习(machine learning，ML)的一种具体实现形式。根据通用近似定理，神经网络理论上可以逼近任意连续函数，从而使得神经网络具备学习任意映射的能力。
以AI模型的类型为神经网络为例，本公开涉及的AI模型可以为深度神经网络(deep neural network，DNN)。传统通信系统需要借助丰富的专家知识来设计通信模块，而基于深度神经网络(deepneuralnetwork，DNN)的深度学习通信系统可以从大量的数据集中自动发现隐含的模式结构，建立数据之间的映射关系，获得优于传统建模方法的性能。
神经网络可以是由神经元组成的，每个神经元都对其输入值做加权求和运算，并加权求和结果通过一个非线性函数产生输出。DNN一般具有多层结构，DNN的每一层都可包含多个神经元，输入层将接收到的数值经过神经元处理后，传递给中间的隐藏层。类似的，隐藏层再将计算结果传递给最后的输出层，产生DNN的最后输出。
DNN一般具有多于一个的隐藏层，隐藏层往往直接影响提取信息和拟合函数的能力。增加DNN的隐藏层数或扩大每一层的宽度都可以提高DNN的函数拟合能力。每个神经元中加权值即为DNN网络模型的参数。模型参数通过训练过程得到优化，从而使得DNN网络具备提取数据特征、表达映射关系的能力。DNN一般使用监督学习或非监督学习策略来优化模型参数。
根据网络的构建方式，DNN可以包括前馈神经网络(feedforward neural network，FNN)、卷积神经网络(convolutional neural networks，CNN)和递归神经网络(recurrent neural network，RNN)等。
CNN是一种专门来处理具有类似网格结构的数据的神经网络。例如，时间序列数据(时间轴离散采样)和图像数据(二维离散采样)都可以认为是类似网格结构的数据。CNN并不一次性利用全部的输入信息做运算，而是采用一个固定大小的窗截取部分信息做卷积运算，这就大大降低了模型参数的计算量。另外根据窗截取的信息类型的不同(如同一副图中的人和物为不同类型信息)，每个窗可以采用不同的卷积核运算，这使得CNN能更好的提取输入数据的特征。
RNN是一类利用反馈时间序列信息的DNN网络。它的输入包括当前时刻的新的输入值和自身在前一时刻的输出值。RNN适合获取在时间上具有相关性的序列特征，特别适用于语音识别、信道编译码等应用。
FNN网络的特点为相邻层的神经元之间两两完全相连，这使得FNN通常需要大量的存储空间、导致较高的计算复杂度。
上述FNN、CNN、RNN为都是以神经元为基础而构造的。如前所述，每个神经元都对其输入值做加权求和运算，并加权求和结果通过一个非线性函数产生输出。神经网络中神经元加权求和运算的权值以及非线性函数称作神经网络的参数。一个神经网络所有神经元的参数构成这个神经网络的参数。
(2)集中式训练：
集中式训练也可以称为中心式训练，是指机器学习模型和训练数据集中在一个计算节点，由该计算节点完成所有的训练步骤。
(3)分布式训练：
分布式训练也可以称为分布式学习，是指利用多个计算节点进行机器学习或者深度学习的算法和系统。
在分布式学习中，一般由多个子计算节点和一个中心计算节点共同参与训练。用户的原始数据留驻在用户本地。子计算节点通过上传神经网络参数与中心计算节点协作完成训练，这避免了原始数据的直接共享，从而保护了用户数据的隐私安全。
(4)联邦学习：
联邦学习是指在多个拥有本地数据样本的分布式边缘设备或计算节点上训练算法，并且允许各个计算节点的数据分布存在差异。
联邦学习是分布式学习中的一种重要框架。在联邦学习中，一个或多个子计算节点在其本地数据上训练神经网络，中心计算节点负责聚合神经网络参数来构建全局模型。
联邦学习包含中心计算节点和子计算节点两个部分。中心计算节点也可以简称为中心节点。子计算节点也可以简称为子节点。M个子节点的本地数据集表示为{D1,D2,···,DM}。M为正整数。其中，子节点k的本地数据集记为其中，xi为模型的输入，即第i个样本，yi为输入xi对应的标签，Nk为Dk的数据量。中心节点与子节点分别拥有各自的神经网络模型f(·,wc)与f(·,wk)。其中，wc为中心节点的神经网络模型的参数，中心节点的神经网络模型也被称为全局模型，wk为子节点k的神经网络模型的参数。子节点的神经网络模型也可以称为本地模型。本地数据集用于本地模型的训练。中心节点和子节点协作起来共同解决如下优化问题：
其中，为子节点k的本地数据集的数据量占总数据量的比例。Lk(w；Dk)表示子节点k的损失函数的本地的损失值。L(w)表示损失值。w表示神经网络模型的参数。
联邦学习的训练过程如下：首先，在训练开始的初始阶段，中心节点将全局模型的参数wc发送给所有子节点。然后，每个子节点的模型都使用本地数据集Dk进行多轮训练，该步骤被称为本地训练。本地训练具体过程包括如下步骤。
(1)在第t次迭代更新时，子节点k首先从本地数据集Dk中随机选取一批训练数据其中，|Bk|为Bk的数据量，即这一批训练数据的数量，|Bk|为正整数，t为正整数。
(2)子节点k对输入xi计算输出并根据损失函数l(.,.)计算损失值其中，表示处于第t次迭代时的本地模型的参数。
(3)子节点k计算Lk(w)对参数w的梯度：表示处于第t次迭代时的Lk(w)对参数w的梯度。
(4)子节点k使用学习率λ更新本地模型的参数：设置t＝t+1，并返回步骤(1)。
上述步骤(1)至(4)循环进行，直到本地数据集中所有样本都被使用过，也即本地模型更新次数达到时，完成本回合的本地训练，其中表示对x执行向下取整。然后，子节点将训练后的模型参数发送给中心节点。中心节点聚合收到的模型参数得到新的模型参数wc。聚合完成后，中心节点与子节点共享聚合后的全局模型的参数wc，然后重复上述步骤，直到模型收敛。
批规范化(batchnormalization，BN)层对隐藏层输出的特征进行规范化，可以缓解过拟合问题，缓解梯度消失问题，并具有正则化作用，有利于提升AI模型的训练性能。对于深度神经网络模型来说，由于参数在训练过程中是不断更新的，即使输入数据已经做过标准化处理，但是对于比较靠后的那些层，其接收到的输入仍然是剧烈变化的，通常会导致数值不稳定，模型很难收敛。批规范化能够使模型的中间层的输出变得更加稳定。在神经网络模型构建时，可以在一个或多个网络层后设置对应的批规范化层。例如，在神经网络模型的第二层后设置一个批规范化层，第四层后设置一个批规范化层。第二层的输出即为第二层对应的批规范化层的输入，第四层的输出即为第四层对应的批规范化层的输入。
批规范化的计算公式为：
其中，xl表示神经网络模型的第l层的输入，hl(xl)表示神经网络模型的第l层的输出，该神经网络模型的第l层的输出即第l层对应的批规范化层的输入。μB表示hl(xl)的均值，表示hl(xl)的方差，ε为一个保证分母不为0的常数，通常取10-6。γl和βl为该批规范化层的可训练参数。表示该批规范化层的输出。B表示数据集中的一批训练数据。该数据集用于该神经网络模型的训练。在上述公式中，μB和可以通过如下公式确定。
其中，|B|表示这批训练数据的数量，表示第l层的第i个样本的输入。
由于子节点的训练数据较少，本地训练容易过拟合，导致中心节点聚合得到的全局模型过拟合，最终导致全局模型的性能较差。因此，通常在本地模型中添加BN层来缓解过拟合问题。
在分布式训练中，子节点的本地数据非独立同分布，即各个子节点的数据分布存在差异，导致本地模型的批规范化层统计的均值与方差存在差异，最终导致分布式训练的性能差于集中式训练的性能。
在相关方案中，子节点在本地训练结束后可以上传本地模型的批规范化层的输入的均值和方差以及训练后的模型参数。中心节点聚合接收到的均值，方差和模型参数，并将聚合后的均值，方差和新的全局模型参数发送给子节点。上述步骤重复执行直到模型收敛。
虽然上述方案缓解了各个子节点的本地模型的批规范化层的输入的均值与方差存在差异的问题，但是通过上述方式得到的聚合后的方差与集中式训练中的方差并不相同。
具体地，对于M个子节点，子节点m可以通过如下公式计算本地模型的批规范化层的输入的均值和方差。
其中，表示神经网络模型的第l层的第i个样本的输入，表示在输入为的情况下子节点m上的神经网络模型的第l层的输出，该神经网络模型的第l层的输出即第l层对应的批规范化层的输入。表示子节点m上的神经网络模型的第l层对应的批规范化层的输入的均值，表示子节点m上的神经网络模型的第l层对应的批规范化层的输入的方差。Bm表示子节点m上的一批训练数据，即第m批数据。|Bm|表示这批训练数据的数量。
中心节点可以使用如下公式聚合M个子节点反馈的均值与方差：
其中，μl,B表示聚合后的神经网络模型的第l层对应的批规范化层的输入的均值，表示聚合后的神经网络模型的第l层对应的批规范化层的输入的方差。
在集中式训练中，神经网络模型的第l层对应的批规范化层的输入的方差可以通过如下公式计算。
其中，B＝B1∪B2∪...∪BM，μ'l,B表示训练数据B对应的第l层对应的批规范化层的输入的均值。为第m批训练数据Bm对应的第l层对应的批规范化层的输入的方差。表示在输入为的情况下神经网络模型的第l层的输出，即第l层对应的批规范化层的输入。
可以通过如下公式计算。
多数机器学习算法，例如，基于DNN的学习算法，需要基于海量数据进行训练。通常采用集中式训练方式来完成AI模型的训练。然而，集中式训练方式存在一些问题。例如，集中式训练需要收集训练数据，训练数据通常来源于用户设备，即需要用户设备上传数据。这些数据的上传开销通常较大，而且，收集和使用这些数据可能会侵犯用户的隐私安全，例如，泄露用户的位置、运动路线和照片等。而在分布式训练过程中，子节点基于本地数据训练本地模型，并上传本地模型的参数至中心节点，与中心节点协作完成训练。在该过程中，子节点上的本地数据可以留驻在子节点本地，避免了本地数据的共享，从而避免了训练数据的上传所带来的开销，同时有利于保护本地数据的隐私安全。分布式训练有望在保障用户的隐私安全的同时使用用户设备的数据和计算能力达到与集中式训练相同的性能。
在分布式训练过程中，子节点的本地模型可以使用BN层来抵抗过拟合以获得较好的性能。
然而，在分布式训练中，各个子计算节点的本地数据非独立同分布，各个子计算节点的BN层统计的均值和方差之间存在差异，影响了批规范化的准确性，从而影响了分布式训练的性能。
在相关方案中，子节点可以在本地训练结束后上传BN层对应的均值和方差以及训练后的本地模型的模型参数，中心节点聚合接收到的均值、方差和模型参数，并将聚合后的均值、方差和新的全局模型的模型参数发送给各个子节点，子节点基于聚合后的均值、方差对新的全局模型进行本地训练，重复上述过程，直至模型收敛。然而，该方式聚合得到的方差和集中式训练的方差不同。具体地，对比上述公式(6)、(8)、(9)和(10)可以看出，两种方式下计算得到的方差存在差异。即在分布式训练中通过聚合的子节点的方差得到的聚合后的方差和集中式训练的方差不同，影响了分布式训练过程中的批规范化处理的准确性，从而影响分布式训练的性能，使其难以达到集中式训练的性能。
有鉴于此，本申请提供一种通信的方法和通信装置，有利于提高分布式训练过程中的批规范化处理的准确性，从而有利于提高分布式训练的性能。
应理解，本申请中，指示包括直接指示(也称为显式指示)和隐式指示。其中，直接指示信息A，是指包括该信息A；隐式指示信息A，是指通过信息A和信息B的对应关系以及直接指示信息B，来指示信息A。其中，信息A和信息B的对应关系可以是预定义的，预存储的，预烧制的，或者，预先配置的。
应理解，本申请中，信息C用于信息D的确定，既包括信息D仅基于信息C来确定，也包括基于信息C和其他信息来确定。此外，信息C用于信息D的确定，还可以间接确定的情况，比如，信息D基于信息E确定，而信息E基于信息C确定这种情况。
此外，本申请各实施例中的“网元A向网元B发送信息A”，可以理解为该信息A的目的端或与目的端之间的传输路径中的中间网元是网元B，可以包括直接或间接的向网元B发送信息。“网元B从网元A接收信息A”，可以理解为该信息A的源端或与该源端之间的传输路径中的中间网元是网元A，可以包括直接或间接的从网元A接收信息。信息在信息发送的源端和目的端之间可能会被进行必要的处理，例如格式变化等，但目的端可以理解来自源端的有效信息。本申请中类似的表述可以做类似的理解，在此不予赘述。
在本申请实施例的方案中，“和/或”，用于描述对应对象的对应关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或复数。
在本申请实施例的方案中，中心节点可以与M个子节点协作完成AI模型的训练。M为大于1的整数。为了便于描述，本申请实施例中主要以M个子节点中的一个子节点(如子节点#1)与中心节点之间的通信为例进行说明。子节点#1可以为该M个子节点中的任一子节点。其他子节点的行为可以参考子节点#1的行为。
作为一种示例，中心节点可以为网络设备侧的AI实体，子节点可以为终端设备侧的AI实体。
网络设备侧包括网络设备，或者，其他与网络设备通信的设备，比如，由网络设备控制或服务于网络设备的设备。该AI实体可以为网络设备本身，或者，与网络设备通信的AI实体。例如，中心节点可以为RIC，OAM或服务器，比如，OTT服务器或云端服务器。近实时RIC设置在RAN节点中，例如，CU/DU中。
终端设备侧包括终端设备，或者，其他与终端设备通信的设备，比如，由终端设备控制或服务于终端设备的设备。该AI实体可以为终端设备本身，或者，为与终端设备通信的AI实体。例如，子节点可以为服务器，比如OTT服务器或云端服务器。
作为另一种示例，中心节点可以为终端设备侧的AI实体，子节点可以为终端设备侧的AI实体。
例如，中心节点可以为服务器，多个子节点可以为与该服务器通信的多个终端设备。
示例性地，AI模型训练完成后，可以部署于终端设备侧，或者，部署于网络设备侧，或者，部署于终端设备侧和网络设备侧。示例性地，AI模型的输出可以包括离散数值或连续数值。例如，AI模型用于执行分类任务时可以输出离散数值。例如，AI模型用于执行回归任务时，可以输出连续数值。
此外，在一些场景中，中心节点也可以基于中心节点上的训练数据集对AI模型进行训练。在该情况下，中心节点也可以视为一个子节点。或者说，该中心节点还承担子节点上与模型训练相关的操作。前述M个子节点可以视为包括该中心节点。
子节点可以称为子计算节点。M个子节点也可以替换为M个设备。
中心节点也可以称为中心计算节点。
中心节点上的AI模型可以称为全局模型。子节点上的AI模型可以称为子节点上的本地模型。中心节点上的AI模型和该M个子节点上的AI模型的结构相同。
图5是本申请提供的一种通信的方法的示意性流程图。示例性地，图5所示的方法500可以应用于分布式训练的场景。在方法500中，中心节点将聚合得到的BN层对应的全局均值下发给子节点，子节点基于BN层对应的全局均值来计算BN层的输入的方差。中心节点可以聚合子节点反馈的BN层的输入的方差，以得到BN层对应的全局方差。
如图5所示，方法500可以包括如下步骤。
510，中心节点向多个子节点(如图5中的子节点#1和子节点#2)发送信息#1(第一信息的一例)。信息#1包括索引#1(第一索引的一例)和指示全局均值#1(第一均值的一例)的信息。
全局均值#1可以用于本地方差#1(第一方差的一例)的计算。子节点的本地方差#1为子节点上的AI模型的网络层#1(第一网络层的一例)的输入的方差。
换言之，该全局均值#1可以用于计算该多个子节点上的AI模型的网络层#1的输入的方差。
其中，索引#1与全局均值#1相关。
根据索引#1可以确定全局均值#1用于计算本地方差#1。
步骤510中的多个子节点为前述M个子节点的部分或全部。
该多个子节点可以包括子节点#1(第一设备的一例)，以子节点#1为例，子节点#1接收到来自中心节点的信息#1，可以根据该全局均值#1计算子节点#1的本地方差#1，即子节点#1上的AI模型的网络层#1的输入的方差。
全局均值#1即为网络层#1对应的全局均值，或者说，与网络层#1相关的全局均值。该全局均值#1可以是通过聚合一个或多个子节点的本地均值#1得到的。子节点的本地均值#1为子节点上的AI模型的网络层#1的输入的均值。全局均值#1的确定方式可以参考后文的描述。
多个子节点也可以替换为多个设备。
520，多个子节点向中心节点发送信息#2(第二信息的一例)。信息#2包括索引#2(第二索引的一例)和指示本地方差#1的信息。索引#2和索引#1具有关联关系。
来自该多个子节点的信息#2分别包括指示该多个子节点的本地方差#1的信息。
步骤520中的多个子节点为前述M个子节点的部分或全部。
步骤520中的多个子节点与步骤510中的多个子节点可以相同，也可以不同。
步骤520中的多个子节点与步骤510中的多个子节点至少部分相同。例如：步骤510中的多个子节点可以包括步骤520中的多个子节点。
该多个子节点可以包括子节点#1，以子节点#1为例，子节点#1向中心节点发送信息#2。来自子节点#1的信息#2所指示的本地方差#1为子节点#1的本地方差#1。
其中，索引#2与本地方差#1相关。根据索引#2和索引#1的关联关系即可确定本地方差#1与全局均值#1相关，或者说，本地方差#1是基于全局均值#1确定的。进一步的，本地方差#1可以基于全局均值#1和网络层#1的输入确定。
分布式训练通常包括多个训练的回合，AI模型中可能存在多个BN层。在本申请实施例中，通过索引可以确定本地方差与全局均值之间的相关性，避免子节点基于不相关的全局均值计算本地方差，例如，避免子节点基于其他BN层对应的全局均值来计算当前BN层的输入的本地方差，或者，避免子节点基于其他回合的全局均值来计算本回合的本地方差，从而有利于保证本地方差的准确性。
进一步地，方法500还可以包括步骤530。
530，中心节点向多个子节点发送信息#3(第四信息的一例)。信息#3包括索引#3(第三索引的一例)和指示全局方差#1(第二方差的一例)的信息。索引#1和索引#3具有关联关系，和/或，索引#2和索引#3具有关联关系。全局均值#1和全局方差#1可以用于子节点上的AI模型的训练。
根据索引#2可以确定本地方差#1用于全局方差#1的计算。
中心节点可以根据来自一个或多个子节点的信息#2所指示的本地方差#1确定全局方差#1。来自多个子节点的信息#2所指示的本地方差#1即该多个子节点的本地方差#1。
具体地，中心节点可以聚合一个或多个子节点的本地方差#1，以得到全局方差#1，并通过信息#3将全局方差#1下发给多个子节点。全局方差也可以称为聚合方差或聚合后的方差。
步骤530中的多个子节点为前述M个子节点的部分或全部。
步骤530中的多个子节点与步骤510中的多个子节点可以相同，也可以不同。
步骤530中的多个子节点与步骤520中的多个子节点可以相同，也可以不同。
该多个子节点可以包括子节点#1，以子节点#1为例，子节点#1接收到来自中心节点的信息#3后，可以根据全局均值#1和全局方差#1对本地模型进行训练。
全局方差#1即为网络层#1对应的全局方差，或者说，与网络层#1相关的全局方差。全局方差#1的确定方式可以参考后文的描述。
其中，索引#3与全局方差#1相关。根据索引#3和索引#1的关联关系以及索引#1和索引#2的关联关系即可确定全局方差#1、全局均值#1以及本地方差#1相关。或者，根据索引#3和索引#2的关联关系以及索引#1和索引#2的关联关系即可确定全局方差#1、全局均值#1以及本地方差#1相关。或者，根据索引#3、索引#1和索引#2的关联关系即可确定全局方差#1、全局均值#1以及本地方差#1相关。
进一步地，在步骤510之前，方法500还可以包括步骤540。
540，多个子节点向中心节点发送信息#4(第五信息的一例)。信息#4包括索引#4(第四索引的一例)和指示本地均值#1(第二均值的一例)的信息。子节点的本地均值#1为子节点上的AI模型的网络层#1的输入的均值。索引#4与以下至少一项具有关联关系：索引#1、索引#2或索引#3。
来自多个子节点的信息#4分别包括指示该多个子节点的本地均值#1的信息。
步骤540中的多个子节点为前述M个子节点的部分或全部。
步骤540中的多个子节点与步骤510中的多个子节点可以相同，也可以不同。
步骤540中的多个子节点与步骤520中的多个子节点可以相同，也可以不同。
步骤540中的多个子节点与步骤530中的多个子节点可以相同，也可以不同。
该多个子节点可以包括子节点#1，以子节点#1为例，子节点#1向中心节点发送信息#4。子节点#1发送的信息#4所指示的本地均值#1即为子节点#1的本地均值#1，即子节点#1上的AI模型的网络层#1的输入的均值。
中心节点可以根据来自一个或多个子节点的信息#4所指示的本地均值#1确定全局均值。来自多个子节点的信息#4所指示的本地均值#1即多个子节点的本地均值#1。
具体地，中心节点可以聚合一个或多个子节点的本地均值#1，以得到全局均值#1。在步骤510中通过信息#1将全局均值#1下发给多个子节点。全局均值也可以称为聚合均值或聚合后的均值。
其中，索引#4与本地均值#1相关。索引#4与索引#1、索引#2或索引#3中的至少一项具有关系联系，根据该关联关系可以确定，本地均值#1与全局均值#1、本地方差#1或全局方差#1中的至少一项相关。
应理解，本申请实施例中步骤的编号仅为描述方便使用，不对本申请实施例中的步骤的执行顺序构成限定。例如，步骤540可以在步骤510之前执行。
AI模型可以包括一个或多个BN层。网络层#1可以为该一个或多个BN层中的任一层。为了便于描述，本申请实施例中仅以一个BN层为例进行说明。在AI模型包括多个BN层的情况下，其他BN层均可以参考与网络层#1相关的处理方式。
在AI模型中，除了输入层之外，一个网络层的输入可以作为另一个网络层的输出。相应地，网络层#1的输入也可以理解为网络层#1的前一层的输出。或者说，网络层#1的输入也可以替换为网络层#2的输出，网络层#2可以为BN层的前一层。
网络层#1的输入可以包括一个或多个特征，或者，一个或多个特征通道。网络层#1的输入也可以替换为网络层#1的特征，或网络层#1的特征通道。例如，网络层#1的输入的均值即网络层#1的特征的均值。网络层#1的输入的方差即网络层#1的特征的方差。
下面对如何确定本地均值#1进行说明。
子节点上的AI模型的网络层#1的输入的均值是基于该子节点上的一批(batch)训练数据确定的。
以子节点m为例，子节点m上的一批训练数据Bm的数据量为|Bm|。|Bm|为正整数。将该批训练数据Bm输入至子节点m上的AI模型中，相应地，可以得到网络层#1的|Bm|个输入，该|Bm|个输入的平均值即可作为子节点m上的AI模型的网络层#1的输入的均值。例如，子节点m可以基于前文中的公式(5)计算本地均值#1，第l层对应的BN层可以视为网络层#1。
下面对如何确定全局均值#1进行说明。
中心节点可以聚合一个或多个子节点的本地均值#1，以得到全局均值#1。
该一个或多个子节点可以为M个子节点的部分或全部，即中心节点可以基于部分或全部子节点的本地均值#1确定全局均值#1。该一个或多个子节点属于步骤510中的多个子节点。
作为一种可能的实现方式，中心节点根据M个子节点的本地均值#1确定全局均值#1。
中心节点可以接收来自M个子节点的信息#4，获取该M个子节点的本地均值#1。中心节点聚合该M个子节点的本地均值#1，以得到全局均值#1。
作为另一种可能的实现方式，中心节点根据该M个子节点中的部分子节点的本地均值#1确定全局均值#1。
作为一个示例，中心节点仅接收到来自该M个子节点中的部分子节点的信息#4。中心节点可以基于接收到的子节点的本地均值#1聚合得到全局均值#1。
例如，子节点#1可能未发送信息#4。再如，子节点#1向中心节点发送信息#4，但中心节点并未接收到子节点#1上报的本地均值#1。在该情况下，中心节点在计算全局均值#1时可以不使用子节点#1的本地均值#1。
作为另一个示例，中心节点可以自行决定根据该M个子节点中的部分子节点的本地均值#1确定全局均值#1。
例如，中心节点虽然接收到了子节点#1上报的本地均值#1，但中心节点可以自行决定不使用子节点#1的本地均值#1来计算全局均值#1。
中心节点可以通过多种聚合方式聚合子节点的本地均值。
示例性地，中心节点聚合一个或多个子节点的本地均值#1，可以包括：中心节点计算该多个子节点的本地均值#1的平均值，将其作为全局均值#1。
例如，以该多个子节点为M个子节点的全部为例，全局均值#1可以满足如下公式：
其中，μl,B表示AI模型的第l层对应的BN层(即网络层#1)对应的全局均值，即全局均值#1。表示AI模型的第l层对应的BN层的输入的均值，即本地均值#1。
可替换地，中心节点聚合该一个或多个子节点的本地均值#1，可以包括：中心节点计算该多个子节点的本地均值#1的加权平均值，将其作为全局均值#1。例如，在加权平均值的计算过程中，各个子节点的本地均值#1对应的权重可以与各个子节点的批训练数据的数据量呈正相关关系。以该一个或多个子节点为M个子节点的全部为例，中心节点可以基于前文中的公式(7)计算全局均值#1。
下面对如何确定本地方差#1进行说明。
全局均值#1可以用于计算多个子节点的本地方差#1。
以子节点#1为例，在接收到全局均值#1后，子节点#1不是基于本地均值#1来计算本地方差#1，而是基于全局均值#1来计算本地方差#1。
例如，本地方差#1可以满足如下公式：
其中，表示子节点m的AI模型的第l层对应的BN层(即网络层#1)的输入的方差，即子节点m的本地方差#1，μl,B表示第l层对应的BN层对应的全局均值，即全局均值#1。表示第l层的第i个样本的输入，表示在输入为的情况下子节点m上的AI模型的第l层的输出，即第l层对应的BN层的输入。
下面对如何确定全局方差#1进行说明。
中心节点可以聚合一个或多个子节点的本地方差#1，以得到全局方差#1。
该一个或多个子节点可以为M个子节点的部分或全部，即中心节点可以基于部分或全部子节点的本地方差#1确定全局方差#1。该一个或多个子节点属于步骤520中的多个子节点。该一个或多个子节点的具体描述可以参考前文中计算全局均值#1的过程的一个或多个子节点的相关描述。应理解，用于计算全局方差#1的本地方差#1所属的一个或多个子节点和用于计算全局均值#1的本地均值#1所属的一个或多个子节点可以相同，也可以不同。
示例性地，中心节点聚合该一个或多个子节点的本地方差#1，可以包括：中心节点计算该多个子节点的本地方差#1的平均值，将其作为全局方差#1。
可替换地，中心节点聚合该一个或多个子节点的本地方差#1，可以包括：中心节点计算该多个子节点的本地方差#1的加权平均值，将其作为全局方差#1。例如，在加权平均值的计算过程中，各个子节点的本地方差#1对应的权重可以与各个子节点的批训练数据的数据量呈正相关关系。以该多个子节点为M个子节点的全部为例，中心节点可以基于前文中的公式(8)计算全局方差#1。
在本申请实施例的方案中，子节点基于来自中心节点的全局均值#1计算与全局均值#1相关的本地方差#1，有利于避免各个子节点的本地数据非独立同分布的影响，得到更准确的本地方差#1，以期使得中心节点能够基于更准确的本地方差#1聚合得到更准确的全局方差#1，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。
下面对索引#1和全局均值#1的相关性进行说明。
分布式训练通常需要多个回合才能完成模型的训练。一个回合可以理解为，从中心节点向各个子节点发送全局模型的模型参数开始到中心节点聚合生成新的全局模型结束。不同的回合可以通过不同的回合的标识区分。示例性地，回合的标识可以为当前回合的编号。例如，当前回合的编号可以体现当前已经历过的回合的数量，即当前的回合数。当前回合的编号可以等于当前的回合数。或者，回合的编号与回合数可以呈正相关关系，比如，回合数越大，回合的编号越大。当前回合的编号也可以与当前的回合数无关，只要能够区分不同的回合即可。为便于描述，本申请实施例中主要以当前回合的编号可以体现当前的回合数为例进行说明。
示例性地，对于中心节点而言，每次向子节点发送全局模型的模型参数，或者，每次更新全局模型，则中心节点进入新的回合，中心节点可以更新中心节点的回合的标识，例如，中心节点处的回合的编号加一，或者说，与全局模型相关的训练的回合的编号加一。对于子节点而言，每次接收到全局模型的模型参数，或者，每次根据收到的全局模型的模型参数更新本地模型，则子节点进入新的回合，子节点可以更新本地的回合的标识，例如，子节点处的回合的编号加一，或者，与本地模型相关的训练的回合的编号加一。或者，中心节点每次向子节点发送全局模型的模型参数时，也可以将更新后的回合的标识下发给子节点，子节点可以基于接收到的回合的标识更新本地的回合的标识。
回合的标识也可以为全局模型被更新的次数，或，中心节点和子节点关于全局模型的模型参数的交互次数。
索引#1和全局均值#1相关，可以理解为，索引#1和全局均值#1均与同一个回合和/或同一个网络层相关。
下面以两个回合为例进行说明。
例如，在第一回合中，中心节点根据多个子节点的本地均值a计算得到全局均值a，中心节点向子节点发送索引a和全局均值a，索引a与第一回合相关，子节点基于全局均值a计算得到本地方差a，并上报给中心节点，中心节点基于多个子节点的本地方差a计算得到全局方差a，并下发给子节点，子节点基于全局方差a和全局均值a对本地模型进行训练。在本地训练完成后，将训练后的本地模型的模型参数上报给中心节点，中心节点聚合多个子节点上报的本地模型的模型参数，以生成新的全局模型。中心节点将新的全局模型的参数下发给子节点，即进入第二回合。在第二回合中，中心节点根据多个子节点的本地均值b计算得到全局均值b，中心节点向子节点发送索引b和全局均值b，索引b与第二回合相关，子节点基于全局均值b计算得到本地方差b，并上报给中心节点，中心节点基于多个子节点的本地方差b计算得到全局方差b，并下发给子节点，子节点基于全局方差b和全局均值b对本地模型进行训练。在上述过程中，索引a即为与全局均值a相关的索引，索引b即为与全局均值b相关的索引。索引a可以视为与全局均值b无关的索引。索引b可以视为与全局均值a无关的索引。
索引#1和全局均值#1的相关性可以通过多种方式体现。
示例性地，索引#1和全局均值#1可以承载于同一信令。即第一信息可以承载于同一信令中。
在该情况下，与全局均值#1承载于同一信令的索引即为与全局均值#1相关的索引#1。
进一步地，若同一信令中承载了多个BN层对应的全局均值及其对应的多个索引，与多个全局均值相关的索引可以为与该多个全局均值的位置具有预设位置关系的索引。
例如，索引#1可以为处于全局均值#1的后面并与全局均值#1相邻的索引。
可替换地，索引#1和全局均值#1可以分别承载于两个信令中。即第一信息可以承载于两个信令中。该两个信令相关。子节点在接收到该两个信令后根据全局均值#1计算本地方差#1。
下面基于两个示例(示例#1和示例#2)对索引#1所指示的信息进行说明。
示例#1：
作为一个示例，索引#1可以指示与全局均值#1相关的训练的回合。
与全局均值#1相关的训练的回合即中心节点处的回合。
可选地，索引#1可以指示与全局均值#1相关的训练的回合的标识。
例如，索引#1可以包括与全局均值#1相关的训练的回合的编号。
子节点#1在接收到信息#1后，可以根据索引#1确定全局均值#1为哪一个回合中计算得到的，进而可以在该回合中基于该全局均值#1计算本地方差#1。
可选地，索引#1可以指示全局均值#1对应的AI模型的标识。
索引#1可以通过指示AI模型的标识来指示与全局均值#1相关的训练的回合。
例如，AI模型的标识可以为该AI模型的版本号。
在分布式训练过程中，每进入一个新的回合，可以为当前的AI模型分配一个与之前回合中的AI模型不同的标识，如设置一个新的版本号。例如，第一回合的AI模型的版本号可以为V1.1，第二回合的AI模型的版本号可以为V1.2，以此类推。或者说，中心节点每次聚合生成新的全局模型，可以为新的全局模型分配与之前的全局模型不同的标识，如设置一个新的版本号。例如，初始的全局模型的版本号可以为V1.0，第一次更新后的全局模型的版本号可以为V1.1，以此类推。
在聚合生成新的全局模型后，中心节点可以将新的全局模型的模型参数和新的全局模型的标识下发到各个子节点中。子节点基于新的全局模型的模型参数更新本地模型，相应地，新的全局模型的标识即作为更新后的本地模型的标识。
子节点#1可以根据全局均值#1对应的AI模型的标识来确定该全局均值#1对应哪个版本的AI模型，即确定该全局均值#1为哪一回合中计算得到的。
示例#2：
作为一个示例，索引#1可以指示网络层#1。即索引#1可以指示全局均值#1对应的网络层。
示例性地，索引#1可以包括网络层#1的标识。例如，网络层的标识可以为网络层的层号或网络层的索引。层号可以为整数。
子节点#1可以根据索引#1确定全局均值#1与哪个网络层关联，从而确定该全局均值#1用于计算子节点#1中的AI模型中该网络层的输入的方差。
应理解，以上仅为示例，网络层#1也可以通过其他方式指示。示例性地，信息#1中指示全局均值#1的信息的位置可以用于指示网络层#1。例如，AI模型可以包括多个BN层，信息#1可以包括用于指示该多个BN层对应的全局均值的信息。在信息#1中，指示该多个BN层对应的全局均值的字段的顺序与该多个BN层在AI模型中的顺序一致。比如，指示第一个BN层对应的全局均值的字段到指示最后一个BN层对应的全局均值的字段在信息#1中按照从前向后的顺序或者从后向前的顺序排列。
图6示出了信息#1中指示全局均值的信息的一个示例。
例如，AI模型包括3个BN层，该3个BN层在AI模型中按照从前到后的顺序排列为：BN层L1、BN层L2和BN层L3，对应的全局均值分别为全局均值a，全局均值a'和全局均值a”。信息#1可以包括指示该3个BN层对应的全局均值的信息。如图6所示，全局均值a，全局均值a'和全局均值a”在信息#1中按照对应的BN层在AI模型中的顺序排列。最左侧的全局均值即BN层L1对应的全局均值a，最右侧的全局均值即为BN层L3对应的全局均值a”。
示例1和示例2也可以结合使用。即索引#1可以指示网络层#1和与全局均值#1相关的训练的回合。
可选地，索引#1可以指示网络层#1的标识和与全局均值#1相关的训练的回合的标识。
示例性地，索引#1可以包括网络层#1的标识和与全局均值#1相关的训练的回合的标识。可选地，索引#1可以包括两个字段(也可称为信元)，分别承载网络层#1的标识和与全局均值#1相关的训练的回合的标识。
示例性地，索引#1可以包括网络层#1的标识和与全局均值#1相关的训练的回合的标识的联合编码结果。可选地，索引#1可以包括一个字段(也可称为信元)，该字段承载网络层#1的标识和与全局均值#1相关的训练的回合的标识的联合编码结果。
例如，索引#1的索引值为01，可以表示网络层#1为第一个BN层，与全局均值#1相关的训练的回合的编号为1；索引#1的索引值为02，可以表示网络层#1为第二个BN层，与全局均值#1相关的训练的回合的编号为1；索引#1的索引值为03，可以表示网络层#1为第一个BN层，与全局均值#1相关的训练的回合的编号为2。
可选地，索引#1可以指示网络层#1的标识和全局均值#1对应的AI模型的标识。
下面对索引#2和本地方差#1的相关性进行说明。
索引#2和本地方差#1相关，可以理解为，索引#2和本地方差#1均与同一个回合和/或同一个网络层相关。
下面以两个回合为例进行说明。
例如，在第一回合中，中心节点根据多个子节点的本地均值a计算得到全局均值a，中心节点向子节点发送全局均值a，子节点基于全局均值a计算得到本地方差a，将索引a和本地方差a上报给中心节点。索引a为第一回合相关。中心节点基于多个子节点的本地方差a计算得到全局方差a，并下发给子节点，子节点基于全局方差a和全局均值a对本地模型进行训练。在本地训练完成后，将训练后的本地模型的模型参数上报给中心节点，中心节点聚合多个子节点上报的本地模型的模型参数，以生成新的全局模型。中心节点将新的全局模型的参数下发给子节点，即进入第二回合。在第二回合中，中心节点根据多个子节点的本地均值b计算得到全局均值b，中心节点向子节点发送全局均值b，子节点基于全局均值b计算得到本地方差b，将索引b和本地方差b上报给中心节点，索引b与第二回合相关，中心节点基于多个子节点的本地方差b计算得到全局方差b，并下发给子节点，子节点基于全局方差b和全局均值b对本地模型进行训练。在上述过程中，索引a即为与本地方差a相关的索引，索引b即为与本地方差b相关的索引。索引a可以视为与本地方差b无关的索引。索引b可以视为与本地方差a无关的索引。
索引#2和本地方差#1的相关性可以通过多种方式体现。
示例性地，索引#2和本地方差#1可以承载于同一信令。即第二信息可以承载于同一信令中。
可替换地，索引#2和本地方差#1可以分别承载于两个信令中。即第二信息可以承载于两个信令中。该两个信令相关。
具体描述可以参考前文中的索引#1和全局均值#1的相关性，此处不再赘述。
下面基于两个示例(示例#3和示例#4)对索引#2所指示的信息进行说明。
示例#3：
作为一个示例，索引#2可以指示与本地方差#1相关的训练的回合。
与本地方差#1相关的训练的回合即子节点的回合。
可选地，索引#2可以指示与本地方差相关的训练的回合的标识。
例如，索引#2可以包括与本地方差#1相关的训练的回合的编号。
中心节点在接收到信息#2后，可以根据索引#2确定本地方差#1为哪一个回合中计算得到的，进而可以在该回合中基于本地方差#1计算全局方差#1。
可选地，索引#2可以指示本地方差#1对应的AI模型的标识。
索引#2可以通过指示本地方差#1对应的AI模型的标识来指示与本地方差#1相关的训练的回合。
本地方差#1对应的AI模型，即该本地方差#1是基于该AI模型计算得到的。
例如，AI模型的标识可以为该AI模型的版本号。
中心节点根据通过本地方差#1对应的AI模型的版本号来确定该本地方差#1对应哪个版本的AI模型，即确定该本地方差#1为哪一回合中计算得到的。
示例#4：
作为一个示例，索引#2可以指示网络层#1。即索引#2可以指示本地方差#1对应的网络层。
示例性地，索引#2可以包括网络层#1的标识。
中心节点可以根据索引#2确定本地方差#1与哪个网络层关联，从而确定该本地方差#1用于计算该网络层对应的全局方差。
应理解，以上仅为示例，网络层#1也可以通过其他方式指示。示例性地，信息#2中指示本地方差#1的信息的位置可以用于指示网络层#1。
示例#3和示例#4也可以结合使用。即索引#2可以指示网络层#1和与本地方差#1相关的训练的回合。
可选地，索引#2可以指示网络层#1的标识和与本地方差#1相关的训练的回合的标识。
例如，索引#2可以包括网络层#1的标识和与本地方差#1相关的训练的回合的标识。可选地，索引#2可以包括两个字段(也可称为信元)，分别承载网络层#1的标识和与本地方差#1相关的训练的回合的标识。
再如，索引#2可以包括网络层#1的标识和与本地方差#1相关的训练的回合的标识的联合编码结果。可选地，索引#2可以包括一个字段(也可称为信元)，该字段承载网络层#1的标识和与本地方差#1相关的训练的回合的标识的联合编码结果。
可选地，索引#2可以指示网络层#1的标识和与本地方差#1相对应的AI模型的标识。
关于索引#2的具体描述可以参考前文的索引#1，此处不再赘述。
索引#3和全局方差#1的相关性可以参考前文索引#1和全局均值#1的相关性，以及索引#2和本地方差#1的相关性的描述，索引#3的描述可以参考索引#1和索引#2的描述，此处不再赘述。
索引#4和本地均值#1的相关性可以参考前文索引#1和全局均值#1的相关性，以及索引#2和本地方差#1的相关性的描述，索引#4的描述可以参考索引#1和索引#2的描述，此处不再赘述。
下面对索引#1和索引#2的关联关系进行说明。
可选地，索引#1和索引#2的索引值相同。
若两个索引的索引值相同，则该两个索引具有关联关系。在该情况下，索引#1和索引#2具有关联关系，即索引#1和索引#2的索引值相同。
可选地，索引#1和索引#2具有预定义的关联关系。
即索引#1和索引#2的关联关系可以是预定义的。
可选地，索引#1和索引#2具有预配置的关联关系。
即方法500还可以包括：中心节点向多个子节点发送信息#5，信息#5指示两个索引之间的关联关系。
示例性地，两个索引之间具有关联关系可以为两个索引的索引值满足预设条件#1。该预设条件#1可以是预定义的，或者，也可以是由信息#5指示的。
例如，两个索引之间具有关联关系可以为两个索引的索引值之间的差值为1。即若两个索引的索引值之间的差值为1，则该两个索引具有关联关系。在该情况下，索引#1和索引#2具有关联关系，即索引#1和索引#2的索引值之间的差值为1。比如，子节点在接收到信息a以及和信息a相关的索引a后，在发送与该信息a相关的信息b时，可以将索引a的索引值加1，以得到索引b的索引值，发送信息b和索引b，中心节点在接收到信息b和索引b时，根据索引b与索引a的差值为1可以根据确定索引a与索引b相关，从而确定信息b与信息a相关。
图7示出了本申请实施例的一种关联关系的格式示意图。示例性地，该格式可以由中心节点下发给子节点。或者，该格式也可以是预定义的。
子节点可以根据该格式确定均值和方差是否相关。例如，如图7所示，子节点可以在“模型的回合索引号t”的位置处填上与当前的本地模型相关的训练的回合的编号。在接收到全局均值#1和索引#1时，可以在“均值的回合索引号t'”和“均值的层序号l”位置处分别填上索引#1指示的与全局均值#1相关的训练的回合的编号和全局均值#1对应的网络层#1的层号。通过对比“模型的回合索引号t”和“均值的回合索引号t'”可以判断与全局均值#1相关的训练的回合的编号与当前的本地模型的回合的编号是否相关，比如是否相同。类似地，子节点在接收到全局方差#1和索引#3时，可以将“方差的回合索引号t””和“方差的层序号l'”位置处分别填上索引#3指示的与全局方差#1相关的训练的回合的编号和全局方差#1对应的网络层#1的层号。通过对比“方差的回合索引号t””和“均值的回合索引号t'”以及“方差的层序号l'”和“均值的层序号l”可以判断接收到的全局方差#1和全局均值#1是否相关，比如，若完全相同，则说明两者相关，否则，两者不相关。
如前所述，索引#3与以下至少一项具有关联关系：索引#1或索引#2。
如前所述，索引#4与以下至少一项具有关联关系：索引#1、索引#2或索引#3。
两个索引之间具有关联关系的描述可以参考索引#1和索引#2，此处不再赘述。
进一步，方法500还可以包括：子节点#1接收来自中心节点的AI模型的模型参数。
各个子节点在接收到来自中心节点的AI模型的模型参数后，可以根据该AI模型的模型参数更新本地模型，经过该更新的本地模型的模型参数即为中心节点下发的AI模型的模型参数，或者说，经过该更新的本地模型即为中心节点下发的AI模型。
示例性地，该AI模型的模型参数可以为步骤540中用于计算本地均值的AI模型的模型参数。
例如，各个子节点可以基于本地的训练数据统计中心节点下发的AI模型的网络层#1的输入的均值，并上报给中心节点。换言之，在进入新的回合后，各个子节点在开始本地训练之前，可以基于中心节点下发的最新的全局模型统计网络层#1的输入的均值，而不是在本地训练开始之后本地模型的参数发生改变时，再统计网络层#1的输入的均值。
在本申请实施例的方案中，各个子节点在本轮训练开始之前，可以基于接收到的AI模型的模型参数计算BN层的输入的均值，有利于保证各个子节点使用统一的AI模型来计算BN层的输入的均值，或者说，有利于保证各个子节点计算本地模型的BN层的输入的均值时各个子节点的本地模型采用统一的AI模型，从而有利于提高全局均值和全局方差的准确性。
示例性地，该AI模型的模型参数可以为步骤510中用于计算本地方差的AI模型的模型参数，或者说，该AI模型的模型参数可以为与全局均值#1对应的AI模型的模型参数。
例如，各个子节点可以基于本地的训练数据和全局均值#1计算中心节点下发的AI模型的网络层#1的输入的方差，并上报给中心节点。换言之，在进入新的回合后，各个子节点在开始本地训练之前，可以基于本地的训练数据和中心节点下发的最新的全局模型计算网络层#1的输入的方差，而不是在本地训练开始之后本地模型的参数发生改变时，再计算网络层#1的输入的方差。
在本申请实施例的方案中，各个子节点在本轮训练开始之前，可以基于接收到的AI模型的模型参数计算BN层的输入的方差，有利于保证各个子节点使用统一的AI模型来计算BN层的输入的方差，或者说，有利于保证各个子节点计算本地模型的BN层的输入的方差时各个子节点的本地模型采用统一的AI模型，从而有利于提高全局方差的准确性。
进一步地，中心节点和/或子节点可以判断子节点上的AI模型与中心节点上的全局模型是否一致。下面以方案#1和方案#2为例进行说明。
方案#1：
在方案#1中可以由子节点判断本地模型与中心节点上的全局模型是否一致。
在一种可能的实现方式中，子节点#1可以接收来自中心节点的信息，该信息可以用于确定子节点#1是否需要从中心节点获取AI模型的模型参数。
该AI模型可以为最新的全局模型，该信息可以用于确定子节点#1上的AI模型是否为最新的全局模型。
作为一个示例，索引#1可以用于确定子节点#1上的AI模型是否为全局均值#1对应的AI模型，或者，是否是最新的全局模型。
全局均值#1对应的AI模型通常为最新的全局模型。
示例性地，索引#1可以用于指示与全局均值#1相关的训练的回合。
子节点#1可以根据索引#1指示的回合和子节点#1的回合是否相同确定本地模型是否为最新的全局模型。
例如，索引#1可以包括与全局均值#1相关的训练的回合的编号，若索引#1指示的回合的编号大于子节点#1的回合的编号，则说明子节点#1未接收到中心节点下发的新的全局模型。由此，子节点#1可以根据索引#1指示的回合的编号和子节点#1的回合的编号来确定本地模型是否为最新的全局模型。
示例性地，索引#1可以用于指示全局均值#1对应的AI模型。
子节点#1可以根据索引#1指示的AI模型的版本和本地模型的版本是否相同确定本地模型是否为最新的全局模型。
例如，索引#1可以包括全局均值#1对应的AI模型的版本号，若本地模型的版本号与索引#1指示的AI模型的版本号不同，则说明子节点#1未接收到中心节点下发的新的全局模型。由此，子节点#1可以根据索引#1指示的AI模型的版本号和本地模型的版本号来确定本地模型是否为最新的全局模型。
以上仅为示例，中心节点还可以发送其他信息，以使得子节点#1能够判断本地模型是否为最新的全局模型。
进一步地，可选地，方法500还可以包括：子节点#1向中心节点发送信息#6(第三信息的一例)。信息#6用于请求AI模型的模型参数。
例如，在子节点#1确定本地模型不是全局均值#1对应的AI模型的情况下，子节点#1可以向中心节点请求该AI模型的模型参数，进而基于该AI模型的模型参数更新本地模型，这样，子节点#1可以基于全局均值#1对应的AI模型来计算本地方差#1。或者说，在子节点#1确定本地模型不是最新的全局模型的情况下，子节点#1可以向中心节点请求最新的全局模型的模型参数，进而基于最新的全局模型的模型参数更新本地模型，这样，子节点#1可以基于最新的全局模型来计算本地方差#1。
下面以索引#1包括与全局均值#1相关的训练的回合的编号为例进行说明。
可选地，在索引#1指示的回合的编号大于子节点#1的回合的编号的情况下，子节点#1向中心节点发送信息#6。
在发送信息#6后子节点#1接收到中心节点发送的AI模型的模型参数。子节点#1基于该模型参数对本地模型进行更新，即将本地模型的模型参数更新为中心节点发送的AI模型的模型参数，然后基于更新后的本地模型计算本地方差#1。
例如，子节点#1接收到来自中心节点的信息#1。该信息#1中的索引#1指示的回合的编号为2，即该信息#1所指示的全局均值#1是第二回合中的全局均值#1。子节点#1的回合的编号为1，即当前子节点#1上的本地模型是基于第一回合接收到的全局模型的模型参数得到的。子节点#1可以确定该全局均值#1是用于在第二回合中计算本地方差#1，即基于第二回合的全局模型计算本地方差#1，而当前的本地模型不是第二回合的全局模型。子节点#1可以向中心节点发送信息#6，以请求第二回合的全局模型的模型参数。中心节点在接收到信息#6后，可以向子节点#1发送第二回合的全局模型的模型参数。子节点#1基于第二回合的全局模型的模型参数对本地模型进行更新，并将与本地模型相关的训练的回合的编号更新为2。子节点#1基于更新后的本地模型根据全局均值#1计算本地方差#1。
在索引#1指示的回合的编号与子节点#1的回合的编号相同的情况下，子节点#1可以基于当前的本地模型计算本地方差#1。
这样，有利于保证各个子节点均是基于统一的AI模型来计算本地方差，从而有利于进一步提高BN层的全局方差的准确性。
方案#2：
在方案#2中可以由中心节点判断本地模型与中心节点上的全局模型是否一致。
在一种可能的实现方式中，子节点#1可以向中心节点发送信息，该信息可以用于确定是否需要向子节点#1发送AI模型的模型参数。
该AI模型可以为最新的全局模型，该信息可以用于确定子节点#1上的AI模型是否为最新的全局模型。
作为一个示例，索引#4可以用于确定子节点#1上的AI模型是否为全局均值#1对应的AI模型，或者，是否是最新的全局模型。
示例性地，索引#4可以用于指示与本地均值#1相关的训练的回合。
中心节点可以根据索引#4指示的回合和中心节点的回合是否相同确定子节点#1上的本地模型是否为最新的全局模型。
例如，索引#4可以包括与本地均值#1相关的训练的回合的编号。索引#4指示的回合的编号小于中心节点的回合的编号，则说明子节点#1未接收到中心节点下发的新的全局模型。由此，中心节点可以根据索引#4指示的回合的编号和中心节点的回合的编号来确定子节点#1上的本地模型是否为最新的全局模型。
示例性地，索引#4可以用于指示本地均值#1对应的AI模型。
中心节点可以根据索引#4指示的AI模型的版本和全局模型的版本是否相同确定子节点#1的本地模型是否为最新的全局模型。
例如，索引#4可以包括本地均值#1对应的AI模型的版本号。若全局模型的版本号与索引#4指示的AI模型的版本号相同，则说明子节点#1未接收到中心节点下发的新的全局模型。由此，中心节点可以根据索引#4指示的AI模型的版本号和全局模型的版本号来确定子节点#1上的本地模型是否为最新的全局模型。
以上仅为示例，子节点#1还可以发送其他信息，以使得中心节点能够判断子节点#1上的本地模型是否为最新的全局模型。
作为另一个示例，索引#2可以用于确定子节点#1上的AI模型是否为全局均值#1对应的AI模型，或者，是否是最新的全局模型。
示例性地，索引#2可以用于指示与本地方差#1相关的训练的回合。
示例性地，索引#2可以用于指示本地方差#1对应的AI模型。
具体的判断方式可以参考前文中的索引#4的相关描述，将索引#4替换为索引#2，将本地均值#1替换为本地方差#1即可。
进一步地，方法500可以包括：中心节点再次向子节点#1发送AI模型的模型参数。
即中心节点向子节点#1发送了两次AI模型的模型参数，该两次AI模型的模型参数相同。
例如，在中心节点确定子节点#1上的本地模型不是最新的全局模型的情况下，中心节点可以向子节点发送最新的全局模型的模型参数，使得子节点#1可以基于最新的全局模型的模型参数更新本地模型。下面以索引#4包括与本地均值#1相关的回合的编号为例进行说明。
可选地，在索引#4指示的回合的编号小于中心节点的回合的编号的情况下，中心节点再次向子节点#1发送AI模型的模型参数。
在该情况下，中心节点在接收到来自子节点#1的信息#4前后向子节点#1发送了相同的AI模型的模型参数。
在发送信息#4后，子节点#1接收到中心节点发送的AI模型的模型参数。子节点#1基于该AI模型的模型参数对本地模型进行更新，即将本地模型的模型参数更新为中心节点发送的AI模型的模型参数，然后基于更新后的本地模型计算本地方差#1。这样，子节点#1可以基于最新的全局模型来计算本地方差#1。
例如，中心节点接收到来自子节点#1的信息#4。该信息#4中的索引#1指示的回合的编号为2，即该信息#4所指示的本地均值#1是第二回合中的本地均值#1。与全局模型相关的训练的回合的编号为3，即当前的全局模型为第三回合聚合得到的全局模型。中心节点可以确定该本地均值#1是用于在第二回合中计算全局均值#1，而当前的全局模型不是第二回合的全局模型。中心节点可以向子节点#1再次发送第三回合的全局模型的模型参数。子节点#1基于第三回合的全局模型的模型参数对本地模型进行更新，并将与本地模型相关的训练的回合的编号更新为3。子节点#1基于更新后的本地模型根据全局均值#1计算本地方差#1。
在该情况下，中心节点可以不使用该信息#4指示的本地均值#1来计算全局均值#1。换言之，中心节点可以忽略该信息#4指示的本地均值#1。
可替换地，子节点#1可以在发送信息#4后接收到最新的全局模型的模型参数后，再次执行步骤540。在该情况下，中心节点可以基于再次发送的信息#4所指示的本地均值#1来计算全局均值#1。或者，中心节点也可以不使用子节点#1再次上报的本地均值#1来计算全局均值#1。
这样，有利于保证用于计算全局均值的各个子节点的本地均值均是基于统一的AI模型来计算的，从而有利于提高BN层的全局均值的准确性。
应理解，以上仅为示例，中心节点也可以使用在再次向子节点#1发送最新的全局模型的模型参数之前接收到的信息#4所指示的本地均值#1来计算全局均值#1。
下面以索引#2包括与本地方差#1相关的训练的回合的编号为例进行说明。
可选地，在索引#2指示的回合的编号小于中心节点的回合的编号的情况下，中心节点再次向子节点#1发送AI模型的模型参数。
在该情况下，中心节点在接收到来自子节点#1的信息#2前后向子节点#1发送了相同的AI模型的模型参数。
在该情况下，中心节点可以不使用该信息#2指示的本地方差#1来计算全局方差#1。换言之，中心节点可以忽略该信息#2指示的本地方差#1。
可替换地，子节点#1可以在发送信息#2后接收到最新的全局模型的模型参数后，基于最新的全局模型的模型参数对本地模型进行更新，然后基于更新后的本地模型计算本地方差#1。这样，子节点#1可以基于最新的全局模型来计算本地方差#1。子节点#1可以再次发送信息#2，在该情况下，中心节点可以基于再次发送的信息#2所指示的本地方差#1来计算全局方差#1。或者，中心节点也可以不使用子节点#1再次上报的本地方差#1来计算全局方差#1。
这样，有利于保证用于计算全局方差的各个子节点的本地方差均是基于统一的AI模型来计算的，从而有利于提高BN层的全局方差的准确性。
应理解，以上仅为示例，中心节点也可以使用在再次向子节点#1发送最新的全局模型的模型参数之前接收到的信息#2所指示的本地方差#1来计算全局方差#1。
以上仅为示例，子节点#1还可以通过其他信息使得中心节点能够判断子节点#1上的AI模型是否为最新的全局模型。进一步地，中心节点可以根据判断结果确定是否需要再次下发最新的全局模型的模型参数。中心节点可以根据判断结果确定子节点#1上报的本地均值#1和/或本地方差#1是否是基于最新的全局模型确定的，并基于此确定是否要使用子节点#1上报的本地均值#1和/或本地方差#1来计算全局均值#1和/或全局方差#1。
全局方差#1和全局均值#1可以用于子节点上的AI模型的训练。
示例性地，子节点#1可以基于全局方差#1和全局均值#1执行一次或多次前向传播和反向传播来更新本地模型的模型参数。具体的训练过程可以参考后文，此处不展开描述。
应理解，图5中仅以两个子节点作为示例，不对本申请实施例中的子节点的数量构成限定。
图8示出了本申请实施例的一种通信的方法的示意性流程图。图8所示的方法800可以视为方法500的一种具体实现方式。具体描述可以参考方法500，为避免重复，在描述方法800时适当省略部分描述。在该方案中，中心节点聚合各个子节点上传的本地模型的参数，以生成新的全局模型。中心节点将新的全局模型的参数下发给各个子节点。子节点可以依据同一个模型计算BN层的特征的均值，并上传该均值给中心节点。中心节点聚合各个子节点上传的BN层的特征的均值，以生成BN层的特征的全局均值，将全局均值下发给各个子节点。子节点根据全局均值计算BN层的特征的方差，并上传该方差给中心节点。中心节点聚合各个子节点上传的BN层的特征的方差，以生成BN层的特征的全局方差。为了便于描述方法800中主要以中心节点为网络设备侧的设备，子节点为终端设备侧的设备为例进行说明。例如，中心节点可以是基站。网络设备侧的设备可以是网络设备，或者是网络设备中的模块(如芯片)，或者是包含有网络设备功能的软件(如控制子系统)，或者是与网络设备通信的其他设备，比如AI网元，该AI网元为服务器，如OTT设备或云服务器等，或者，包括网络设备和与网络设备通信的其他设备。终端设备侧的设备可以是终端设备，或者是终端设备中的模块(如芯片)，或者是包含有终端设备功能的软件(如控制子系统)，或者是与终端设备通信的其他设备，比如AI网元，该AI网元为服务器，如OTT设备或云服务器等，或者，包括终端设备和与终端设备通信的其他设备。中心节点也可以替换为网络设备侧的设备以外的其他设备，子节点也可以替换为终端设备侧的设备以外的其他设备。具体描述参考前文，此处不再赘述。
为了便于描述，图8中仅以一个子节点为例进行说明，其他子节点的行为可以参考该子节点。
如图8所示，方法800可以包括如下步骤。
801，中心节点向子节点发送模型配置信息和模型训练信息。
示例性地，中心节点可以以广播的方式向所有子节点发送相同的模型配置信息和模型训练信息。
示例性地，模型配置信息可以包括以下至少一项：初始AI模型的类型、初始AI模型的结构信息或初始AI模型的模型参数。例如，初始AI模型的类型可以包括以下至少一项：DNN、CNN或转换器(Transformer)模型。
例如，初始AI模型的结构信息可以包括以下至少一项：DNN的隐藏层数量、每一层或部分层神经元数量与激活函数或批规范化层所在的位置。
初始AI模型的类型和初始AI模型的结构信息可以以配置文本方式体现，也可以是可以编译出相应初始AI模型的代码脚本。
初始AI模型的模型参数可以是由中心节点通过某一种策略生成的。例如，初始AI模型的模型参数可以是由中心节点随机生成的或预训练生成的。或者，初始AI模型的模型参数也可以是由中心节点从其他实体获取的。其中，其他实体可以是与中心节点通信的其他设备，比如服务器，如OTT设备或云服务器等。
示例性地，模型训练信息可以包括以下至少一项：子节点执行梯度更新所使用的优化器，正则化惩罚项(如L2惩罚项)，初始学习率，梯度更新批大小，或者，数据增广方法。
例如，优化器可以为以下任一项：随机梯度下降(stochastic gradient descent，SGD)、均方根传播(root mean square propagation，RMSprop)或自适应动量估计(adaptive moment estimation，Adam)。
数据增广方法可以为一种，也可以为多种。
中心节点下发模型配置信息，该模型配置信息可以指示初始AI模型的模型参数。例如，中心节点处的回合的编号可以设置为1，中心节点进入分布式训练的第一回合。即与初始AI模型相关的训练的回合的编号为1。
进一步地，子节点还可以向中心节点发送本地的训练样本集的数据量。
进一步地，中心节点还可以向子节点发送批规范化层的均值与方差的关联关系的格式。
或者，批规范化层的均值与方差的关联关系的格式也可以是预定义的。
子节点在接收到模型配置信息后，可以根据初始AI模型的模型参数更新本地模型，并将本地的回合的编号设置为1，子节点进入分布式训练的第一回合。即与本地模型相关的训练的回合的编号为1。
802，子节点执行本地模型的训练。
在一个回合中，子节点基于本地的训练数据集执行本地模型的训练，以更新本地模型的参数。该更新过程即迭代过程。在一个回合的训练过程中，本地模型的参数可以进行一次或多次更新。
803，子节点向中心节点反馈更新后的本地模型的模型参数。
作为一个示例，子节点可以在满足条件#1的情况下，向中心节点反馈更新后的本地模型的模型参数。
该条件#1可以理解为模型参数的上传条件。子节点#1在更新本地模型的模型参数后，可以判断是否满足条件#1，若满足则上传更新后的本地模型的模型参数。
示例性地，该条件#1可以为与本地模型的更新次数相关的条件。
例如，条件#1可以为本地模型的更新次数t为常数T的n倍。T为正整数。n为正整数。T可以是预定义的，也可以是由中心节点配置的，或者，还可以是子节点自行确定的。n可以是预定义的，也可以是由中心节点配置的，或者，还可以是子节点自行确定的。
示例性地，条件#1可以与更新后的本地模型的模型参数相关的条件。
例如，条件#1可以为更新后的本地模型的模型参数与当前回合训练开始前的本地模型的模型参数之间的差异大于或等于阈值#1。
当前回合训练开始前的本地模型的模型参数即当前回合中心节点下发的全局模型的模型参数。
阈值#1可以是预定义的，也可以是由中心节点配置的，或者，还可以是子节点自行确定的。
示例性地，条件#1可以为一个回合中本地模型训练的终止条件。在满足终止条件时，可以终止本回合的本地模型的训练。即在本地模型的训练终止的情况下，向中心节点反馈更新后的本地模型的模型参数。该终止条件可以是预定义的，也可以是由中心节点配置的(例如，由模型训练信息指示)，或者，还可以是子节点自行确定的。
作为另一个示例，中心节点可以向各个子节点发送指示信息，以通知各个子节点反馈更新后的本地模型的模型参数。
子节点在接收到该指示信息后，可以将当前的本地模型的模型参数反馈给中心节点。
以上仅为示例，本申请实施例对子节点上报模型参数的触发方式不做限定。
804，中心节点聚合子节点反馈的模型参数，以更新全局模型。
中心节点在接收到一个或多个子节点反馈的模型参数后，执行聚合操作，生成新的全局模型。例如，聚合操作可以为对该多个子节点反馈的模型参数计算平均值。再如，聚合操作可以为对该多个子节点反馈的模型参数计算加权平均值。
该一个或多个子节点可以通过多种方式确定。例如，该接收过程可以维持一段时间后结束，该段时间内接收的子节点反馈的模型参数可以用于聚合得到全局模型。或者，中心节点在接收到一定数量的子节点反馈的模型参数后结束。该一定数量的子节点反馈的模型参数可以用于聚合得到全局模型。
本申请实施例对聚合操作的具体实现方式不做限定。
805，中心节点向子节点发送更新后的全局模型的模型参数。
中心节点在更新全局模型后，或者，中心节点在下发更新后的全局模型的模型参数后，中心节点处的回合的编号加1，即与该更新后的全局模型相关的训练的回合的编号加1。
例如，中心节点可以向子节点广播更新后的全局模型的模型参数。
806，子节点根据接收到的模型参数更新本地模型。
子节点在接收到来自中心节点的全局模型的模型参数后，或者，子节点在根据接收到的模型参数更新本地模型后，本地的回合的编号加1，即与本地模型相关的训练的回合的编号加1。
807，子节点计算本地模型的BN层(如网络层#1)的输入的均值(如本地均值#1)。
子节点计算本地模型中的一个或多个BN层的输入的均值。
808，子节点向中心节点发送指示BN层的输入的均值的信息和相关的索引(如索引#4)。
与一个均值相关的索引可以用于指示与该均值相关的训练的回合的编号以及与该均值相关的BN层的标识。根据该索引可以确定该均值对应于哪一个回合以及对应于哪一个BN层，或者说，根据该索引可以确定该均值用于哪一个回合的哪一个BN层对应的全局均值的计算。
图9示出了一个信息#4的格式的示意图。图9仅以AI模型包括两个BN层为例。该两个BN层的输入的均值分别为均值b和均值b'。
如图9所示，信息#4包括两个均值，即均值b和均值b'。在一个均值之后与该均值相邻的索引即为与该均值相关的索引。如图9所示，与均值b相关的索引的索引值包括回合的编号c和层号L1，与均值b'相关的索引的索引值包括回合的编号c和层号L2。
809，中心节点根据聚合子节点反馈的均值，以得到全局均值。
以图9中的均值b为例，中心节点根据信息#4可以确定与均值b相关的训练的回合的编号为c，与均值b相关的层号为L1，即均值b用于第c回合的层号为L1的BN层对应的全局均值的计算。
中心节点可以根据索引聚合各个子节点的各个BN层的输入的均值，以得到各个BN层对应的全局均值。
810，中心节点向子节点发送指示全局均值(如全局均值#1)的信息和相关的索引(如索引#1)。
与一个全局均值相关的索引可以用于指示与该全局均值相关的训练的回合的编号以及与该全局均值相关的BN层的标识。根据该索引可以确定该全局均值对应于哪一个回合以及对应于哪一个BN层，或者说，根据该索引可以确定该全局均值用于哪一个回合的哪一个BN层的输入的方差的计算。
图10示出了一个信息#1的格式的示意图。图10仅以AI模型包括两个BN层为例。该两个BN层对应的全局均值分别为全局均值a和全局均值a'。
如图10所示，信息#1包括两个均值，即全局均值a和全局均值a'。在一个均值之后与该均值相邻的索引即为与该均值相关的索引。如图10所示，与全局均值a相关的索引的索引值包括回合的编号c'和层号L1，与全局均值b相关的索引的索引值包括回合的编号c'和层号L2。
811，子节点根据索引判断是否需要获取最新的全局模型的模型参数。
若是，则执行步骤812，若否，则执行步骤815。
示例性地，子节点可以根据索引#1所指示的回合的编号确定是否需要获取最新的全局模型的模型参数。以图9和图10为例，子节点向中心节点发送信息#4，其中，索引#4指示的回合的编号为编号c，即子节点处于第c回合，与该子节点的本地模型相关的训练的回合的编号为c。中心节点向该子节点发送信息#1，其中，索引#1指示的回合的编号为c'，即中心节点处于第c'回合，与全局模型相关的训练的回合的编号为编号c'。若c和编号c'相同，则子节点可以确定当前的本地模型即为最新的全局模型，执行步骤815。若c小于编号c'，则子节点可以确定当前的本地模型不是最新的全局模型，则执行步骤812。
812，子节点向中心节点发送信息(如信息#6)以请求AI模型的模型参数。
子节点可以向中心节点发送信息以请求中心节点下发最新的全局模型的模型参数。
813，中心节点向子节点发送最新的全局模型的模型参数。
该模型参数即为步骤805中下发的模型参数。
814，子节点根据接收到的模型参数更新本地模型。
子节点在接收到来自中心节点的全局模型的模型参数后，或者，子节点在根据接收到的模型参数更新本地模型后，本地的回合的编号加1，即与本地模型相关的训练的回合的编号加1。
815，子节点根据全局均值计算本地模型的BN层的输入的方差(如本地方差#1)。
示例性地，子节点可以在与全局均值相关的索引所指示的回合的编号等于与本地模型相关的训练的回合的编号的情况下执行步骤815。
可替换地，子节点可以在发送信息#6之后接收到中心节点下发的模型参数的情况下执行步骤815。
子节点可以根据索引#1确定中心节点下发的全局均值与本地模型的各个BN层的输入的方差之间的关联关系，进而根据各个BN层对应的全局均值计算本地模型中的各个BN层的输入的方差。
以图10中的全局均值a为例，中心节点根据信息#1可以确定与全局均值a相关的训练的回合的编号为c'，与全局均值a相关的层号为L1，即全局均值a用于第c'回合的本地模型中层号为L1的BN层的输入的方差的计算。
应理解，以上步骤811至步骤814为可选步骤。若方法800包括步骤811至步骤814，有利于保证在步骤815中各个子节点可以基于最新的全局模型的参数来计算本地方差，即有利于保证各个子节点基于统一的AI模型来计算本地方差。若方法800不包括步骤811至步骤814，则子节点可能会基于过时的模型来计算本地方差，该过时的模型是根据之前回合下发的全局模型得到的。
816，子节点向中心节点发送指示BN层的输入的方差的信息和相关的索引(如索引#2)。
与一个方差相关的索引可以用于指示与该方差相关的训练的回合的编号以及与该方差相关的BN层的标识。根据该索引可以确定该方差对应于哪一个回合以及对应于哪一个BN层，或者说，根据该索引可以确定该方差用于哪一个回合的哪一个BN层对应的全局方差的计算。
图11示出了一个信息#2的格式的示意图。图11以AI模型包括两个BN层为例。该两个BN层的输入的方差分别为方差d和方差d'。
如图11所示，信息#2包括两个方差，即方差d和方差d'。在一个方差之后与该方差相邻的索引即为与该方差相关的索引。如图11所示，与方差d相关的索引的索引值包括回合的编号c”和层号Ll，与方差d'相关的索引的索引值包括回合的编号c”和层号L2。
817，中心节点根据索引聚合子节点反馈的方差，以得到全局方差(如全局方差#1)。
以图11中的方差a为例，中心节点根据信息#2可以确定与方差d相关的训练的回合的编号为c”，与方差d相关的层号为L1，即方差d用于第c”回合的层号为L1的BN层对应的全局方差的计算。
中心节点可以根据索引聚合各个子节点的各个BN层的输入的方差，以得到各个BN层对应的全局方差。
818，中心节点向子节点发送指示全局方差的信息和相关的索引(如索引#3)。
与一个全局方差相关的索引可以用于指示与该全局方差相关的训练的回合的编号以及与该全局方差相关的BN层的标识。根据该索引可以确定该全局方差对应于哪一个回合以及对应于哪一个BN层，或者说，根据该索引可以确定该全局方差在训练过程中用作哪一个回合的哪一个BN层的输入。
信息#3的格式可以参考信息#1，此处不再赘述。
返回步骤802，重复执行步骤802至步骤818，直至全局模型训练完成，例如，直至全局模型收敛，比如模型性能达到需求。
在步骤802中，子节点在接收到全局均值和全局方差后，可以基于全局均值和全局方差进行本地模型的训练。
下面对接收到全局均值和全局方差的情况下的训练过程进行示例性说明。
示例性地，在一个回合中，训练过程可以分为两个阶段：阶段1和阶段2。在阶段1中，子节点可以使用各个BN层对应的全局均值和全局方差执行一次或多次前向传播和反向传播更新本地模型的模型参数。以前文中的公式(2)为例，在阶段1中，将公式(2)中的均值和方差分别替换为全局均值和全局方差，以计算各个BN层的输入，并基于此进行执行一次或多次前向传播和反向传播更新以本地模型的模型参数。在阶段2中，可以基于阶段1更新后的本地模型统计各个BN层的输入的均值和方差。以一个BN层为例，即将一批训练数据输入至更新后的本地模型中，以得到基于该批训练数据的BN层的输入，进而计算该BN层的输入的均值和方差。子节点可以基于统计得到的各个BN层的输入的均值与方差执行一次或多次前向传播和反向传播以更新本地模型的参数。例如，在阶段2中，可以采用公式(2)计算各个BN层的输出，并基于此执行一次或多次前向传播和反向传播更新以本地模型的模型参数。在该训练过程后继续执行步骤803至步骤818。
图12是本申请提供的另一种通信的方法的示意性流程图。示例性地，图12所示的方法1200可以应用于分布式训练的场景。在方法1200中，中心节点聚合子节点反馈的BN层的输入的均值和BN层的输入的平方的均值，根据聚合结果得到BN层对应的全局方差。
如图12所示，方法1200可以包括如下步骤。
1210，多个子节点(如图中的子节点#1和子节点#2)向中心节点发送信息#7(第七信息的一例)。信息#7包括本地均值#2(第三均值的一例)和本地均值#3(第四均值的一例)的信息。本地均值#2为子节点上的AI模型中的网络层#1的输入，本地均值#3为该输入的平方的均值。
方法1200中的网络层#1的描述可以参考方法500中的网络层#1的描述，方法1200中的本地均值#2的描述可以参考方法500中的本地均值#1的描述，在方法1200中不再赘述。
本地均值#2和本地均值#3可以用于全局方差#2(第三方差的一例)的计算。
来自多个子节点的信息#7分别包括指示该多个子节点的本地均值#2和本地均值#3的信息。或者说，来自多个子节点的信息#7所指示的本地均值#2和本地均值#3分别为该多个子节点的本地均值#2和本地均值#3。
步骤1210中的多个子节点为前述M个子节点中的部分或全部。
该多个子节点可以包括子节点#1，以子节点#1为例，子节点#1向中心节点发送信息#7。来自子节点#1的信息#7所指示的本地均值#2为子节点#1的本地均值#2。来自子节点#1的信息#7所指示的本地均值#3为子节点#1的本地均值#3。
多个子节点也可以替换为多个设备。
进一步地，方法1200还可以步骤1220。
1220，中心节点根据来自一个或多个子节点的信息#7所指示的本地均值#2和本地均值#3确定全局方差#2。
全局方差#2即为网络层#1对应的全局方差，或者说，与网络层#1相关的全局方差。全局方差#2的确定方式可以参考后文的描述。
步骤1220中的一个或多个子节点为前述M个子节点的部分或全部。
步骤1220中的一个或多个子节点属于步骤1210中的多个子节点。
进一步地，方法1200还可以包括步骤1230。
1230，中心节点向多个子节点发送信息#8(第八信息的一例)。信息#8包括指示全局均值#2(第五均值的一例)和全局方差#2的信息。
全局均值#2即为网络层#1对应的全局均值，或者说，与网络层#1相关的全局均值。全局均值#2的相关描述可以参考方法500中的全局均值#1。
全局均值#2和全局方差#2可以用于子节点上的AI模型的训练。
步骤1230中的多个子节点为前述M个子节点的部分或全部。
步骤1230中的多个子节点与步骤1210中的多个子节点可以相同，也可以不同。
该多个子节点可以包括子节点#1，以子节点#1为例，子节点#1接收到来自中心节点的信息#8后，可以根据全局均值#2和全局方差#2对本地模型进行训练。
下面对如何确定本地均值#3进行说明。
子节点上的AI模型的网络层#1的输入的平方的均值是基于该子节点上的一批(batch)训练数据确定的。
以子节点m为例，子节点m上的一批训练数据Bm的数据量为|Bm|。|Bm|为正整数。将该批训练数据Bm输入至子节点m上的AI模型中，相应地，可以得到网络层#1的|Bm|个输入，该|Bm|个输入的平方的平均值即可作为子节点m上的AI模型的网络层#1的输入的平方的均值。
下面对如何确定全局均值#2进行说明。
中心节点可以聚合一个或多个子节点的本地均值#2，以得到全局均值#2。
该一个或多个子节点可以为M个子节点的部分或全部，即中心节点可以基于部分或全部子节点的本地均值#2确定全局均值#2。
全局均值#2的具体确定方式可以参考方法500中的全局均值#1的确定方式，将全局均值#1替换为全局均值#2，将本地均值#1替换为本地均值#2，将信息#4替换为信息#7即可。
下面对如何确定全局方差#2进行说明。
中心节点可以聚合一个或多个子节点的本地均值#2和本地均值#3，根据本地均值#2的聚合结果和本地均值#3的聚合结果计算全局方差#2。本地均值#2的聚合结果可以为全局均值#2。
该一个或多个子节点可以为M个子节点的部分或全部。
示例性地，中心节点聚合该一个或多个子节点的本地均值#2，可以包括：中心节点计算该多个子节点的本地均值#2的平均值，将其作为本地均值#2的聚合结果。
可替换地，中心节点聚合该一个或多个子节点的本地均值#2，可以包括：中心节点计算该多个子节点的本地均值#2的加权平均值，将其作为本地均值#2的聚合结果。例如，在加权平均值的计算过程中，各个子节点的本地均值#2对应的权重可以与各个子节点的批训练数据的数据量呈正相关关系。
示例性地，中心节点聚合该一个或多个子节点的本地均值#3，可以包括：中心节点计算该多个子节点的本地均值#3的平均值，得到本地均值#3的聚合结果。
可替换地，中心节点聚合该一个或多个子节点的本地均值#3，可以包括：中心节点计算该多个子节点的本地均值#3的加权平均值，将其作为本地均值#3的聚合结果。例如，在加权平均值的计算过程中，各个子节点的本地均值#3对应的权重可以与各个子节点的批训练数据的数据量呈正相关关系。
根据本地均值#2的聚合结果和本地均值#3的聚合结果计算全局方差#2，可以为，根据本地均值#2的聚合结果的平方以及本地均值#3的聚合结果计算全局方差#2。
可选地，全局方差#2满足如下公式：
其中，表示AI模型的第l层对应的BN层(即网络层#1)对应的全局方差，即全局方差#2。表示该BN层的输入的平方的均值，即来自一个或多个子节点的信息#7所指示的本地均值#3的聚合结果，Ε[hl(x)]表示该BN层对应的全局均值，即全局均值#2，即来自一个或多个子节点的信息#7所指示的本地均值#2的聚合结果。B表示多个子节点上的训练数据集。
在本申请实施例的方案中，中心节点基于子节点上的AI模型的BN层的输入的均值和该输入的平方的均值来计算与全局方差#2，有利于避免各个子节点的本地数据非独立同分布的影响，以期使得中心节点能够聚合得到更准确的全局方差#2，从而有利于提高分布式训练过程中的批规范化处理的准确性，提高分布式训练的性能。
同时，在本申请实施例中，子节点和中心节点交互次数更少，有利于减少交互开销。
可选地，信息#7还包括索引#5(第五索引的一例)。
索引#5与以下至少一项相关：本地均值#2或本地均值#3。
根据索引#5可以确定本地均值#2和本地均值#3用于计算全局方差#2。
下面对索引#5和本地均值#2的相关性进行说明。
索引#5和本地均值#2相关，可以理解为，索引#5和本地均值#2均与同一个回合和/或同一个网络层相关。
索引#5和本地均值#2的相关性可以通过多种方式体现。
示例性地，索引#5和本地均值#2可以承载于同一信令。即信息#7可以承载于同一信令中。
可替换地，索引#5和本地均值#2可以分别承载于两个信令中。即信息#7可以承载于两个信令中。该两个信令相关。
下面基于两个示例(示例#5和示例#6)对索引#1所指示的信息进行说明。
示例#5：
作为一个示例，索引#5可以指示与本地均值#2和/或本地均值#3相关的训练的回合。
中心节点在接收到信息#7后，可以根据索引#5确定本地均值#2和本地均值#3为哪一个回合中计算得到的，进而可以在该回合中基于该本地均值#2和本地均值#3计算全局方差#2。
可选地，索引#5可以指示与本地均值#2和/或本地均值#3相关的训练的回合的标识。
例如，索引#5可以包括与本地均值#2和/或本地均值#3相关的训练的回合的编号。
可选地，索引#5可以指示本地均值#2和/或本地均值#3对应的AI模型的标识。
索引#5可以通过指示本地均值#2和/或本地均值#3对应的AI模型的标识来指示与本地均值#2和/或本地均值#3相关的训练的回合。
中心节点在接收到信息#7后，可以根据索引#5确定本地均值#2和本地均值#3对应于哪个版本的AI模型，即确定该本地均值#2和本地均值#3对应哪一个回合。
示例#6：
作为一个示例，索引#5可以指示网络层#1。即索引#5可以指示本地均值#2或本地均值#3对应的网络层。
示例性地，索引#5可以包括网络层#5的标识。
中心节点可以根据索引#5确定本地均值#2和本地均值#3与哪个网络层关联，从而确定该本地均值#2和本地均值#3用于计算该网络层对应的全局方差。
应理解，以上仅为示例，网络层#1也可以通过其他方式指示。示例性地，信息#7中指示本地均值#2或本地均值#3的信息的位置可以用于指示网络层#1。
示例#5和示例#6也可以结合使用。即索引#5可以指示网络层#1，和，与本地均值#2或本地均值#3相关的训练的回合。
可选地，索引#5可以指示网络层#1的标识，和，与本地均值#2或本地均值#3相关的训练的回合的标识。
示例性地，索引#5可以包括网络层#1的标识和与本地均值#2或本地均值#3相关的训练的回合的标识。可选地，索引#5可以包括两个字段(或信元)，分别承载网络层#1的标识和与本地均值#2或本地均值#3相关的训练的回合的标识。
示例性地，索引#5可以包括网络层#1的标识和与本地均值#2或本地均值#3相关的训练的回合的标识的联合编码结果。可选地，索引#5可以包括一个字段(或信元)，承载网络层#1的标识和与本地均值#2或本地均值#3相关的训练的回合的标识的联合编码结果。
可选地，索引#5可以指示网络层#1的标识和本地均值#2或本地均值#3对应的AI模型的标识。
本地均值#2或本地均值#3中的至少一项与索引#5的相关性的具体描述可以参考方法500中的索引#1和全局均值#1的相关性，以及索引#2和本地方差#1的相关性的描述，索引#5的描述可以参考方法500中的索引#1和索引#2的描述，此处不再赘述。
可选地，信息#8还包括索引#6(第六索引的一例)。
索引#6与以下至少一项相关：全局均值#2或全局方差#2。
全局均值#2或全局方差#2中的至少一项与索引#6的相关性的具体描述可以参考方法500中的索引#1和全局均值#1的相关性，以及索引#2和本地方差#1的相关性的描述，索引#6的描述可以参考方法500中的索引#1和索引#2的描述，此处不再赘述。
可选地，索引#5和索引#6具有关联关系。
根据索引#5和索引#6的关联关系即可确定本地均值#2或本地均值#3中的至少一项与全局方差#2相关，或者说，全局方差#2是基于本地均值#2和本地均值#3确定的。
下面对索引#5和索引#6的关联关系进行说明。
可选地，索引#5和索引#6的索引值相同。
可选地，索引#5和索引#6具有预定义的关联关系。
可选地，索引#5和索引#6具有预配置的关联关系。
即方法1200还可以包括：中心节点向多个子节点发送信息#9，信息#9指示两个索引之间的关联关系。
示例性地，两个索引之间具有关联关系可以为两个索引的索引值满足预设条件#2。该预设条件#2可以是预定义的，或者，也可以是由信息#9指示的。
两个索引之间具有关联关系的描述可以参考方法500中的索引#1和索引#2，此处不再赘述。
进一步，方法1200还可以包括：子节点#1接收来自中心节点的AI模型的模型参数。
示例性地，该AI模型的模型参数可以为用于计算本地均值#2和本地均值#3的AI模型的模型参数。
例如，各个子节点可以基于本地的训练数据统计中心节点下发的AI模型的网络层#1的输入的均值和该输入的平方的均值，并上报给中心节点。换言之，在进入新的回合后，各个子节点在开始本地训练之前，可以基于中心节点下发的最新的全局模型统计网络层#1的输入的均值和输入的平方的均值，而不是在本地训练开始之后本地模型的参数发生改变后，再统计网络层#1的输入的均值和输入的平方的均值。
在本申请实施例的方案中，各个子节点在本轮训练开始之前，可以基于接收到的AI模型的模型参数计算BN层的输入的均值和该输入的平方的均值，有利于保证各个子节点使用统一的AI模型来计算BN层的输入的均值和输入的平方的均值，或者说，有利于保证各个子节点计算本地模型的BN层的输入的均值和输入的平方的均值时各个子节点的本地模型采用统一的AI模型，从而有利于提高全局均值和全局方差的准确性。
进一步地，中心节点和/或子节点可以判断子节点上的AI模型与中心节点上的全局模型是否一致。下面以方案#3为例进行说明。
方案#3：
在方案#3中可以由中心节点判断本地模型与中心节点上的全局模型是否一致。
在一种可能的实现方式中，子节点#1可以向中心节点发送信息，该信息可以用于确定是否需要向子节点#1发送AI模型的模型参数。
该AI模型可以为最新的全局模型，该信息可以用于确定子节点#1上的AI模型是否为最新的全局模型。
作为一个示例，索引#5可以用于确定子节点#1上的AI模型是否为全局均值#2和/或全局方差#2对应的AI模型，或者，是否是最新的全局模型。
示例性地，索引#5可以用于指示与本地均值#2和/或本地均值#3相关的训练的回合。
中心节点可以根据索引#5指示的回合和中心节点的回合是否相同确定子节点#1上的本地模型是否为最新的全局模型。
示例性地，索引#5可以用于指示本地均值#2和/或本地均值#3对应的AI模型。
中心节点可以根据索引#5指示的AI模型的版本和全局模型的版本是否相同确定子节点#1的本地模型是否为最新的全局模型。
具体判断方式可以参考方法500中索引#4的相关描述，此处不再赘述。
以上仅为示例，子节点#1还可以发送其他信息，以使得中心节点能够判断子节点#1上的本地模型是否为最新的全局模型。
进一步地，方法500可以包括：中心节点再次向子节点#1发送AI模型的模型参数。
即中心节点向子节点#1发送了两次AI模型的模型参数，该两次AI模型的模型参数相同。
例如，在中心节点确定子节点#1上的本地模型不是最新的全局模型的情况下，中心节点可以向子节点发送最新的全局模型的模型参数，使得子节点#1可以基于最新的全局模型的模型参数更新本地模型。
可选地，索引#5可以包括与本地均值#2和/或本地均值#3相关的训练的回合的编号，在索引#5指示的回合的编号小于中心节点的回合的编号的情况下，中心节点再次向子节点#1发送AI模型的模型参数。
在该情况下，中心节点在接收到来自子节点#1的信息#7前后向子节点#1发送了相同的AI模型的模型参数。
在发送信息#7后，子节点#1接收到中心节点发送的AI模型的模型参数。子节点#1基于该AI模型的模型参数对本地模型进行更新，即将本地模型的模型参数更新为中心节点发送的AI模型的模型参数。
在该情况下，中心节点可以不使用该信息#7指示的本地均值#2和本地均值#3来计算全局均值#2和/或全局方差#2。换言之，中心节点可以忽略来自子节点#1的信息#7。
可替换地，子节点#1可以在发送信息#7后接收到最新的全局模型的模型参数后，基于最新的全局模型的模型参数计算本地均值#2和本地均值#3，再次执行步骤1210。在该情况下，中心节点可以基于再次发送的信息#7所指示的本地均值#2和本地均值#3来计算全局均值#2和/或全局方差#2。或者，中心节点也可以不使用子节点#1再次上报的本地均值#2和本地均值#3来计算全局均值#2和/或全局方差#2。
这样，有利于保证用于计算全局均值和全局方差的各个子节点的BN层的输入的均值和BN层的输入的平方的均值均是基于统一的AI模型来计算的，从而有利于提高BN层对应的全局均值和全局方差的准确性。
应理解，以上仅为示例，中心节点也可以使用在再次向子节点#1发送最新的全局模型的模型参数之前接收到的信息#7所指示的本地均值#2和本地均值#3。
以上仅为示例，子节点#1还可以通过其他信息使得中心节点能够判断子节点#1上的AI模型是否为最新的全局模型。进一步地，中心节点可以根据判断结果确定是否需要再次下发最新的全局模型的模型参数。中心节点可以根据判断结果确定子节点#1上报的本地均值#2和本地均值#3是否是基于最新的全局模型确定的，并基于此确定是否要使用子节点#1上报的本地均值#2和本地均值#3来计算全局均值#1和全局方差#1。
或者，也可以由子节点判断本地模型是否为最新的全局模型。例如，索引#6可以用于确定子节点#1上的AI模型是否为全局均值#2和/或全局方差#2对应的AI模型，或者，是否是最新的全局模型。
全局方差#2和全局均值#2可以用于子节点上的AI模型的训练。
示例性地，子节点#1可以基于全局方差#2和全局均值#2执行一次或多次前向传播和反向传播来更新本地模型的模型参数。具体的训练过程可以参考前文，此处不再赘述。
应理解，图12中以两个子节点作为示例，不对本申请实施例中的子节点的数量构成限定。
图13示出了本申请实施例的一种通信的方法的示意性流程图。图13所示的方法1300可以视为方法1200的一种具体实现方式。具体描述可以参考方法1200，为避免重复，在描述方法1300时适当省略部分描述。在该方案中，中心节点聚合各个子节点上传的本地模型的参数，以生成新的全局模型。中心节点将新的全局模型的参数下发给各个子节点。子节点可以依据同一个全局模型计算BN层的特征的均值和BN层的特征的平方的均值，并上传给中心节点。中心节点聚合各个子节点上传的BN层的特征的均值，以生成BN层的特征的全局均值。中心节点根据全局均值和各个子节点上传的BN层的特征的平方的均值计算BN层的特征的全局方差。为了便于描述，方法1300中主要以中心节点为网络设备侧设备，子节点为终端设备侧设备为例进行说明。中心节点也可以替换为网络设备侧设备以外的其他设备，子节点也可以替换为终端设备侧设备以外的其他设备。具体描述参考前文，此处不再赘述。
为了便于描述，图13中以一个子节点为例进行说明，其他子节点的行为可以参考该子节点。
如图13所示，方法1300可以包括如下步骤。其中步骤1301至步骤1306的描述与步骤801至步骤806一致，具体描述可以参考步骤801至步骤806。
1301，中心节点向子节点发送模型配置信息和模型训练信息。
1302，子节点执行本地模型的训练。
1303，子节点向中心节点反馈更新后的本地模型的模型参数。
1304，中心节点聚合子节点反馈的模型参数，以更新全局模型。
1305，中心节点向子节点下发更新后的全局模型的模型参数。
1306，子节点根据接收到的模型参数更新本地模型。
1307，子节点计算本地模型的BN层(如网络层#1)的输入的均值(如本地均值#2)和该输入的平方的均值(如本地均值#3)。
子节点计算本地模型中的一个或多个BN层的输入的均值以及该输入的平方的均值。
1308，子节点向中心节点发送指示BN层的输入的均值的信息和指示该输入的平方的均值的信息以及相关的索引(如索引#5)。
1309，中心节点根据索引聚合子节点反馈的BN层的输入的均值和该输入的平方的均值，以确定全局均值(如全局均值#2)和全局方差(全局方差#2)。
中心节点可以根据索引聚合各个子节点的各个BN层的输入的均值，以得到各个BN层对应的全局均值。中心节点可以根据索引聚合各个子节点的各个BN层的输入的平方的均值，以得到各个BN层的输入的平方的均值的聚合结果。中心节点根据各个BN层对应的全局均值和各个BN层的输入的平方的均值的聚合结果确定全局方差。
1310，中心节点向子节点发送指示全局均值的信息和指示全局方差的信息以及相关的索引(如索引#6)。
返回步骤1302，重复执行步骤1302至步骤1310，直至全局模型训练完成，例如，直至全局模型收敛。其中，在步骤1302中，子节点在接收到全局均值和全局方差后，可以基于全局均值和全局方差进行本地模型的训练，训练过程的具体描述可以参考前文的步骤802。
可以理解，在上述一些实施例中，涉及到的信息名称，仅是一种示例，不对本申请实施例的保护范围造成限定。
还可以理解，在本申请各个实施例中涉及到的公式仅是示例性说明，其不对本申请实施例的保护范围造成限定。在计算上述各个涉及的参数的过程中，也可以根据上述公式进行计算，或者基于上述公式的变形进行计算，也可以根据其它方式进行计算以满足公式计算的结果。
还可以理解，本申请的各实施例中的一些可选的特征，在某些场景下，可以不依赖于其他特征，也可以在某些场景下，与其他特征进行结合，不作限定。
还可以理解，本申请的各实施例中的方案可以进行合理的组合使用，并且实施例中出现的各个术语的解释或说明可以在各个实施例中互相参考或解释，对此不作限定。
还可以理解，在本申请的各实施例中的各种数字序号的大小并不意味着执行顺序的先后，仅为描述方便进行的区分，不应对本申请实施例的实施过程构成任何限定。
还可以理解，上述各个方法实施例中，由设备实现的方法和操作，也可以由可由设备的组成部件(例如芯片或者电路)来实现。
相应于上述各方法实施例给出的方法，本申请实施例还提供了相应的装置，所述装置包括用于执行上述各个方法实施例相应的模块。该模块可以是软件，也可以是硬件，或者是软件和硬件结合。可以理解的是，上述各方法实施例所描述的技术特征同样适用于以下装置实施例。
图14是本申请实施例提供的一种通信的装置1900的示意图。该装置1900包括收发单元1910和处理单元1920。收发单元1910可以用于实现相应的通信功能。收发单元1910还可以称为通信接口或通信单元等。处理单元1920可以用于实现相应的处理或控制功能，如配置资源。
可选地，该装置1900还包括存储单元，该存储单元可以用于存储指令和/或数据，处理单元1920可以读取存储单元中的指令和/或数据，以使得装置实现前述各个方法实施例中设备或网元的动作。
该装置1900可以是中心节点，也可以是应用于中心节点或者和中心节点匹配使用，能够实现中心节点侧执行的通信方法的通信装置；或者，该装置1900可以是子节点，也可以是应用于子节点或者和子节点匹配使用，能够实现子节点侧执行的通信方法的通信装置。
该装置1900应用于中心节点时，装置1900可实现对应于上文方法实施例中的中心节点执行的步骤或者流程。其中，收发单元1910可用于执行上文方法实施例中中心节点的收发相关的操作，处理单元1920可用于执行上文方法实施例中中心节点的处理相关的操作。
该装置1900应用于子节点时，装置1900可实现对应于上文方法实施例中的子节点执行的步骤或者流程，其中，收发单元1910可用于执行上文方法实施例中子节点的收发相关的操作，处理单元1920可用于执行上文方法实施例中子节点的处理相关的操作。
应理解，各单元执行上述相应步骤的具体过程在上述各方法实施例中已经详细说明，为了简洁，在此不再赘述。
还应理解，这里的装置1900以功能单元的形式体现。这里的术语“单元”可以指ASIC、电子电路、用于执行一个或多个软件或固件程序的处理器(例如共享处理器、专有处理器或组处理器等)和存储器、合并逻辑电路和/或其它支持所描述的功能的合适组件。在一个可选例子中，本领域技术人员可以理解，装置1900可以具体为上述实施例中的子节点，可以用于执行上述各方法实施例中与子节点对应的各个流程和/或步骤；或者，装置1900可以具体为上述实施例中的中心节点，可以用于执行上述各方法实施例中与中心节点对应的各个流程和/或步骤，为避免重复，在此不再赘述。
上述各个方案的装置1900具有实现上述方法中设备(如子节点，又如中心节点)所执行的相应步骤的功能。所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个与上述功能相对应的模块；例如收发单元可以由收发机替代(例如，收发单元中的发送单元可以由发送机替代，收发单元中的接收单元可以由接收机替代)，其它单元，如处理单元等可以由处理器替代，分别执行各个方法实施例中的收发操作以及相关的处理操作。
此外，上述收发单元1910还可以是收发电路(例如可以包括接收电路和发送电路)，处理单元1920可以是处理电路。处理电路可以包括一个或多个处理器，或者，一个或多个处理器中用于处理或控制功能的电路等。
需要指出的是，图14中的装置可以是前述实施例中的网元或设备，也可以是芯片或者芯片系统，例如：SoC。其中，收发单元可以是输入输出电路、通信接口；处理单元为该芯片上集成的处理器或者微处理器或者集成电路。在此不做限定。
图15是本申请实施例提供另一种通信的装置2000的示意图。该装置2000包括处理器2010，处理器2010用于执行存储器2020存储的计算机程序或指令，或读取存储器2020存储的数据/信令，以执行上文各方法实施例中的方法。可选地，处理器2010为一个或多个。
可选地，如图15所示，该装置2000还包括存储器2020，存储器2020用于存储计算机程序或指令和/或数据。该存储器2020可以与处理器2010集成在一起，或者也可以分离设置。可选地，存储器2020为一个或多个。
可选地，如图15所示，该装置2000还包括收发电路2030，收发电路2030用于信号的接收和/或发送。例如，处理器2010用于控制收发电路2030进行信号的接收和/或发送。处理器2010也可以替换为处理电路。
装置2000可以为前述实施例中的网元或设备，也可以是芯片或芯片系统。当装置2000为前述实施例中的网元或设备时，收发电路2030可以为收发器。当装置2000为芯片或芯片系统时，收发电路2030可以为接口电路或输入输出接口。
作为一种方案，该装置2000可以应用于中心节点，具体装置2000可以是中心节点，也可以是能够支持中心节点，实现上述涉及的任一示例中中心节点的功能的装置。该装置2000用于实现上文各个方法实施例中由中心节点执行的操作。
例如，处理器2010用于执行存储器2020存储的计算机程序或指令，以实现上文各个方法实施例中中心节点的相关操作。
作为另一种方案，该装置2000可以应用于子节点，具体装置2000可以是子节点，也可以是能够支持子节点，实现上述涉及的任一示例中子节点的功能的装置。该装置2000用于实现上文各个方法实施例中由子节点执行的操作。
例如，处理器2010用于执行存储器2020存储的计算机程序或指令，以实现上文各个方法实施例中子节点的相关操作。
应理解，本申请实施例中提及的处理器可以是中央处理单元(central processing unit，CPU)，还可以是其他通用处理器、数字信号处理器(digital signal processor，DSP)、ASIC、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
还应理解，本申请实施例中提及的存储器可以是易失性存储器和/或非易失性存储器。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)。例如，RAM可以用作外部高速缓存。作为示例而非限定，RAM包括如下多种形式：静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。
需要说明的是，当处理器为通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件时，存储器(存储模块)可以集成在处理器中。
还需要说明的是，本文描述的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
本申请实施例还提供一种计算机可读存储介质，其上存储有用于实现上述各方法实施例中由通信设备执行的方法的计算机指令。
例如，该计算机程序被计算机执行时，使得该计算机可以实现上述方法各实施例中由子节点执行的方法。
又如，该计算机程序被计算机执行时，使得该计算机可以实现上述方法各实施例中由中心节点执行的方法。
本申请实施例还提供一种计算机程序产品，包含指令，该指令被计算机执行时以实现上述各方法实施例中由设备(如子节点，又如中心节点)执行的方法。
本申请实施例还提供一种通信的系统，包括前述的子节点和中心节点。子节点和中心节点可以实现前述任一示例中所示的通信的方法。
可选地，该系统中还包括与上述子节点和/或中心节点通信的设备。
上述提供的任一种装置中相关内容的解释及有益效果均可参考上文提供的对应的方法实施例，此处不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。此外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。例如，所述计算机可以是个人计算机，服务器，或者网络设备等。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD)等。例如，前述的可用介质包括但不限于：U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (44)

  1. 一种通信的方法，其特征在于，应用于子节点，所述方法包括：
    接收第一信息，所述第一信息包括第一索引和指示第一均值的信息，所述第一均值用于第一方差的计算，所述第一方差为人工智能AI模型中的第一网络层的输入的方差；
    发送第二信息，所述第二信息包括第二索引和指示所述第一方差的信息，所述第一索引和所述第二索引具有关联关系。
  2. 根据权利要求1所述的方法，其特征在于，所述第一索引和所述第一均值相关。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第一索引和所述第二索引的索引值相同，或者，所述第一索引和所述第二索引具有预定义或预配置的关联关系。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述第一索引指示所述第一网络层的标识。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述第一索引指示与所述第一均值相关的训练的回合。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述方法还包括：
    发送第三信息，所述第三信息用于请求所述AI模型的模型参数。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述发送第二信息后，所述方法还包括：
    接收第四信息，所述第四信息包括第三索引和指示第二方差的信息，所述第一索引和所述第二索引中的至少一项与所述第三索引具有关联关系，所述第一均值和所述第二方差用于所述AI模型的训练。
  8. 根据权利要求1至7中任一项所述的方法，其特征在于，所述接收第一信息之前，所述方法还包括：
    发送第五信息，所述第五信息包括第四索引和指示第二均值的信息，所述第二均值为所述AI模型中的所述第一网络层的输入的均值，所述第一索引和所述第二索引中的至少一项与所述第四索引具有关联关系。
  9. 一种通信的方法，其特征在于，应用于中心节点，所述方法包括：
    向多个设备发送第一信息，所述第一信息包括第一索引和指示第一均值的信息，所述第一均值用于第一方差的计算，所述第一方差为所述多个设备中每个设备的人工智能AI模型中的第一网络层的输入的方差；
    接收来自所述多个设备中的每个设备的第二信息，所述第二信息包括第二索引和指示所述第一方差的信息，所述第一索引和所述第二索引具有关联关系。
  10. 根据权利要求9所述的方法，其特征在于，所述第一索引和所述第一均值相关。
  11. 根据权利要求9或10所述的方法，其特征在于，所述第一索引和所述第二索引的索引值相同，或者，所述第一索引和所述第二索引具有预定义或预配置的关联关系。
  12. 根据权利要求9至11中任一项所述的方法，其特征在于，所述第一索引指示所述第一网络层的标识。
  13. 根据权利要求9至12中任一项所述的方法，其特征在于，所述第一索引指示与所述第一均值相关的训练的回合。
  14. 根据权利要求9至13中任一项所述的方法，其特征在于，所述方法还包括：
    接收来自所述多个设备中的第一设备的第三信息，所述第三信息用于请求所述AI模型的模型参数。
  15. 根据权利要求9至14中任一项所述的方法，其特征在于，所述方法还包括：
    向所述多个设备发送第四信息，所述第四信息包括第三索引和指示第二方差的信息，所述第一索引和所述第二索引中的至少一项与所述第三索引具有关联关系，所述第一均值和所述第二方差用于所述AI模型的训练。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    根据来自所述多个设备中的每个设备的第二信息所指示的所述第一方差确定所述第二方差。
  17. 根据权利要求16所述的方法，其特征在于，所述第二方差满足：
    其中，表示所述第二方差，M表示所述多个设备的数量，|Bj|表示所述多个设备中的第j个设备上的训练样本的数量，表示来自所述第j个设备的第二信息所指示的所述第一方差，M为正整数，|Bj|为正整数。
  18. 根据权利要求9至17中任一项所述的方法，其特征在于，所述向多个设备发送第一信息之前，所述方法还包括：
    接收来自所述多个设备中的每个设备的第五信息，所述第五信息包括第四索引和指示第二均值的信息，所述第二均值为所述多个设备中每个设备的所述AI模型中的所述第一网络层的输入的均值，所述第一索引和所述第二索引中的至少一项与所述第四索引具有关联关系。
  19. 根据权利要求18所述的方法，其特征在于，还包括：
    基于所述多个设备中的每个设备的第五信息确定所述第一信息，其中，所述第一均值基于所述第二均值。
  20. 一种通信的方法，其特征在于，应用于子节点，所述方法包括：
    发送第七信息，所述第七信息包括指示第三均值和第四均值的信息，所述第三均值为人工智能AI模型中的第一网络层的输入的均值，所述第四均值为所述第一网络层的输入的平方的均值。
  21. 根据权利要求20所述的方法，其特征在于，还包括：
    接收第八信息，所述第八信息包括指示第五均值和第三方差的信息，所述第五均值和所述第三方差与所述第一网络层相关，所述第五均值和所述第三方差用于所述AI模型的训练。
  22. 根据权利要求21所述的方法，其特征在于，所述第三方差基于所述第三均值和所述第四均值。
  23. 根据权利要求21或22所述的方法，其特征在于，所述第七信息还包括第五索引，所述第八信息还包括第六索引，所述第五索引和所述第六索引具有关联关系。
  24. 根据权利要求23所述的方法，其特征在于，所述第三均值和所述第四均值中的至少一项和所述第五索引相关。
  25. 根据权利要求23或24所述的方法，其特征在于，所述第五索引和所述第六索引的索引值相同，或者，所述第五索引和所述第六索引具有预配置或预定义的关联关系。
  26. 根据权利要求23至25中任一项所述的方法，其特征在于，所述第五索引指示所述第一网络层的标识。
  27. 根据权利要求23至26中任一项所述的方法，其特征在于，所述第五索引指示与所述第三均值和/或所述第四均值相关的训练的回合。
  28. 一种通信的方法，其特征在于，应用于中心节点，所述方法包括：
    接收来自多个设备中的每个设备的第七信息，所述第七信息包括指示第三均值和第四均值的信息，所述第三均值为所述多个设备中的每个设备的人工智能AI模型中的第一网络层的输入，所述第四均值为所述第一网络层的输入的平方的均值。
  29. 根据权利要求28所述的方法，其特征在于，还包括：
    向所述多个设备发送第八信息，所述第八信息包括指示第五均值和第三方差的信息，所述第五均值和所述第三方差与所述第一网络层相关，所述第五均值和所述第三方差用于所述AI模型的训练。
  30. 根据权利要求29所述的方法，其特征在于，还包括：
    根据来自所述多个设备中的每个设备的所述第七信息所指示的所述第三均值和所述第四均值确定所述第三方差。
  31. 根据权利要求29或30所述的方法，其特征在于，所述第三方差满足：
    其中，表示所述第三方差，表示第六均值，所述第六均值为来自所述多个设备中的每个设备的所述第七信息所指示的所述第四均值的平均值，Ε[hl(x)]表示所述第五均值，所述第五均值为来自所述多个设备中的每个设备的所述第七信息所指示的所述第三均值的平均值，B表示所述多个设备上的训练数据集。
  32. 根据权利要求29至31中任一项所述的方法，其特征在于，所述第七信息还包括第五索引，所述第八信息还包括第六索引，所述第五索引和所述第六索引具有关联关系。
  33. 根据权利要求32所述的方法，其特征在于，所述第三均值和所述第四均值中的至少一项和所述第五索引相关。
  34. 根据权利要求32或33所述的方法，其特征在于，所述第五索引和所述第六索引的索引值相同，或者，所述第五索引和所述第六索引具有预配置或预定义的关联关系。
  35. 根据权利要求32至34中任一项所述的方法，其特征在于，所述第五索引指示所述第一网络层的标识。
  36. 根据权利要求32至35中任一项所述的方法，其特征在于，所述第五索引指示与所述第三均值和/或所述第四均值相关的训练的回合。
  37. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括指令，当所述指令被处理器运行时，使得如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法被实现。
  38. 一种通信装置，其特征在于，所述通信装置包括与存储介质耦合的处理器，所述存储介质存储有指令，所述指令被所述处理器运行时，使得所述通信装置执行如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法。
  39. 一种通信装置，其特征在于，包括执行如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法的模块。
  40. 一种通信装置，其特征在于，包括一个或多个处理器，所述一个或多个处理器用于处理数据和/或信息，以使得如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法被实现。
  41. 一种芯片，其特征在于，包括处理器，所述处理器用于运行程序或指令，以使得如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法被实现。
  42. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序代码或指令，当所述计算机程序代码或指令被运行时，使得如权利要求1至8，权利要求9至19，权利要求20至27，或者权利要求28至36中任一项所述的方法被实现。
  43. 一种通信系统，其特征在于，包括，执行如权利要求1至8中任一项所述的方法的通信装置，和，执行如权利要求9至19中任一项所述的方法的通信装置。
  44. 一种通信系统，其特征在于，包括，执行如权利要求20至27中任一项所述的方法的通信装置，和，执行如权利要求28至36中任一项所述的方法的通信装置。
PCT/CN2025/098029 2024-06-05 2025-05-29 通信的方法和通信装置 Pending WO2025251996A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410725330.0 2024-06-05
CN202410725330.0A CN121077596A (zh) 2024-06-05 2024-06-05 通信的方法和通信装置

Publications (1)

Publication Number Publication Date
WO2025251996A1 true WO2025251996A1 (zh) 2025-12-11

Family

ID=97838988

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/098029 Pending WO2025251996A1 (zh) 2024-06-05 2025-05-29 通信的方法和通信装置

Country Status (2)

Country Link
CN (1) CN121077596A (zh)
WO (1) WO2025251996A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019149376A1 (en) * 2018-02-02 2019-08-08 Toyota Motor Europe Method and system for processing input data using a neural network and normalizations
US20200125949A1 (en) * 2018-10-19 2020-04-23 Google Llc Training neural networks using distributed batch normalization
CN114722427A (zh) * 2022-06-07 2022-07-08 腾讯科技（深圳）有限公司 联邦学习中的隐私去重方法、装置、设备及存储介质
CN115734244A (zh) * 2021-08-30 2023-03-03 华为技术有限公司 一种通信方法及装置
CN117474116A (zh) * 2022-07-21 2024-01-30 华为技术有限公司 模型的训练方法及通信装置
CN117858123A (zh) * 2023-12-28 2024-04-09 大连理工大学 一种d2d辅助的双阶段空中计算模型聚合方法
US20240171991A1 (en) * 2022-11-17 2024-05-23 Qualcomm Incorporated User equipment grouping for federated learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019149376A1 (en) * 2018-02-02 2019-08-08 Toyota Motor Europe Method and system for processing input data using a neural network and normalizations
US20200125949A1 (en) * 2018-10-19 2020-04-23 Google Llc Training neural networks using distributed batch normalization
CN115734244A (zh) * 2021-08-30 2023-03-03 华为技术有限公司 一种通信方法及装置
CN114722427A (zh) * 2022-06-07 2022-07-08 腾讯科技（深圳）有限公司 联邦学习中的隐私去重方法、装置、设备及存储介质
CN117474116A (zh) * 2022-07-21 2024-01-30 华为技术有限公司 模型的训练方法及通信装置
US20240171991A1 (en) * 2022-11-17 2024-05-23 Qualcomm Incorporated User equipment grouping for federated learning
CN117858123A (zh) * 2023-12-28 2024-04-09 大连理工大学 一种d2d辅助的双阶段空中计算模型聚合方法

Also Published As

Publication number Publication date
CN121077596A (zh) 2025-12-05

Similar Documents

Publication Publication Date Title
CN117693021A (zh) 一种波束管理方法
CN116418686A (zh) 模型的数据处理方法及装置
WO2025007932A1 (zh) 通信方法和装置
WO2025251996A1 (zh) 通信的方法和通信装置
WO2025241999A1 (zh) 通信的方法和通信装置
WO2025209305A9 (zh) 信息传输方法、装置及系统
WO2026007948A1 (zh) 通信的方法和通信装置
WO2025039193A1 (zh) 神经网络训练的方法和通信装置
WO2025209331A1 (zh) 信息传输方法、装置及系统
WO2026067235A1 (zh) 通信方法及通信装置
WO2026067180A1 (zh) 通信方法及通信装置
WO2026031746A1 (zh) 通信方法及通信装置
CN119721157A (zh) 人工智能模型的训练方法与通信装置
WO2025237350A1 (zh) 数据处理方法及装置
WO2025261029A1 (zh) 通信方法及相关装置
WO2025261090A1 (zh) 一种通信方法和通信装置
WO2026031954A1 (zh) 一种通信方法及通信装置
WO2026040738A9 (zh) 用于信道测量的方法和通信装置
WO2025092630A1 (zh) 通信方法和通信装置
KR20260053524A (ko) 신경망 훈련 방법 및 통신 장치
WO2025161598A1 (zh) 一种用于网络训练的方法和通信装置
WO2026032252A1 (zh) 通信方法及装置
WO2025232813A1 (zh) 一种通信方法和通信装置
CN121283472A (zh) 通信的方法和通信装置
WO2026026766A1 (zh) 一种通信方法和相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25819049

Country of ref document: EP

Kind code of ref document: A1