KR20220107690A - Bayesian federated learning driving method over wireless networks and the system thereof - Google Patents

Abstract

The present invention relates to a method and system for driving Bayesian federated learning through a wireless network to optimally aggregate quantified gradient information. The method includes the steps of: calculating a local gradient, by a mobile device, using training dataset of the mobile device and global model parameters received from a server; compressing, by the mobile device, the local gradient using a 1-bit quantifier and a scalar quantifier and transmitting the local gradient to the server; and representing, by the server, a local gradient sum for the local gradient using a Bayesian aggregation function, and updating the global model parameter.

Description

Bayesian federated learning driving method and system through wireless network

The present invention relates to a method and system for driving Bayesian federated learning through a wireless network, and to a technique for proposing a Bayesian federated learning algorithm for optimally aggregating quantified gradient information.

Federated learning is a distributed approach that trains machine learning models on a server using a distributed, heterogeneous learning dataset deployed on a mobile device without sharing raw data with the server. This federated learning has received considerable interest in the industry due to applications that require the privacy of user-generated data. Federated learning repeatedly performs distributed model training through two tasks: model optimization using a local data set and model aggregation, that is, model averaging. For example, in every round, the server sends a global model to available mobile devices, each mobile device optimizing the global model with locally available data and then communicating updated model parameters (or updated local gradients). Send it to the server through the link. Accordingly, the server averages the local model or gradient received from the mobile device, updates the global model, and repeatedly shares it.

As such, optimizing global models using local data sets deployed on numerous mobile devices is a difficult task. The biggest of these is the high communication cost of updating local computations on mobile devices and servers. The cost of communication is proportional to the size of the global model parameters and the number of mobile devices connected to the server.

Also, while lossy compression of the local gradient is a practical solution to reduce the message size per communication round, compression can be performed through sparsity or quantification. Despite significant progress in communication efficiency improvement through compression, existing studies did not consider the aggregation of local gradients by jointly accommodating heterogeneous local data distributions, various communication link reliability and quantification effects by device. Since most existing studies mainly focus on the design approach of separate communication and learning systems, we decoded the local gradients and aggregated them independently.

Over the air aggregation is a new paradigm for jointly designing communication and learning systems by utilizing the overlapping properties of wireless media. This approach opens new opportunities for jointly designing radio and learning systems, but is limited to time-division duplex radio systems that can use channel state information from mobile devices for uplink transmission.

Accordingly, the present invention considers a federated learning system in which a mobile device sends a local gradient to a server every communication round using orthogonalized multiple access channels each having a heterogeneous radio link quality.

J. Konehcn`y, H. B. McMahan, F. X. Yu, P. Richt´arik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.

An object of the present invention is to propose a Bayesian Federated Learning (BFL) algorithm for optimally aggregating quantified local gradient information to minimize mean-squared error (MSE).

Another object of the present invention is to perform joint detection and aggregation of local gradients using a Bayesian framework as a federated learning system that sends a local gradient from a mobile device to a server.

In the method for driving Bayesian Federated Learning (BFL) through a wireless network composed of a server and a plurality of mobile devices according to an embodiment of the present invention, the mobile device receives its own training dataset and the server Calculating a local gradient using the global model parameter obtained by compressing the local gradient using a 1-bit quantifier and a scalar quantifier in the mobile device and sending it to the server, and in the server, and representing the local gradient sum for the local gradient using a Bayesian aggregation function, and updating the global model parameter.

In the method for driving scalable-Bayesian Federated Learning (scalable-BFL) through a wireless network composed of a server and a plurality of mobile devices according to an embodiment of the present invention, in the mobile device, its own training data calculating a local gradient using the set and global model parameters received from the server, and parameterizing it as a scalar for the mobile device, in the mobile device, using a 1-bit quantifier and a scalar quantifier Quantifying the local gradient and the scalar and transmitting to the server, and in the server, using an MMSE (Minimum Mean-Squared Error) aggregation function to represent the sum for the quantified local gradient and the quantified scalar, the global updating model parameters.

In the Bayesian Federated Learning (BFL) driving system through a wireless network composed of a server and a plurality of mobile devices according to an embodiment of the present invention, a global model parameter received from its own training dataset and the server Calculates a local gradient using Represents the sum and includes a server that updates the global model parameters.

In the scalable-Bayesian Federated Learning (scalable-BFL) driving system through a wireless network composed of a server and a plurality of mobile devices according to an embodiment of the present invention, A mobile device that quantifies the local gradient and scalar calculated using the received global model parameter using a 1-bit quantifier and a scalar quantifier and transmits it to the server and a Minimum Mean-Squared Error (MMSE) ) representing the sum for the quantified local gradient and the quantified scalar using an aggregation function, and a server for updating the global model parameter.

According to an embodiment of the present invention, by performing joint detection and aggregation of a local gradient using a Bayesian aggregation function, it is possible to minimize the mean squared error per communication round.

In addition, according to an embodiment of the present invention, by using the Bayesian aggregation function of scalable Bayesian joint learning (scalable-BFL), it is possible to improve the local gradient data distribution and communication link quality according to the expansion of the number of mobile devices.

1 is a diagram for explaining the associative learning.
2 is a flowchart illustrating an operation of a method for driving Bayesian federated learning through a Bayesian federated learning driving system according to an embodiment of the present invention.
3 is a flowchart illustrating an operation of an extensible Bayesian federated learning driving method through an extensible Bayesian federated learning driving system according to an embodiment of the present invention.
4 is a diagram illustrating a neural network structure of scalable Bayesian joint learning (scalable-BFL) according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments, and is not intended to limit the present invention. In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

1 is a diagram for explaining the associative learning.

With federated learning, it is possible to jointly learn using a common prediction model while managing prediction data on a mobile device, so there is no need to store data on a separate server or cloud.

Referring to FIG. 1 , all mobile devices download the current common prediction model (or global model) ( S101 ). The mobile device may improve the learning of the predictive model based on data (or training dataset) of the terminal according to the user's use ( S102 ). After improving the learning of the predictive model, the mobile device may generate these changes as update data (S103). Numerous predictive models of mobile devices can be learned by reflecting various usage environments and user characteristics (S104). The update data of each mobile device is transmitted to the cloud server through communication, which can be used to improve the common prediction model (S105). The improved common prediction model may be distributed to each mobile device again (S106). The mobile device can develop and share the common predictive model while repeating the steps of re-learning the predictive model and improving the common predictive model.

2 is a flowchart illustrating an operation of a method for driving Bayesian federated learning through a Bayesian federated learning driving system according to an embodiment of the present invention.

Referring to FIG. 2 , a Bayesian Federated Learning (BFL) driving method through a wireless network configured with a server and a plurality of mobile devices according to an embodiment of the present invention is a Bayesian Federated Learning (BFL) method. ) provides a framework for Bayesian federated learning according to an embodiment of the present invention provides an optimal aggregation method of local gradients from a Bayesian point of view, and proposes an optimal aggregation method of local gradients in the sense of minimizing Mean-Squared Error (MSE). do.

The mobile device 200 downloads the global model (or common prediction model) from the server 300 (step S201).

In step S202 , the mobile device 200 calculates a local gradient using its own training dataset and global model parameters received from the server 300 . The mobile device 200 may calculate a local gradient using prior information (prior) or posterior information (posterior) of the local gradient based on the training dataset and global model parameters.

)의 사전 분포는

이다. 이때, 사전 분포를 특성화하는 것은 손실 함수(

)뿐만 아니라 데이터 표본의 기본 분포에 의존하기 때문에 매우 어렵다. 이러한 어려움을 극복하기 위해, 모바일 장치(200)는 적절한 모멘트 매칭 기법을 사용하여 로컬 기울기의 분포를 가우스 사전 분포로 모델링하여 로컬 기울기의 사전 분포를 특성화한다. 특히, 사전 정보(prior)는 평균 벡터(

) 및 공분산 행렬(

)을 갖는 다변량 정규 분포인 하기의 [수식 1]이라고 가정한다. In more detail, unlike the existing federated learning, the mobile device 200 uses prior data of the local gradient in step S202 according to an embodiment of the present invention. Local gradient computed with mobile device k in communication round t (

) is the prior distribution of

to be. In this case, characterizing the prior distribution is a loss function (

) as well as it is very difficult because it depends on the underlying distribution of the data sample. To overcome this difficulty, the mobile device 200 uses an appropriate moment matching technique to model the distribution of the local gradient as a Gaussian prior distribution to characterize the prior distribution of the local gradient. In particular, the prior information (prior) is an average vector (

) and the covariance matrix (

) is assumed to be the following [Equation 1], which is a multivariate normal distribution with .

[Formula 1]

The prior distribution may change in communication rounds depending on the underlying distribution of the data sample and the local loss function.

에서 로컬 기울기

와

는 글로벌 모델 파라미터(

)에서 평가되기 때문에 통계적으로 상관 관계가 있다. 이를 설명하기 위해 선형 회귀 손실 함수를 고려한다.

local gradient in

Wow

is the global model parameter (

), so they are statistically correlated. To explain this, consider a linear regression loss function.

[Equation 2]

에 대해

를 나타내고,

를 나타낸다. 또한,

에 대한 모든 로컬 데이터 분포(

)는 각각

와

으로 독립적이라고 가정한다. 이 경우, 글로벌 모델 파라미터(

)를 사용한 모바일 장치(

)에서의 로컬 기울기는

를 나타낸다. here,

About

represents,

indicates In addition,

All local data distributions for (

) is each

Wow

is assumed to be independent. In this case, the global model parameter (

) using a mobile device (

) is the local gradient at

indicates

)에서 조건화된

와

사이의 상관 행렬은 다음과 같이 계산된다. global model parameters (

) conditioned in

Wow

The correlation matrix between

[Equation 3]

와

가 모바일 장치와 상관되어 있다면

와

사이의 상관 관계는 더욱 복잡할 수 있다. At this time, the data distribution

Wow

is associated with a mobile device

Wow

The correlation between them can be more complex.

In step S203, the mobile device 200 compresses the local gradient using a 1-bit quantifier.

The mobile device 200 compresses the local gradient using a 1-bit quantifier to minimize the communication cost, and first performs normalization with an average of 0 to evenly distribute the 1-bit quantified output.

[Equation 4]

신호를 사용하여 정량화한다.Then, the mobile device 200 calculates the normalized slope

The signal is used to quantify.

[Equation 5]

Thereafter, the mobile device 200 transmits the updated local gradient to the server 300 (step S204).

)와 공분산 행렬(

)의 파라미터와 함께 이진 기울기 정보(binary gradient information,

)를 서버(300)로 전송할 수 있다. In step S204, the mobile device 200 determines the prior distribution of the mean vector (

) and the covariance matrix (

) along with the parameters of the binary gradient information (binary gradient information,

) may be transmitted to the server 300 .

The server 300 calculates the sum of the local gradients for the local gradients using the Bayesian aggregation function (step S205), and updates the global model parameters (step S206).

에 대한 채널 출력(

)을 활용하여 글로벌 모델 파라미터를 업데이트한다. 이때, 서버(300)는 채널 분포(

), 로컬 기울기의 한계 분포(

) 및 로컬 기울기의 공통 분포(

)를 알고 있다고 가정하며, 이는 MSE 최적 집계 함수를 도출하기 위한 가정(genie-aided assumption)을 나타낸다.Server 300 in the t-th communication round,

Channel output for (

) to update global model parameters. At this time, the server 300 is channel distribution (

), the marginal distribution of the local gradient (

) and the common distribution of local gradients (

), which represents a genie-aided assumption for deriving the MSE optimal aggregate function.

가 로컬 기울기의 합(

)을 추정하기 위한 집계 함수라 하면, 서버(300)는 다음과 같이 집계 함수(

)를 획득할 수 있다.

is the sum of local gradients (

) is an aggregate function for estimating, the server 300 performs the aggregate function (

) can be obtained.

[Equation 6]

The MSE-optimal aggregate function for the optimization problem of [Equation 6] is obtained by conditional expectation.

[Equation 7]

[Equation 8]

)에서 조건화된

에 대한

사이의 독립성을 이용하여 다음과 같이 인수분해된다.The conditional distribution in [Equation 8] is based on the chain rule and the normalized local gradient (

) conditioned in

for

Using the independence between

[Equation 9]

사실로부터 마르코프 체인을 형성하고,

에 대한

는 공동 분포(

)의 주변화로 획득될 수 있다.Here, the final equality is

form a Markov chain from facts,

for

is the joint distribution (

) can be obtained by marginalization of

를 사용하여 글로벌 모델 파라미터를 하기의 [수식 10]과 같이 확률적 경사 강하법에 기반하여 업데이트한다. In this case, the server 300 connects [Equation 9] to [Equation 8] to provide a Bayesian aggregation function that is an optimal gradient aggregation function from a Bayesian point of view. In addition, the server 300 in [Equation 7]

is used to update the global model parameters based on the stochastic gradient descent method as shown in [Equation 10] below.

[Equation 10]

Thereafter, in step S207 , the server 300 may share the updated global model.

3 is a flowchart illustrating an extensible Bayesian federated learning driving method through an extensible Bayesian federated learning driving system according to an embodiment of the present invention, and FIG. 4 is an expandable Bayesian federated learning ( It shows the neural network structure of scalable-BFL).

Referring to FIG. 3 , a method for driving scalable Bayesian Federated Learning (BFL) through a wireless network composed of a server and a plurality of mobile devices according to an embodiment of the present invention is A framework for BFL) is presented. The scalable Bayesian federated learning according to an embodiment of the present invention is scalable to the number of mobile devices and has strong heterogeneity in both local data distribution and communication link quality.

The mobile device 200 downloads a global model (or a common prediction model) from the server 300 (step S301).

In step S302, the mobile device 200 normalizes a local gradient calculated using its own training dataset and global model parameters received from the server 300, and parameterizes it as two scalars.

)과 분산(

)을 사용하여 독립 분포(independent identically distributed; i.i.d.) 가우스처럼 로컬 기울기(

)의 모든 요소를 모델링한다. More specifically, the mobile device 200 has a common average (

) and variance (

) using the independent identically distributed (iid) Gaussian-like local gradient (

) to model all elements of

[Equation 11]

의 샘플 평균과 분산을 다음과 같이 계산하여 평균과 분산을 추정한다.Then, the mobile device 200

Estimate the mean and variance by calculating the sample mean and variance of

[Equation 12]

[Equation 13]

에 대한 두 개의 스칼라

와

로 각 모바일 장치(200)의 사전 분포에 대한 뚜렷한 통계 정보를 획득할 수 있다. 이러한 단순화는 집계 기능의 이전 및 복잡성에 대한 정보를 제공하는 통신 비용을 크게 절감할 수 있다. 특히, 각 모바일 장치(200)는 서버(300)에 1비트 로컬 기울기 외에 2개의 스칼라

와

만 전송하면 된다. 이에 따라서, 확장 가능한 베이지안 연합학습(Scalable-BFL)은 베이지안 연합학습(BFL)에서, 큰 차원의 평균 벡터(

)와 공분산 행렬(

)을 보내는 것에 비해 통신 비용을 감소시킬 수 있다.Pre-distribution for mobile devices

two scalars for

Wow

With this method, clear statistical information about the prior distribution of each mobile device 200 can be obtained. This simplification can significantly reduce the cost of communication that informs the migration and complexity of aggregation functions. In particular, each mobile device 200 has two scalars in addition to the 1-bit local gradient to the server 300 .

Wow

You only need to send Accordingly, Scalable-BFL is a large-dimensional mean vector (

) and the covariance matrix (

) can reduce the communication cost compared to sending

The mobile device 200 performs normalization with an average of 0 as follows before 1-bit quantification.

[Equation 14]

In step S303, the mobile device 200 compresses the local gradient into a binary vector using a 1-bit quantifier, and quantifies two scalars using a scalar quantifier.

)를 사용하여 정규화된 로컬 기울기(

)를 이진 벡터로 압축한다. 또한, 모바일 장치(200)는 B비트 스칼라 정량기를 사용하여 두 개의 스칼라

와

를 정량화한다. 이때,

와

를 각각 정량화된 출력과 빈 경계(bin boundary)의 집합으로 설정한다. 그런 다음, 정량화 함수(

)는 입력을 다음과 같이 Q의 이산값 출력에 매핑한다.After normalization through [Equation 14], the mobile device 200 is a 1-bit quantifier (

) using the normalized local gradient (

) to a binary vector. In addition, the mobile device 200 uses a B-bit scalar quantifier to

Wow

to quantify At this time,

Wow

is set as a set of quantified outputs and bin boundaries, respectively. Then, the quantification function (

) maps the input to the discrete output of Q as

[Equation 15]

및

은 각각 두 개의 스칼라

와

의 정량기 출력을 나타낸다. Accordingly,

and

are two scalars each

Wow

represents the output of the quantifier.

Thereafter, the mobile device 200 transmits the updated local gradient and the quantified two scalars to the server 300 (step S304).

와

가 포함된 정량화된 로컬 기울기(

)를 통신 라운드 t에서 서버(300)로 전송한다.Referring to FIG. 4, an uplink transport packet structure is shown. Mobile device (200) is a two scalar

Wow

quantified local slope with (

) to the server 300 in a communication round t.

와

를 전송할 때, 모바일 장치(200)는 강력한 채널 코드(예를 들면, 극성 모드)를 사용하여 정량화된 두 개의 스칼라에 대한 2B 정보 비트를 인코딩하여 전송하며, 서버(300)는 이를 완벽하게 디코딩한다. 예를 들면, 코드 속도가 r<1로 고정될 때,

이진 코드 부호의 총량은 정량화된 로컬 기울기(

)를 전송하기 위한 M 이진 기호와 함께 추가로 전송된다. 이때, 글로벌 모델 크기(

)가

보다 훨씬 크기 때문에, 추가 통신 오버헤드는 무시할 수 있다. Two quantified scalars

Wow

When sending, the mobile device 200 encodes and transmits the 2B information bits for the two scalars quantified using a strong channel code (eg, polar mode), and the server 300 decodes them perfectly. . For example, when the code rate is fixed at r<1,

The total amount of binary code sign is the quantified local slope (

) is additionally transmitted along with the M binary symbol to transmit At this time, the global model size (

)go

Since it is much larger than , the additional communication overhead is negligible.

The server 300 calculates the sum of the quantified local gradient and the quantified two scalars using the Bayesian aggregation function (step S305), and updates the global model parameter (step S306). Thereafter, in step S307, the server 300 may share the updated global model.

와

를 모두 사용하여 요소별 MMSE(Minimum Mean-Squared Error) 추정을 수행한다. 이는 비선형 베이지안 집계 함수에 대한 폐쇄형 표현식을 획득할 수 있으며, 단순화된 사전 분포를 통해 확장 가능한 MMSE 집계 함수를 획득할 수 있다.Server 300 quantifies two scalars

Wow

MMSE (Minimum Mean-Squared Error) estimation for each element is performed using all It is possible to obtain a closed expression for the nonlinear Bayesian aggregate function, and to obtain a scalable MMSE aggregate function through a simplified prior distribution.

The server 300 sets the MMSE aggregation function in a closed form under the simplified previous distribution assumption.

는

와

에 대한 정량화된 두 개의 스칼라

및

와 함께 i.i.d. 가우스 랜덤 변수라 하며,

를 경사 집계 함수라 한다. 이에 의해,

를 최소화하는 집계 함수는 다음과 같다.For example,

Is

Wow

Two quantified scalars for

and

together with iid is called a Gaussian random variable,

is called the gradient aggregation function. Thereby,

The aggregate function that minimizes

[Equation 16]

Here, 1M represents an all-one vector of dimension M.

) 및 통신 채널 파라미터(

)를 사용하여

에서

로 비선형 매핑을 수행한다. 두 번째 단계는 로컬 기울기(

)의 추정치를 얻기 위해 평균

으로

를 합한다. The aggregation method as described above is implemented through a two-step operation. The first step is the prior distribution parameters (

) and communication channel parameters (

)use with

at

to perform non-linear mapping. The second step is the local gradient (

) to get an estimate of the mean

by

add up

)를 추정해야 한다. 채널 파라미터(

)는 기존의 파일럿 신호를 사용하여 추정할 수 있으며, 추정 정확도는 파일럿 신호의 길이에 선형적으로 비례한다. 따라서, 업링크 스펙트럼 효율을 희생하여 채널 파라미터(

)를 정확하게 추정할 수 있다. In order to implement a scalable Bayesian federated learning (Scalable-BFL), the server 300 is a communication channel parameter (

) should be estimated. Channel parameters (

) can be estimated using the existing pilot signal, and the estimation accuracy is linearly proportional to the length of the pilot signal. Thus, at the expense of uplink spectral efficiency, channel parameters (

) can be accurately estimated.

Referring to FIG. 4 , the aggregation function as described above may be implemented through a two-layer neural network.

에 대한

와

가 상관관계가 있는 가우스일 때, [수식 16]에서 도출된 집계 함수는 [수식 7]에 설명된 상관관계가 있는 이전 가정(

)에 따라 차선책이 된다는 점에 유의한다. 그럼에도 불구하고, 로컬 기울기와 상관관계가 있는 사전 분포 하에서, 확장 가능한 베이지안 연합학습(Scalable-BFL)은 모바일 장치의 수와 MSE 성능 손실을 감수하는 모델 크기 모두에서 계산적 확장성으로 인해 유망한 알고리즘으로 제안된다.

for

Wow

When is a correlated Gaussian, the aggregate function derived from [Equation 16] is

) to be suboptimal. Nevertheless, under a prior distribution correlated with local gradients, scalable Bayesian federated learning (Scalable-BFL) is proposed as a promising algorithm due to its computational scalability in both the number of mobile devices and the model size at the cost of MSE performance loss. do.

The system or apparatus described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (14)

In a Bayesian Federated Learning (BFL) driving method through a wireless network composed of a server and a plurality of mobile devices,
calculating, in the mobile device, a local gradient using its own training dataset and global model parameters received from a server;
at the mobile device, compressing the local gradient using a 1-bit quantifier and a scalar quantifier and transmitting it to the server; and
Representing the local gradient sum for the local gradient using a Bayesian aggregation function, in the server, and updating the global model parameter
A Bayesian federated learning driving method comprising According to claim 1,
The step of calculating the local gradient is
Based on the training dataset and the global model parameter, the Bayesian federated learning driving method, characterized in that the local gradient is calculated using prior or posterior information of the local gradient. 3. The method of claim 2,
The step of calculating the local gradient is
A Bayesian joint learning driving method, characterized in that the distribution of the local gradient is modeled as a Gaussian prior distribution using the moment matching technique to characterize the prior or posterior distribution of the local gradient. According to claim 1,
The step of compressing the local gradient and transmitting it to the server is
A Bayesian federated learning driving method that performs normalization with a mean of 0 and quantifies the normalized local gradient to uniformly distribute the 1-bit quantified output. According to claim 1,
The step of compressing the local gradient and transmitting it to the server is
A Bayesian federated learning driving method, characterized in that the mobile device transmits binary gradient information together with the sample moment parameter of the gradient, which is information about the prior distribution, from the mobile device to the server. According to claim 1,
The step of updating the global model parameter is
In the server through the channel distribution, the marginal distribution of the local gradient, and the joint distribution of the local gradient, using the Bayesian aggregation function, which is the optimal gradient aggregation function from a Bayesian point of view, through the MSE-optimal aggregation function estimated by conditional expectation, Bayesian joint learning driving method, characterized in that it represents the sum of the local gradients with respect to the local gradients. 7. The method of claim 6,
The step of updating the global model parameter is
A method for driving Bayesian federated learning, updating the global model parameters using the MSE-optimal aggregation function. In a scalable-Bayesian Federated Learning (scalable-BFL) driving method through a wireless network composed of a server and a plurality of mobile devices,
calculating, in the mobile device, a local gradient using its own training dataset and global model parameters received from a server, and parameterizing the local gradient as a scalar for the mobile device;
quantifying, in the mobile device, the local gradient and the scalar using a 1-bit quantifier and a scalar quantifier and transmitting the quantification to the server; and
Representing, in the server, the sum of the quantified local gradient and the quantified scalar using a Minimum Mean-Squared Error (MMSE) aggregation function, updating the global model parameter;
A scalable Bayesian federated learning driving method including 9. The method of claim 8,
Normalizing the local gradient and parameterizing it as a scalar comprises
A scalable Bayesian joint learning driving method that uses a common mean and variance to model all elements of the local gradient, such as an independent distribution (iid) Gaussian distribution. 10. The method of claim 9,
Normalizing the local gradient and parameterizing it as a scalar comprises
A method for driving scalable Bayesian joint learning, characterized in that normalization with an average of 0 is performed before quantifying the local gradient by one bit. 11. The method of claim 10,
The step of sending to the server is
compressing the normalized local gradient into a heterogeneous vector using the 1-bit quantifier, and quantifying two scalars of a common mean and variance using a B-bit scalar quantifier, wherein the quantified two scalars are included. A method for driving scalable Bayesian federated learning, characterized in that it transmits the quantified local gradient to the server. 9. The method of claim 8,
The step of updating the global model parameter is
A scalable Bayesian joint learning driving method for updating the global model parameter by performing summation on the quantified local gradient and the quantified scalar using the Minimum Mean-Squared Error (MMSE) aggregation function. In a Bayesian Federated Learning (BFL) driving system through a wireless network composed of a server and a plurality of mobile devices,
a mobile device that calculates a local gradient using its own training dataset and global model parameters received from a server, compresses the local gradient using a 1-bit quantifier and a scalar quantifier, and transmits it to the server; and
A server that represents the local gradient sum for the local gradient using a Bayesian aggregation function, and updates the global model parameter
Bayesian federated learning driving system including In a scalable-Bayesian Federated Learning (scalable-BFL) driving system through a wireless network composed of a server and a plurality of mobile devices,
A mobile device that quantifies the local gradient and scalar using a 1-bit quantifier and a scalar quantifier and transmits the local gradient calculated using its own training dataset and global model parameters received from the server to the server ; and
A server that represents the sum of the quantified local gradient and the quantified scalar using a Minimum Mean-Squared Error (MMSE) aggregation function, and updates the global model parameter
A scalable Bayesian federated learning driving system that includes

Images