KR102348948B1 - Mean field game framework based techinique for control of transmission power for machine type communication - Google Patents

Abstract

A transmission power control method for Machine Type Communication (MTC) is provided. The disclosed transmit power control method is that the Fokker-Planck-Kolmogorov (FPK) equation of the mean field representing the statistical distribution of the system state across a number of MTC devices located in a network environment is and obtaining, using the finite difference method, a solution of the mean field game coupled as a constraint to the Hamilton-Jacobi-Bellman (HJB) equation governing the determination of the optimal control policy.

Description

MEAN FIELD GAME FRAMEWORK BASED TECHINIQUE FOR CONTROL OF TRANSMISSION POWER FOR MACHINE TYPE COMMUNICATION

The present disclosure relates to a technique for controlling transmit power for Machine Type Communication (MTC), and more specifically, transmit power for MTC based on a Mean Field Game (MFG) framework. It's about how to control it.

The development of new algorithms and network architectures for evolving applications has rapidly increased with the development of next-generation wireless cellular networks. This can be attributed to the rapid increase in the gap between new service categories and service requirements as defined by the International Telecommunication Union Radiocommunication Sector (ITU-R). In contrast, the applications the previous generation focused on were spectrally efficient but bandwidth consuming. 5th generation (5 ^th Generation: 5G) network is foreseen arranged densely as a result of various service requirements, energy depletion, and to encounter various challenges, including the increasing lack of spectrum and user capacity.

Providing enhanced Mobile BroadBand (eMBB) along with ultra-reliable Low Latency Communication (uRLLC) and massive Machine-Type Communication (mMTC) enables a variety of user applications. It requires a complex network architecture with a dynamic service structure for Machine-Type Communication (MTC) is defined as a communication design in which user devices generate, exchange, and analyze data without human intervention or devices. MTC is roughly classified as either mMTC or uRLLC, given certain Quality of Service (QoS) constraints. What mMTC typically supports is a more dense user environment characterized by low complexity and power requirements with changing QoS. Applications of such devices include monitoring systems, smart agriculture, smart instrumentation, sensor networks, remote control and diagnostics. On the other hand, the uRLLC device is very sensitive to end-to-end latency and requires high reliability with a relatively medium service rate. Some use cases for uRLLC are autonomous vehicles/drones, remote healthcare, automated cyber-physical systems and traffic management.

The 3 ^rd Partnership Project (3GPP) and Institute of Electrical and Electronics Engineers (IEEE) have developed specific features for MTC to facilitate network design, such as time-dependent infrequent transmission. transmission) and low mobility were defined. The resource and traffic requirements of an MTC device are usually differentiated from devices that are not, such as Human Type Communication (HTC) devices. HTC is a communication in which the transmission of a signal is determined by a person, whereas in MTC, each device decides to transmit a signal on its own periodically or based on an event without human intervention. As the traffic requirements of MTC and HTC devices are differentiated in this way, in order to accommodate a wide range of needs of MTC devices in terms of low latency and reliability as well as large-scale connectivity, a more advanced reconstruction of the infrastructure of the deployed network modeling is required. For example, Ericsson's Ericsson Mobility Report, November 2018 predicts that the number of MTC devices will reach 4.1 billion by 2024, and presents the motivation for research and development related to MTC traffic trends in future networks. In order to avoid network overload in such a dense MTC network, a distributed resource management approach would be appropriate from the point of view of traffic demand and processing. Key goals for distributed design are the ability to accommodate scalability, random placement of devices, minimal signaling overhead and limited energy supply, heterogeneity between devices, and availability of imperfect information. includes

As described above, as services provided to cellular subscribers evolve, the need for an MTC-HTC coexistence cellular network is becoming a reality. The main goal of HTC devices is to improve data throughput for services such as High Definition (HD) video, telepresence and virtual reality, and the data rate ( Spectrally efficient but energy consuming service architecture is required to improve data rate. On the other hand, since it is desirable to improve the battery lifetime expectancy of an MTC device performing remote sensing, it is necessary to design an energy-efficient power control method in consideration of this.

A technique for controlling transmit power for Machine Type Communication (MTC) is disclosed in this document.

According to at least one embodiment, a transmission power control method for Machine Type Communication (MTC) includes a Mean Field (Mean Field) representing a statistical distribution of a system state across a plurality of MTC devices located in a network environment: Fokker-Planck-Kolmogorov (FPK) equation of MF) combined as a constraint to the Hamilton-Jacobi-Bellman (HJB) equation governing the determination of the optimal control policy of the transmit power of each of the multiple MTC devices over a time interval. obtaining a solution of a Mean Field Game (MFG) using a Finite Difference Method (FDM), wherein the system state is 2 defined by the remaining battery energy and the associated interference. is a dimensional state, and the step of obtaining the solution comprises a grid space in which a three-dimensional domain space defined by the interval of the remaining battery energy and the interval of the associated interference, along with the time interval, is discretized. for each of a plurality of grid points in a grid space, calculating an updated value of the mean field forward in time using a discretized version of the FPK equation; for each grid point, calculating an update of the transmit power backward in time based on the discretized Lagrangian for the average field game.

The network environment may be configured for access of a Human Type Communication (HTC) device in addition to communication of the respective MTC device, wherein the associated interference is caused by the respective MTC device to be experienced by other MTC devices. Interference, of course, may also indicate interference caused by the respective MTC device and experienced by the HTC device.

The discretized version of the FPK equation may include a term dependent on the transmit power and a term dependent on an interference gain, wherein the associated interference is the transmit power multiplied by the interference gain. it will be done

The optimal control policy of the transmit power of each MTC device is a signal-to-interference-plus-Noise Ratio (SINR) of the transmission from the corresponding MTC device and a utility function based on the transmit power. A utility function may minimize the accumulated cost value over the time interval.

The discretized Lagrangian may depend on the utility function.

Calculating the update value of the transmit power may include calculating an update value of a Lagrange multiplier only based on a relational expression indicating that the rate of change of the discretized Lagrangian with respect to the average field is zero; and calculating the update value of the transmit power based on a relational expression indicating that the discretized rate of change of the Lagrangian with respect to .

The obtaining of the solution includes: obtaining initial values of each of the average length, the Lagrange multiplier, and the transmit power; and after obtaining the initial values, calculating the updated values of the average length; , performing a loop iteration repeating the steps of calculating the update value of the Lagrange multiplier and calculating the update value of the transmit power, wherein the loop iteration is the loop iteration. It ends when it is repeated by this maximum number of repetitions.

Even if the number of times the loop iteration is repeated does not reach the maximum number of iterations, if the difference between the updated value of the average field and the previous value of the average field is smaller than a preset value, the loop iteration may be terminated.

The solution may be obtained in such a way that batch processing is performed on the plurality of MTC devices for future reference.

The transmission power control method includes the steps of providing the obtained solution in the form of a data structure to be referenced later by an MTC device in the network environment as an indexing operation for MTC transmission conforming to the optimal control policy may include more.

According to at least one embodiment, an apparatus for controlling transmit power for Machine Type Communication (MTC) includes a processor and a memory storing processor-executable instructions, wherein the processor-executable instructions are executed by the processor. When executed, causes the processor to perform the transmit power control method described above.

The previous summary is provided to introduce in a simplified form some aspects that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to delineate the scope of the claimed subject matter. Furthermore, claimed subject matter is not limited to implementations that provide any or all advantages discussed herein.

According to an embodiment of the present invention, an energy-efficient transmission power control mechanism is provided in consideration of the battery energy of the MTC device in a network environment in which the MTC device coexists with the HTC device.

According to an embodiment of the present invention, the transmit power control problem for MTC is converted into a mean-length game, and a solution of the mean-length game can be obtained through a finite difference method.

According to an embodiment of the present invention, the solution of the average field game can be provided in the form of a data structure that can be referenced by the MTC device for MTC transmission corresponding thereto. The delay time can be shortened.

1 illustrates a network architecture for a hybrid cellular MTC and HTC access framework, in accordance with an embodiment of the present invention.
2 shows a method for controlling transmit power for MTC in the form of pseudocode according to an embodiment of the present invention.
3 is a block diagram of an apparatus for performing transmission power control for MTC according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The present invention is capable of several embodiments, some of which are disclosed herein. However, these are provided by way of illustration and not limitation of the present invention, and should be understood to cover all modifications, equivalents and substitutions falling within the spirit and scope of the present invention. Certain details are provided in the following detailed description to aid a comprehensive understanding of a method, apparatus, and/or system in accordance with disclosed embodiments, wherein some embodiments may be practiced without some or all of these details. In addition, detailed descriptions of well-known techniques may be omitted so as not to unnecessarily obscure various aspects of the present invention.

The terms described below are used only to describe specific embodiments, and are not intended to be considered in a limiting sense. Expressions in the singular form include the meaning of the plural unless expressly used otherwise. Also, in this document, terms such as "comprise" or "have" are intended to indicate the presence of some feature, number, step, operation, element, information, or a combination thereof, one or more other features, It should be understood as not excluding the presence or possibility of numbers, steps, acts, elements, information, or combinations thereof.

System overview

1 illustrates a network architecture for a hybrid cellular MTC and HTC access framework, in accordance with an embodiment of the present invention. In an embodiment, as shown in FIG. 1 , the exemplary wireless communication network environment 100 is served by a Base Station (BS) 120 (eg, an evolved Node B (eNB)). It includes a served (served) cell (110). Also, in an embodiment, one or more HTC devices (eg, User Equipment (UE) 141 , 142 , 143 ) and one or more MTC nodes (eg, MTC Device: MTCD) 151 , 152 , 161 , 162 , 163 , and MTC Gateway (MTCG) 160 ) may be present. In some example implementations, it may be understood that an MTC node is housed within an existing cellular network for an HTC device.

In an embodiment, the uRLLC service node (eg, MTCD 151 , 152 ) is provided with direct connectivity to BS 120 to ensure low latency and reliable communication link. On the other hand, direct access to the BS 120 by the mMTC device may cause a serious congestion problem due to large-scale simultaneous access by all devices, so for example, the mMTC device is grouped and relayed to cope with congestion in the BS A clustering approach may be considered. In this approach, the MTCG 160 is responsible for relaying the data of the connected MTCD (eg, the MTCDs 161, 162, 163) to the BS 120, which is for a seamless access experience. MTCG 160 may itself be an MTC device or may be an HTC device. In some example implementations, for privacy and data integrity, an mMTC device may be selected as a gateway node, which may lead to improved battery life and energy consumption. Further, as an example, although not shown, there may be MTC devices that are paired with each other and communicate directly with each other.

According to an embodiment, the following may be considered for implementation of an energy-efficient access scheme in each MTC node. First, all MTC devices obtain their respective channel responses to enable accurate modeling and information understanding of the current network environment 100 . In some example implementations, to avoid service disruption, service and node constraints may be considered so that the MTC node effectively integrates with traditional HTC devices. Second, provision of access to the MTC node is performed after determination of device and context information. To this end, for example, scheduling for mMTC and uRLLC nodes may be considered according to their specific QoS constraints. In some example implementations, mMTC devices may be arranged into clusters (eg, cluster 130 ) to simplify resource access and to handle congestion conditions that are likely to occur due to the large-scale connectivity requirements of these devices. Similarly, in some example implementations, uRLLC devices may be scheduled according to the requirements of these devices due to the event-driven nature of their traffic. Third, a Stochastic Differential Game (SDG) for energy-aware design is configured with an emphasis on controlling the transmit power of the MTC device. In particular, for some example implementations, Mean Field (MF) approximation is presented as an approach to finding a possible solution to the constructed SDG, which is entirely based on the state dynamics of MTC nodes within the HTC/MTC combination cellular infrastructure. It depends on how accurate the MF modeling of state dynamics is. Fourth, an optimal control policy based on the mean-field approximated solution is established, which entails using a finite difference method (FDM) to solve the MF equation to obtain a converging energy-aware power control.

For the sake of discussion, as shown in FIG. 1 , a single cell in which MTC and HTC devices coexist with different service requirements are variously distributed is considered. As such, in the exemplary network environment 100 of FIG. 1 , a large number of MTC devices are randomly distributed over the entire cell area, and the BS 120 may be said to be the center of the cell environment. Now, assume that there are a total of N users, that is, J typical HTC devices and K MTC devices within the coverage of the BS 120 .

In an embodiment, interference associated with an HTC device and/or an MTC device in a network environment 100 where MTC and HTC devices coexist as shown in FIG. 1 may be modeled based on the interaction in such network environment 100 . can Such interference is referred to as inter-domain interference or intra-domain interference, respectively, depending on whether the interference is associated with both the HTC device and the MTC device, or the interference associated with either the MTC device or the HTC device. is called

For example, the intra-domain interference introduced by the remaining MTC devices to a transmission from the kth MTC device at any time t (which may be referred to as the kth MTC transmission hereinafter) can be expressed as follows.

는 제i MTC 디바이스의 송신 전력이고,

는 제i MTC 디바이스 및 제k MTC 송신의 수신단(가령, 네트워크 환경(100)에서의 uRLLC 디바이스(151)로부터의 업링크 송신을 위한 시나리오에서, BS(120)) 간에 겪는 채널 이득이다.here,

is the transmit power of the i-th MTC device,

is the channel gain experienced between the i th MTC device and the receiving end of the k th MTC transmission (eg, BS 120 in the scenario for uplink transmission from uRLLC device 151 in network environment 100 ).

In addition, the inter-domain interference introduced by the HTC device to the transmission from the kth MTC device at any time t may be defined as follows.

는 제j HTC 디바이스의 송신 전력이고

는 제j HTC 디바이스 및 제k MTC 송신의 수신단 간에 겪는 채널 이득이다.here

is the transmit power of the jth HTC device,

is the channel gain experienced between the j th HTC device and the receiving end of the k th MTC transmission.

의 송신 전력으로 이득이

인 채널을 통해 잡음 분산

의 신호를 송신하는 제k MTC 디바이스에 대해, 달성가능한 신호 대 간섭 및 잡음 비율(Signal-to-Interference-plus-Noise Ratio: SINR)은 다음과 같다.Then, at time t

gain with the transmit power of

Dissipate noise through the in-channel

For a kth MTC device transmitting a signal of , the achievable Signal-to-Interference-plus-Noise Ratio (SINR) is

Differential game framework for power control

An energy efficient transmit power control mechanism for MTC devices in the network environment 100 may be designed in an energy-aware manner that takes into account the remaining battery energy of each MTC device as described above, where the energy efficiency of the wireless device is its It is based on the fact that there is a directly proportional relationship to battery life.

Specifically, according to an embodiment, the problem of how to improve energy consumption by controlling the transmit power of an MTC device in a network environment in which HTC and MTC coexist depends on the dynamics of both the interference associated with each MTC device and its residual energy. It can be defined as a stochastic differential game (SDG) based on Such an SDG framework utilizes the concept of maximizing some utility (or minimizing cost value) for each player in the game, namely an MTC device. In other words, solving the above power control problem means determining the optimal power control policy for each player.

는 플레이어 세트

={1,...,K}(가령, 전술된 전력 제어 SDG에서, 제1 내지 제K MTC 디바이스를 나타냄), 그리고 플레이어와 연관된 액션(action) 세트

및 대응하는 상태 공간

로 구성되며, 플레이어 세트

내의 각 플레이어에 의해 취해지는 액션의 성능을 분석하는 수단으로서 효용 함수(또는 비용 함수)

가 주어지며, 제어 정책

는 시간 구간 [T_i,T_f]에 걸친 비용 값(즉, 이 시간 구간에 걸쳐 효용 함수가 누적된 값)을 최소화하고자 각 플레이어에 의해 취해지도록 정해진 일련의 액션(가령, 전술된 전력 제어 SDG에서, 각 플레이어의 송신 전력 값의 세트)이다.Hereinafter, for discussion, an SDG for controlling the transmit power of the MTC device is defined for all time points t in the time interval _{from the start time Ti} to the end time T _{f .} Typically, a game of stochastic differentiation

is a set of players

={1,...,K} (eg, in the power control SDG described above, representing the first through Kth MTC devices), and an action set associated with the player

and the corresponding state space

It consists of a set of players

A utility function (or cost function) as a means of analyzing the performance of an action taken by each player in

is given, and the control policy

is a set of actions (e.g., the power control SDG described above) that are determined to be taken by each player to minimize the cost value over the time interval [T _i ,T _{f ] (i.e. the accumulated utility function over this time interval).} , is the set of transmit power values of each player).

의 상태 동역학, 곧 잔여 배터리 에너지 및 소모된 전력 간의 관계는 다음과 같이 베터리의 잔여 에너지 레벨

가 송신 전력 소모

에 따라 감소한다고 두더라도 무방하다.According to an embodiment, the power control SDG has a modeled state space based on the residual battery energy and associated interference of each MTC device. First, the MTC device only transmits for a finite amount of time due to limited battery energy. In general, the available energy of the battery of the kth MTC device (k is 1, 2, ..., or K) at any time t where _{t∈[T i} ,T _{f ].}

The state dynamics of , that is, the relationship between the remaining battery energy and the power dissipated, is as follows:

transmit power consumption

It is safe even if we leave it to decrease according to

는, 아래의 수학식 5에 표현된 바와 같이, 제k MTC 디바이스에 의해 유발된 간섭(이 예에서, 제k MTC 디바이스에 의해 다른 MTC 디바이스로부터의 송신에 도입된 도메인내 간섭 및 제k MTC 디바이스에 의해 HTC 디바이스로부터의 송신에 도입된 도메인간 간섭을 포함함)일 수 있다. 다만, 본 발명의 실시예는 이 점에 한정되지 않는데, 몇몇 다른 예시적인 구현에서 제k MTC 디바이스와 연관된 간섭

는 제k MTC 송신에 대해 다른 디바이스에 의해 야기된 간섭(가령, 도메인내 간섭

및 도메인간 간섭

의 합)일 수 있다.In some example implementations, interference associated with the kth MTC device

is, as expressed in Equation 5 below, the interference caused by the kth MTC device (in this example, intra-domain interference introduced by the kth MTC device to transmission from another MTC device and the kth MTC device) including inter-domain interference introduced in transmissions from HTC devices by However, embodiments of the present invention are not limited in this respect, and in some other exemplary implementations, interference associated with the kth MTC device

is the interference caused by other devices on the kth MTC transmission (eg, intra-domain interference)

and inter-domain interference

sum of) can be

는 제k MTC 디바이스의 송신 전력이고,

는 제k MTC 디바이스 및 제i MTC 디바이스로부터의 송신의 수신단 간에 겪는 채널 이득이며,

는 제k MTC 디바이스 및 제j HTC 디바이스로부터의 송신의 수신단 간에 겪는 채널 이득이다. 또한, 위 수학식은 간섭 레벨

이 송신 전력

와 간섭 이득

의 곱으로서 나타내어질 수 있음을 보여준다. 그러면, 간섭 상태의 동역학은 다음과 같이 나타낼 수 있다.here

is the transmit power of the kth MTC device,

is the channel gain experienced between the k th MTC device and the receiving end of the transmission from the i th MTC device,

is the channel gain experienced between the receiving end of the transmission from the kth MTC device and the jth HTC device. In addition, the above equation is the interference level

this transmit power

with interference gain

shows that it can be expressed as a product of Then, the dynamics of the interference state can be expressed as

According to an embodiment, the state space for the power control SDG is defined in two dimensions as follows.

As mentioned above, each player's goal in the SDG is to maximize their respective utility to determine an optimal control policy. In an embodiment, each MTC device seeks to minimize power consumption and ultimately maximize battery life, which can be achieved using an appropriate utility function. For example, the utility function can be defined as

는 수학식 3에서 정의된 SINR이고,

는 신뢰성 있는 통신 링크를 수립하는 데에 요구되는 SINR 임계 값(가령, 이는 모든 MTC 디바이스에 대해 동일하다고 가정될 수 있음)이며,

는 제k MTC 디바이스의 송신 전력

및 SINR

간의 차이의 단위를 조화시키기 위해 포함된 상수이다. 이 효용 함수에는 제k MTC 디바이스가 경험하는 통신 성능을 특징짓는 SINR 및 제k MTC 디바이스에 의해 취해지는 송신 전력이 모두 반영되어 있다는 점이 인식될 것이다. 또한, 위와 같이 주어진 효용 함수가

에 대해 볼록(convex)임은 쉽게 증명될 수 있다.here

is the SINR defined in Equation 3,

is the SINR threshold required to establish a reliable communication link (eg, it can be assumed to be the same for all MTC devices),

is the transmit power of the kth MTC device

and SINR

A constant included to reconcile the units of difference between them. It will be appreciated that this utility function reflects both the SINR, which characterizes the communication performance experienced by the kth MTC device, and the transmit power taken by the kth MTC device. Also, the utility function given above is

It can be easily proved that it is convex with respect to .

를 결정하는 문제로 귀결된다. 그러한 최적 제어 문제는 다음과 같이 표현될 수 있다.Now, according to an embodiment, the power control problem is an optimal transmit power control policy in which the utility function given by Equation (8) minimizes the accumulated cost value during the _{time interval [T i} ,T _{f ]}

comes down to the problem of determining Such an optimal control problem can be expressed as

는 최종 시간 T_f에서의 효용 함수의 값이다.here

is the value of the utility function at the final time T _{f .}

In an embodiment, the following value function is introduced to deal with this optimal control problem.

는 최종 시간 T_f에서의 최종 상태

에서 가치 함수가 갖는 값이다.here

is the final state at the final time T _f

is the value of the value function in .

(단, k∈

)이 미분 게임

의 내쉬 균형(Nash Equilibrium: NE)이기 위한 필요 충분 조건은 다음과 같다.Then, the optimal control profile

(However, k∈

) this differential game

The necessary and sufficient conditions for a Nash Equilibrium (NE) of

는 제k MTC 디바이스를 제외한 모든 MTC 디바이스의 송신 전력 벡터이다. 어떤 MTC 디바이스도 최적 제어 정책 외의 다른 전력 제어 정책을 채택하여 효용 함수를 더 낮출 수는 없다.here

is the transmit power vector of all MTC devices except the kth MTC device. No MTC device can adopt a power control policy other than the optimal control policy to further lower the utility function.

Specifically, according to the optimal control theory and Bellman's optimality principle, the value function of Equation 10 satisfies the partial differential equation known as the Hamilton-Jacobi-Bellman (HJB) equation. Since the value function is a solution of the HJB equation, it results in maximum utility (ie, minimum cost) for a dynamic differential game system with an associated utility function as in equation (8). In an embodiment, the HJB equation is:

Here, the Hamiltonian is defined as follows.

에서 적어도 하나의 NE가 존재한다.The existence of NE is guaranteed for the above-described differential game if a unique solution of the HJB equation can be found. A solution to the HJB equation of Equation 12 exists when the Hamiltonian is smooth. Since the utility function defined above has continuity, the Hamiltonian of Equation 13 is smooth. After all, the power-controlled differential game

There is at least one NE in

However, in a network environment including a total of K MTC devices, finding a converged solution that leads to the NE of the above differential game means finding K partial differential equations at the same time, so the dimensional difficulty (that is, the It is difficult to realize in practice because the complexity increases exponentially as the number of MTC devices increases. Therefore, the exemplary transmit power control technique is to be implemented in a way to solve such an SDG by approximating it to a Mean Field Game (MFG) based on the Mean Field (MF) concept, as discussed below. can

Mean Field Game Framework for Power Control

According to the MF concept, any differential game becomes suitable for solving based on the MFG theory if the number of players becomes very large. MFG is expressed as a system in which the HJB equation and the Fokker-Planck-Kolmogorov (FPK) equation are combined. The FPK equation is used to describe the evolution of the MF by modeling the collective behavior of all players according to the action taken, and the HJB equation is used to model the relationship between the individual player and the MF, so that player over a given time interval. determines the optimal control policy of

에 대해, 평균장은 다음과 같이 임의의 주어진 시간에서 플레이어 세트에 걸친 그 게임의 상태의 통계적 분포로 정의된다.In an embodiment, the power control differential game described above

For , the mean field is defined as the statistical distribution of the state of the game over a set of players at any given time as

은 주어진 조건이 참이면 1이고 그렇지 않으면 0인 지표 함수를 나타내고,

는 수학식 7에서 주어진 바와 같이 전력 제어 게임의 상태 공간이다.here

denotes an index function that is 1 if the given condition is true and 0 otherwise,

is the state space of the power control game as given in equation (7).

In an embodiment, converting the power control differential game for MTC device to MFG is premised on the following characteristics. First, individual players act in a rational and independent manner, choosing an optimal control policy. Second, each player interacts with the average player rather than engaging in individual interactions with the other players. Third, the large number of MTC devices provided in power control guarantees the continuity of the average field model. Fourth, the player's actions are modeled using a mean-length approximation, with the interchangeability that changing them does not affect the outcome.

In an embodiment, to deal with a large number of MTC devices, the following mean-field approximated interference may be considered.

는 MF 근사화에서 제i MTC 디바이스에 대해 요구되는 송신 전력이고,

는 MF 근사화된 채널 이득이다. 예를 들어, 업링크 송신 테스트 시나리오에서, 각 MTC 디바이스의 송신 전력이 주어지고 제k MTC 디바이스로부터 BS로 의도된 신호가

의 송신 전력으로 송신된다고 가정하자. 그러면, BS에서 수신된 전력

는 다음 수학식과 같이 수신된 의도된 신호의 전력

과 그러한 신호에 대한 총 간섭의 전력의 합이다.here

is the transmit power required for the i th MTC device in the MF approximation,

is the MF approximated channel gain. For example, in the uplink transmit test scenario, the transmit power of each MTC device is given and the signal intended from the kth MTC device to the BS is

Assume that it is transmitted with a transmit power of . Then, the power received from the BS

is the power of the received intended signal as in the following equation

and the power of the total interference for such a signal.

가 추정될 수 있다. 또한, 추정된 채널 이득을 사용하여, 수학식 3과 마찬가지 형태를 갖는 MF 근사화된 SINR

가, 그리고 결국 MF 근사화된 효용 함수

가 주어질 수 있다.Therefore, the channel gain

can be estimated. In addition, using the estimated channel gain, the MF approximated SINR having the same form as Equation 3

a, and eventually MF the approximated utility function

can be given

가 주어지면, 전력 제어 MFG는 HJB 방정식과 FPK 방정식의 결합으로 정의될 수 있다. 우선, FPK 방정식은 다음과 같다.As above, the utility function

Given , the power control MFG can be defined as a combination of the HJB equation and the FPK equation. First, the FPK equation is:

And, the HJB equation combined with the above FPK equation is given in the same form as in Equation 12.

가 획득되며, FPK는 시간에 따라 순방향으로 전개되어

가 획득된다.The FPK and HJB equations are also referred to as forward and backward equations, respectively, depending on the nature of their interaction with the mean field. That is, the HJB equation is always solved in the reverse direction in time.

is obtained, and the FPK develops forward with time

is obtained

및 평균장

를 통해 표현되는 안정적이고 수렴되는 최적 제어 정책을 평균장 균형(Mean Field Equilibrium: MFE)이라고 정의할 수 있다. 실시예에서, 전력 제어 MFG의 MFE는 유한 차분법을 사용하여 달성할 수 있다. 특히, 몇몇 예시적인 구현에서, MFG의 HJB-FPK 방정식 조합을 푸는 데에 다음과 같이 Lax-Freidrichs 기법이 적용된다. 우선, 시간 구간 [0, T_f](편의상, MFG의 시작 시간 T_i는 0이라고 가정됨), 에너지 상태 구간 [0, E_max] 및 간섭 상태 구간 [0, β_max]가 각각 3차원 도메인 공간의 x축, y축 및 z축 상으로 이산화되어 맵핑된다. 예를 들어, 이산화된 3차원 공간은 x축, y축 및 z축에서의 스텝 크기(step size)

,

및

가 각각 다음과 같이 균일한 격자 공간(grid space)일 수 있다.Furthermore, the optimal value function

and average length

A stable and convergent optimal control policy expressed through ? can be defined as Mean Field Equilibrium (MFE). In an embodiment, the MFE of the power control MFG may be achieved using a finite difference method. In particular, in some exemplary implementations, the Lax-Freidrichs technique is applied as follows to solve the combination of HJB-FPK equations in MFG. First, the time interval [0, T _f ] (for convenience, _{it is assumed that the start time T i} of the MFG is 0), the energy state interval [0, E _max ], and the interference state interval [0, β _max ] are three-dimensional domains, respectively. It is discretized and mapped onto the x-axis, y-axis, and z-axis of space. For example, a discretized three-dimensional space has a step size in the x-axis, y-axis, and z-axis.

,

and

Each may be a uniform grid space as follows.

Then, obtaining a solution of the MFG involves solving the FPK and HJB equations for each grid point in the three-dimensional grid space. When this technique is applied, the FPK equation is discretized as

,

및

는 각각 격자 공간 내의 격자점 (i,j,k)(즉, 시간 i, 에너지 레벨 j 및 간섭 상태 k)에서의 평균장, 송신 전력 및 간섭 이득을 나타낸다.here

,

and

denotes the average field, transmit power and interference gain at grid points (i, j, k) (ie, time i, energy level j, and interference state k) in grid space, respectively.

Since the finite difference method cannot be directly applied to solve the HJB equation due to the Hamiltonian, the optimal control problem is newly defined as an MFG problem in which the FPK equation is combined with the HJB equation as a constraint as follows.

=0으로 설정하고 라그랑주 승수(Lagrange multiplier)

를 도입함으로써 라그랑지안(Lagrangian)

이 획득된다.Then, as can be seen from the following equation,

=0 and Lagrange multiplier

Lagrangian by introducing

this is obtained

Now, by applying the finite difference method to solving Equation 23 like the transformation of Equation 21 for solving the FPK equation, the optimal control problem newly defined as MFG can be solved. Accordingly, the discretized Lagrangian for this MFG is given as

,

및

는 각각 다음과 같다.here

,

and

are each as follows.

,

,

및

는 각각 격자 공간 내의 격자점 (i,j,k)(즉, 시간 i, 에너지 레벨 j 및 간섭 상태 k)에서의 평균장, 송신 전력, 효용 함수 값 및 라그랑주 승수를 나타낸다.in these formulas

,

,

and

denotes the average field, transmit power, utility function value and Lagrange multiplier at grid points (i, j, k) (ie, time i, energy level j and interference state k) in grid space, respectively.

For the converged optimal control policy, the optimal decision variables P ^* , M ^* , λ ^* for the lattice space must satisfy the Karush-Kuhn-Tucker (KKT) condition. At any point (i, j, k) in the grid space, the optimal transmission control policy is such that the following equation is zero.

으로부터 라그랑주 승수에 관한 다음의 관계가 도출된다.Additionally,

From , the following relation with respect to the Lagrange multiplier is derived.

Exemplary transmit power control technique

2 shows a method for controlling transmit power for MTC in the form of pseudocode according to an embodiment of the present invention. An exemplary transmit power control technique is to optimally adjust the transmit power of an individual MTC device in a network environment (eg, network environment 100 ) to minimize battery energy consumption during a transmit period. To this end, interference associated with the MTC device in the network environment may be considered (such interference is caused by the MTC device in a network environment configured for access of the HTC device in addition to the communication of the MTC device, as well as the interference experienced by other MTC devices It also shows the interference experienced by the HTC device).

In an embodiment, the transmit power control technique is an FPK equation of a mean field representing the statistical distribution of a system state (which may be a two-dimensional state defined by residual battery energy and associated interference) across multiple MTC devices located within a network environment. Using the finite difference method, the solution of the mean field game coupled as a constraint to the HJB equation governing the determination of the optimal control policy of the transmit power of each MTC device over this time interval (e.g., [0, T _{f ])} It entails acquiring The optimal control policy of the transmit power of each MTC device minimizes the cost value accumulated over the time interval by a utility function based on the transmit power and the SINR of the transmission from that MTC device. According to the exemplary transmit power control scheme, as shown in FIG. 2 , in addition to a time interval, an interval of remaining battery energy (eg, [0, E _max ]) and an interval of associated interference (eg, [0, β _max ]) For each of a plurality of lattice points in the lattice space where the three-dimensional domain space defined by Then, based on the discretized Lagrangian for the mean field game (which is based on the utility function as shown in Equation 24), the update value of the transmit power in the reverse direction in time can be calculated, in order to obtain a convergence solution. Equation 21, Equation 28 and Equation 29 may be repeatedly used in calculations. Specifically, the solution of such MFG can be obtained as follows.

According to an embodiment, as indicated by step 205 , initial values of each of the average field, Lagrange multiplier and transmit power are first obtained. Since the average field develops forward in time according to the FPK equation, the initial value of the average field is given for a grid point (0,0,0) in grid space. Since the HJB equation is calculated in the reverse direction in time, the initial values of the transmit power and each of the Lagrange multipliers are given for the grid point (X+1,0,0).

Thereafter, a loop iteration repeating steps 210 , 215 , and 220 is performed. This loop iteration is repeated until the iteration termination condition is satisfied. For example, such a termination condition may be that the loop iterates for a preset maximum number of iterations (Iter _max ). In some exemplary implementations, this loop iteration may be terminated when other independent iteration termination conditions are satisfied, for example, even if the number of iterations of the loop (Iter) does not reach the maximum number of iterations, the update value of the average field and If the difference between its previous value ^{is less than a preset value (eg, 10 -5} ), the loop iteration may end.

는 시간에 있어서 순방향으로 계산된 것임이 이해될 수 있다. 추가로, 잔여 배터리 에너지 상태만 스텝 크기만큼 변화시킨 송신 전력

가 0이 아니라면 시간 및 두 상태 변수 모두 각자의 스텝 크기만큼 변화시킨 평균장

은 0이 될 수 있고, 그렇지 않으면 그러한 변화가 없을 때의 평균장

과 동일하게 유지될 수 있다.In step 210, for each grid point in grid space, an update of the mean field is computed using a discretized FPK equation (e.g., Equation 21), the discretized version of the FPK equation being a term dependent on transmit power and a term dependent on the interference gain. In particular, from Equation (21), the update value of the average length

It can be understood that is calculated in the forward direction in time. In addition, transmit power with only the remaining battery energy state changed by the step size

If is non-zero, the average length of time and both state variables changed by their respective step sizes

can be zero, otherwise the average field with no such change

can remain the same as

)이 0임을 나타내는 관계식(가령, 수학식 29)에 기반할 수 있다. 특히, 라그랑주 승수의 갱신치

는 시간에 있어서 역방향으로 계산된 것임이 이해될 수 있다.In step 215, for each grid point in grid space, an update of the Lagrange multiplier is computed, which is only the rate of change of the discretized Lagrangian with respect to the mean field (i.e.,

) may be based on a relational expression indicating that 0 is 0 (eg, Equation 29). In particular, the update value of the Lagrange multiplier

It can be understood that is calculated in the reverse direction in time.

)을 나타내는 관계식에 기반할 수 있다. 특히, 송신 전력의 갱신치

는 시간에 있어서 역방향으로 계산된 것임이 이해될 수 있다.In step 220, for each grid point in grid space, an update of transmit power is computed, which only means that the rate of change of the discretized Lagrangian with respect to transmit power (e.g., Equation 28) is zero (i.e.,

) can be based on a relational expression representing In particular, the update value of the transmit power

It can be understood that is calculated in the reverse direction in time.

In an embodiment, the solution of the MFG may be obtained in such a way that batch processing is performed for multiple MTC devices for future reference. In addition, the obtained solution is to be provided in the form of a data structure (eg, Look-Up Table (LUT)) to be referenced as an indexing operation for MTC transmission conforming to the optimal control policy by the MTC device in the network environment later. This allows the MTC device to transmit with optimally controlled power. In some example implementations, in a network environment where the MTC device coexists with the HTC device, the MFG solution is determined according to the transmit power control scheme illustrated in FIG. 2 using the battery energy level available at the MTC device and measurements of interference associated with the MTC device It is obtained in a batch processing manner, and the obtained solution may be stored in a LUT form so that it can be used by any MTC device that will exist in the future in a network environment to perform MTC transmission with optimal transmission power. Accordingly, such an MTC device can obtain the advantage of reduced latency over the implementation of the real-time MFG solving method by individually referencing the already obtained MFG solution in the LUT, especially in applications requiring low latency (e.g., uRLLC ) will be advantageous.

Exemplary transmit power control device

3 is a block diagram of an apparatus for performing transmission power control for MTC according to an embodiment of the present invention. The exemplary transmit power control apparatus 300 may be configured to determine an optimal transmit power control policy for an MTC device in a wireless communication network environment (eg, network environment 100 ) using the transmit power control technique described above. . For example, the wireless communication apparatus 300 may be or be implemented within a device located within a network environment, or other device communicatively coupled thereto. Referring to FIG. 3 , the apparatus 300 for controlling transmit power is exemplified as including a processor 310 and a memory 320 . In an embodiment, the processor 310 may operate the apparatus 300 according to the transmit power control scheme of FIG. 2 . For example, memory 320 may store computer-executable (or processor-executable) instructions, which when executed by processor 310 cause device 300 to perform the transmit power control techniques described above. can do.

Example embodiments may be implemented as a computer-readable storage medium comprising a computer program embodied in the acts, techniques, processes, or any aspect or portion thereof described herein. The computer-readable storage medium may include program instructions, local data files, local data structures, and the like alone or in combination. A program that may implement or use the disclosed acts, techniques, processes, or any aspect or portion thereof, is a programming language of any type (e.g., compiled or interpreted) that can be executed by a computer, such as: It may be implemented in assembly, machine language, procedural language, object-oriented language, etc., and may be combined with hardware implementation. The term “computer-readable storage medium” is used to store instructions for execution by a computing device (which, when executed, cause the computing device to perform the disclosed techniques), and to store data structures used by or associated with such instructions. It may include any medium that can be Examples of computer-readable storage media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROM, RAM, flash memory, memory devices such as, but not limited to, solid-state memory.

Although some embodiments of the present invention have been described in detail above, these should be regarded as illustrative and not restrictive. Those of ordinary skill in the art to which the present invention pertains will understand that various changes can be made to the details of the disclosed embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the following claims and their equivalents.

100: network environment
120: base station
141, 142, 143: HTC devices
151, 152, 161, 162, 163: MTC device

Claims (8)

A transmission power control method for Machine Type Communication (MTC), comprising:
Model the Fokker-Planck-Kolmogorov (FPK) equation of the mean field representing the statistical distribution of the system state across multiple MTC devices located in a network environment and the relationship between each of the multiple MTC devices and the mean field to obtain the solution of the average field game defined by the combination of the Hamilton-Jacobi-Bellman (HJB) equations that govern the determination of the optimal control policy of the transmit power of each MTC device over the time interval using the finite difference method wherein the system state is a two-dimensional state defined by residual battery energy and associated interference, and wherein obtaining the solution comprises:
A discretized version of the FPK equation, for each of a plurality of grid points in the grid space in which the three-dimensional domain space defined by the interval of residual battery energy and the interval of the associated interference, along with the time interval, is discretized. calculating an updated value of the average field in a forward direction in time using a version;
for each grid point, calculating an update value of the transmit power in the reverse direction in time based on the discretized Lagrangian for the average field game;
How to control transmit power. According to claim 1,
The network environment is configured for the access of a Human Type Communication (HTC) device in addition to the communication of the respective MTC device, and the associated interference is caused by the respective MTC device such that the interference experienced by other MTC devices is Of course, also indicating the interference caused by the respective MTC device and experienced by the HTC device,
How to control transmit power. According to claim 1,
wherein the discretized version of the FPK equation includes a term dependent on the transmit power and a term dependent on an interference gain, wherein the associated interference is the transmit power multiplied by the interference gain.
How to control transmit power. According to claim 1,
The optimal control policy of the transmit power of each MTC device is a signal-to-interference-plus-Noise Ratio (SINR) of the transmission from the corresponding MTC device and a cost function based on the transmit power minimizes a cost value accumulated over the time interval, and the discretized Lagrangian depends on the cost function;
How to control transmit power. According to claim 1,
Calculating the update value of the transmit power comprises:
calculating an update value of a Lagrange multiplier such that only the rate of change of the discretized Lagrangian with respect to the average field becomes 0;
calculating the update value of the transmit power such that only the rate of change of the discretized Lagrangian with respect to the transmit power is zero.
How to control transmit power. 6. The method of claim 5,
The step of obtaining the solution is
obtaining an initial value of each of the average field, the Lagrange multiplier, and the transmit power;
A loop iteration of repeating the steps of calculating the update value of the average length, calculating the update value of the Lagrange multiplier, and calculating the update value of the transmit power after the step of obtaining the initial value further comprising: when the difference between the updated value of the average field and the previous value of the average field is smaller than a preset value, the loop iteration is terminated,
How to control transmit power. According to claim 1,
The solution is obtained in such a way that batch processing is performed on the plurality of MTC devices for future reference;
How to control transmit power. A transmission power control device for Machine Type Communication (MTC), comprising:
processor and
a memory having stored thereon processor-executable instructions;
The processor-executable instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 7 for controlling transmit power.
Transmit Power Control Unit.

Images