KR101845398B1 - Method and apparatus for setting barring factor for controlling access of user equipment based on machine learning - Google Patents

Abstract

The present invention provides a method and a device for setting a barring factor based on machine learning in an access control technique of an access class barring manner. According to an embodiment of the present invention, the method performs learning according to a Q-learning algorithm by putting connection success probability as a state, putting increase of the barring factor, maintenance of the barring factor and reduction of the barring factor as a selective behavior, and putting collision probability and an average access delay time as a compensation. When an actual barring factor is set, optical behavior in a current state derived from a training result is performed. According to an embodiment of the present invention, the barring factor suitable for an environment in which a base station and terminals are faced can be set through the machine learning, thereby reducing overload and congestion of a communication network.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for setting a blocking factor for a terminal access control based on a machine learning,

The present invention relates to a method and apparatus for setting a blocking factor for terminal access control based on a machine learning, and more particularly, to a method and apparatus for setting a blocking factor for a terminal access control based on a machine learning based on machine learning in an access class barring To a method and apparatus for setting an argument.

1 is a block diagram illustrating a mobile communication system. The mobile communication system includes a core network (CN) 106, a base station 102, and terminals 104a to 104n. A base station 102 is a fixed station that communicates with terminals 104a-104n and may be referred to in other terms such as an evolved-NodeB (eNB), a base transceiver system (BTS), an access point, have. One base station 102 may have more than one cell. One cell is set to one of the bandwidths of 1.25, 2.5, 5, 10, and 20 MHz, and provides a downlink or uplink transmission service to a plurality of UEs. At this time, different cells may be set to provide different bandwidths. Terminals 104a-104n may be fixed or mobile and may be referred to by other terms such as a mobile station (MS), a user terminal (UT), a subscriber station (SS), a mobile terminal (MT) .

Machine-type communication (hereinafter referred to as 'MTC') refers to communication between small devices without human intervention, and can be performed using existing mobile communication networks. Hereinafter, the terminal will be referred to as an MTC device and the base station will be referred to as an eNB.

MTC devices transmit and receive a small amount of data by using LTE-A communication network. However, since a large number of devices attempt to access the same channel (RACH: Random Access Channel) at the same time, overload and congestion congestion phenomenon. A random access procedure will be described with reference to FIG. FIG. 2 is a data flow chart showing a simplified random access procedure. FIG.

64 preambles are used for communication in the RACH in which the MTC apparatus UE tries to access. Each MTC device selects an arbitrary preamble out of 64 and notifies the eNB (Evolved NodeB) of its transmission intention by attempting to access the RACH channel (S21). The eNB checks whether there are different devices using the same preamble. Devices using the same preamble will fail to access the RACH due to a collision, and only devices that do not collide will be given access to the network, so that desired data can be transmitted to the eNB. If there is no collision, the eNB transmits a random access response to the corresponding MTC device using the same preamble (S22). Then, the MTC device sends its ID and requests a RRC (Radio Resource Control) connection (S23). Then, the eNB performs contention resolution (S24).

However, if more devices are attempted to connect, the frequency of collision increases and the data transmission efficiency becomes lower. Among the methods for preventing collision at the time of random access are Access Class Barring (ACB). In the ACB, the eNB periodically transmits ACB-related parameters in a SIB (System Information Block), and this parameter includes a barring factor (a value between 0 and 1) and a barring duration . The ACB scheme is activated when the congestion degree of the network (congestion = 1 - (number of successful connection terminals in a predetermined period / number of connection request terminals in a predetermined period) increases and the communication network is overloaded. Related to the ACB technique is disclosed in, for example, Japanese Laid-Open Patent Publication No. 10-2017-0008665.

When the ACB method is activated, the devices to be connected to the network generate random numbers from 0 to 1, and if the value is smaller than the blocking factor, the random access continues. If the value is large, the random access is restored during the blocking period, The random access is attempted again. Therefore, how to set the blocking factor value has a great effect on reducing the overload and congestion of the communication network.

It is an object of one embodiment of the present invention to provide a method for obtaining an optimum blocking factor value through machine learning in an access class barring access control method in a wireless communication network.

Another object of an embodiment of the present invention is to reduce an overload and a congestion phenomenon of a communication network by setting an optimum blocking factor value in a connection control method of an access class barring scheme in a wireless communication network will be.

A method according to an embodiment of the present invention is a method for setting a blocking factor in an access class barric access control scheme in a wireless communication network, The learning of the Q-learning algorithm is performed by setting the collision probability and the average access delay time as compensations, and the optimal behavior to be selected in each state is obtained. And a blocking factor setting step for performing an optimal behavior derived from the training result according to the current state.

An apparatus according to an embodiment of the present invention includes a processor, a memory, and a wireless communication unit for wireless communication with terminals. The processor places the probability of connection success in the state and puts the action to be able to select the increase of the blocking factor, the maintenance factor of the blocking factor and the reduction of the blocking factor, and sets the compensation probability and the average access delay time as compensation, Learning is performed a predetermined number of times or more to obtain optimal behavior to be selected in each state and stored in a memory, and a blocking factor setting step for performing the optimal behavior stored in the current state is performed, Setting the argument.

According to an embodiment of the present invention, it is possible to set a blocking factor suitable for the environments in which the base station and the terminals are located through machine learning, thereby reducing the overload and congestion of the communication network.

1 is a block diagram showing a configuration of a mobile communication system to which a machine type communication is applied.
FIG. 2 is a data flow chart showing a simplified random access procedure. FIG.
3 is a flowchart illustrating an operation flow of a blocking factor setting method for terminal connection control according to an embodiment of the present invention.
FIG. 4 is a graph showing performance in an embodiment to which the blocking factor setting method of the present invention is applied.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

In one embodiment of the present invention, a congestion coefficient is used to enable or disable an overload control method such as ACB (Access Class Barring). The congestion coefficient is expressed as a ratio of the number of colliding preambles to the total number of preamble transmissions. If the congestion coefficient exceeds a predetermined threshold value, the overload control method is enabled. If the congestion coefficient is below the threshold value, the overload control method is disabled.

In the present invention, a blocking factor is efficiently set using a Q-learning algorithm. The Q-learning algorithm is a kind of TD-learning (Temporal Difference learning) that uses the difference of fitness according to time as a learning. It is a learning method to maximize the reward given from the environment through trial and error under dynamic environment . An agent acting in the Q-learning environment takes one of the possible actions in a particular state and moves to another state. On the move, you receive compensation for the cost of action from the environment. The goal of Q-learning is to maximize the sum of these rewards. In other words, the action is chosen so that the instantaneous reward value is maximized.

The terminal (MTC device) competes for the RACH on a time slot basis. The eNB takes the action that maximizes the expected compensation in the current state when the overload control method is enabled.

Let S be a set of possible states in the environment, and let A be a set of possible actions. In time slot t, the state of the environment is s _t and the state s _t belongs to the set S of states (s _t = s∈S). In time slot t, the eNB recognizes the state s _t of the environment and takes the behavior a _t that maximizes the expected compensation based on the recognized state s _t and past experience. The action a _t belongs to the set A of actions (a _t = a∈A).

If the behavior a _t operates and the environment is changed to s _{(t + 1)} , the system will compensate r _t . The goal is to find the optimal policy π ^* (s) that can compensate for the maximum r _t for each state s. The optimal policy can be defined as Equation (1).

Where Q ^* (s, a) is the optimal Q value. The Q value is defined by Equation (2).

는 (0≤

≤1)의 범위를 갖는 할인인자(discount factor)를 말한다. 학습율(learning rate)이 0이라면 Q값은 업데이트 하지 않는다. 학습율(learning rate)을 높게 설정하면 학습이 빠르게 이루어진다. 할인인자(discount factor)를 높게 설정하면 즉각적인 보상의 미래의 보상에 대한 가중치가 더 커진다.In this equation, a is a learning rate having a range of (0?? 1)

(0?

Lt; 1). ≪ / RTI > If the learning rate is zero, the Q value is not updated. Setting a high learning rate makes learning faster. Setting a high discount factor increases the weighting of future compensation of immediate compensation.

The eNB applies Q-learning algorithms to set blocking factors to maximize system performance. To this end, the present invention defines states, actions, and rewards as follows.

로 정의한다.

는 시간 슬롯 t에서 RACH에 대해 경합하는 장치들의 수에 대한 RACH에 성공적으로 접속한 장치들의 수의 비율을 나타내며 (0≤

≤1)의 범위를 갖는다. 접속성공확률은 최소값(0)과 최대값(1) 사이에서 유한한 수(N)의 상태로 균등하게 분할되어 할당된다. 예를 들어, 상태의 수를 4개로 하는 경우(N=4)에는 접속성공확률값에 따라 0.75≤

≤1.0, 0.5≤

<0.75, 0.25≤

<0.5, 0≤

<0.25 의 4개의 상태 중의 하나로 할당된다. 따라서 eNB는 현재의 접속성공확률에 따라서 지금 어떠한 상태에 속해 있는지 알 수 있다. State is the probability of success

.

Represents the ratio of the number of devices successfully connected to the RACH to the number of devices contending for RACH in time slot t (0 <

Lt; = 1). The connection success probability is equally divided and allocated in a finite number (N) state between the minimum value (0) and the maximum value (1). For example, if the number of states is four (N = 4), the probability of connection is 0.75

? 1.0, 0.5?

&Lt; 0.75, 0.25

<0.5, 0

&Lt; 0.25. Therefore, the eNB can know what status it is in now according to the current connection success probability.

Action can take three actions: 1) increase of barring factor p value, 2) decrease of p factor value, and 3) maintenance of previous value of p factor value, and increase of p value And the reduction can be made by δ _i (δ _i {0.1, 0.2, 0.3, 0.4, 0.5}).

One way to select behavior in the training phase is to use the epsilon-greedy algorithm. The Epsilon - Greedy algorithm randomly chooses the behavior that can be selected from the current state with probability ε. That is, when choosing an action, in most cases the action with the highest Q value is selected in the current state, but the behavior is chosen randomly with probability ε.

The compensation must be defined in order for the eNB to select a behavior a with a maximum Q value in the current state s. In one embodiment, compensation is defined to maximize system performance in terms of collision probability and average access delay. That is, the compensation of the eNB for the state s and the behavior a in the time slot t is expressed by Equation (3).

는 시간 슬롯 t에서 RACH에 대해 경합하는 장치들의 수에 대한 RACH에서 충돌이 발생한 장치들의 수의 비율로 정해진다 (0≤

≤1). 평균 접속지연시간 delay _t 는 시간 슬롯 t에서 RACH 접속에 성공한 기기들에 대하여, 각 기기가 처음 랜덤 액세스를 시도한 시간부터 접속을 최종 성공한 시간까지의 지연 시간들의 평균을 나타낸다. delay _max 는 시스템에서 허용하는 최대 접속 지연 시간이며, β는 평활인자(smoothing factor)이다. (0≤β≤1)Collision probability

Is defined as the ratio of the number of devices with collisions in the RACH to the number of devices competing for RACH in time slot t (0 &lt;

1). The average access delay time delay _t represents the average of the delay times from the time when each device first attempts random access to the time when the device succeeds in connection to the devices that succeed in RACH connection in time slot t. delay _max is the maximum access delay allowed by the system, and β is the smoothing factor. (0??? 1)

Next, with reference to FIG. 3, a method of setting a blocking factor for terminal access control based on a machine learning according to an embodiment of the present invention will be described.

The method is performed in an eNB (base station). The eNB includes a processor, a memory, a wireless communication unit for wireless communication with the terminal, and a core network connection unit for connection to the core network.

As described above, the processor divides the connection success probability into a finite number of states and assigns it as an action capable of selecting an increase in blocking factor, maintenance of blocking factor, and reduction in blocking factor, Learning is performed according to the Q-learning algorithm, and the training result is stored in the memory (step S310).

That is, for all the actions a (a∈A) that can be selected in the current state s when the delay _max is the maximum access delay time allowed by the system, and β is a smoothing factor with a value between 0 and 1, mathematical compensation is represented by the following formula 3 r _t (s, a) for calculating and subjected to then choose to bring the greatest compensation in the calculated compensation action or greater than a predetermined number of times, obtain the best action selected in each state memory .

When the training is finished, it is checked whether the congestion coefficient exceeds a predetermined threshold value, when the ratio of the number of colliding preambles to the total number of preamble transmissions is a congestion coefficient (step S320).

When the congestion coefficient exceeds a predetermined threshold value, the optimum behavior stored in the memory for the current state is read and performed (step S330). Optimal behavior is one of increasing, decreasing, and maintaining the blocking factor, and the width of the increase or decrease δ _i can be selected, for example, {0.1, 0.2, 0.3, 0.4, 0.5}. On the other hand, according to the embodiment, it is possible to constitute such that the result is continuously reflected on the training result while performing the step S330.

≤1.0, 0.5≤

<0.75, 0.25≤

<0.5, 0≤

<0.25)의 4개의 상태로 이루어지고, 행동의 집합 A가 (δ_i=0.2일때, 0.2 감소, 유지, +0.2 증가)의 3개의 행동으로 이루어진다고 정의되었을 때의 성능을 보인 것이다. Figure 4 shows that the set of states S is (0.75 <

? 1.0, 0.5?

&Lt; 0.75, 0.25

<0.5, 0

<0.25), and the performance when the set of actions A is defined to be composed of three behaviors (δ _i = 0.2, decrease 0.2, keep, increase +0.2).

For compensation, the delay _max is set to the maximum number of timeslot size x preamble transmission, with beta = 0.5. In FIG. 4, the success probability represents a ratio of the number of devices that have succeeded in RACH connection to the number of devices attempting RACH connection for the entire service time T. Fail probability indicates the ratio of the number of devices that finally fail the RACH connection to the number of devices attempting RACH connection for the entire service time. The training was repeated 5 times, and the more the training was repeated, the better the performance.

The features, structures, effects and the like described in the embodiments are included in one embodiment of the present invention and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects and the like illustrated in the embodiments can be combined and modified by other persons skilled in the art to which the embodiments belong. Therefore, it should be understood that the present invention is not limited to these combinations and modifications.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of illustration, It can be seen that various modifications and applications are possible. For example, each component specifically shown in the embodiments can be modified and implemented. It is to be understood that all changes and modifications that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

102 base station
104 core network
106 terminal

Claims (9)

delete A method for setting a blocking factor in an access class blocking (Access Class Barring) access control scheme in a wireless communication network,
The probability of connection success is set as a state, and the action to select the increase of the blocking factor, the maintenance factor of the blocking factor, and the reduction factor of the blocking factor is set as the compensation and the learning is performed according to the Q-learning algorithm A training step of obtaining an optimum behavior to be selected in each state and storing it as a training result,
A blocking factor setting step of performing an optimal behavior derived from the training result according to a current state
And,
In the training step
Connection success probability in time slot t

Is the ratio of the number of devices successfully connected to the RACH to the number of devices competing for RACH in time slot t,
The probability of collision in time slot t

Is the ratio of the number of collision-causing devices in the RACH to the number of devices competing for RACH in time slot t,
The average access delay time delay _t in the time slot t is the average of the delay times from the time when each device first attempts random access to the time when the device succeeds in connection to the devices that succeed in RACH connection in time slot t,
The connection success probability is divided into a minimum value and a maximum value and is allocated in a finite number of states,
The training step comprises:
For all behaviors a (a∈A) that can be selected in the current state s when the delay _max is the maximum access delay time allowed by the system and β is a smoothing factor with a value between 0 and 1, calculating a compensation r _t (s, a), which is represented by,

A step of selecting and carrying out the action that brings the largest compensation out of the calculated compensation
Is performed a predetermined number of times or more to obtain and store an optimal behavior to be selected in each state,
Wherein the blocking factor setting step performs the optimal behavior stored for the current state.
3. The method of claim 2,
Wherein the connection success probability is equally divided between a minimum value and a maximum value and is allocated in a finite number of states. delete The method of claim 3,
Wherein the increasing and decreasing width of the blocking factor is one of 0.1, 0.2, 0.3, 0.4 and 0.5. The method according to any one of claims 2, 3, and 5,
And selecting a randomly selected behavior from the current state to a probability ε in the training step. The method according to any one of claims 2, 3, and 5,
Further comprising the step of reflecting the execution result in the blocking factor setting step to the training result. The method according to any one of claims 2, 3, and 5,
Wherein the blocking factor setting step is performed only when the ratio of the number of collided preambles to the total number of preamble transmissions is a congestion coefficient and the congestion coefficient exceeds a predetermined threshold value . A processor,
A memory,
And a wireless communication unit for wireless communication with the terminals,
The processor comprising:
The probability of connection success is set as the state, and the action of selecting the increase of the blocking factor, the maintenance factor of the blocking factor, and the reduction factor of the blocking factor is set as a compensation, the collision probability and the average access delay time are compensated, A training step of performing an operation more than a predetermined number of times,
A blocking factor setting step of performing the optimal behavior stored for the current state
A blocking factor for terminal connection control is set,
In the training step
Connection success probability in time slot t

Is the ratio of the number of devices successfully connected to the RACH to the number of devices competing for RACH in time slot t,
The probability of collision in time slot t

Is the ratio of the number of collision-causing devices in the RACH to the number of devices competing for RACH in time slot t,
The average access delay time delay _t in the time slot t is the average of the delay times from the time when each device first attempts random access to the time when the device succeeds in connection to the devices that succeed in RACH connection in time slot t,
The connection success probability is divided into a minimum value and a maximum value and is allocated in a finite number of states,
The training step comprises:
For all behaviors a (a∈A) that can be selected in the current state s when the delay _max is the maximum access delay time allowed by the system and β is a smoothing factor with a value between 0 and 1, calculating a compensation r _t (s, a), which is represented by,

A step of selecting and carrying out the action that brings the largest compensation out of the calculated compensation
Is performed a predetermined number of times or more to obtain and store an optimal behavior to be selected in each state,
The blocking factor setting step performs the stored optimal behavior for the current state.

Images