KR20220158916A - Method of radio resource scheduling for beyond-5g network using artificial intelligence, recording medium and device for performing the method - Google Patents

Abstract

A wireless communication resource allocation method using artificial intelligence comprises the steps of: determining the scheduling priority of a user between heterogeneous services of eMBB and URLLC using selectively input learning data; selecting subcarrier spacing (SCS) numerology and an orthogonal frequency division multiplex (OFDM) symbol number for a user using a URLLC service by using the determined scheduling priority as an input of an intelligent scheduler; calculating a compensation value of a state according to a combination of the SCS numerology and the OFDM symbol number that minimizes a normalized load (NL) in each slot of each user; and updating an operation value and learning data of the state by reflecting the calculated compensation value. Accordingly, it is possible to efficiently allocate resources to heterogeneous eMBB and URLLC in a B5G environment.

Description

Method for allocating resources for wireless communication using artificial intelligence, recording medium and device for performing the same

The present invention relates to a wireless communication resource allocation method using artificial intelligence, and to a recording medium and apparatus for performing the same, and more particularly, to a minislot TTI (Transmission Time Interval) scheduling for coexistence of eMBB and URLLC in a B5G environment. An intelligent next-generation NodeB (gNB) that models the selection of optimal OFDM symbol count and numerology and utilizes a Q-learning (QL) algorithm to solve it.

Ultra-Reliable and Low Latency Communication (URLLC) and enhanced Mobile Broadband (eMBB) are two categories classified by the International Telecommunication Union (ITU) for beyond 5G (B5G). This is the main category.

These two categories differ from each other in terms of required data rate, data packet size, throughput, latency and reliability. Therefore, coexistence and scheduling of eMBB and URLLC services is one of the major challenges of next-generation networks.

의 확장 가능한 부반송파 간격(SCS) 수비학을 제공한다. 또한, NR은 URLLC 트래픽이 산발적이고 eMBB에 비해 데이터 크기가 작기 때문에 다양한 수의 OFDM 심볼로 URLLC 전송을 위한 확장 가능한 미니 슬롯을 허용한다. NR에서 SCS를 늘리거나 미니 슬롯에서 OFDM 심볼 수를 줄이면 전송 시간 간격(TTI; Transmission Time Interval)이 줄어들 수 있다. The 3rd Generation Partnership Project (3GPP) standardizes New Radio (NR) as a radio interface for B5G networks. NR is

Provides a scalable subcarrier spacing (SCS) numerology of In addition, NR allows scalable mini-slots for URLLC transmission with a variable number of OFDM symbols because URLLC traffic is sporadic and the data size is small compared to eMBB. The Transmission Time Interval (TTI) can be reduced by increasing the SCS in the NR or reducing the number of OFDM symbols in the minislot.

Optimal selection of SCS and OFDM symbols in minislots is very important for coexistence of eMBB and URLLC. 3GPP has standardized a puncturing mechanism to meet URLLC latency and reliability. The puncturing mechanism aborts the ongoing eMBB traffic to send URLLC traffic in the minislot without notifying the eMBB user. However, the puncturing mechanism has a problem of degrading quality of service (QoS) of eMBB users.

KR 10-2020-0015381 A KR 10-2017-0128143 A WO 2018/204344 A1

Accordingly, the technical problem of the present invention is conceived in this respect, and an object of the present invention is to provide a wireless communication resource allocation method using artificial intelligence.

Another object of the present invention is to provide a recording medium on which a computer program for performing the wireless communication resource allocation method using artificial intelligence is recorded.

Another object of the present invention is to provide an apparatus for performing the wireless communication resource allocation method using artificial intelligence.

A wireless communication resource allocation method using artificial intelligence according to an embodiment for realizing the object of the present invention is to determine a user's scheduling priority between heterogeneous services of eMBB and URLLC using selectively input learning data. step; selecting the number of SubCarrier Spacing (SCS) numerology and Orthogonal Frequency Division Multiplex (OFDM) symbols for a user using the URLLC service by using the determined scheduling priority as an input of the intelligent scheduler; Calculating a compensation value of a state according to a combination of SCS numerology and OFDM symbol number that minimizes a normalized load (NL) in each slot of each user; and updating the operation value and learning data of the state by reflecting the calculated compensation value.

In an embodiment of the present invention, the step of determining the user's scheduling priority includes: inputting the arrival rate of each service type user's packet/second, packet length, queue weight and CSI of each user for each slot; calculating each user's priority according to the priority function for service types eMBB and URLLC; and determining the order in descending order according to the priorities of each user using the service types eMBB and URLLC.

In an embodiment of the present invention, the priority function may be based on at least one of arrival rate, data size and queue length.

In an embodiment of the present invention, the step of selecting the SCS numerology and the number of OFDM symbols may include allocating a time slot to an eMBB user having the highest priority in priority setting for each time slot; generating a random number between 0 and 1 for each minislot with time slots for a URLLC user; and comparing the generated random number with a preset threshold and learning the selected operation.

In an embodiment of the present invention, the step of comparing the generated random number with a preset threshold and learning with the selected operation may include, when the generated random number is less than or equal to a preset threshold, in a state in which SCS numerology and OFDM symbol numbers are randomly selected. The operation of is explored, and if the generated random number is greater than a preset threshold, the operation in the state with the highest Q-value can be exploited.

In an embodiment of the present invention, the step of calculating the compensation value of the state may further include calculating a normalized load of each user using service types URLLC and eMBB of each time slot.

In an embodiment of the present invention, the length of the slot may be the sum of the length of the mini-slot for the URLLC service and the length of the slot for the eMBB service.

A computer program for performing the wireless communication resource allocation method using artificial intelligence is recorded in a computer-readable storage medium according to an embodiment for realizing another object of the present invention.

An apparatus for allocating resources for wireless communication using artificial intelligence according to an embodiment for realizing another object of the present invention described above determines the user's scheduling priority between heterogeneous services of eMBB and URLLC using selectively input learning data. a priority calculation unit that determines; an optimal state selection unit that selects the number of SubCarrier Spacing (SCS) numerology and Orthogonal Frequency Division Multiplex (OFDM) symbols for a user using the URLLC service using the determined scheduling priority as an input; A compensation value calculation unit for calculating a compensation value of a state according to a combination of SCS numerology and OFDM symbol number that minimizes a normalized load (NL) in each slot of each user; and an optimal value returning unit that updates the operating value and learning data of the state by reflecting the calculated compensation value.

In an embodiment of the present invention, the priority calculation unit takes as input the arrival rate of packets/second of each service type user, the packet length, the queue weight and CSI of each user for each slot, and determines the priority for the service types eMBB and URLLC. The priority of each user may be calculated according to the ranking function, and the ranking may be determined in descending order according to the priority of each user.

In an embodiment of the present invention, the optimal state selector allocates a time slot to an eMBB user having the highest priority in priority setting for each time slot, and 0 for each mini-slot with a time slot for a URLLC user. By generating a random number between 1 and 1, the generated random number can be compared with a preset threshold and learned as a selected operation.

In an embodiment of the present invention, when the generated random number is less than or equal to a preset threshold value, the optimal state selector explores an operation in a state in which SCS numerology and the number of OFDM symbols are randomly selected, and the generated random number is previously set. If it is greater than the set threshold, the operation in the state with the highest Q-value may be exploited.

According to such a wireless communication resource allocation method using artificial intelligence, efficient resource allocation for heterogeneous eMBB and URLLC in a B5G environment is achieved by selecting the optimal state of SCS and OFDM symbols for mini-slots to provide URLLC applications. It is possible.

1 is a block diagram of a wireless communication resource allocation apparatus using artificial intelligence according to an embodiment of the present invention.
2 is a conceptual diagram of a downlink single cell cellular network providing heterogeneous services (URLLC and eMBB).
3 is a diagram showing the frame structure of New Radio (NR) with possible SCS numerologies for various spectrums and deployments.
4 is a diagram showing a queue model describing PHY/MAC layer operation using a puncturing mechanism in a gNB for 5G applications.
5 is a diagram showing a state space diagram (speed adaptation scheme) of the present invention.
6 is a diagram showing the agent-environment interaction of the intelligent gNB proposed in the present invention.
7 is a diagram showing a general QL algorithm.
8 is a flowchart of a priority calculation process according to an embodiment of the present invention.
9 is a flowchart of a wireless communication resource allocation method using artificial intelligence according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable one skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a block diagram of a wireless communication resource allocation apparatus using artificial intelligence according to an embodiment of the present invention.

A wireless communication resource allocation apparatus (10, hereinafter) using artificial intelligence according to the present invention selects the optimal number of OFDM symbols and numerology for mini-slot TTI (Transmission Time Interval) scheduling for coexistence of eMBB and URLLC in a B5G environment It is about an intelligent next-generation NodeB (gNB) that models and utilizes a Q-learning (QL) algorithm to solve it.

Referring to FIG. 1 , an apparatus 10 according to the present invention includes a priority calculation unit 110, an optimum state selection unit 130, a compensation value calculation unit 150, and an optimum value return unit 170.

In the apparatus 10 of the present invention, software (application) for performing wireless communication resource allocation using artificial intelligence may be installed and executed, and the priority calculation unit 110, the optimal state selection unit 130, Configurations of the compensation value calculation unit 150 and the optimal value return unit 170 may be controlled by software for performing wireless communication resource allocation using the artificial intelligence executed in the device 10 .

The device 10 may be a separate terminal or a part of a module of the terminal. In addition, the configuration of the priority calculation unit 110, the optimal state selection unit 130, the compensation value calculation unit 150, and the optimum value return unit 170 are formed as an integrated module or as one or more modules. It can be done. However, on the contrary, each component may be composed of a separate module.

The device 10 may be mobile or stationary. The apparatus 10 may be in the form of a server or engine, and may be a device, an apparatus, a terminal, a user equipment (UE), a mobile station (MS), or a wireless device. It can be called by other terms such as wireless device, handheld device, etc.

The device 10 may execute or manufacture various software based on an operating system (OS), that is, a system. The operating system is a system program for enabling software to use the hardware of the device, and is a mobile computer operating system such as Android OS, iOS, Windows mobile OS, Bada OS, Symbian OS, Blackberry OS, and Windows-based, Linux-based, Unix-based, It can include all computer operating systems such as MAC, AIX, and HP-UX.

The International Telecommunication Union (ITU) classifies heterogeneous traffic in fifth generation (5G) cellular communications into three categories: Enhanced Mobile Broadband (eMBB), Ultra Reliable and Low Latency Communication (URLC), and Massive Machine-Type Communication (mMTC). .

eMBBs include bandwidth-hungry applications such as high-volume video streaming and augmented/virtual reality (AR/VR). mMTC addresses sensing, measurement, monitoring, measurement and calibration applications to support large-scale deployments of the Internet of Things (IoT), while URLLC supports latency- and reliability-sensitive applications such as autonomous vehicles and drones.

The coexistence of eMBB and URLLC applications is very important in 5G networks. 2 illustrates a downlink transmission scenario of a single-cell next-generation NodeB (gNB). The most stringent requirements for URLLC defined by 3GPP (3rd Generation Partnership Project) are as follows.

1. Low end-to-end latency should be as low as 1 ms when the air interface latency is 0.5 ms.

2. Reliability should be as high as 99.99%, corresponding to a 10 ^-9 packet error rate. This reflects a reliability failure declaration if one of the 10 ⁹ packets is not delivered within 1 ms.

The minimum requirements for eMBB are defined as:

1. Maximum Data Rate: Downlink: 20 Gbit/s

2. Latency up to 4 ms

3. Spectral efficiency 30bit/s/Hz

(여기서, n = {0, 1, 2, 3, ...})의 순서로 확장 가능하다. 지금까지는 n = 4가 표준에 포함되었다. 3GPP standardized NR (New Radio) as a radio interface for B5G networks. As shown in FIG. 3, NR follows the same frame structure and Orthogonal Frequency Division Multiplex (OFDM) transmission as Long Term Evolution-Advanced (LTE-A). The subcarrier spacing (SCS) numerology in NR is

(Here, n = {0, 1, 2, 3, ...}) can be extended in the order. So far, n = 4 has been included in the standard.

개의 슬롯을 포함한다. 슬롯은 TTI(Transmission Time Interval) 내에서 물리적 신호가 전송 및 반복되는 14 개의 OFDM 심볼을 포함하는 기본 프레임 구조이다.However, n > 4 is expected to be included in the next-generation standard, and the radio frame of NR has a time length of 10 ms and a sub-frame of 1 ms, and each sub-frame is

contains two slots. A slot is a basic frame structure including 14 OFDM symbols in which a physical signal is transmitted and repeated within a Transmission Time Interval (TTI).

A slot of NR corresponds to a subframe containing 14 OFDM symbols of 1 ms at 15 kHz SCS as shown in FIG. 3, and different SCS numerologies correspond to different slot lengths ranging from 1 ms at 15 kHz to 125 μs at 120 kHz Enables shorter TTI. The concept of non-slotted transmission was introduced to NR representing mini-slots in NR.

A mini-slot can start at any OFDM symbol and can carry a variable number of OFDM symbols (eg, 2, 4 or 7 symbols). Minislots with a low number of OFDM symbols enable fast transmission, providing a viable solution for low-latency applications such as URLLC, regardless of SCS coverage.

To meet the requirements of 5G, 3GPP in release 15 standardizesto standardize the TTI to be reduced from 1 ms to a few symbols to reduce physical and medium access control (MAC) layer latency. The reduced TTI enables faster user scheduling on both the uplink and downlink, and reduces the Hybrid Automatic Repeat Requests (HARQ) timeline, increasing network capacity and reducing latency. Additionally, more retransmissions can be accommodated within the latency threshold, improving packet error rate or accuracy.

In addition, increasing the SCS increases the usable bandwidth as shown in FIG. 3 . The maximum usable bandwidth goes from 50 MHz at 15 kHz SCS to 400 MHz at 120 kHz SCS. In NR, the number of subcarriers (SC) of a physical resource block (PRB) is fixed to 12. Since the number of SCs is defined by SCS, the total number of PRBs is also defined by SCS. The bandwidth of the PRB also depends on the SCS, which is 180 kHz at n = 0 and 2.88 MHz at n = 4, as shown in FIG. The present invention addresses the selection of SCS and OFDM symbols for minislots to serve URLLC applications.

End-to-end delays for successful transmission between MAC layers in downlink transmission include scheduling delay, queuing delay, transmission delay, processing delay, decoding delay, and HARQ round-trip time (RTT). Queue delays are caused by statistical multiplexing of multiple user data. The data flow of different users can be bursty and sporadic because of the different traffic patterns of different users in B5G heterogeneous networks. In order to obtain high reliability, sufficient HARQ retransmission is required.

As the user's data grows, the standby delay increases to maximize spectral efficiency. Therefore, while designing the 5G network, the latency issue must be addressed.

In downlink transmission, the gNB reserves the arriving user's packet, buffers it in the user's first transmit queue, and waits for the reservation for the first HARQ retransmission. If the first HARQ fails, the packet is available for second retransmission after RTT. Whenever a packet buffered in the gNB misses the deadline, the packet is discarded, reducing reliability. In addition, data packets that cannot be decoded by the receiving end after n HARQs are declared as failure by the gNB, and thus reliability may decrease.

Figure 4 shows the operation of the signaling queue model with heterogeneous traffic in a B5G network. In all scheduling, the gNB allocates frequency and time resources to new transmissions and retransmissions of buffered packets. However, the buffer is bounded, and packets are dropped at the gNB if the queue delay is greater than the delay requirement.

To meet stringent latency and reliability requirements, 3GPP has standardized a puncturing mechanism in NR for URLLC traffic to suspend ongoing eMBB traffic to send URLLC traffic in minislots without informing the eMBB User Equipment (UE).

The puncturing mechanism degrades the quality of service (QoS) of eMBB users. Sometimes the QoS degradation experienced by eMBB UEs outweighs the gains obtained by sending URLLC traffic. Figure 4 shows the puncturing mechanism and efficient scheduling is required to meet the QoS requirements of B5G network services.

The priority calculation unit 110 determines a user's scheduling priority between heterogeneous services of eMBB and URLLC using selectively input learning data.

The optimal state selector 130 selects the number of SubCarrier Spacing (SCS) numerology and Orthogonal Frequency Division Multiplex (OFDM) symbols for the user using the URLLC service by taking the determined scheduling priority as an input.

The system model is shown in FIG. 4 . The present invention considers the downlink dynamic scheduling of the eMBB-URLLC coexistence system, and users associated with the eMBB service generate continuous traffic (i.e. full buffer traffic) with an infinite packet size. Users associated with the URLLC service generate small bursts.

, 여기서

,

및

,

). 패킷/초 단위의 URLLC 및 eMBB 데이터 패킷의 평균 도착률은 도 4에서와 같이 M/G/Δ큐잉 모델에서 포이즌(Poison) 분포에 따른 λ_r 및 λ_e이다. 패킷 길이 L_d는 bits/packets를 의미한다. 미니 슬롯의 OFDM 심볼 수는 m = {2,4,7}로 표시된다. NR의 서브 프레임에 있는 각 슬롯의 길이는 б로 표시되고 아래의 수학식 1에 따라 계산된다. SCS의 수비는 n = {0,1,2,3,...}와 같이 n으로 표시된다.Packets following the Poisson Point Process (PPP) with an arrival rate of λ (packets/sec). Two types of services, namely URLLC and eMBB, are denoted by d _r and d _e (where

, here

,

and

,

). Average arrival rates of URLLC and eMBB data packets in units of packets/second are λ _r and λ _e according to a poison distribution in the M/G/Δ queuing model as shown in FIG. 4 . Packet length L _d means bits/packets. The number of OFDM symbols in a minislot is denoted by m = {2,4,7}. The length of each slot in the subframe of NR is denoted by б and is calculated according to Equation 1 below. The defense of SCS is denoted by n such that n = {0,1,2,3,...}.

[Equation 1]

The queue length of user service type t is represented by Q _d in units of bits and is used to determine the queue weight ω according to Equation 2. The priority function for eMBB and _URLLC users is denoted by φd. Priorities for eMBB and URLLC are defined as Equations 3 and 4 below, respectively.

[Equation 2]

[Equation 3]

[Equation 4]

where τ is the scheduling delay for URLLC users, i.e., in the 3GPP standard there are two minislots after which URLLC packets are dropped. The symbol period of OFDM is given by Equation 5 below.

[Equation 5]

The length of the minislot ζ for the URLLC service is given by Equation 6 below.

[Equation 6]

The length of the slot ρ for the eMBB service is given by Equation 7 below.

[Equation 7]

The present invention aims to choose the number of SCS numerologies n and OFDM symbols m for minislots in a URLLC service to minimize the normalized load (NL) of all users d. In addition, the priority function is defined in Equations 3 and 4. Consider the characteristics of user data by considering arrival rate, data size, and queue length in order to allocate resources fairly and balance data speed, latency, and reliability.

In the present invention, the combination of n and m is defined as speed adaptation method k as shown in FIG. 5 . The number of ODFM symbols in a slot for eMBB service is o = 14-m. NLs for eMBB and URLLC are given as Equations 8 and 9 below.

[Equation 8]

[Equation 9]

The compensation value calculator 150 calculates a compensation value of a state according to a combination of SCS numerology and OFDM symbol number that minimizes a normalized load (NL) in each slot of each user.

The optimal value returning unit 170 updates the operation value and learning data of the state by reflecting the calculated compensation value.

Reinforcement Learning (RL) is a type of machine learning (ML) in which learners (agents) do not have prior knowledge of the actions they will take to maximize the numerical reward (movement in the direction of the main goal). However, the agent must find the action to perform that yields the maximum reward with a hit and test methodology. RL has three main components: agent, environment and reward.

The Q-Learning (QL) algorithm is an effective RL method for real-time scenarios, which converges quickly on stationary, independent, and randomly distributed traffic, and is an out-of-policy time-difference RL methodology. In QL, agents interact with an unknown environment to learn and optimize system performance. Out of policy refers to the action of the agent and directly optimizes the action value Q independently of the policy. This approach simplifies the algorithm and enables fast convergence.

A problem can be modeled as a Markov Decision Process (MDP) represented by a tuple (K, A, P, R). If K represents a finite state space, then A is the agent's finite action space (set of possible actions). P is the transition probability matrix that determines the transition probability from the current state k _t to the next state k _(t+1) , and R represents the reward function that determines the reward for the agent while moving from one state to another.

The dynamics (transient probabilities) of the real-time environment are unknown. One of the effective RL algorithms for solving the Bellman optimality equation is QL. Policy π uses the Bellman equation to determine and update the action values of the state-action pairs performed at each iteration with a lookup table.

(0; 1]의 값이다. γ is a depreciation factor that limits compensation, and 0≤γ≤1. The depreciation factor determines the present value of future rewards. When the value of γ is set to 0, the agent is considered more for the immediate reward, i.e. r _k (t). As γ approaches 1, the agent considers the long-term reward, which is the future reward, α is the step size, also called the learning rate,

It is a value of (0; 1].

In Equations 10 and 11, when γ is set to 0, the agent does not learn, and when γ is a high value such as 0.9, it can be seen that the agent learns quickly.

[Equation 10]

[Equation 11]

는 SCS 수비학 n 및 TTI 동안 사용된 OFDM 심볼 수 m의 조합이다(도 5 참조). 동작

은 모든 슬롯에서

의 NL을 최소화하는 상태 k(속도 적응 체계)를 선택하기 위해 에이전트에 의해 수행된다. 도 6은 본 발명에서 제안된 지능형 gNB의 에이전트-환경 상호 작용을 보여준다.In the present invention, the agent is the gNB scheduler, and the environment's state-space

is a combination of the SCS numerology n and the number of OFDM symbols used during TTI m (see FIG. 5). movement

in all slots

Performed by the agent to select the state k (rate adaptation scheme) that minimizes the NL of 6 shows the agent-environment interaction of the intelligent gNB proposed in the present invention.

는 각 사용자

의 각 슬롯에서 NL을 최소화하는 SCS 수비학 n과 TTI(slot) 당 심볼 수인 상태 k를 선택하여 얻은 보상이다. Reward is a quantitative performance indicator of an action for a particular state. compensation in the present invention

is for each user

It is a reward obtained by selecting SCS numerology n that minimizes NL in each slot of and state k, which is the number of symbols per TTI (slot).

에 대한 두 값, 즉 x가 항상 양수이고, y보다 큰 x와 y 즉, x > y가 고려된다. 예를 들어, x = 1 및 y = 0이다. 보상은 다음의 수학식 12와 같이 정의된다.We prioritize users for scheduling and minimize the NL to meet the delay and reliability requirements at each time step t. compensation

Two values for x are always considered positive, x and y greater than y, i.e. x > y. For example, x = 1 and y = 0. Compensation is defined as in Equation 12 below.

[Equation 12]

는 현재 상태와 이전 상태의 NL 간의 차이를 나타낸다.

represents the difference between the NL in the current state and the previous state.

To maximize reward, agents prefer actions learned in the past that provide effective rewards. This is called exploitation. When an agent randomly explores different actions in order to select a better action, it is called exploration. One of the best ways to balance exploration and exploitation is to use ∈-greedy.

The agent searches with ∈ probability, and exploits with 1-∈. ∈-greedy prevents premature convergence of the system. In addition, the possibility of selecting an unsearched operation increases. During exploitation, the agent uses Equation 10 to select an action according to Equation 14 below.

[Equation 13]

The general QL algorithm is shown in FIG. 7 .

8 is a flowchart of a priority calculation process according to an embodiment of the present invention. 9 is a flowchart of a wireless communication resource allocation method using artificial intelligence according to an embodiment of the present invention.

The wireless communication resource allocation method using artificial intelligence according to the present embodiment may be performed in substantially the same configuration as the device 10 of FIG. 1 . Accordingly, components identical to those of the apparatus 10 of FIG. 1 are given the same reference numerals, and repeated descriptions are omitted.

In addition, the wireless communication resource allocation method using artificial intelligence according to the present embodiment may be executed by software (application) for performing wireless communication resource allocation using artificial intelligence.

의 패킷/초의 도착률

, 사용자 d의 패킷 길이

(비트/초), 각 사용자의 큐 가중치

및 CSI를 입력한다(단계 S100). Referring to FIG. 8, in the wireless communication resource allocation method using artificial intelligence according to the present embodiment, the priority calculation process is performed on the priority calculator for each service type user.

of packets/second arrival rate

, the packet length of user d

(bits/sec), each user's queue weight

and CSI are input (step S100).

를 계산한다(단계 S300). 이후, 서비스 유형 eMBB

및 URLLC

로 각 사용자를 우선 순위에 따라 내림차순으로 순위를 정한다(단계 S400).If all are entered for each time slot t (step S200), the priority of each user d using Equations 3 and 4

Calculate (step S300). After, service type eMBB

and URLLC

Each user is ranked in descending order according to the priority (step S400).

의 패킷/초의 도착률

, 사용자 d의 패킷 길이

(비트/초), 단계 크기 α, 감가 계수 γ, 엡실론

및 서비스 유형

및

의 우선 순위가 설정된 사용자 집합을 입력한다(단계 S10).Referring to FIG. 8, in the wireless communication resource allocation method using artificial intelligence according to this embodiment, each service type user for an intelligent scheduler

of packets/second arrival rate

, the packet length of user d

(bits/sec), step size α, decrement factor γ, epsilon

and service type

and

Enter a set of users whose priorities are set (step S10).

에게 시간 슬롯을 할당한다(단계 S11).For each time slot, the eMBB user with the highest rank in Priority Settings

A time slot is allocated to (step S11).

인 사용자에 대한 시간 슬롯이 있는 각 미니 슬롯에 대해 0에서 1 사이의 난수(random number) β를 생성한다(단계 S12). 생성된 난수가 β≤∈인지 확인한다(단계 S13).URLLC service type

A random number β between 0 and 1 is generated for each mini-slot in which there is a time slot for user . It is checked whether the generated random number is β≤∈ (step S13).

(속도 적응 방식: SCS 수비학 및 OFDM 심볼 수의 조합)를 무작위로 선택하여

의 동작을 취하여 탐색한다(단계 S14). 아니라면 지능형 gNB는 수학식 13에 따라 가장 높은 Q-값을 가진 상태

를 선택하여

의 동작을 취하여 착취한다(단계 S15).Then the intelligent gNB states

(rate adaptation method: combination of SCS numerology and OFDM symbol count) at random

Search by taking the action of (step S14). Otherwise, the intelligent gNB has the highest Q-value according to Equation 13

by selecting

Takes the action of and exploits it (step S15).

및

을 사용하여 각 사용자의 정규화된 부하를 계산한다(단계 S16).Type of service in time slot t using Equation 8 and Equation 9

and

Calculate the normalized load of each user by using (step S16).

이다. 그렇다면 보상

를 x로 업데이트하고, 그렇지 않으면

를 y로 업데이트한다.It is checked whether η<0 (step S17). here,

to be. if so reward

to x, otherwise

update to y.

를 선택하여 동작

의 Q-값

를 업데이트한다(단계 S18). 모든 상태

의 Q-값을 업데이트하고(단계 S19), 모든 동작

의 동작-보상 매트릭스를 업데이트한다(단계 S20). situation

Select to operate

Q-value of

is updated (step S18). all status

Update the Q-value of (step S19), and all operations

Updates the motion-compensation matrix of (step S20).

Thereafter, returning to step S11, the above process is repeated for all time slots.

The present invention models the selection of the optimal number of OFDM symbols and numerology for mini-slot TTI (Transmission Time Interval) scheduling for coexistence of eMBB and URLLC in a B5G environment, and utilizes a Q-learning (QL) algorithm to solve this problem. Provides an intelligent next-generation NodeB (gNB) that

Such a wireless communication resource allocation method using artificial intelligence may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. You will understand.

The present invention provides a resource allocation method for the coexistence of eMBB and URLLC in a B5G environment, and dynamically learns using a Q-learning (QL) algorithm for this purpose, so it can be usefully used for next-generation wireless communication.

10: Wireless communication resource allocation device using artificial intelligence
110: priority calculation unit
130: optimal state selection unit
150: compensation value calculation unit
170: optimal value return unit

Claims (12)

Determining a user's scheduling priority between heterogeneous services of eMBB and URLLC using selectively input learning data;
selecting the number of SubCarrier Spacing (SCS) numerology and Orthogonal Frequency Division Multiplex (OFDM) symbols for a user using the URLLC service by using the determined scheduling priority as an input of the intelligent scheduler;
Calculating a compensation value of a state according to a combination of SCS numerology and OFDM symbol number that minimizes a normalized load (NL) in each slot of each user; and
A method of allocating resources for wireless communication using artificial intelligence, comprising: updating operating values and learning data of states by reflecting the calculated compensation values.
The method of claim 1, wherein determining the scheduling priority of the user comprises:
inputting the arrival rate of each service type user in packets/second, the packet length, the queue weight of each user and the CSI for each slot;
calculating each user's priority according to the priority function for service types eMBB and URLLC; and
A wireless communication resource allocation method using artificial intelligence, comprising: determining a rank in descending order according to the priorities of each user using the service type eMBB and URLLC.
According to claim 2,
Wherein the priority function is based on at least one of an arrival rate, a data size, and a queue length, a wireless communication resource allocation method using artificial intelligence.
The method of claim 1, wherein selecting the SCS numerology and the number of OFDM symbols comprises:
allocating a time slot to an eMBB user with the highest priority in a priority setting for each time slot;
generating a random number between 0 and 1 for each minislot with time slots for a URLLC user; and
A method of allocating resources for wireless communication using artificial intelligence, comprising: comparing the generated random number with a preset threshold and learning the selected operation.
The method of claim 4, wherein the step of comparing the generated random number with a preset threshold and learning as a selected operation comprises:
When the generated random number is less than or equal to a preset threshold, the operation in the state of randomly selecting the number of SCS numerology and OFDM symbols is explored, and when the generated random number is greater than the preset threshold, the highest Q-value is selected. A wireless communication resource allocation method using artificial intelligence that exploits operation in a state of being.
The method of claim 1, wherein calculating the compensation value of the state comprises:
Further comprising, wireless communication resource allocation method using artificial intelligence; calculating a normalized load of each user using the service type URLLC and eMBB of each time slot.
According to claim 1,
The length of the slot is the sum of the length of the mini-slot for the URLLC service and the length of the slot for the eMBB service, wireless communication resource allocation method using artificial intelligence.
A computer-readable storage medium on which a computer program for performing the wireless communication resource allocation method using artificial intelligence according to any one of claims 1 to 7 is recorded.
a priority calculation unit for determining a user's scheduling priority between heterogeneous services of eMBB and URLLC using selectively input learning data;
an optimal state selection unit that selects the number of SubCarrier Spacing (SCS) numerology and Orthogonal Frequency Division Multiplex (OFDM) symbols for a user using the URLLC service using the determined scheduling priority as an input;
A compensation value calculation unit for calculating a compensation value of a state according to a combination of SCS numerology and OFDM symbol number that minimizes a normalized load (NL) in each slot of each user; and
An apparatus for allocating resources for wireless communication using artificial intelligence, comprising: an optimal value return unit for updating the operation value and learning data of the state by reflecting the calculated compensation value.
10. The method of claim 9, wherein the priority calculation unit,
Calculate the priority of each user according to the priority function for the service types eMBB and URLLC, for each slot, taking as inputs the arrival rate of each service type user in packets/sec, the packet length, the queue weight and the CSI of each user, An apparatus for allocating resources for wireless communication using artificial intelligence, which determines the ranking in descending order according to the priority of each user.
10. The method of claim 9, wherein the optimal state selection unit,
For each time slot, allocate a time slot to the eMBB user with the highest priority in the priority setting, and generate a random number between 0 and 1 for each mini slot with a time slot for a URLLC user; An apparatus for allocating resources for wireless communication using artificial intelligence, wherein the generated random number is compared with a preset threshold and learned as a selected operation.
The method of claim 11, wherein the optimal state selection unit,
When the generated random number is less than or equal to a preset threshold, the operation in the state of randomly selecting the number of SCS numerology and OFDM symbols is explored, and when the generated random number is greater than the preset threshold, the highest Q-value is selected. A wireless communication resource allocation device using artificial intelligence that exploits operation in a state of being.

Images