US10805931B2 - Resource allocation method and apparatus for wireless backhaul network based on reinforcement learning - Google Patents
Resource allocation method and apparatus for wireless backhaul network based on reinforcement learning Download PDFInfo
- Publication number
- US10805931B2 US10805931B2 US16/286,844 US201916286844A US10805931B2 US 10805931 B2 US10805931 B2 US 10805931B2 US 201916286844 A US201916286844 A US 201916286844A US 10805931 B2 US10805931 B2 US 10805931B2
- Authority
- US
- United States
- Prior art keywords
- base stations
- resource allocation
- neural network
- reinforcement learning
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- H04W72/048—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
- H04W16/02—Resource partitioning among network components, e.g. reuse partitioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/16—Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W72/00—Local resource management
- H04W72/50—Allocation or scheduling criteria for wireless resources
- H04W72/51—Allocation or scheduling criteria for wireless resources based on terminal or device properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
- H04W16/24—Cell structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W64/00—Locating users or terminals or network equipment for network management purposes, e.g. mobility management
- H04W64/003—Locating users or terminals or network equipment for network management purposes, e.g. mobility management locating network equipment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
Definitions
- the following description relates to a resource allocation technique for a wireless backhaul network.
- a backhaul for the plurality of small base stations being installed in a wired manner makes problems in complexity and economic efficiency. Accordingly, a technology for supporting a plurality of small base stations with a single powerful ultra-wide-area wireless backhaul network has been proposed.
- the ultra-wide-area wireless backhaul network is a technology for supporting group beams, narrow beams, and multi-stream wireless communication based thereon through three-step beamforming.
- the technology includes forming a large group beam using analog beamforming, forming narrow beams in the group beam using digital beamforming, and supporting multi-stream wireless communication with a corresponding base station using each of the narrow beams.
- Beamforming may be performed considering channel situations of all small base stations (full digital beamforming).
- full digital beamforming the scheme causes a problem if the small base stations include a mobile base station so that a channel situation changes even slightly or if the number of small base stations is large.
- the following description provides a technique for allocating resources to a plurality of small base stations including a mobile station in an ultra-wide-area wireless backhaul network.
- a reinforcement learning-based resource allocation method for a wireless backhaul network which is performed by a resource allocation apparatus, the method including estimating locations of a plurality of small base stations on the basis of channel state information (CSI) measured by the plurality of small base stations, and allocating resources of the wireless backhaul network to the plurality of small base stations using a reinforcement learning neural network having the locations as an input.
- CSI channel state information
- a reinforcement learning-based resource allocation apparatus for a wireless backhaul network
- the apparatus including a communication apparatus configured to receive CSI measured by a plurality of small base stations, a storage apparatus configured to store a first neural network for estimating locations of the plurality of small base stations using a plurality of pieces of CSI and a second neural network for determining allocation of resources to the plurality of small base stations using reinforcement learning on the basis of the estimated locations, and a computational apparatus configured to estimate the locations of the plurality of small base stations by inputting the CSI to the first neural network and determine allocation of resources of the wireless backhaul network to the plurality of small base stations on the basis of a reward for a current state determined by inputting the estimated locations to the second neural network.
- FIG. 1 illustrates an example of a plurality of small base stations using a wireless backhaul network.
- FIG. 2 illustrates an example flowchart of a resource allocation method using a wireless backhaul network.
- FIG. 3 illustrates an example of a neural network configuration process for estimating locations of small base stations.
- FIG. 4 illustrates an example of channel state information (CSI) measured by a small base station.
- CSI channel state information
- FIG. 5 illustrates an example flowchart showing a process of estimating a location of a small base station using a neural network.
- FIG. 6 illustrates an example of a neural network used to estimate a location of a small base station.
- FIG. 7 illustrates an example flowchart of a method of allocating resources using a reinforcement learning neural network.
- FIG. 8 illustrates an example of a state used in reinforcement learning.
- FIG. 9 illustrates an example of a reinforcement learning neural network.
- FIG. 10 illustrates an example block diagram showing a configuration of a resource allocation apparatus.
- components that will be described in the specification are discriminated merely according to functions mainly performed by the components or conventionally carried out according to common knowledge of related technical fields. That is, two or more components which will be described later can be integrated into a single component. Furthermore, a single component which will be explained later can be separated into two or more components. Moreover, each component which will be described can additionally perform some or all of a function executed by another component in addition to the main function thereof. Some or all of the main function of each component which will be explained can be carried out by another component. Accordingly, presence/absence of each component which will be described throughout the specification should be functionally interpreted.
- FIG. 1 illustrates an example of a plurality of small base stations using a wireless backhaul network.
- a wireless backhaul apparatus 50 provides a wireless backhaul network to a plurality of small base stations 11 , 12 , 13 , and 14 .
- FIG. 1 shows four small base stations.
- Each of the small base stations receives allocated resources from the wireless backhaul network and provides a network service to a user (terminal) located in a coverage area thereof.
- Each of the small base stations provides a coverage area to a small cell.
- Each of the small base stations may be called by various names, such as a femtocell and a picocell, depending on an available coverage size.
- Each of the small base stations may be a stationary base station or a mobile base station.
- the wireless backhaul apparatus 50 provide radio resources to the small base stations 11 , 12 , 13 , and 14 according to a specific communication scheme. There may be various communication schemes such as 3G, Long-Term Evolution (LTE, 4G) and 5G
- the wireless backhaul apparatus 50 may provide a large group beam using analog beamforming and form narrow beams in the group beam using digital beamforming.
- the following description is not limited to the size, type, etc. of the beams provided by the wireless backhaul apparatus 50 .
- the small base stations 11 , 12 , 13 , and 14 shown in FIG. 1 belong to one group beam. Therefore, it is assumed that the following description provides resources to the small base stations shown in FIG. 1 using narrow beams formed using digital beamforming.
- the wireless backhaul apparatus 50 may manage the allocation of resources to the small base stations 11 , 12 , 13 , and 14 .
- a wireless backhaul apparatus, small base stations 11 , 12 , 13 , and 14 , and a control apparatus 80 are shown.
- the control apparatus 80 may allocate radio resources to the small base stations 11 , 12 , 13 , and 14 on the basis of given information.
- the control apparatus 80 may be an apparatus belonging to a core network.
- control apparatus 80 may be an individual apparatus added to the wireless backhaul apparatus 50 .
- a resource allocation apparatus is described as determining to allocate resources to the small base stations.
- the resource allocation apparatus described herein may be the above-described wireless backhaul apparatus 50 or control apparatus 80 .
- the resource allocation apparatus uses a machine learning model to determine values for location estimation and resource allocation for the small base stations.
- the resource allocation apparatus may use a learning model such as a neural network.
- a small base station is to be provided with a wireless backhaul network.
- a target to which resources are to be allocated by a wireless backhaul network is not limited to a small cell. Accordingly, the following description may be applied to cells of various shapes and sizes such as a macrocell, a femtocell, a microcell, and the like.
- FIG. 2 illustrates an example flowchart of a resource allocation method 100 using a wireless backhaul network.
- a small base station measures channel state information (CSI) ( 110 ).
- a plurality of small base stations measure CSI respectively.
- the CSI measured by the small base station has two types.
- a small base station measures CSI with respect to its own wireless backhaul network (hereinafter referred to as first CSI) and CSI with respect to its neighboring small base station (hereinafter referred to as second CSI).
- first CSI CSI with respect to its own wireless backhaul network
- second CSI CSI with respect to its neighboring small base station
- the resource allocation apparatus estimates a location of the small base station on the basis of the CSI measured by the small base station ( 120 ).
- the small base station may be a mobile base station.
- the resource allocation apparatus estimates the current location of the small base station.
- the resource allocation apparatus may estimate the location of the small base station using a pre-learned neural network.
- the resource allocation apparatus may estimate locations of the plurality of small base stations.
- the resource allocation apparatus determines narrow beams and resources for the plurality of small base stations on the basis of the estimated locations of the small base stations ( 130 ). As will be described later, the resource allocation apparatus may allocate resources to the plurality of small base stations using a reinforcement learning-based neural network. Elements used for reinforcement learning are a state, an action, and a reward. The resource allocation apparatus defines a state on the basis of the location of the small base station and determines a corresponding action and a reward for the action.
- the resource allocation apparatus may estimate the location of the small base station using a pre-learned neural network.
- An apparatus for providing the neural network may be the resource allocation apparatus or a separate apparatus.
- the resource allocation apparatus provides the neural network.
- FIG. 3 illustrates an example of a neural network configuration process 200 for estimating locations of small base stations.
- the resource allocation apparatus inputs training data including CSI information and location information to a neural network ( 210 ).
- the CSI information is information input to the neural network, and the location information is information output from the neural network.
- the resource allocation apparatus inputs the CSI information to the neural network and determines the location of a small base station ( 220 ).
- the resource allocation apparatus adjusts a weight for a parameter of the neural network while comparing a location output from the neural network to an actual location included in the training data ( 230 ).
- Various methods may be utilized for the adjustment of the weight of the neural network.
- the resource allocation apparatus may adjust the weight by using a method such as stochastic gradient descent.
- the stochastic gradient descent is a method of updating a gradient using some input data as a sample.
- the resource allocation apparatus When an error rate of a value output from the neural network is less than a reference value (e.g., the error rate is less than 1%), the resource allocation apparatus completes establishment of the neural network. When an error rate for the location output from the neural network is greater than or equal to the reference value, the resource allocation apparatus repeats a process of learning the neural network ( 240 ).
- the reference value corresponds to a value capable of securing location accuracy required for the resource allocation.
- conventional location measurement systems for radio signals generally measured a position of a receiving end by using a triangulation method based on received signal strength indicators (RSSIs) or using an angle of arrival (AOA) of a radio signal in the case of a multiple-input, multiple-output (MIMO) system.
- RSSIs received signal strength indicators
- AOA angle of arrival
- MIMO multiple-input, multiple-output
- the resource allocation apparatus estimates a location of a small base station using two types of CSI information.
- the neural network estimates the location of the small base station by using the two types of CSI information as input values.
- the two types of CSI information include CSI with respect to a wireless backhaul network (first CSI) and CSI with respect to a neighboring small base station (second CSI).
- FIG. 4 illustrates an example of CSI measured by a small base station.
- the section (A) of FIG. 4 shows an example in which small base stations 11 , 12 , 13 , and 14 measure their own first CSI with respect to a wireless backhaul network.
- the small base stations 11 , 12 , 13 , and 14 each calculate their own first CSI with respect to the backhaul network on the basis of a reference signal transmitted from the backhaul network.
- a wireless backhaul apparatus may transmit reference signals through only an appropriate number (e.g., five) of antennas so that the complexity is not high enough.
- the number of antennas that transmit the reference signals may vary dynamically depending on system performance or communication conditions.
- the section (A) of the FIG. 4 shows an example in which the small base stations 11 and 12 receive a reference signal from only an antenna located in region A of a wireless backhaul apparatus 50 . Also, the section (A) of the FIG.
- the antenna that transmits the reference signal may be an antenna located in a specific region or a specific random number of antennas. Consequently, it can be said that the small base stations 11 , 12 , 13 , and 14 measure partial CSI with respect to the wireless backhaul network.
- the small base stations 11 , 12 , 13 , and 14 measure second CSI with respect to their adjacent neighboring small base stations.
- the section (B) of FIG. 4 shows an example in which the small base stations 11 , 12 , 13 , and 14 measure CSI with respect to their adjacent neighboring base stations (second CSI).
- the range of, or the distance from, the neighboring small base station may vary depending on settings. The range may be a range capable of ensuring sufficient accuracy for second CSI and measuring CSI within a predetermined time.
- FIG. 5 illustrates an example flowchart showing a process 300 of estimating a location of a small base station using a neural network.
- Each small base station measures first CSI with respect to a wireless backhaul network using reference signals transmitted by some antennas of the wireless backhaul network ( 310 ). Whether all small base stations to which resources are to be allocated have measured the first CSI is checked ( 320 ). In this case, the small base stations that have to measure the first CSI may be base stations belonging to one group beam. The small base stations measures second CSI with respect to their neighboring small base stations ( 330 ).
- FIG. 5 shows an example in which the first CSI and then the second CSI are measured, but the first CSI and the second CSI need not be measured in the order.
- the second CSI and then the first CSI may be measured.
- the small base stations transmit the measured first CSI and second CSI to the wireless backhaul apparatus ( 340 ).
- the resource allocation apparatus estimates location(s) of the small base station(s) by inputting the first CSI and the second CSI to a pre-learned neural network ( 350 ).
- FIG. 6 illustrates an example showing a neural network used to estimate a location of a small base station.
- FIG. 6 shows a recurrent neural network (RNN) as an example.
- the neural network may use various topologies.
- an RNN is suitable for reflecting CSI information characteristics.
- the RNN receives CSI information of a target base station at any time point.
- the received CSI information includes the above-described first CSI and second CSI.
- One characteristic of the RNN is that there is remembrance (data) of a previous state, unlike other artificial intelligent neural networks. This may be represented by arrows between modules R of FIG. 6 . For example, when a base station in FIG.
- the RNN utilizes information about a previous location of the base station when estimating a next location of the base station.
- Such an RNN characteristic is well suited for the current situation in which locating is performed in real time. For example, when the small base station is present at a specific time, the small base station is likely to be present near a previously located point at an adjacent time. Accordingly, the RNN, which reflects such a characteristic, may have high accuracy in estimating a location of a small base station.
- a reinforcement learning-based neural network which will be described below, is referred to as a second neural network.
- the resource allocation apparatus uses the estimated locations of the small base stations to allocate resources to the small base stations.
- An actual communication environment has a variety of cases, and thus it is almost impossible to find a heuristic method in consideration of all the cases. Accordingly, the resource allocation apparatus may use reinforce learning in which the best result value for a given situation may be output by learning an action capable of being taken in each situation and a corresponding reward.
- Q-learning is a reinforcement learning algorithm that consists of an environment, an agent, a state, an action, and a reward.
- the agent the resource allocation apparatus
- receives two rewards i.e., an immediate reward and a future reward
- the immediate reward is a reward that immediately occurs due to the action taken by the agent
- the future reward is a reward for a future environment caused by the action.
- the final goal of the agent is to update a quality (Q) value so that the two rewards may be maximally received.
- Q quality
- Equation 1 An equation for updating the Q-value, which is the most basic in the Q-learning is expressed by Equation 1 below:
- ⁇ is a discount factor, which ranges from 0 to 1.
- the conventional Q-learning has a characteristic of storing Q values for all possible situations, so that it is difficult to apply to an actual communication situation in which there are infinitely many possible situations.
- An actual communication situation includes various base stations and users, required communication traffic, etc., and transitions to another state when such items are changed.
- Q-values are stored individually for such many situations, storage space and computational complexity increase exponentially.
- a method called Deep Q-Network (DQN) has been proposed.
- the DQN is a method of substituting a neural network for a part for storing a Q value for each situation in Q-learning.
- the DQN is an algorithm for adding a value network to a Q-learning technique and performing reinforcement learning on wide state space.
- the neural network used for the DQN is characterized in that any arbitrary function may be approximated sufficiently accurately. That is, a Q value is determined through a method of approximating a function between a situation and a Q value, instead of a Q value being stored for each situation in Q-learning.
- the resource allocation apparatus allocates resources to small base stations using the DQN.
- the DQN corresponds to the above-described second neural network.
- FIG. 7 illustrates an example flowchart of a resource allocation process 700 using a reinforcement learning neural network.
- FIG. 7 is an example of a schematic process, by the resource allocation apparatus, of allocating resources to a small base station using the DQN.
- the resource allocation apparatus may preset an appropriate reward function suitable for each service type, network situation, and the like ( 410 ). The reward function will be described below.
- the resource allocation apparatus checks a current state on the basis of the location of the small base station ( 420 ).
- the resource allocation apparatus acquires a Q value by inputting the current state to the pre-learned DQN ( 430 ).
- the resource allocation apparatus determines an action of specific resource allocation corresponding to the Q value ( 440 ).
- the resource allocation apparatus computes a reward corresponding to the action ( 450 ).
- the resource allocation apparatus compares the reward for the current action to a previously computed reward (a reward for a previous action) ( 460 ). When the reward corresponding to the current action is greater than the previous reward, the resource allocation apparatus changes the resources allocated to the small base station ( 470 ).
- the state which is a communication situation, may be distribution and required traffic of small base stations.
- the action may be a distribution (allocation) status of resources and narrow beams.
- the reward may be the capacity of a current communication network. Further, considering a reward part as another factor, it is also possible to perform services suitable for various communication environments.
- FIG. 8 illustrates an example showing a state used in reinforcement learning.
- FIG. 8 shows a situation in which five small base stations are distributed in one group beam.
- C is a required capacity of each small base station.
- a reason for not using the CSI is that the CSI is changed when the location of the small base station is changed after the narrow beam is allocated in the following process. That is, the reason is that the state is changed by the action.
- the reward may be designed on the basis of various values. For example, while increasing an average capacity to be used by the small base station, it is possible to design a reward capable of optimizing various variables.
- C min % indicates the average capacity of base stations with a network usage less than or equal to a certain reference value (%).
- C min10% corresponds to the average capacity of 10% of base stations with the lowest network usage.
- the reference value is 10%.
- ⁇ , ⁇ , and ⁇ are weights.
- ⁇ , ⁇ , and ⁇ are variables for defining the importance of the variables. As the variables increases, the variables behind the variables are considered important. For example, when ⁇ has a large value, a reward value is significantly reduced although there is a little interference. This greatly reduces the interference even at the expense of average capacity.
- ⁇ is a variable for capacities of 10% of base stations with the lowest network usage. Increasing ⁇ means that the resources are distributed such that a network capacity variance is reduced.
- ⁇ is a variable for total consumed energy. When ⁇ increases, communication is performed using the minimum energy.
- the resource allocation apparatus may perform resource allocation according to a service characteristic or network situation by adjusting the weights ⁇ , ⁇ , and ⁇ in the reward function.
- the DQN should be pre-learned.
- the resource allocation apparatus checks a current state S(t). The agent acquires a Q value using the DQN. The resource allocation apparatus selects an action for determining a resource type allocated according to the Q value. Subsequently, the resource allocation apparatus observes a reward corresponding to the action. When the learning is not complete, the resource allocation apparatus stores its own action and a reward corresponding to the action. This process is repeated until the learning is complete. Through this process, the resource allocation apparatus provides a DQN for resource allocation. The resource allocation apparatus may perform learning while performing resource allocation in an actual environment. Also, the resource allocation apparatus may use certain sample data in order to perform pre-learning.
- FIG. 9 illustrates an example of a reinforcement learning neural network.
- FIG. 9 shows an example of a DQN having a convolutional neural network (CNN) structure.
- CNN convolutional neural network
- FIG. 9 is an example DQN, and various neural networks having other structures may be used.
- the CNN is composed of some convolutional layers and some fully connected layers. Each of the convolutional layers extracts a feature from a state observed through a convolution mask and shared weights. By superimposing the convolutional layers, it is possible to find more complex features. Each of the fully connected layers may derive a Q value using the found complex features.
- the first convolutional layer receives the current location and required communication capacity of a small base station as an input.
- This layer finds a low level feature using a 5 ⁇ 5 convolution mask.
- the low feature level means, for example, a simple feature such as distribution and density of terminals between any two access points (APs).
- the subsequent two layers find a high level feature using a 3 ⁇ 3 convolution mask.
- 2 ⁇ 2 max pooling is performed.
- the max pooling is a task of leaving only one maximum value in the n ⁇ n mask and can be seen as a task of reducing accuracy by reducing data.
- all output values are input to the fully connected layers.
- the first fully connected layer may have a dimension of 100, and the subsequent fully connected layers may have dimensions of 100 and 10. This is to gradually reduce the number of output values of neurons to leave only important features.
- a Q value is derived by collecting a plurality of output values into one neuron.
- FIG. 10 illustrates an example block diagram showing a configuration of a resource allocation apparatus 500 .
- the resource allocation apparatus 500 may be a wireless backhaul apparatus 50 or a separate control apparatus 80 .
- the resource allocation apparatus 500 corresponds to a computer apparatus capable of data processing and computation.
- the resource allocation apparatus 500 may be a separate computer apparatus built in the wireless backhaul apparatus 50 .
- the resource allocation apparatus 500 may be a chip embedded into the wireless backhaul apparatus 50 .
- the embedded chip may contain a program for the above-described resource allocation for the wireless backhaul network.
- the resource allocation apparatus 500 may include a communication apparatus 510 , a storage apparatus 520 , and a computational apparatus 530 .
- the communication apparatus 510 receives CSI information measured by each of a plurality of small base stations.
- the CSI information includes first CSI and second CSI.
- the communication apparatus 510 may receive the CSI information from a small base station directly or via another object.
- the communication apparatus 510 may include a communication module according to a communication scheme in which the CSI information is delivered. Further, the communication apparatus 510 may include an antenna when the CSI information is received in a wireless manner.
- the storage apparatus 520 stores the above-described first neural network and second neural network.
- the first neural network and the second neural network are pre-learned.
- the first neural network estimates a location of a small base station using the CSI measured by the small base station.
- the second neural network determines allocation of resources to the plurality of small base stations using reinforcement learning on the basis of the estimated location.
- the storage apparatus 520 may store the received CSI information, the location of the small base station, and information regarding the allocation of resources to the small base stations.
- the storage apparatus 520 may be implemented with an apparatus such as a hard disk, a flash memory, and the like.
- the resource allocation apparatus 500 may include a memory.
- the memory may store data used during the resource allocation process and data occurring during the resource allocation process.
- the computational apparatus 530 may determine allocation of resources to small base stations using the received information and the neural network stored in the storage apparatus 520 .
- the computational apparatus 530 estimates the locations of the plurality of small base stations by inputting the CSI to the first neural network.
- the computational apparatus 530 determines states and actions for the small base stations on the basis of the estimated locations of the plurality of small base stations.
- An action indicates allocation of resources to a small base station.
- the computational apparatus 530 may change the allocation of resources to the small base station while comparing a reward of a current state action to a previous reward.
- the computational apparatus 530 may include a memory for storing a program for location estimation and resource allocation for the small base station and a processor for processing data and information.
- the computational apparatus 530 may be an apparatus such as a processor for processing data and performing certain computation, an AP, or a chip in which a program is embedded.
- the above-described resource allocation method for the wireless backhaul network may be implemented with software including an executable program that may be executed by a computer.
- the program may be stored and provided in a non-transitory computer readable medium.
- the software may include a computer program, a piece of code, an instruction, or some combination thereof, for independently or collectively instructing or configuring the processing device to operate as desired.
- Software and data may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, computer storage medium or device, or in a propagated signal wave capable of providing instructions or data to or being interpreted by the processing device.
- the software also may be distributed over network coupled computer systems so that the software is stored and executed in a distributed fashion.
- the software and data may be stored by one or more computer readable recording mediums.
- the computer readable recording medium may include any data storage device that can store data which can be thereafter read by a computer system or processing device.
- Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices. Also, functional programs, codes, and code segments for accomplishing the examples disclosed herein can be easily construed by programmers skilled in the art to which the examples pertain based on and using the flow diagrams and block diagrams of the figures and their corresponding descriptions as provided herein.
- a storage or memory may include a computer component and a recording medium used to retain digital data.
- Examples of a computer storage or memory include semiconductor storage read-write random-access memory, such as DRAM, hard disk drives, optical disc drives, and the like.
Abstract
Description
where Q is quality for each state, s is a time state, α is an action, and r is a reward. s indicates a current distribution situation and required communication traffic of small base stations, and α indicates a resource assignment status of a backhaul network. Q is a function of how well the current distribution situation and the resource assignment status match. γ is a discount factor, which ranges from 0 to 1. When γ is close to 0, the importance of compensation for the present is emphasized. When γ is close to 1, the importance of compensation for the future is emphasized. Also, αt is a learning ratio, which ranges from 0 to 1, and determines the learning ratio of the Q value. For example, when αt=0, the agent does not perform learning. When αt=1, the agent performs learning by utilizing the most recent information. The agent has to perform learning through previous Q values, and thus it is assumed that αt=1.
S(t)=(X,Y,Z,C)t. [Equation 2]
A(t)=({x 1 ,y 1 ,R 1 }, . . . ,{x n ,y n ,R n})t ,ΣR n =R [Equation 3]
R t+1 =C avg −αI avg−β(C avg −C min %)−γE tot [Equation 4]
where Cavg is an average capacity of the entire network, Iavg is an average interference level measured by the small base stations, Cmin % is an average capacity of min % of base stations with the lowest network usage, and Etot is total energy used in communication. Also, Iavg is a value corresponding to interference between beams. Also, Cmin % indicates the average capacity of base stations with a network usage less than or equal to a certain reference value (%). For example, Cmin10% corresponds to the average capacity of 10% of base stations with the lowest network usage. For convenience of description, it is assumed that the reference value is 10%.
Claims (17)
R t+1 =C avg −αI avg−β(C avg −C min %)−γE tot
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR10-2018-0024400 | 2018-02-28 | ||
KR1020180024400A KR102030128B1 (en) | 2018-02-28 | 2018-02-28 | Resource allocating method for wireless backhaul network and apparatus based on machine learning |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190268894A1 US20190268894A1 (en) | 2019-08-29 |
US10805931B2 true US10805931B2 (en) | 2020-10-13 |
Family
ID=67684933
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/286,844 Active 2039-04-13 US10805931B2 (en) | 2018-02-28 | 2019-02-27 | Resource allocation method and apparatus for wireless backhaul network based on reinforcement learning |
Country Status (2)
Country | Link |
---|---|
US (1) | US10805931B2 (en) |
KR (1) | KR102030128B1 (en) |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102174190B1 (en) * | 2019-09-09 | 2020-11-04 | 주식회사 맥데이타 | Performance visualization method, apparatus and system for 5g based network |
US11678272B2 (en) * | 2019-10-30 | 2023-06-13 | University Of Ottawa | System and method for joint power and resource allocation using reinforcement learning |
CN111464869B (en) * | 2020-04-21 | 2022-06-14 | 海信视像科技股份有限公司 | Motion position detection method, screen brightness adjustment method and intelligent device |
KR20210138947A (en) | 2020-05-13 | 2021-11-22 | 한양대학교 에리카산학협력단 | Method And System For Allocating Multi-Channel Resource |
KR20210145514A (en) | 2020-05-25 | 2021-12-02 | 삼성전자주식회사 | Method and apparatus for power controlling for network energy optimization |
CN111726811B (en) * | 2020-05-26 | 2023-11-14 | 国网浙江省电力有限公司嘉兴供电公司 | Slice resource allocation method and system for cognitive wireless network |
CN112165344B (en) * | 2020-09-16 | 2021-09-21 | 杭州电子科技大学 | Mixed precoding method based on gradient descent method |
CN114390580A (en) * | 2020-10-20 | 2022-04-22 | 维沃移动通信有限公司 | Beam reporting method, beam information determining method and related equipment |
US20240063840A1 (en) * | 2020-12-29 | 2024-02-22 | Lg Electronics Inc. | Intelligent signal transmission or reception method and device therefor |
CN113242602B (en) * | 2021-05-10 | 2022-04-22 | 内蒙古大学 | Millimeter wave large-scale MIMO-NOMA system resource allocation method and system |
US20220417948A1 (en) * | 2021-06-23 | 2022-12-29 | Microsoft Technology Licensing, Llc | Learned scheduler for flexible radio resource allocation to applications |
CN114079953B (en) * | 2021-10-28 | 2023-05-23 | 深圳大学 | Resource scheduling method, device, terminal and storage medium of wireless network system |
KR102626094B1 (en) * | 2021-11-22 | 2024-01-16 | 아주대학교산학협력단 | DQN-based resource management method for wireless communication |
KR102530179B1 (en) * | 2021-12-06 | 2023-05-10 | 홍익대학교 산학협력단 | Method for controlling hvac based on reinforcement learning |
WO2023184463A1 (en) * | 2022-04-01 | 2023-10-05 | Qualcomm Incorporated | Techniques for machine learning scheme changing based at least in part on dynamic network changes |
KR20230143329A (en) | 2022-04-05 | 2023-10-12 | 금오공과대학교 산학협력단 | Reinforcement Learning inspired Quantum Neural Network for Communication Resource Optimization |
CN115173922B (en) * | 2022-06-30 | 2024-03-15 | 深圳泓越信息科技有限公司 | Multi-beam satellite communication system resource allocation method based on CMADDQN network |
KR102561345B1 (en) * | 2023-02-02 | 2023-07-27 | 홍익대학교 산학협력단 | Apparatus and method for controlling hvac based on drqn |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008042897A (en) | 2006-07-13 | 2008-02-21 | Nec Commun Syst Ltd | Resource allocation apparatus, centralized control apparatus, wireless base station, wireless communication system, resource allocation method, and resource allocation program |
US20130143541A1 (en) | 2011-12-06 | 2013-06-06 | At&T Mobility Ii Llc | Centralized femtocell optimization |
KR20140059258A (en) | 2011-08-25 | 2014-05-15 | 후아웨이 테크놀러지 컴퍼니 리미티드 | Dynamic frequency spectrum optimization method, device and system |
KR101415670B1 (en) | 2012-07-03 | 2014-07-04 | 한국과학기술원 | System and method for assigning beam/polarized wave resource by use of polarized wave characteristics |
US20190014488A1 (en) * | 2017-07-06 | 2019-01-10 | Futurewei Technologies, Inc. | System and method for deep learning and wireless network optimization using deep learning |
US20190230046A1 (en) * | 2018-01-19 | 2019-07-25 | Ciena Corporation | Autonomic resource partitions for adaptive networks |
US20190364492A1 (en) * | 2016-12-30 | 2019-11-28 | Intel Corporation | Methods and devices for radio communications |
US20200084683A1 (en) * | 2017-11-16 | 2020-03-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Target Link Ranking in a Wireless Communication Network |
-
2018
- 2018-02-28 KR KR1020180024400A patent/KR102030128B1/en active IP Right Grant
-
2019
- 2019-02-27 US US16/286,844 patent/US10805931B2/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008042897A (en) | 2006-07-13 | 2008-02-21 | Nec Commun Syst Ltd | Resource allocation apparatus, centralized control apparatus, wireless base station, wireless communication system, resource allocation method, and resource allocation program |
KR20140059258A (en) | 2011-08-25 | 2014-05-15 | 후아웨이 테크놀러지 컴퍼니 리미티드 | Dynamic frequency spectrum optimization method, device and system |
US20130143541A1 (en) | 2011-12-06 | 2013-06-06 | At&T Mobility Ii Llc | Centralized femtocell optimization |
KR101415670B1 (en) | 2012-07-03 | 2014-07-04 | 한국과학기술원 | System and method for assigning beam/polarized wave resource by use of polarized wave characteristics |
US20190364492A1 (en) * | 2016-12-30 | 2019-11-28 | Intel Corporation | Methods and devices for radio communications |
US20190014488A1 (en) * | 2017-07-06 | 2019-01-10 | Futurewei Technologies, Inc. | System and method for deep learning and wireless network optimization using deep learning |
US20200084683A1 (en) * | 2017-11-16 | 2020-03-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Target Link Ranking in a Wireless Communication Network |
US20190230046A1 (en) * | 2018-01-19 | 2019-07-25 | Ciena Corporation | Autonomic resource partitions for adaptive networks |
Non-Patent Citations (1)
Title |
---|
Korean Office Action for related KR Application No. 10-2018-0024400 dated Mar. 17, 2019 from Korean Intellectual Property Office. |
Also Published As
Publication number | Publication date |
---|---|
KR102030128B1 (en) | 2019-11-08 |
US20190268894A1 (en) | 2019-08-29 |
KR20190103681A (en) | 2019-09-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10805931B2 (en) | Resource allocation method and apparatus for wireless backhaul network based on reinforcement learning | |
CN111666149A (en) | Ultra-dense edge computing network mobility management method based on deep reinforcement learning | |
US11399376B2 (en) | Method and system in a wireless communications for estimating quality based on learning model | |
CN111683375B (en) | Unmanned aerial vehicle deployment optimization method for unmanned aerial vehicle-assisted wireless cellular network | |
US8934931B2 (en) | Apparatus and method for allocating resources to nodes in a communication system using an update of iteration resource weights | |
US11140563B2 (en) | Dynamic quantized signature vector selection for a cloud radio access network | |
Kim et al. | Online learning-based downlink transmission coordination in ultra-dense millimeter wave heterogeneous networks | |
US11265731B1 (en) | Small cell telecommunications network design | |
US20090003284A1 (en) | Call Admission Control in an Ad Hoc Network | |
Gendia et al. | UAV positioning with joint NOMA power allocation and receiver node activation | |
CN103648105A (en) | Method and system for obtaining wireless local area network (WLAN) access point (AP) disposition scheme | |
EP3162113B1 (en) | Methods, nodes and system for enabling redistribution of cell load | |
JP2017158141A (en) | Program to be executed by computer, computing device equipped with the program, and computer-readable recording medium recording the program | |
US8654704B2 (en) | System and method for distributed control of spectrum power in a multi-cell communication system | |
KR101549309B1 (en) | Power-optimized vertical handover scheme for heterogeneous wireless networks | |
US10904766B2 (en) | Method of determining node location in cognitive radio network environment and node location determination device for implementing the method | |
CN111835388B (en) | Communication adjustment method, terminal, device, base station and medium in MIMO system | |
CN113597799B (en) | Apparatus, method and computer readable medium for adjusting a beamforming profile | |
KR101537891B1 (en) | Energy-efficient transmission power management scheme for multicast service over wireless access network | |
US20210160823A1 (en) | Resource management method and apparatus in user-centric wireless network | |
KR101680487B1 (en) | Multicell beamforming method and system for ofdma small cell networks | |
Morita et al. | Channel Capacity Prediction Using Point of Interest for Design and Operation Support of Network | |
KR20160124553A (en) | Method for providing communication service and apparatus for performing the method | |
KR101430306B1 (en) | Method and apparatus for grouping femtocell base stations | |
CN116017717A (en) | Hybrid intelligent spectrum management method in unmanned system cluster network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHO, DONG HO;JI, DONG JIN;LEE, HYUK JOON;SIGNING DATES FROM 20190225 TO 20190226;REEL/FRAME:048451/0715 Owner name: KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY, KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHO, DONG HO;JI, DONG JIN;LEE, HYUK JOON;SIGNING DATES FROM 20190225 TO 20190226;REEL/FRAME:048451/0715 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |