JP7388634B2 - Optimization method for wireless communication system, wireless communication system and program for wireless communication system - Google Patents

Description

The present invention relates to a method for optimizing a wireless communication system, a wireless communication system, and a program for a wireless communication system, and in particular, a method for optimizing a wireless communication system that uses multi-level evaluation learning to optimize communication conditions, and a wireless communication system. Related to communication systems and programs for wireless communication systems.

More specifically, the present invention aims to optimize communication in an environment where different wireless communication systems coexist and interfere with each other. Here, when different requirements are imposed on each wireless communication system, optimization is performed in consideration of each of the requirements. This optimization is performed for one or more wireless communication systems by learning using a computer, such as machine learning or reinforcement learning.

Wireless LAN is a wireless communication system that can be used at low cost in an unlicensed band. For this reason, the popularity of wireless LAN has rapidly progressed, and a situation has arisen in which many wireless LAN terminals coexist in the same area. As a result, wireless LAN terminals are interfering with each other, which has become a problem. In response to these issues, many techniques have been proposed to minimize the effects of interference between wireless LAN terminals and expand individual or overall system capacity.

For example, FIG. 1 shows an example in which wireless communication terminals 1 to N are wireless LAN base stations (AP: Access Points) that interfere with each other. Note that the wireless communication terminals N+1 to N+M shown in the lower part of FIG. 1 are user terminals such as smartphones that establish communication with the above AP. In this example, each of the wireless communication terminals 1 to N functioning as an AP acquires interference information in their surroundings and information on the success or failure of connection with the wireless communication terminals N+1 to N+M, and sends it to the control server 10 as wireless environment information. Send.

The control server 10 calculates allocation of frequency channels and transmission power values so that the throughput of the AP group including the wireless communication terminals 1 to N is maximized, and sends the results back to each AP as control information.

By the way, when it comes to applications and devices that use wireless LAN, the items that should be emphasized may differ depending on the usage scenario. For example, communication speed is not important in wireless communication systems that include IoT sensors. On the other hand, when installing a large number of IoT sensors within an area, it is important to increase the number of communications that can be established within the area.

Therefore, rather than aiming to expand system capacity, it is necessary to select a frequency channel with less interference even if it is a narrow band. Furthermore, when it is desired to communicate over a wide range, control is required to maximize transmission power within a range that does not affect other wireless communication systems, rather than focusing on interference from other wireless communication systems. In this way, the required control policy differs depending on the application of the wireless communication system used.

In particular, multiple wireless communication systems coexist in the 920MHz band, which is currently open to RFID and IoT in Japan. Specifically, this band is used, for example, in the following systems.
1. A wireless communication system that periodically transmits sensor information such as location information and temperature 2. Wireless communication system that transmits videos using surveillance cameras 3. Wireless communication systems that require network construction over wide areas such as mountainous areas and oceans.

These wireless communication systems each have different requirements, and at the same time, it is assumed that they coexist on the same frequency channel. Therefore, control information for these needs to be calculated on the assumption that they coexist on the same frequency resource.

In the configuration example of the wireless communication system shown in FIG. 1, a plurality of wireless communication terminals 1 to N are connected to a control server 10 in an environment where they interfere with each other. Furthermore, the wireless communication terminals 1 to N can perform data communication with other wireless communication terminals N+1 to N+M using wireless communication.

In this system configuration, each of the wireless communication terminals 1 to N transmits wireless environment information to the control server 10. For example, in the case of wireless LAN communication, the wireless environment information means SSID, frequency channel usage information such as channel usage rate, and parameters set in the wireless communication terminal. The parameters include the frequency channel in use, channel bandwidth, transmit power value, etc. The wireless environment information is acquired by the wireless communication terminals 1 to N by performing carrier sense on their respective surroundings.

In the conventional method, after collecting wireless environment information, the control server 10 performs optimization on the assumption that all wireless communication terminals 1 to N have the same specifications and require the same communication capacity. Perform calculations. Optimization calculations are performed on, for example, the position and width of frequency channels, and transmission power. The calculation results are transmitted from the control server 10 to the wireless communication terminals 1 to N as control information. The wireless communication terminals 1 to N that have received the control information change the corresponding setting values according to the control information.

When control is executed periodically or due to some trigger, the optimal parameters are calculated again based on the initial value calculation method and updated information such as frequency channel usage information. . Then, control information is determined by the calculation and transmitted to each of the wireless communication terminals 1 to N.

FIG. 2 is a flowchart of a conventional control example. In S100, wireless environment information of each of wireless communication terminals 1 to N functioning as an AP is collected.

In S102, optimal parameters are calculated based on the collected information. In conventional control, it is determined that all wireless communication terminals 1 to N require the same communication capacity. Therefore, the same evaluation function is used for all wireless communication terminals 1 to N, and wireless communication parameters considered to be optimal are calculated by heuristic methods such as repeated calculations and genetic algorithms.

The calculated parameter information is transmitted to the wireless communication terminals 1 to N as control information (S104).

Thereafter, a standby process is adopted (S110) until the occurrence of a control trigger is recognized (S106), and unless it is determined that the control has ended (S108). Then, if a control trigger occurs, the processes from S100 onwards are executed again. However, since it is difficult to fully consider the environment and control information of multiple wireless communication systems using heuristic methods alone, optimization using machine learning and reinforcement learning has also been proposed.

Liang, Le et al., “Multi-Agent Reinforcement Learning for Spectrum Sharing in Vehicular Networks”, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 1-5, 2019. Cheng Wu, Kaushik Chowdhury, Marco Di Felice, and Waleed Meleis, “Spectrum management of cognitive radio using multi-agent reinforcement learning”, In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Industry track (AAMAS '10), 1705-1712, 2010.

Methods that incorporate iterative calculations and machine learning have been proposed to optimize wireless communication resources for a variety of communication devices and services and applications that use wireless communication. These methods assume that all wireless communication systems have a unified purpose, such as increasing communication capacity.

However, in reality, in an environment where a plurality of different wireless communication systems coexist, the same requirements are not necessarily imposed on each system. That is, in reality, a situation may arise in which the purpose or goal of optimization differs from system to system. Conventional methods that aim at single optimization are insufficient for such situations. In this case, it is necessary to realize optimization that reflects the requirements for each wireless device or the requirements for each usage scene.

The present invention aims to improve the configuration parameters and usage conditions of a wireless communication system in a situation where multiple wireless communication systems with multiple different constraints or different requirements coexist in the same environment while interfering with each other. The purpose of the present invention is to provide a method for optimizing a wireless communication system using computer-based learning.

In order to achieve the above object, a first invention is a method for optimizing a wireless communication system that is executed in a wireless communication environment where a plurality of different wireless communication systems coexist, the method comprising: Detecting wireless environment information including a state related to wireless communication from a communication terminal; and providing the wireless environment information to each of the agents prepared in accordance with conditions imposed on each of the plurality of wireless communication systems. a step of causing a computer to perform reinforcement learning applying the conditions and the wireless environment information to each of the agents; The method preferably includes the steps of causing a computer to calculate control parameters based on the results of the reinforcement learning, and providing the control parameters to the wireless communication terminal belonging to the corresponding wireless communication system.

Further, a second invention is a wireless communication system that operates in a wireless communication environment in which a plurality of different wireless communication systems coexist, the wireless communication system receiving wireless environment information from the plurality of wireless communication systems, and receiving wireless environment information from the plurality of wireless communication systems. a control server that provides control information to the plurality of wireless communication systems; A process of providing the wireless environment information to each of the agents prepared in accordance with the conditions imposed on each of the agents, and reinforcement learning in which the conditions and the wireless environment information are applied to each of the agents. a process of calculating optimal control parameters under the wireless communication environment for each of the plurality of wireless communication systems based on the results of the reinforcement learning; It is desirable to perform a process provided to the wireless communication terminal belonging to the wireless communication terminal.

Further, a third invention is a program for a wireless communication system implemented in a control server that receives wireless environment information from a plurality of wireless communication systems and provides control information to the plurality of wireless communication systems, the program A process for detecting wireless environment information including a state related to wireless communication from wireless communication terminals belonging to the plurality of wireless communication systems, and a process prepared in response to conditions imposed on each of the plurality of wireless communication systems. a process of providing the wireless environment information to each of the agents; a process of implementing reinforcement learning in which the conditions and the wireless environment information are applied to each of the agents; and a process of performing reinforcement learning on each of the multiple wireless communication systems. A process of calculating optimal control parameters under a wireless communication environment in which a plurality of wireless communication systems are operating based on the results of the reinforcement learning; It is desirable that the communication terminal executes the processing provided to the communication terminal.

According to the present invention, when a plurality of different wireless communication systems coexist in a wireless communication environment, an agent for reinforcement learning is prepared for each condition imposed on the wireless communication system. Reinforcement learning can then proceed by applying conditions corresponding to each agent. Therefore, according to the present invention, in a situation where a plurality of wireless communication systems coexist and interfere with each other in the same environment, the setting parameters and usage conditions of the wireless communication systems can be optimized for each condition. .

1 is a diagram illustrating a configuration example of a wireless communication system. 3 is a flowchart of an example of control of a wireless communication system. FIG. 2 is a diagram for explaining an example of a conventional reinforcement learning model. FIG. 2 is a diagram for explaining an example of a reinforcement learning model implemented in Embodiment 1 of the present invention. 5 is a diagram for explaining an example of an environment to which the model shown in FIG. 4 is applied. FIG. 6 is a diagram for explaining aspects of aggregation that are allowed in the environment shown in FIG. 5. FIG. FIG. 3 is a diagram illustrating an example of channel allocation determined by a conventional method. FIG. 3 is a diagram showing an example of channel allocation determined by the method of Embodiment 1 of the present invention. FIG. 7 is a diagram for explaining an example of a reinforcement learning model implemented in Embodiment 2 of the present invention.

Embodiment 1.
[Configuration of Embodiment 1]
The wireless communication system according to the first embodiment of the present invention can be realized by the configuration example shown in FIG. In FIG. 1, wireless communication terminals 1 to N shown in the middle row each function as an access point (AP). These can communicate with wireless communication terminals N+1 to N+M shown in the lower part of FIG. Wireless communication terminals N+1 to N+M are composed of smartphones, IoT sensors, smart meters, etc. In this way, the configuration shown in FIG. 1 includes a plurality of wireless communication systems that share the same frequency resource but have different standards and specifications.

The wireless communication system of this embodiment includes a control server 10. The control server 10 includes hardware such as a communication interface, a processor unit, and a memory. The control server 10 realizes the functions described below by having these hardware proceed with processing according to programs stored in memory.

The control server 10 can provide control information to the wireless communication terminals 1 to N functioning as APs. The control information includes, for example, information such as available frequency resources and transmission power. On the other hand, the wireless communication terminals 1 to N can transmit wireless environment information to the control server 10. The wireless environment information includes interference information in the vicinity of each of the wireless communication terminals 1 to N, and information regarding success or failure of connection with the wireless communication terminals N+1 to N+M.

The control server 10 also has a learning function for optimizing various parameters to be included in the control information based on wireless environment information, etc., and a function for determining these various parameters based on the results of the learning. ing.

[Overview of reinforcement learning]

In this embodiment, reinforcement learning is used to optimize various parameters included in the control information. FIG. 3 shows a general reinforcement learning model diagram. The model shown in FIG. 3 includes an agent 12 as a learning target. The agent 12 calculates and executes the action A(t+1) from the current state S(t) and the reward R(t) in the unique environment 14, with the observation timing of the event being t. As a result, state S(t+1) is realized. From this state S(t+1), a reward R(t+1) for evaluating the behavior is obtained, and the next behavior is calculated.

In the following description, it is assumed that s and S represent states, a and A represent actions, and r and R represent rewards, respectively. Here, lowercase letters mean parameters for individual agents (optimization targets), and uppercase letters mean parameters for the set (multiple agents). Also, the subscript t of each parameter indicates that the parameter is the value at observation timing t, and St, At, and Rt are the same as S(t), A(t), and R(t), respectively. shall be taken as a thing.

The reinforcement learning shown in FIG. 3 proceeds by repeating the following steps.
1. The agent 12 receives the state S(t) and the reward R(t) from the environment 14, and returns to the environment 14 an action A(t) determined based on the policy π.
2. The environment 14 changes to the next state S(t+1) based on the action A(t) received from the agent 12 and the current state S(t), and changes to the next state S(t+1) and reward R(t+1) after the transition. ) to the agent 12. Note that the reward R is a scalar amount that indicates the quality of the immediately preceding action A.

Assuming that the agent's action for a certain state S is A, the sum of the rewards R that can be obtained from the present moment to the infinite future, that is, the profit G, is as shown in the following equation.

However, γ is 0≦γ≦1, and is a parameter that adjusts to what extent the influence of future remuneration is evaluated as profit.

In Q learning using reinforcement learning, the value of action a is evaluated using the following function.

However, E is a function indicating an expected value. Further, Q ^π is a value function (hereinafter referred to as “Q function”) representing the expected value when an agent who takes action a from state s takes action according to policy π.

The reinforcement learning shown in FIG. 3 is performed so as to maximize this Q function. This learning can proceed, for example, by finding a Q function for estimating the profit G when performing action a in state s using the following algorithm.

Here, p is a parameter called the learning rate, which is an algebra determined by the machine learning designer. Usually it is set to a small value less than 1. Moreover, maxQ indicates the maximum value of the Q function that is considered to be ideally obtained. Learning of the Q function involves estimating all the Q values obtained by actions taken at the next time t+1 at each time t, and updating the Q value using the largest value among them.

[Features of Embodiment 1]
FIG. 4 shows a reinforcement learning model implemented in the wireless communication system of this embodiment. In this embodiment, optimization is aimed at a plurality of wireless communication systems with different conditions. Multiple wireless communication systems can be grouped based on their respective conditions. In the model shown in FIG. 4, there are three groups, and an agent exists for each group.

Agents 12-1, 12-3, and 12-3 shown in FIG. 4 evaluate the behavior of user i belonging to each group. For example, agent 12-1 can calculate reward R and action A from state S of user i included in group 1. Furthermore, the agents 12-2 and 12-3 determine the action A by calculating the reward R from the state S of the user i belonging to each agent.

The three agents 12-1, 12-2, and 12-3 shown in FIG. 4 may output different actions A based on the reward R and state S provided to each agent. In the model shown in FIG. 4, different rewards R and different states S may be provided to each of the agents 12-1, 12-2, and 12-3 from the same environment 14.

Hereinafter, with reference to FIGS. 5 and 6, the description will be continued using as an example a case where there are four different wireless communication terminals that meet the same wireless communication standard. FIG. 5 is a diagram arranging and expressing the requirements for the four terminals to be controlled in this example. Further, FIG. 6 shows an example of aggregation allowed in this example.

In this example, the method of allocating frequency resources (frequency channel position and frequency channel width) to all terminals is controlled. There are four unit channels, channels 1 to 4, as frequency resources. These channels can be used by aggregating two or four.

Further, the requirements for the four wireless communication terminals are as follows.
1. Since the wireless communication terminal 1 is used as a parent device in a sensor network, a requirement is to "maximize the success rate of uplink transmission from a large number of terminals (sensors)."
2. Since the wireless communication terminal 2 is used in a wide area sensor network, "maximization of transmission range" is a required condition.
3. Since the wireless communication terminals 3 and 4 are used as master devices for data distribution, a requirement is to "maximize the downlink throughput to subordinate terminals."

Note that, in addition to the above-mentioned requirements, several other requirements are naturally imposed on each wireless communication terminal. The above requirements are the conditions that should be given the highest priority among the several conditions required of each wireless communication terminal, depending on the characteristics of each.

When wireless communication terminal 1 is assumed to be agent 1, wireless communication terminal 2 is assumed to be agent 2, and wireless communication terminals 3 and 4 are assumed to be agent 3, the calculation of each reward can be set as follows.

As mentioned above, the requirement for agent 1 is to "maximize the success rate of uplink transmission from a large number of terminals." Therefore, the agent 1 determines the action A so that the transmission success rate of the transmitting terminal that performs uplink communication under the control of the wireless communication terminal 1 is maximized. In this example, it is assumed that all terminals can perform carrier sense over wireless LAN. In this case, the transmission success rate can be expressed by the following equation.

However, N shown in the above formula is the total number of transmitting terminals existing in the same channel. Here, for uplink traffic, all terminals under the master device within the same channel become transmitting terminals. Furthermore, in the case of downlink traffic, all base stations within the same channel become transmitting terminals. The above N is the sum of the number of uplink traffic transmission terminals and the number of downlink traffic transmission terminals. Further, τ in the above formula is the probability that each transmitting terminal transmits. This probability can be calculated based on the total number N mentioned above. Furthermore, n' in the above formula is the number of transmitting terminals connected to the master device being controlled.

The requirement of agent 2 is "maximization of transmission reach." For this reason, the agent 2 considers propagation characteristics, power density, frame error rate, etc. When it is possible to select from a plurality of frequency channels, the reward in consideration of propagation characteristics, that is, the transmission reach D, can be expressed by the following formula.

However, L in the above formula is a function for determining attenuation due to propagation, and L ^-1 is its inverse function. Further, fc is the center frequency of the transmission signal, and B is the bandwidth.

As for agent 3, the required condition is "maximization of downlink throughput to subordinate terminals". For this reason, agent 3 evaluates the throughput in the case where, for example, as in the case of agent 1, all terminals can perform carrier sense over the wireless LAN. In this case, the throughput can be calculated using the following equation.

However, E[L] in the above formula is the average number of bits at the time of successful transmission, and E[I] is the average waiting time. Furthermore, Ts in the formula is the average transmission frame time, and Tc is the average time wasted due to collisions.

Optimization control using the conventional method is performed with the aim of maximizing throughput equally for all terminals. In this case, channel allocation is determined so that interference between terminals does not occur. More specifically, as shown in FIG. 7, one channel is allocated to each of wireless communication terminals 1 to 4. That is, for wireless communication terminals 3 and 4 that mainly perform downlink transmission, frequency resources are allocated with a narrow bandwidth even though there is almost no collision between transmitting terminals. As a result, the throughput of the wireless communication terminals 3 and 4 ends up being lower than the throughput that could originally be achieved.

In contrast, in this embodiment, channel allocation is determined as shown in FIG. 8, for example. Here, channel 2 is assigned to radio communication terminal 1, and channel 1 is assigned to radio communication terminal 2. Since the wireless communication terminals 1 and 2 are components of a sensor network, they are susceptible to interference from the outside. For this reason, these terminals 1 and 2 are allocated unit channels with narrow bandwidths. Furthermore, the wireless communication terminal 2 is required to communicate over a wide area. The lower the communication center frequency, the smaller the signal propagation loss becomes. Channel 1 assigned to the wireless communication terminal 2 is a channel that has the lowest frequency and is thought to have the lowest signal propagation loss.

On the other hand, channels 3+4, which are two unit channels aggregated, are assigned to wireless communication terminals 3 and 4 for which it is important to maximize throughput. According to this assignment, wireless communication terminals 3 and 4 will interfere with each other. However, since the main traffic in all of them is downlink traffic, there are mainly two transmitting terminals, the base unit. In this case, the probability that a collision will occur due to coexistence within the same frequency channel is low. Therefore, even if two terminals coexist on the same channel, unless the scenario is such that they are constantly competing with each other for the channel, increasing the bandwidth through aggregation can be expected to have the effect of increasing instantaneous throughput.

As described above, according to the wireless communication system of this embodiment, reinforcement learning for optimization can be performed by setting different agents for a plurality of wireless communication terminals with different requirements and the like. Then, the behavior of each terminal can be individually and independently optimized according to the requirements for each terminal. Therefore, according to the wireless communication system of the present embodiment, in an area where different types of terminals with different requirements etc. coexist, it is possible to make each terminal exhibit maximum performance in relation to the requirements required of each terminal. I can do it.

Note that since this example is a simple example, modeling using, for example, a Markov process is also possible, but the actual environment is complex and difficult to model due to hidden terminals and the like. For this reason, reinforcement learning is required when assuming a real environment. Furthermore, if the environment is complex, a method that uses simulation or measurement results in real space, or a method that measures states and rewards using a database may be used instead of a mathematical model.

Further, although this example shows an example of control for allocating frequency resources, the present invention is not limited to this. In addition to the above examples, parameters related to wireless communication systems such as transmission power, transmission frequency (or average waiting time required for random access), wireless LAN RTS/CTS settings, number of terminals that can be connected, power consumption values, etc. , resources and setting values used for wireless communication can be controlled.

Embodiment 2.
Next, a second embodiment of the present invention will be described with reference to FIG. 9 together with FIG. The wireless communication system of this embodiment can be realized by the configuration shown in FIG. 1, as in the case of Embodiment 1.

FIG. 9 shows a reinforcement learning model implemented in the wireless communication system of this embodiment. The model shown in FIG. 9 includes a plurality of agents 12-1, 12-2, and 12-3, as in the first embodiment. This model assumes that multiple different environments will be subject to evaluation. More specifically, in the model shown in FIG. 9, there are a plurality of environments in which the actions selected by each of the agents 12-1, 12-2, and 12-3 are returned. In this case, it becomes necessary to select an environment in order to evaluate the selected behavior.

When each of the agents 12-1, 12-2, and 12-3 is defined by different requirements, the environment can be defined by different standards or communication systems. For example, wireless communication systems for IoT include IEEE 802.11ah, Wi-SUN, and LoRa. These systems have different prescribed frequency bandwidths and modulation/demodulation methods. Therefore, even if the received power value and the interference power value are the same, the probability that a transmitted frame will have an error will be a different value for each system.

Therefore, in an area where different wireless communication systems coexist and interference occurs between them, it is necessary to estimate the influence of interference on one system to be greater than the influence on the other system. A similar situation occurs, for example, regarding evaluation of power consumption. That is, when comparing actual devices, the hardware configurations of wireless communication terminals are not necessarily uniform, and devices with large battery capacities and devices with small battery capacities may coexist in the same area. The influence of power consumption needs to be estimated to be larger for a terminal with a small battery capacity than for a terminal with a large battery capacity.

Furthermore, if a wireless communication terminal with multi-RF functionality is to be controlled, it becomes necessary to evaluate the environment for each frequency band. For example, for wireless communication terminals that operate in tri-bands of 920 MHz, 2.4 GHz, and 5 GHz, it is necessary to switch the environmental evaluation method depending on which of these frequency bands the terminal operates in.

In the model shown in FIG. 9, three environments 14-1, 14-2, and 14-3 are prepared. These environments 14-1, 14-2, and 14-3 are arranged to cover environments that may exist for a plurality of wireless communication terminals to be controlled. Therefore, in this embodiment, all wireless communication terminals under the control of the control server 10 are operating under one of the environments 14-1, 14-2, and 14-3.

The model shown in FIG. 9 includes an environment selection section 16. The environment selection unit 16 selects the environment in which the agent 12-1 is placed from among the three environments 14-1, 14-2, and 14-3, and provides action _A1 to the selected environment. As a result, even in a situation where a plurality of different environments coexist, the action _A1 of the agent 12-1 is returned to the correct environment. The environment selection unit 16 performs similar environment selection for the agents 12-2 and 12-3. As a result, the actions A ₂ and A ₃ selected by the agents 12-2 and 12-3 are also returned to their respective appropriate environments. The function of the environment selection unit 16 is realized by the control server 10 determining the environment in which the wireless communication terminals 1 to N are placed based on wireless environment information.

The model shown in FIG. 9 further includes an agent selection section 18. The agent selection unit 18 provides states S ₁ , S ₂ , S ₃ and rewards R ₁ , R ₂ , R 3 provided from the environments 14-1, 14-2, and _14-3 to appropriate agents. do. The function of the agent selection unit 18 is realized by the control server 10 providing control information to an appropriate one of the wireless communication terminals 1 to N.

As explained above, according to the model shown in FIG. 9, when performing optimization control targeting multiple wireless communication systems that coexist within the same frequency resource, multiple environments can be appropriately switched to determine the selected behavior. A can be evaluated. Similarly, even when the control target includes a wireless communication terminal that operates by switching between multiple frequency bands as appropriate, the model shown in FIG. be able to appropriately evaluate gender.

10 Control server 12, 12-1, 12-2, 12-3 Agent 14, 14-1, 14-2, 14-3 Environment 16 Environment selection section 18 Agent selection section

Claims (7)

A wireless communication system optimization method executed in a wireless communication environment where a plurality of different wireless communication systems coexist while sharing the same frequency resource, the method comprising:
detecting wireless environment information including status regarding wireless communication from one or more wireless communication terminals belonging to each of the plurality of wireless communication systems;
Each of the agents prepared in accordance with the conditions imposed on each of the plurality of wireless communication systems receives the wireless communication detected from the wireless communication terminal belonging to the wireless communication system to which the condition corresponding to the agent is imposed. providing environmental information;
causing a computer to perform reinforcement learning applying the conditions and the wireless environment information to each of the agents;
for each of the plurality of wireless communication systems, causing a computer to calculate control parameters for satisfying the conditions imposed on the wireless communication system under the wireless communication environment, based on the results of the reinforcement learning;
providing the control parameters to the wireless communication terminal belonging to a corresponding wireless communication system;
including;
A wireless communication system optimization method , wherein the conditions include a plurality of mutually different conditions . The plurality of wireless communication systems include wireless communication systems to which different conditions are imposed,
2. The optimization method according to claim 1, wherein the reinforcement learning aims at optimizing the agent's reward, state, and behavior by setting them for each same condition. The conditions imposed on each of the plurality of wireless communication systems include a plurality of requirements,
3. The optimization method according to claim 2, wherein the reinforcement learning aims at optimization by applying the highest priority requirement for each of the wireless communication systems to the corresponding agent. The wireless communication environment includes a mixture of multiple wireless communication systems that comply with different wireless communication standards,
3. The optimization method according to claim 2, wherein the reinforcement learning aims at optimization by setting different conditions for each of the wireless communication standards. The wireless communication environment includes a mixture of different environments,
2. The optimization method according to claim 1, wherein the reinforcement learning aims at optimization by setting an environment evaluation corresponding to each of the agents for each of the plurality of environments. A wireless communication system that operates in a mixed wireless communication environment where a plurality of different wireless communication systems share the same frequency resource ,
comprising a control server that receives wireless environment information from each of the plurality of wireless communication systems and provides control information to each of the plurality of wireless communication systems,
The control server is
a process of detecting wireless environment information including a state related to wireless communication from one or more wireless communication terminals belonging to each of the plurality of wireless communication systems;
Each of the agents prepared in accordance with the conditions imposed on each of the plurality of wireless communication systems receives the wireless communication detected from the wireless communication terminal belonging to the wireless communication system to which the condition corresponding to the agent is imposed. processing for providing environmental information;
A process of performing reinforcement learning applying the conditions and the wireless environment information to each of the agents;
A process of calculating, for each of the plurality of wireless communication systems, a control parameter for satisfying the condition imposed on the wireless communication system under the wireless communication environment, based on the result of the reinforcement learning;
a process of providing the control parameter to the wireless communication terminal belonging to a corresponding wireless communication system;
Run
A wireless communication system in which the conditions include a plurality of mutually different conditions . In a wireless communication environment where a plurality of different wireless communication systems coexist while sharing the same frequency resource, receiving wireless environment information from each of the plurality of wireless communication systems and providing control information to each of the plurality of wireless communication systems. A program for a wireless communication system implemented in a control server,
To the control server,
a process of detecting wireless environment information including a state related to wireless communication from one or more wireless communication terminals belonging to each of the plurality of wireless communication systems;
Each of the agents prepared in accordance with the conditions imposed on each of the plurality of wireless communication systems receives the wireless communication detected from the wireless communication terminal belonging to the wireless communication system to which the condition corresponding to the agent is imposed. processing for providing environmental information;
A process of performing reinforcement learning applying the conditions and the wireless environment information to each of the agents;
a process of calculating, for each of the plurality of wireless communication systems, control parameters for satisfying the conditions imposed on the wireless communication system under the wireless communication environment, based on the results of the reinforcement learning; ,
providing the control parameters to the wireless communication terminal belonging to the corresponding wireless communication system ;
A program for a wireless communication system , wherein the conditions include a plurality of mutually different conditions .

Images