TW202224472A - Beam selection method based on neural network and management server - Google Patents

Abstract

The present invention provides a beam selection method based on a neural network and a management server. The method includes: obtaining a neural network that have experienced first and second training processes; determining a transmission beam of each base station and accordingly measuring a signal intensity distribution map of each base station in a specific field; obtaining a position distribution map of multiple user devices in the specific field; generating a first matrix based on the position distribution map and the signal intensity distribution map of each base station; inputting the first matrix to the neural network, wherein the neural network outputs a second matrix based on the first matrix; determining an optimum transmitting beam corresponding to each base station based on the second matrix; and controlling each base station to use the corresponding optimum transmitting beam to transmit signals.

Description

Neural-like network-based beam selection method and management server

The present invention relates to a beam selection technology for a base station, and more particularly, to a neural network-based beam selection method and a management server.

Due to the increasing demand for mobile network data by users, the mobile communication network has developed towards a Heterogeneous Network (HetNet). Under this circumstance, the multi-level and high-density base station deployment method is the development trend of various operators, and its purpose is to improve the spectral efficiency per unit area.

However, in the dense deployment environment of base stations, the problem of interference between base stations is one of the important issues that operators need to face. When the interference between base stations is reduced, the transmission quality and spectrum efficiency of the system will be improved, thereby improving the overall user's transmission experience.

In the case of high-density base station deployment, small cell is one of the options for operators. It features low power, plug-and-play, and easy deployment. However, operators control it. less difficult. In addition, because the miniaturized base station has relatively simple functions, although it has a plug-and-play function, if the transmission beam is not properly selected for the miniaturized base station, it will easily cause interference to other surrounding base stations.

In view of this, the present invention provides a neural network-based beam selection method and a management server, which can be used to solve the above-mentioned technical problems.

The present invention provides a beam selection method based on a neural network, which is suitable for managing a management server of multiple base stations deployed in a specific field, and each base station has multiple transmit beams . The method includes: obtaining a first-type neural network that has undergone a first training process and a second training process; determining a first transmit beam of each base station, and measuring each base station in a specific field accordingly a specific signal strength distribution map of a plurality of user devices in a specific field; obtain a specific location distribution map of a plurality of user devices in a specific field; generate a first specific matrix according to the specific location distribution map and the specific signal intensity distribution map of each base station; The specific matrix is input to the first type of neural network, wherein the first type of neural network outputs a second specific matrix according to the first specific matrix, wherein the second specific matrix includes a probability that each base station selects each transmission beam; based on the second specific matrix The specific matrix determines an optimal transmission beam corresponding to each base station; and controls each base station to use the corresponding optimal transmission beam to transmit signals.

The present invention provides a management server, which manages multiple base stations deployed in a specific field, and each base station has multiple transmit beams. The management server includes a storage circuit and a processor. The storage circuit stores a plurality of modules. The processor is coupled to the storage circuit, and accesses the plurality of modules to perform the following steps: obtaining a first-type neural network undergoing a first training process and a second training process; determining a first type of neural network of each base station Transmit beams, and measure a specific signal strength distribution map of each base station in a specific field; obtain a specific location distribution map of multiple user devices in a specific field; according to the specific location distribution map and each base station The specific signal intensity distribution map generates a first specific matrix; the first specific matrix is input to the first type of neural network, wherein the first type of neural network outputs a second specific matrix according to the first specific matrix, wherein the second specific matrix It includes a probability that each base station selects each transmit beam; determines an optimal transmit beam corresponding to each base station based on the second specific matrix; and controls each base station to use the corresponding optimal transmit beam to transmit signals.

Please refer to FIG. 1 , which is a schematic diagram of a communication system according to an embodiment of the present invention. In FIG. 1 , the communication system 10 may include a management server 100 and base stations 111-11H, wherein the base stations 111-11H are, for example, deployed in a specific field (for example, an area with a length of N and a width of M, but not limited to this), and can be used to serve multiple user devices in a specific field.

In the embodiment of the present invention, each of the base stations 111 to 11H may have a plurality of selectable transmit beams (corresponding to different transmit directions), and each of the base stations 111 to 11H may select one of the transmit beams to transmit. Signal. For convenience of description, it is assumed below that the base stations 111 to 11H individually have three transmit beams (hereinafter referred to as transmit beam 1 to transmit beam 3 ) for selection. That is, each base station 111-11H can select one of the three transmit beams to transmit signals.

In addition, as shown in FIG. 1 , the management server 100 can be connected to the base stations 111 - 11H, and can be used to manage the base stations 111 - 11H. In some embodiments, the management server 110 may receive the information reported by each of the base stations 111 - 11H, and determine a suitable transmit beam for each of the base stations 111 - 11H accordingly. For the convenience of description, it is assumed that the number of base stations 111 to 11H is 4 (ie, H is 4), but the present invention may not be limited thereto.

In FIG. 1 , the management server 100 includes a storage circuit 102 and a processor 104 . The storage circuit 102 is, for example, any type of fixed or removable random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), flash memory (Flash memory), hard drive A disc or other similar device or a combination of these devices may be used to record multiple code or modules. The processor 104 is coupled to the storage circuit 102 and may be a graphic processing unit (graphical processing unit, GPU) or the like.

In the embodiment of the present invention, the processor 104 can access the modules and program codes recorded in the storage circuit 102 to implement the neural network-like beam selection method proposed by the present invention, the details of which are described below.

Please refer to FIG. 2 , which is a flowchart of a method for beam selection based on a neural network-like network according to an embodiment of the present invention. The method of this embodiment can be executed by the management server 100 in FIG. 1 , and the details of each step in FIG. 2 will be described below in conjunction with the components shown in FIG. 1 .

First, in step S210, the processor 104 can obtain the first type of neural network that has undergone the first training process and the second training process. In the embodiment of the present invention, the first type of neural network is, for example, a convolutional neural network, and the related first training process and second training process will be described later with reference to FIG. 4 and FIG. 5 .

Next, in step S220, the processor 104 may determine the first transmit beams of the base stations 111-114, and measure the specific signal intensity distribution diagram of the base stations 111-114 in the specific field accordingly. In one embodiment, the processor 104 may randomly select one of the three transmit beams as the first transmit beam for each base station 111 - 114 . For example, the processor 104 may select the transmit beams 1, 2, 3, and 1 for the base stations 111-114, respectively, as the first transmit beams of the base stations 111-114, but it is not limited thereto.

In order to facilitate the understanding of the concept of the present invention, the following further description is supplemented with FIG. 3 . Please refer to FIG. 3 , which is an application scenario diagram according to an embodiment of the present invention. As shown in FIG. 3 , after determining the first transmit beams of the base stations 111 - 114 , the processor 104 can extract the required data from the database of the relevant mobile network to generate the data in the specific field of the base stations 111 - 114 Domain-specific signal strength distribution maps ST1~ST4.

In FIG. 3 , the specific signal strength distribution diagrams ST1 to ST4 can be represented as an N×M matrix, and each matrix element can represent the signal strength of the corresponding base station at the corresponding position in the specific field, but not limited to this.

Afterwards, in step S230, the processor 104 may obtain the specific location distribution map PD1 of the user device in the specific field. In one embodiment, the processor 104 can obtain, for example, measurement reports (MRs) reported by the served user equipments from the base stations 111 - 114 to estimate the positions of the user equipments in the specific field, and then construct the The specific location distribution map PD1, but not limited to this.

In step S240, a first specific matrix SM1 is generated according to the specific position distribution map PD1 and the specific signal intensity distribution maps ST1-ST4 of each base station 111-114.

In FIG. 3 , the specific location distribution map PD1 corresponding to the specific field (the dimension of which is, for example, NxM) may include, for example, a plurality of black and white dots, wherein the black dots represent the positions where there is no user device in the specific field, The white dots represent the position of a certain user device in a specific field, but it is not limited to this.

In one embodiment, the processor 104 may represent the specific location distribution map PD1 as three maps corresponding to RGB (the respective dimensions of which are also NxM), wherein the map corresponding to R is, for example, the specific location distribution map PD1. The R component map, the map corresponding to G is, for example, the G component map of the specific position distribution map PD1, and the map corresponding to B is, for example, the B component map of the specific position distribution map PD1.

Similarly, the processor 104 can also decompose the specific signal intensity distribution maps ST1 to ST4 into three maps corresponding to RGB respectively. In other words, the processor 104 can convert the specific position distribution map PD1 and the specific signal intensity distribution maps ST1 to ST4 into 5 sets of RGB images (ie, 15 images in total).

In one embodiment, the processor 104 may superimpose the above-mentioned five sets of RGB images into a 15×N×M three-dimensional matrix to serve as the first specific matrix SM1 , but it is not limited thereto.

Afterwards, in step S250, the processor 104 may input the first specific matrix SM1 to the first type of neural network NN1. In one embodiment, the first type of neural network NN1 can output a second specific matrix SM2 according to the first specific matrix SM1, wherein the second specific matrix SM2 can include the probability that each base station 111-114 selects each transmission beam.

）的3個元素分別可代表基地台111~114中的第n個基地台選用發射波束1、2、3的機率（其總和為1）。舉例而言，當n為1時，第二特定矩陣SM2的第1列的3個元素分別可代表基地台111選用發射波束1、2、3的機率（其總和為1），而當n為2時，第二特定矩陣SM2的第2列的3個元素分別可代表基地台112選用發射波束1、2、3的機率（其總和為1），但可不限於此。 In one embodiment, since the number of base stations 111 to 114 is 4, and each of them has 3 transmit beams, the second specific matrix SM2 is, for example, a 4×3 two-dimensional matrix, wherein the nth of the second specific matrix SM2 is List(

The three elements of ) respectively represent the probability that the nth base station among the base stations 111 to 114 selects the transmit beams 1, 2, and 3 (the sum of which is 1). For example, when n is 1, the three elements in the first column of the second specific matrix SM2 can represent the probability that the base station 111 selects transmit beams 1, 2, and 3 respectively (the sum of which is 1), and when n is When 2 is 2, the three elements in the second column of the second specific matrix SM2 may respectively represent the probability that the base station 112 selects the transmit beams 1, 2, and 3 (the sum of which is 1), but is not limited thereto.

Afterwards, in step S260, the processor 104 may determine the optimal transmit beam corresponding to each base station 111-114 based on the second specific matrix SM2. In one embodiment, the processor 104 may take the argmax operator on the second specific matrix SM2 to obtain the transmit beam parameter vector, and determine the optimal transmit beam corresponding to each base station 111-114 based on the transmit beam parameter vector.

的發射波束引數向量。 For example, it is assumed that the content of the second specific matrix SM2 can be represented as Table 1 below. transmit beam 1 transmit beam 2 transmit beam 3 Base station 111 0.3 0.3 0.4 Base station 112 0.4 0.5 0.1 Base station 113 0.7 0.2 0.1 Base station 114 0.8 0.1 0.1 Table I. In this case, after the processor 104 takes the argmax operator on the second specific matrix SM2, for example, it can be obtained

The transmit beam argument vector.

From the above transmit beam argument vector, it can be seen that base station 111 is most suitable for selecting transmit beam 3, base station 112 is most suitable for selecting transmit beam 2, base station 113 is most suitable for selecting transmit beam 1, and base station 114 is most suitable for selecting transmit beam 1, but not limited to this. In other words, the optimal transmit beams of the base stations 111 to 114 are, for example, transmit beams 3, 2, 1, and 1, respectively, but not limited thereto.

Afterwards, in step S270, the processor 104 may control each base station 111-114 to transmit signals using the corresponding optimal transmit beam. For example, the processor 104 can control the base stations 111-114 to use the transmit beams 3, 2, 1, and 1 to transmit signals, respectively.

In an embodiment of the present invention, by appropriately designing the first training process and the second training process of the first type of neural network NN1, the first type of neural network NN1 can be made to respond to the output of the first specific matrix SM1 The second specific matrix SM2 can represent how each base station 111 - 114 should select transmit beams so as to reduce mutual interference, thereby improving the transmission quality and spectrum efficiency in a specific field, and further improving the transmission experience of the user equipment in the specific field . The following will further describe the first training process and the second training process.

Please refer to FIG. 4 , which is a schematic diagram of a first training process according to an embodiment of the present invention. The method of this embodiment can be executed by the management server 100 in FIG. 1 , and the details of each step in FIG. 4 will be described below in conjunction with the components shown in FIG. 1 .

In the embodiment of the present invention, the first training process may include several recursive procedures, and the content of FIG. 4 corresponds to the i-th recursive procedure in the first training process, but it is not limited thereto.

First, in step S411, the processor 104 may determine a plurality of first neuron weights of the first type of neural network NN1. In one embodiment, when i is 1, the processor 104 may randomly generate the above-mentioned first neuron weight, but it is not limited thereto.

In step S412, the processor 104 can determine the second transmit beams of the base stations 111-114, and simulate the first reference signal intensity distribution map of the base stations 111-114 in a specific field accordingly. In one embodiment, when i is 1, the processor 104 may randomly select one of the transmit beams 1, 2, and 3 for each base station 111-114 as the corresponding second transmit beam. For details, please refer to the previous embodiment. The description in , will not be repeated here. Afterwards, the processor 104 can simulate the first reference signal intensity distribution diagram of each base station 111-114 in a specific field by running a specific simulation software. In this embodiment, the form of the first reference signal intensity distribution map corresponding to each base station 111 - 114 may be similar to the specific signal intensity distribution map ST1 - ST4 in FIG. 3 , but not limited thereto.

In step S413, the processor 104 may obtain a reference position distribution map of a plurality of reference user equipments in a specific field. In some embodiments, the processor 104 may determine the respective positions of the reference user equipments in the specific field according to the MRs provided by the reference user equipments, and construct the reference position distribution map accordingly, but it is not limited thereto. In one embodiment, the form of the reference location distribution map may be similar to the specific location distribution map PD1 in FIG. 3 , but it is not limited thereto.

In step S414, the processor 104 may generate a first reference matrix according to the reference position distribution map and the first reference signal intensity distribution map of each base station 111-114. In this embodiment, the processor 104 can convert the reference position distribution map and the first reference signal intensity distribution map of each base station 111 - 114 into corresponding RGB maps individually in a manner similar to that mentioned in the previous embodiment, and It is superimposed into a 15xNxM three-dimensional matrix as the first reference matrix, but it is not limited to this.

In step S415, the processor 104 may input the first reference matrix to the first type of neural network, wherein the first type of neural network may output a second reference matrix according to the first reference matrix, wherein the second reference matrix includes each base Stations 111 to 114 select the probability of each transmit beam. In this embodiment, the form of the second reference matrix may be similar to the form of the second specific matrix SM2, so the details thereof will not be repeated here.

In step S416, the processor 104 may take the argmax operator on the second reference matrix to obtain the first reference vector, and determine the third transmit beams corresponding to the base stations 111-114 based on the first reference vector. In this embodiment, the form of the first reference vector may be similar to the above-mentioned transmit beam argument vector, and the manner in which the processor 104 determines the third transmit beam corresponding to each base station 111-114 is similar to that of the processor 104 based on the above-mentioned transmit beam The argument vector is a way for each base station 111 to 114 to determine the optimal transmission beam, so the details thereof will not be repeated here.

In step S417, the processor 104 can simulate the second reference signal intensity distribution diagram of each base station 111-114 in a specific field according to the third transmit beam corresponding to each base station 111-114. Similarly, the processor 104 can simulate the second reference signal intensity distribution map of each base station 111-114 in a specific field by running a specific simulation software. In this embodiment, the form of the second reference signal intensity distribution map corresponding to each base station 111 - 114 may be similar to the specific signal intensity distribution map ST1 - ST4 in FIG. 3 , but not limited thereto.

In step S418, the processor 104 may generate a third reference matrix according to the reference position distribution map and the second reference signal intensity distribution map of each base station 111-114, and obtain a first average communication of the plurality of reference user equipments quality. In this embodiment, the third reference matrix is, for example, a 15×N×M three-dimensional matrix, and the generation method thereof is similar to that of the first reference matrix, so it will not be repeated here. In addition, the above-mentioned first average communication quality is, for example, the average transmission speed of the plurality of reference user devices or other data that can be used to represent the communication quality, but it is not limited thereto.

In step S419, the processor 104 may determine a plurality of second neuron weights of the second type of neural network. In one embodiment, the parameters of the second type of neural network may be substantially the same as the first type of neural network, but the weight of each neuron is different from that of the first type of neural network, but not limited thereto. In some embodiments, when i is 1, the processor 104 may randomly generate the above-mentioned second neuron weight, but it is not limited thereto.

In step S420, the processor 104 may input the third reference matrix to the second type of neural network, wherein the second type of neural network may output a fourth reference matrix according to the third reference matrix, and the fourth reference matrix includes each base Stations 111 to 114 select the probability of each transmit beam. In this embodiment, the form of the fourth reference matrix may be similar to the form of the second specific matrix SM2, so the details thereof will not be repeated here.

In step S421, the processor 104 may determine the first selection probability vector of the plurality of base stations for the plurality of transmit beams based on the fourth reference matrix, and based on the first selection probability vector, the first average communication quality and the first selection probability vector A reference vector determines a first mean squared error value. In one embodiment, the processor 104 may perform a max operator on the fourth reference matrix to obtain the first selection probability vector.

的第一選用機率向量（其維度為4x1），但可不限於此。 For example, it is assumed that the content of the fourth reference matrix can be represented as Table 2 below. transmit beam 1 transmit beam 2 transmit beam 3 Base station 111 0.3 0.3 0.4 Base station 112 0.4 0.5 0.1 Base station 113 0.7 0.2 0.1 Base station 114 0.8 0.1 0.1 Table II. In this case, after the processor 104 takes the max operator on the fourth reference matrix, for example, it can be obtained

The first selection probability vector of (its dimension is 4x1), but not limited to this.

In addition, in one embodiment, the processor 104 can obtain the first nonlinear combination value of the first selection probability vector and the first average communication quality. For example, assuming that the first average communication quality is Z, the processor 104 can convert the first average communication quality into a 4×1 vector, and each element of this vector is Z, and then combine this vector with the first selected The probability vectors are non-linearly combined to generate the first non-linear combination value described above. In different embodiments, the processor 104 may use different ways to non-linearly combine the first selection probability vector and the first average communication quality according to the designer's requirements. For example, if the designer attaches more importance to the first average communication quality, a higher weight/coefficient can be set for the first average communication quality accordingly, so as to enhance the influence of the first average communication quality on the above-mentioned first nonlinear combination value, but But not limited to this.

Afterwards, the processor 104 may estimate the mean square error value of the first nonlinear combination value and the first reference vector as the first mean square error value of the ith recursive procedure of the first training process.

Next, in step S422, the processor 104 may use the first reference matrix, the third reference matrix and the first average communication quality as training data for the ith recursive procedure.

In one embodiment, in response to determining that i is greater than 1, the first type of neural network in the i-th recursive procedure can be set by the processor 104 to be the same as in the (i-1)-th recursive procedure and the second type of neural network in the ith recursive procedure can be set by the processor 104 to be the same as the second type of neural network in the (i-1)th recursive procedure road. In short, the processor 104 uses the first-type neural network NN1 and the second-type neural network in the previous recursive procedure as the first-type neural network NN1 and the second-type neural network in the current recursive procedure. Internet, but not limited to this.

In one embodiment, in response to determining that i is greater than 1, the first reference signal intensity distribution map in the i-th recursive procedure can be set by the processor 104 to be the same as the (i-1)-th recursive procedure The second reference signal strength profile in . In short, the processor 104 uses the second reference signal intensity distribution map in the previous recursive process as the first reference signal intensity distribution map in the current recursive process, but it is not limited thereto.

In one embodiment, in response to determining that i reaches a predetermined synchronization number (eg, 6000), the processor 104 may further set the second type of neural network to be the same as the first type of neural network NN1. Specifically, since the parameter adjustment of the first/second type of neural network is prone to oscillations (for example, the weights/parameters of the first type of neural network are updated quickly, while the weights/parameters of the second type of neural network The update is relatively slow), and through the above technical means, the second type of neural network can follow the learning progress of the first type of neural network in a timely manner, but it is not limited to this.

In one embodiment, in response to determining that i reaches a first preset number (for example, 100), the processor 104 may further determine the i-th recursion after acquiring the training data of the i-th recursion procedure From the first mean square error value of the procedure to the first specific probability of the first mean square error value of the (i-Y+1)th recursive procedure, and determine the training data of the i-th recursive procedure accordingly Individual first reference weights of training data up to the (i-Y+1)th recursive procedure, where Y is a first predetermined number (eg, 100).

，其中

為所述第i次遞迴程序的第一均方誤差值，

為第一訓練過程中的第k次遞迴程序的第一均方誤差值。由上可知，所述第i次遞迴程序的第一特定機率可正相關於所述第i次遞迴程序的第一均方誤差值。 In one embodiment, the first specific probability of the i-th recursive procedure can be characterized as

,in

is the first mean square error value of the i-th recursive procedure,

is the first mean square error value of the k-th recursive procedure in the first training process. It can be seen from the above that the first specific probability of the ith recursive procedure may be positively related to the first mean square error value of the ith recursive procedure.

，其中BS為訓練資料容量，而

為所述第i次遞迴程序的第一特定機率。在一實施例中，所述訓練資料容量例如是管理伺服器100可容納的訓練資料總數。舉例而言，假設訓練資料容量為10000，則在處理器104收集到第10001筆訓練資料時，處理器104可相應地拋棄第1筆訓練資料，但可不限於此。 In addition, the first reference weight of the i-th recursive procedure can be characterized as

, where BS is the training data capacity, and

is the first specific probability of the i-th recursive procedure. In one embodiment, the training data capacity is, for example, the total number of training data that the management server 100 can accommodate. For example, assuming that the capacity of training data is 10000, when the processor 104 collects the 10001st piece of training data, the processor 104 can correspondingly discard the first piece of training data, but it is not limited to this.

Afterwards, the processor 104 may select the first reference weight of the i-th recursive procedure based on the training data of the i-th recursive procedure to the respective first reference weights of the training data of the (i-Y+1)-th recursive procedure Part of the training data to the training data of the (i-Y+1)th recursive procedure trains the first type neural network NN1 and the second type neural network to update the plurality of first neuron weights and all the plurality of second neuron weights. In one embodiment, the probability that the training data of the ith recursive procedure is selected for training the first type of neural network and the second type of neural network is positively related to the first Reference weight.

As can be seen from the above, in the first training process, the processor 104 will only train the first type of neural network NN1 and The second type of neural network, and the probability that the training data are individually selected for training the first type of neural network NN1 and the second type of neural network is the corresponding first reference weight.

In the embodiment of the present invention, the processor 104 can continuously execute the recursive procedure in the first training process until the obtained first average communication quality satisfies a specified service condition (for example, a certain transmission speed is reached). It is determined that the first training process has been completed, but may not be limited thereto.

In one embodiment, after completing the first training process, the processor 104 may continue to perform the second training process of the first type of neural network, the details of which are described below.

Please refer to FIG. 5 , which is a schematic diagram of a second training process according to an embodiment of the present invention. The method of this embodiment can be executed by the management server 100 in FIG. 1 , and the details of each step in FIG. 5 will be described below in conjunction with the components shown in FIG. 1 .

In an embodiment of the present invention, the second training process may include several recursive procedures, and the content of FIG. 5 corresponds to the j-th recursive procedure in the second training process, but it is not limited thereto.

In step S511, the processor 104 may obtain the first type of neural network NN1 that has undergone the first training process. In one embodiment, since the first type of neural network NN1 has undergone the first training process, the instability of the first type of neural network NN1 in the early stage of the second training process can be reduced.

In step S512, the processor 104 may determine the fourth transmit beam of each base station 111-114, and measure the first actual signal intensity distribution map of each base station 111-114 in a specific field accordingly. In one embodiment, when j is 1, the processor 104 may randomly select one of the transmit beams 1, 2, and 3 for each base station 111-114 as the corresponding fourth transmit beam. For details, please refer to the previous embodiment. The description in , will not be repeated here. Then, different from generating the first reference signal intensity distribution map of each base station 111-114 in a specific field in a simulated manner in FIG. 4, the first actual signal intensity distribution map of each base station 111-114 in this embodiment It is obtained through actual measurement, and the form of the first actual signal intensity distribution map corresponding to each base station 111 - 114 may be similar to the specific signal intensity distribution maps ST1 - ST4 in FIG. 3 , but not limited thereto.

In step S513, the processor 104 may obtain a location distribution map of the plurality of reference user equipments in a specific field. In one embodiment, the processor 104 can obtain, for example, the MRs reported by the served user equipments from the base stations 111-114 to estimate the location of each user equipment in a specific field, and then construct the above-mentioned location distribution map, but it may not limited to this. In one embodiment, the form of the above-mentioned location distribution map may be similar to the specific location distribution map PD1 in FIG. 3 , but it is not limited thereto.

In step S514, the processor 104 may generate a first matrix according to the location distribution map and the first actual signal intensity distribution map of each base station 111-114. In this embodiment, the processor 104 can individually convert the above-mentioned location distribution map and the first actual signal intensity distribution map of each base station 111-114 into corresponding RGB images in a manner similar to that mentioned in the previous embodiment, and It is superimposed into a 15xNxM three-dimensional matrix as the first matrix, but it is not limited to this.

In step S515, the processor 104 may input the first matrix to the first type of neural network NN1, wherein the first type of neural network NN1 may output a second matrix according to the first matrix, and the second matrix includes each base station 111 ~114 selects a probability for each transmit beam. In this embodiment, the form of the second matrix may be similar to the form of the second specific matrix SM2, so the details thereof will not be repeated here.

In step S516, the processor 104 may take the argmax operator on the second matrix to obtain the first vector, and determine the fifth transmit beam corresponding to each base station 111-114 based on the first vector. In this embodiment, the form of the first vector may be similar to the above-mentioned transmit beam parameter vector, and the manner in which the processor 104 determines the fifth transmit beam corresponding to each base station 111-114 is similar to the manner in which the processor 104 determines the fifth transmit beam corresponding to the above-mentioned transmit beam The number vector is a way for each base station 111 to 114 to determine the optimal transmission beam, so the details thereof will not be repeated here.

In step S517, the processor 104 may actually measure the second actual signal intensity distribution diagram of each base station 111-114 in a specific field according to the fifth transmit beam corresponding to each base station 111-114. Similarly, the processor 104 can actually measure the second actual signal strength distribution diagram of each base station 111-114 in a specific field. In this embodiment, the form of the second actual signal intensity distribution map corresponding to each base station 111 - 114 may be similar to the specific signal intensity distribution map ST1 - ST4 in FIG. 3 , but not limited thereto.

In step S518, the processor 104 may generate a third matrix according to the location distribution map and the second actual signal strength distribution map of each base station 111-114, and obtain the second average communication quality of the plurality of reference user equipments. In this embodiment, the third matrix is, for example, a three-dimensional matrix of 15×N×M, and the generation method thereof is similar to that of the first matrix, so it will not be repeated here. In addition, the above-mentioned second average communication quality is, for example, the average transmission speed of the plurality of reference user devices or other data that can be used to represent the communication quality, but it is not limited thereto.

In step S519, the processor 104 may obtain the second type of neural network that has undergone the first training process. In step S520, the processor 104 may input the third matrix to the second type of neural network, wherein the second type of neural network may output a fourth matrix according to the third matrix, and the fourth matrix includes each of the base stations 111-114 The probability of selecting each transmit beam. In this embodiment, the form of the fourth matrix may be similar to the form of the second specific matrix SM2, so the details thereof will not be repeated here.

In step S521, the processor 104 may determine the second selection probability vector of the plurality of base stations for the plurality of transmit beams based on the fourth matrix, and based on the second selection probability vector, the second average communication quality and the first The vector determines the second mean squared error value. In one embodiment, the processor 104 may perform a max operator on the fourth matrix to obtain the second selection probability vector.

In addition, in one embodiment, the processor 104 can obtain the second nonlinear combination value of the second selection probability vector and the second average communication quality. For example, assuming that the first average communication quality is Z', the processor 104 can convert the second average communication quality into a 4x1 vector, and each element of the vector is Z', and then combine this vector with the first The second selects the probability vector to perform nonlinear combination to generate the above-mentioned second nonlinear combination value. In different embodiments, the processor 104 may use different ways to non-linearly combine the second selection probability vector and the second average communication quality according to the designer's needs. For example, if the designer pays more attention to the second average communication quality, a higher weight/coefficient can be set for the second average communication quality accordingly to enhance the influence of the second average communication quality on the second non-linear combination value, but But not limited to this.

Afterwards, the processor 104 may estimate the mean square error value of the second nonlinear combination value and the first vector as the second mean square error value of the j-th recursive procedure of the second training process.

，其中

為所述多個機率向量中對應於第a個基地台的第a個機率向量。 In step S522, the processor 104 may determine a plurality of entropy values based on the second matrix. In one embodiment, the second matrix may include a plurality of probability vectors corresponding to the base stations 111 to 114 respectively, and the a-th entropy value among the plurality of entropy values may be represented as

,in

is the a-th probability vector corresponding to the a-th base station among the plurality of probability vectors.

For example, it is assumed that the content of the second matrix can be represented as Table 3 below. transmit beam 1 transmit beam 2 transmit beam 3 Base station 111 0.3 0.3 0.4 Base station 112 0.4 0.5 0.1 Base station 113 0.7 0.2 0.1 Base station 114 0.8 0.1 0.1 Table 3

，對應於基地台112的機率向量例如是

，對應於基地台113的機率向量例如是

，對應於基地台114的機率向量例如是

。在此情況下，基地台111的熵值可表徵為

，基地台112的熵值可表徵為

，基地台113的熵值可表徵為

，基地台114的熵值可表徵為

，但可不限於此。 In Table 3, the probability vector corresponding to the base station 111 is, for example,

, the probability vector corresponding to the base station 112 is, for example,

, the probability vector corresponding to the base station 113 is, for example,

, the probability vector corresponding to base station 114 is, for example,

. In this case, the entropy value of the base station 111 can be characterized as

, the entropy value of the base station 112 can be characterized as

, the entropy value of the base station 113 can be characterized as

, the entropy value of the base station 114 can be characterized as

, but not limited to this.

In step S523, the processor 104 may estimate a combined value of the second mean square error value and the plurality of entropy values. For example, the processor 104 may add the second mean square error value and the plurality of entropy values as a combined value of the j-th recursive procedure of the second training process, but it is not limited thereto.

Next, in step S524, the processor 104 can use the first matrix, the third matrix and the second average communication quality as training data for the j-th recursive procedure.

In one embodiment, the first type of neural network NN1 in the j-th recursive procedure can be set by the processor 104 to be the same as the first type of neural network in the (j-1)-th iteration procedure NN1, and the second type of neural network in the j-th recursive procedure can be set by the processor 104 to be the same as the second-type neural network in the (j-1)-th iteration procedure. In short, the processor 104 uses the first-type neural network NN1 and the second-type neural network in the previous recursive procedure as the first-type neural network NN1 and the second-type neural network in the current recursive procedure. Internet, but not limited to this.

In one embodiment, in response to determining that j reaches a preset synchronization number (eg, 6000), the processor 104 may further set the second type of neural network to be the same as the first type of neural network NN1, so that the second type of neural network The neural network follows the learning progress of the first type of neural network, but is not limited to this.

In one embodiment, in response to determining that j reaches a second predetermined number (for example, 100), after obtaining the training data of the j-th iteration procedure, the processor 104 may further determine the j-th iteration procedure. Return the combined value of the procedure to the second specific probability of the combined value of the second mean square error value of the (j-X+1)th iteration procedure, and determine the training data of the jth iteration procedure accordingly A second reference weight to the training data of the (j-X+1)th iteration, where X is a second predetermined number.

，其中

為所述第j次遞迴程序的組合值，

為第二訓練過程中的第k’次遞迴程序的組合值。由上可知，所述第j次遞迴程序的第二特定機率可正相關於所述第j次遞迴程序的組合值。 In one embodiment, the second specific probability of the j-th recursive procedure can be characterized as

,in

is the combined value of the j-th recursive procedure,

Combined values for the k'th recursive procedure in the second training process. It can be seen from the above that the second specific probability of the j-th recursive procedure may be positively related to the combined value of the j-th recursive procedure.

，其中BS為訓練資料容量，

為所述第j次遞迴程序的第二特定機率。 In addition, the second reference weight of the j-th recursive procedure can be characterized as

, where BS is the training data capacity,

is the second specific probability of the j-th recursive procedure.

Afterwards, the processor 104 may select the jth recursive procedure based on the individual second reference weights from the training data of the j-th recursive procedure to the training data of the (j-X+1)-th recursive procedure A part of the training data to the training data of the (j-X+1)th recursive procedure trains the first-type neural network NN1 and the second-type neural network to update the plurality of first neuron weights and all the plurality of second neuron weights. In one embodiment, the probability that the training data of the jth recursive procedure is selected for training the first type of neural network NN1 and the second type of neural network is positively related to the probability of the jth iteration procedure. Two reference weights.

It can be seen from the above that in the second training process, the processor 104 will only train the first type of neural network NN1 and The second type of neural network, and the probability that the training data are individually selected for training the first type of neural network NN1 and the second type of neural network is its corresponding second reference weight.

In the embodiment of the present invention, the processor 104 can continue to execute the recursive procedure in the second training process until the acquired second average communication quality meets the specified service conditions (for example, reaches a certain transmission speed, etc.). It is determined that the second training process has been completed, but may not be limited thereto.

After the second training process is completed, the processor 104 can package the relevant neural network parameters of the first type of neural network NN1 with Docker, and can be used to execute steps S210-S270 to determine the optimum for each base station 111-114 transmit beam.

To sum up, the present invention aims to use the advantages of deep reinforcement learning to reduce the influence of the increase in the beam selection space index caused by the increase in the number of base stations. The processor can extract the database data and draw various signal intensity distribution maps, which can be used as the processing basis for the deep reinforcement learning computing module in the future. Based on this deep reinforcement learning operation module, a feedback deep neural network can be trained as a model for outputting optimal beam selection. In addition, the present invention also has a re-sampling mechanism and a weight complementing mechanism, which can accelerate the learning speed of the first/second type of neural network and improve the learning ability, and through this method, the past training data can be fully utilized to improve the utilization rate of the training data. . This capability is particularly important because the collection of data in wireless communication environments is often characterized by omission or duplication.

In addition, the present invention can reduce the time for determining the optimal transmission beam of each base, and can also perform parameter correction or structure adjustment in real time, so as to achieve the purpose and advantages of online learning.

The present invention is a multi-base station joint optimization method to solve the problem of inter-base station interference. Since the parameters of the mobile network are interlinked, compared to the individual optimization of each base station, the joint optimization between the base stations can obtain the best solution.

The present invention also considers that there are occasional special events in the field of wireless communication, and it is necessary to perform additional re-sampling processing for rare events, otherwise the first/second type neural network may not be learned, or the learning time may be increased, so the present invention has Additional probability estimation mechanism and weight adjustment function.

The method proposed in the present invention can utilize the historical data, which is different from some intelligent methods that directly perform gradient operation on the objective function, so that the parameter adjustment mechanism can only accept the current fresh data. This kind of method is not suitable in the wireless communication environment, because wireless There are often omissions or repetitions in the collection of communication environment data, and the method of the present invention is not limited to this. Whether it is current fresh data or past historical data, it can be used, which significantly increases data utilization efficiency.

The present invention can output highly vectorized trained neural network parameters, which can accelerate the operation of signal processing with the GPU installed in the base station, and the trained neural network parameters can obtain an operation result at the millisecond level in an actual environment to meet immediate service requirements.

Although the present invention has been disclosed above by the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the scope of the appended patent application.

10: Communication system 100: Manage Servers 111~11H: base station 102: Storage circuit 104: Processor PD1: Location-specific distribution map ST1~ST4: Distribution map of specific signal strength SM1: First specific matrix SM2: Second specific matrix NN1: Neural Networks of the First Kind S210~S270, S411~S422, S511~S523: Steps

FIG. 1 is a schematic diagram of a communication system according to an embodiment of the present invention. FIG. 2 is a flowchart of a method for beam selection based on a neural network according to an embodiment of the present invention. FIG. 3 is an application scenario diagram according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a first training process according to an embodiment of the present invention. FIG. 5 is a schematic diagram of a second training process according to an embodiment of the present invention.

S210~S270: Steps

Claims (26)

A method for beam selection based on a neural network, suitable for managing a management server of a plurality of base stations, the base stations are deployed in a specific field, and each of the base stations has a plurality of transmission beams, the method include: obtaining a first-type neural network undergoing a first training process and a second training process; determining a first transmit beam of each of the base stations, and measuring a specific signal intensity distribution map of each of the base stations in the specific field; obtaining a distribution map of a specific location of a plurality of user devices in the specific field; generating a first specific matrix according to the specific location distribution map and the specific signal strength distribution map of each base station; The first specific matrix is input to the first type of neural network, wherein the first type of neural network outputs a second specific matrix according to the first specific matrix, wherein the second specific matrix includes a probability of the transmit beam; determining an optimal transmit beam corresponding to each of the base stations based on the second specific matrix; and Control each of the base stations to transmit signals using the corresponding optimal transmit beam. The method of claim 1, wherein the ith recursive procedure in the first training process of the first type of neural network comprises: determining a plurality of first neuron weights of the first type of neural network; determining a second transmit beam of each of the base stations, and simulating a first reference signal intensity distribution map of each of the base stations in the specific field accordingly; obtaining a reference position distribution map of a plurality of reference user equipments in the specific field; generating a first reference matrix according to the reference position distribution map and the first reference signal intensity distribution map of each base station; The first reference matrix is input to the first type of neural network, wherein the first type of neural network outputs a second reference matrix according to the first reference matrix, wherein the second reference matrix includes a probability of the transmit beam; An argmax operator is taken on the second reference matrix to obtain a first reference vector, and a third transmit beam corresponding to each base station is determined based on the first reference vector. The method of claim 2, wherein the ith recursive procedure in the first training process of the first type of neural network further comprises: simulating a second reference signal intensity distribution diagram of each of the base stations in the specific field according to the third transmit beams corresponding to each of the base stations; generating a third reference matrix according to the reference position distribution map and the second reference signal strength distribution map of each base station, and obtaining a first average communication quality of the reference user equipments; determining a plurality of second neuron weights of a second type of neural network; The third reference matrix is input to the second type of neural network, wherein the second type of neural network outputs a fourth reference matrix according to the third reference matrix, wherein the fourth reference matrix includes a probability of the transmit beam; A first selection probability vector of the base stations for the transmit beams is determined based on the fourth reference matrix, and a first average selection probability vector is determined based on the first selection probability vector, the first average communication quality and the first reference vector square error value; The first reference matrix, the third reference matrix and the first average communication quality are used as training data for the ith recursive procedure. The method according to claim 2, wherein in response to determining that i reaches a first preset number, after obtaining the training data of the i-th recursive procedure, the method further comprises: Determining a first specific probability from the first mean square error value of the i-th recursive procedure to the first mean-square error value of the (i-Y+1)-th recursive procedure, and determining accordingly a first reference weight from the training data of the i-th recursive procedure to the training data of the (i-Y+1)-th recursive procedure, wherein Y is the first predetermined number; Based on the training data of the i-th recursive procedure to the training data of the (i-Y+1)-th recursive procedure, the first reference weights individually select the training data of the i-th recursive procedure to the (i-Y+1)-th recursive procedure Part of the training data of i-Y+1) recursive procedure trains the first type neural network and the second type neural network to update the first neuron weights and the second neuron weights . The method of claim 4, wherein the first specific probability of the ith recursive procedure is characterized by

,in

is the first mean square error value of the i-th recursive procedure,

is the first mean square error value of the k-th recursive procedure in the first training process. The method of claim 4, wherein the first reference weight of the ith recursive procedure is characterized by

, where BS is a training data capacity,

is the first specific probability of the i-th recursive procedure. The method of claim 4, wherein the probability that the training data of the i-th recursive procedure is selected for training the first type of neural network and the second type of neural network is positively related to the i-th iteration This first reference weight for the recursive procedure. The method of claim 3, wherein in response to determining that i is 1, the weights of the first neurons and the weights of the second neurons are randomly generated. The method of claim 3, wherein in response to determining that i is greater than 1, the first type of neural network in the ith recursive procedure is the same as the neural network in the (i-1)th recursive procedure The first type of neural network, and the second type of neural network in the ith recursive procedure is the same as the second type of neural network in the (i-1)th recursive procedure. The method of claim 3, wherein in response to determining that i is greater than 1, the first reference signal intensity distribution in the i-th recursive procedure is the same as the (i-1)-th recursive procedure The second reference signal intensity distribution map. The method of claim 3, wherein in response to determining that i reaches a predetermined synchronization number, the method further comprises setting the second type of neural network to be the same as the first type of neural network. The method of claim 3, wherein the step of determining the first selection probability vectors of the base stations for the transmit beams based on the fourth reference matrix comprises taking a max operator on the fourth reference matrix to obtain The first selection probability vector. The method of claim 3, wherein the step of determining the first mean square error value based on the first selection probability vector, the first average communication quality and the first reference vector comprises: obtaining a first nonlinear combination value of the first selection probability vector and the first average communication quality; A mean square error value between the first nonlinear combination value and the first reference vector is estimated as the first mean square error value. The method of claim 1, wherein the j-th recursive procedure of the second training process of the first type of neural network comprises: (a2) obtaining the first type of neural network that has undergone the first training process; (b2) determining a fourth transmission beam of each of the base stations, and actually measuring a first actual signal strength distribution map of each of the base stations in the specific field according to it; (c2) obtaining a location distribution map of the reference user equipments in the specific field; (d2) generating a first matrix according to the location distribution map and the first actual signal strength distribution map of each of the base stations; (e2) Inputting the first matrix to the first type of neural network, wherein the first type of neural network outputs a second matrix according to the first matrix, wherein the second matrix includes that each base station selects each of the a probability of transmitting the beam; (f2) taking an argmax operator from the second matrix to obtain a first vector, and determining a fifth transmit beam corresponding to each base station based on the first vector. The method of claim 14, wherein the j-th recursive procedure in the second training process of the first type of neural network further comprises: actually measuring a second actual signal strength distribution map of each of the base stations in the specific field according to the fifth transmission beam corresponding to each of the base stations; generating a third matrix according to the location distribution map and the second actual signal strength distribution map of each base station, and obtaining a second average communication quality of the reference user equipments; obtaining the second type of neural network undergoing the first training process; The third matrix is input to the second type of neural network, wherein the second type of neural network outputs a fourth matrix according to the third matrix, wherein the fourth matrix includes the transmission beam selected by each base station. probability; A second selection probability vector of the base stations for the transmit beams is determined based on the fourth matrix, and a second mean square error is determined based on the second selection probability vector, the second average communication quality and the first vector value; determining a plurality of entropy values based on the second matrix; estimating a combined value of the second mean squared error value and the entropy values; The first matrix, the third matrix and the second average communication quality are used as training data for the j-th recursive procedure. The method according to claim 15, wherein in response to determining that j reaches a second preset number, after obtaining the training data of the j-th recursive procedure, the method further comprises: Determining a second specific probability of the combined value of the combined value of the j-th recursive procedure to the second mean square error value of the (j-X+1)-th recursive procedure, respectively, and determining accordingly A second reference weight from the training data of the j-th recursive procedure to the training data of the (j-X+1)-th recursive procedure, wherein X is the second predetermined number; Based on the training data of the j-th recursive procedure to the training data of the (j-X+1)-th recursive procedure, the individual second reference weights select the training data of the j-th recursive procedure to the (j-X+1)-th recursive procedure A part of the training data of j-X+1) recursive procedure trains the first type neural network and the second type neural network to update the first neuron weights and the second neuron weights . The method of claim 16, wherein the second specific probability of the j-th recursive procedure is characterized by

,in

is the combined value for the j-th recursive procedure,

This combined value for the k'th recursion procedure in the second training process. The method of claim 16, wherein the second reference weight of the j-th recursive procedure is characterized by

, where BS is a training data capacity,

is the second specific probability of the j-th recursive procedure. The method of claim 16, wherein the probability that the training data of the jth recursive procedure is selected for training the first type of neural network and the second type of neural network is positively related to the jth iteration This second reference weight for the recursive procedure. The method of claim 15, wherein the first type of neural network in the jth iteration is the same as the first type of neural network in the (j-1)th iteration, and The second-type neural network in the j-th recursive procedure is the same as the second-type neural network in the (j-1)-th recursive procedure. The method of claim 15, wherein in response to determining that j reaches a predetermined synchronization number, the method further comprises setting the second type of neural network to be the same as the first type of neural network. The method of claim 15, wherein the step of determining the second selection probability vectors of the base stations for the transmit beams based on the fourth matrix comprises taking a max operator on the fourth matrix to obtain the first Second, select probability vectors. The method of claim 15, wherein the step of determining the second mean square error value based on the second selection probability vector, the second average communication quality and the first vector comprises: obtaining a second nonlinear combination value of the second selection probability vector and the second average communication quality; A mean square error value between the second nonlinear combination value and the first vector is estimated as the second mean square error value. The method of claim 15, wherein the second matrix includes a plurality of probability vectors respectively corresponding to the base stations, and the a-th entropy value among the entropy values is represented as

,in

is the a-th probability vector corresponding to the a-th base station among the probability vectors. The method of claim 1, wherein the step of determining the optimal transmit beam corresponding to each of the base stations based on the second specific matrix comprises: An argmax operator is taken on the second specific matrix to obtain a transmit beam argument vector, and the optimal transmit beam corresponding to each base station is determined based on the transmit beam argument vector. A management server, which manages a plurality of base stations deployed in a specific field, and each of the base stations has a plurality of transmission beams, the management server includes: a storage circuit storing a plurality of modules; and A processor, coupled to the storage circuit, accesses the modules to perform the following steps: obtaining a first-type neural network undergoing a first training process and a second training process; determining a first transmit beam of each of the base stations, and measuring a specific signal intensity distribution map of each of the base stations in the specific field; obtaining a distribution map of a specific location of a plurality of user devices in the specific field; generating a first specific matrix according to the specific location distribution map and the specific signal strength distribution map of each base station; The first specific matrix is input to the first type of neural network, wherein the first type of neural network outputs a second specific matrix according to the first specific matrix, wherein the second specific matrix includes a probability of the transmit beam; determining an optimal transmit beam corresponding to each of the base stations based on the second specific matrix; and Control each of the base stations to transmit signals using the corresponding optimal transmit beam.

Images