CN112714064A - Power line communication network topology control method, device, equipment and medium - Google Patents
Power line communication network topology control method, device, equipment and medium Download PDFInfo
- Publication number
- CN112714064A CN112714064A CN202011603518.6A CN202011603518A CN112714064A CN 112714064 A CN112714064 A CN 112714064A CN 202011603518 A CN202011603518 A CN 202011603518A CN 112714064 A CN112714064 A CN 112714064A
- Authority
- CN
- China
- Prior art keywords
- action
- node
- value
- state
- noise ratio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 81
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000009471 action Effects 0.000 claims abstract description 90
- 230000008569 process Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000003111 delayed effect Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 description 15
- 230000008859 change Effects 0.000 description 6
- 241000239290 Araneae Species 0.000 description 4
- 230000006855 networking Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 241000854291 Dianthus carthusianorum Species 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/12—Shortest path evaluation
- H04L45/123—Evaluation of link metrics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
- H04L45/08—Learning-based routing, e.g. using neural networks or artificial intelligence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application discloses a power line communication network topology control method, a device, equipment and a medium, wherein the method comprises the following steps: calculating the signal-to-noise ratio of the current link; if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link; acquiring the state and the action of the node at the current moment, and observing the state of the next moment corresponding to the action at the current moment, wherein the action comprises sending information to a neighbor node; selecting the action with the highest immediate report and the state of the next moment corresponding to the action, and updating the Q value; if the updated Q value is not converged, updating the state and the corresponding Q value at the next moment again; and selecting the most reliable node for communication topology selection until the Q values of all the nodes are converged. The method and the device selectively find out the reliable node to carry out communication topology selection.
Description
Technical Field
The present application relates to the field of power line communication technologies, and in particular, to a method, an apparatus, a device, and a medium for controlling topology of a power line communication network.
Background
Because users can change the power supply range and the power supply object frequently, the physical topology of the low-voltage distribution network has certain time-varying property. Meanwhile, the load in the low-voltage distribution network is changed frequently, and the change can cause the change of a communication channel and even cause the interruption of a communication link, thereby causing the change of the logic topology of the low-voltage distribution network. This reduces the reliability of the power line communication.
In recent years, researchers tend to use a flattened architecture in the aspect of topology control, and the proposed method has no learning capability, so that the topology cannot be dynamically adjusted according to the change of a scene to a certain extent, and an effective control mechanism is also lacking in the aspect of network self-healing.
In a traditional clustering algorithm, a network is provided with certain associated node sets to form clusters, and each cluster comprises 1 cluster head node and a plurality of member nodes in the cluster. The cluster head nodes are selected through a certain algorithm or rule, information in the cluster is collected, data fusion processing is carried out, and therefore the nodes which coordinate and manage the work of member nodes in the cluster are completed, and the function of forwarding among clusters needs to be achieved.
The non-overlapping clustering algorithm is used for researching and discussing the aspects of selection, establishment, automatic routing and the like of a narrow-band power line communication data logical link of a low-voltage distribution network, and provides a dynamic routing algorithm and a network reconstruction algorithm based on non-overlapping clustering. The algorithm can dynamically establish, maintain and optimize the power line communication network route according to the change of the channel quality, and ensure the effectiveness of the communication network.
The artificial spider web topology control algorithm provides a communication model based on artificial spider webs, a logic topology network formed by the artificial spider webs is used for replacing a part of layer networks which are mainly responsible for communication protocols in a traditional low-voltage power distribution network, a new automatic routing protocol is formulated, and an artificial spider web simulation model is established.
The genetic algorithm is improved, the thought of a graph traversal algorithm and the advantages of the Dijkstra algorithm are fused, the ecological niche technology optimal retention principle is combined, and the ecological niche dynamic environment adaptation method can be well achieved.
Disclosure of Invention
The application provides a topology control method, a device, equipment and a medium of a power line communication network, which can selectively find out reliable nodes to select communication topology.
In view of the above, a first aspect of the present application provides a power line communication network topology control method, including:
s01: calculating the signal-to-noise ratio of the current link;
s02: if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link;
s03: acquiring the state and action of a node at the current moment, and observing the state of the next moment corresponding to the action of the current moment, wherein the action comprises sending information to a neighbor node;
s04: selecting the action with the highest immediate return and the state of the next moment corresponding to the action, and updating the Q value;
s05: if the updated Q value does not converge, the process returns to step S03;
s06: and if the Q values of all the nodes are converged, selecting the most reliable node for communication topology selection.
Optionally, the method further includes:
if the signal-to-noise ratio does not belong to the preset signal-to-noise ratio window range, calculating the communication success rate of the current link;
and judging the reliability of the link according to the communication success rate.
Optionally, the calculating the Q value of each node in the current link includes:
in the formula: stIs the state of the node at time t, St∈S*，S*Representing a set of states, i.e. the current state StAll possible states of (a); a istFor the node's action at time t, atE.g. A, A represents action set; r is instant report at time t; gamma is a discount factor used for determining the relative proportion of delayed return and immediate return, and gamma is more than or equal to 0 and less than or equal to 1; alpha is a learning factor, 0<α≤1；Qt+1And QtQ values of the nodes at t +1 and t are respectively;
representing the maximum Q value at which the node takes action a at t + 1.
Optionally, the selecting the action with the highest immediate report and the state of the next moment corresponding to the action, and updating the Q value includes:
select action a with the highest immediate returntAnd the state S of the next time corresponding to the actiont+1Updating the Q value:
optionally, if the Q values of all the nodes converge, selecting the most reliable node for communication topology selection includes:
acquiring a first action with optimal instant return under the condition that the Q value of a node is converged, and acquiring an optimal neighbor node corresponding to the first action;
and selecting communication topology according to the optimal node corresponding to each node.
A second aspect of the present application provides a power line communication network topology control apparatus, the apparatus including:
the first calculating unit is used for calculating the signal-to-noise ratio of the current link;
the second calculation unit is used for calculating the Q value of each node in the current link when the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window;
the first acquisition unit is used for acquiring the current time state and action of the node and observing the state of the next time corresponding to the execution of the current time action, wherein the action comprises the step of sending information to a neighbor node;
the updating unit is used for selecting the action with the highest instant report and the state of the next moment corresponding to the action, and updating the Q value;
the iteration unit is used for returning to the first acquisition unit when the updated Q value is not converged;
and the topology selection unit is used for selecting the most reliable node for communication topology selection when the Q values of all the nodes are converged.
Optionally, the method further includes:
the third calculating unit is used for calculating the communication success rate of the current link when the signal-to-noise ratio does not belong to the range of the preset signal-to-noise ratio window;
and the judging unit is used for judging the reliability of the link according to the communication success rate.
Optionally, the topology selecting unit includes:
the second acquisition unit is used for acquiring a first action with optimal instant return under the condition that the Q value of the node is converged and acquiring an optimal neighbor node corresponding to the first action;
and the second topology selection unit is used for carrying out communication topology selection according to the optimal node corresponding to each node.
A third aspect of the present application provides a power line communication network topology control device, the device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the steps of the power line communication network topology control method according to the first aspect, according to instructions in the program code.
A fourth aspect of the present application provides a computer-readable storage medium for storing program code for performing the method of the first aspect.
According to the technical scheme, the method has the following advantages:
the application provides a topology control method of a power line communication network, which calculates the signal-to-noise ratio of a current link; if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link; acquiring the state and the action of the node at the current moment, and observing the state of the next moment corresponding to the action at the current moment, wherein the action comprises sending information to a neighbor node; selecting the action with the highest immediate report and the state of the next moment corresponding to the action, and updating the Q value; if the updated Q value is not converged, updating the state and the corresponding Q value at the next moment again; and selecting the most reliable node for communication topology selection until the Q values of all the nodes are converged.
According to the method and the device, the reliability of the link can be judged by utilizing the Q learning algorithm model under the condition that the power line communication network is not stable, and the reliable node can be selectively found out so as to select the communication topology mode. Q learning is carried out only in the range of a preset signal-to-noise ratio window by setting the signal-to-noise ratio window to judge the reliability of the link; and the values lower than the window interval and higher than the window interval directly determine the reliability of the link according to the threshold value, thereby reducing the Q learning calculation amount.
Drawings
Fig. 1 is a flowchart of a method of an embodiment of a topology control method for a power line communication network according to the present application;
fig. 2 is a flowchart of a method of another embodiment of a topology control method of a power line communication network according to the present application;
fig. 3 is a device structure diagram of an embodiment of a topology control device of a power line communication network according to the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a flowchart of a method of an embodiment of a topology control method for a power line communication network according to the present application, as shown in fig. 1, where fig. 1 includes:
s01: calculating the signal-to-noise ratio of the current link;
it should be noted that, the signal-to-noise ratio of the current link is calculated, and the present application may determine whether the current link is a reliable link through the signal-to-noise ratio.
S02: if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link;
it should be noted that, if the calculated signal-to-noise ratio falls within the preset signal-to-noise ratio window range, the Q value of each node in the link is calculated, and the calculation method of the Q value of the node is as follows:
in the formula: stIs the state of the node at time t, Ste.S (S represents a state set); a istFor the node's action at time t, ate.A (A represents the action set); r is instant report at time t; γ ∈ [ s, a) is a discount factor for determining the relative proportion of delayed return to immediate return, γ ≦ 1 being 0 ≦ γ, if γ is 1, then being as important for future rewards (delayed returns) and current rewards (immediate returns); if gamma is equal to 0, the method does not belong to reinforcement learning, but is a prediction problem, namely, a binary prediction problem which corresponds to any reward when doing what action in what state; alpha is a learning factor, 0<α≤1；Qt+1And QtQ values of the nodes at t +1 and t are respectively; can pass through StAnd atThe system state at the next moment is obtained,
representing the maximum Q value at which the node takes action a at t + 1.
In the formula, the report function r (i, j) represents an instant report obtained after a node directly/indirectly registers with a gateway, and specifically includes:
in the formula wiThe link weight of the node i receiving the learning message is obtained; w is ajThe link weight of the node j which sends the learning message; chi shapeijIs the hop count between the node j sending the learning message and the node i receiving the learning message. The formula comprehensively considers factors such as link quality, distance and the like and reflects the return acquired by the node when the network changes.
S03: acquiring the state and action of a node at the current moment, and observing the state of the next moment corresponding to the action of the current moment, wherein the action comprises sending information to a neighbor node;
s04: selecting the action with the highest immediate report and the state of the next moment corresponding to the action, and updating the Q value;
it should be noted that, because different application scenarios may have different resources, adjusting α may update the Q value, and the method converges. When α is 1, which is a learning rule in deterministic return, the update formula of Q value is:
the Q learning algorithm recursively learns the best Q value with the winning state and reward and selecting an action at each time t, a representing the learning rate a ∈ (0,1), Q(s) if each state pair is accessed an indefinite number of times and the learning rate a decreases to zerot,at) Convergence to the optimum merit function Q(s) with probability 1t,at) (ii) a The Q learning algorithm is characterized in that historical experiences are accumulated through continuous perception of the environment, and a learning subject can autonomously select an optimal action target through continuous trial and error and continuous reinforcement. The process is as follows: establishing a Q value table, and initializing the Q value Q(s) corresponding to each state-action pairt,at) 0; and judging whether the Q value is converged. If the convergence is reached, the algorithm is completed, and the program is exited; if not, the method comprises the following steps: 1) observing the state s at the current momentt(ii) a 2) Selecting action a based on current state and policyt(ii) a 3) Performing action atObtaining an immediate report r(s)t,at) (ii) a 4) Observing a new state st+1And updating the Q value table until the Q value converges.
S05: if the updated Q value does not converge, the process returns to step S03;
it should be noted that, when the updated Q value is not converged, the new state is taken as the new current time state, an action is selected again according to the current state and the policy, the action is executed to obtain an immediate response, the action with the highest immediate response rate and the state at the next time corresponding to the action are continuously selected, and the Q value is updated until the Q value is converged.
S06: and if the Q values of all the nodes are converged, selecting the most reliable node for communication topology selection.
It should be noted that when the Q values of all nodes are converged, the optimal action of each node can be obtained, and since the optimal action refers to an action that the node sends information to a neighboring node and obtains the highest immediate return, the optimal neighboring node transmitted by the node in the next step can be obtained according to the optimal action, that is, the communication topology selection can be performed according to the optimal neighboring node transmitted by the node in the next step.
According to the method and the device, the reliability of the link can be judged by utilizing the Q learning algorithm model under the condition that the power line communication network is not stable, and the reliable node can be selectively found out so as to select the communication topology mode. Q learning is carried out only in the range of a preset signal-to-noise ratio window by setting the signal-to-noise ratio window to judge the reliability of the link; and the values lower than the window interval and higher than the window interval directly determine the reliability of the link according to the threshold value, thereby reducing the Q learning calculation amount.
The present application further provides another embodiment of a topology control method for a power line communication network, as shown in fig. 2, where fig. 2 includes:
s21: calculating the signal-to-noise ratio of the current link;
s22: if the signal-to-noise ratio does not belong to the range of the preset signal-to-noise ratio window, calculating the communication success rate of the current link;
s23: judging the reliability of the link according to the communication success rate;
it should be noted that, if the signal-to-noise ratio of the current link does not belong to the range of the preset signal-to-noise ratio window, the communication success rate of the current link is calculated, and the reliability of the link is determined according to the communication success rate.
Specifically, the channel state may be defined as 3 states, and when the state is 1, the communication success rate CSR is large, the channel condition is good, and the group frame can be correctly received; when the state is 2, the CSR is small, the channel condition is poor, the networking frame can be received, but the analysis error code is large, and the data error exists. When the state is 3, it indicates that the CSR is small, the channel condition is bad, and the networking frame cannot be transmitted.
The channel state is switched between 1, 2 and 3 under the influence of various factors. According to the equivalent state, the group frame is not retransmitted only when the channel state is in the equivalent state 1. When the equivalent state is 2 or 3, the networking frame is considered to be failed to be sent, and retransmission is needed.
In a power line communication network channel environment, setting the communication success rate CSR thresholds of an uplink/downlink as lambda 1 and lambda 2 respectively, and only when the conditions of the CSR thresholds of the uplink/downlink are met, taking corresponding networking action can determine the reliability of a link.
Setting a window from 0 to the maximum signal-to-noise ratio, and only performing Q learning in a window interval to judge the reliability of the link; and if the values are lower than the window interval and higher than the window interval, the reliability of the link is directly judged according to the threshold value, and the calculated amount is reduced.
S24: if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link;
s25: acquiring the state and the action of the node at the current moment, and observing the state of the next moment corresponding to the action at the current moment, wherein the action comprises sending information to a neighbor node;
s26: selecting the action with the highest immediate report and the state of the next moment corresponding to the action, and updating the Q value;
s27: if the updated Q value does not converge, the process returns to step S03;
s28: and acquiring a first action with optimal instant return under the condition that the Q value of the node is converged, and acquiring an optimal neighbor node corresponding to the first action.
S29: and selecting communication topology according to the optimal node corresponding to each node.
It should be noted that when the Q values of all nodes are converged, a first action that each node has the optimal immediate return can be obtained, and since the first action refers to that when the node sends information to a neighboring node, an action with the highest immediate return is obtained, an optimal neighboring node transmitted by the node in the next step can be obtained according to the optimal action, that is, communication topology selection can be performed according to the optimal neighboring node transmitted by each node in the next step.
The above is an embodiment of the method of the present application, and the present application further provides an embodiment of a topology control device for a power line communication network, as shown in fig. 3, where fig. 3 includes:
a first calculating unit 301, configured to calculate a signal-to-noise ratio of a current link;
a second calculating unit 302, configured to calculate a Q value of each node in the current link when the signal-to-noise ratio belongs to a preset signal-to-noise ratio window range;
a first obtaining unit 303, configured to obtain a current state and an action of a node, and observe a next state corresponding to the execution of the current action, where the action includes sending information to a neighboring node;
an updating unit 304, configured to select the action with the highest immediate report and a state of a next moment corresponding to the action, and update the Q value;
an iteration unit 305, configured to return to the first obtaining unit when the updated Q value is not converged;
and a topology selecting unit 306, configured to select the most reliable node for communication topology selection when the Q values of all the nodes converge.
In a specific embodiment, the method further comprises the following steps:
the third calculating unit is used for calculating the communication success rate of the current link when the signal-to-noise ratio does not belong to the range of the preset signal-to-noise ratio window;
and the judging unit is used for judging the reliability of the link according to the communication success rate.
The topology selection unit includes:
the second acquisition unit is used for acquiring a first action with optimal instant return under the condition that the Q value of the node is converged and acquiring an optimal neighbor node corresponding to the first action;
and the second topology selection unit is used for carrying out communication topology selection according to the optimal node corresponding to each node.
The application also provides a topology control device of a power line communication network, which comprises a processor and a memory:
the memory is used for storing the program codes and transmitting the program codes to the processor; the processor is used for executing a power line communication network topology control method in the embodiment of the application according to instructions in the program code.
The present application also provides a computer-readable storage medium for storing program codes for executing a power line communication network topology control method in the embodiments of the present application.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A topology control method for a power line communication network, comprising:
s01: calculating the signal-to-noise ratio of the current link;
s02: if the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window, calculating the Q value of each node in the current link;
s03: acquiring the state and action of a node at the current moment, and observing the state of the next moment corresponding to the action of the current moment, wherein the action comprises sending information to a neighbor node;
s04: selecting the action with the highest immediate return and the state of the next moment corresponding to the action, and updating the Q value;
s05: if the updated Q value does not converge, the process returns to step S03;
s06: and if the Q values of all the nodes are converged, selecting the most reliable node for communication topology selection.
2. The power line communication network topology control method according to claim 1, further comprising:
if the signal-to-noise ratio does not belong to the preset signal-to-noise ratio window range, calculating the communication success rate of the current link;
and judging the reliability of the link according to the communication success rate.
3. The topology control method for power line communication network according to claim 1, wherein said calculating the Q value of each node in the current link comprises:
in the formula: stIs the state of the node at time t, St∈S*，S*Representing a set of states, i.e. the current state StAll possible states of (a); a istFor the node's action at time t, atE.g. A, A represents action set; r is instant report at time t; gamma is a discount factor used for determining the relative proportion of delayed return and immediate return, and gamma is more than or equal to 0 and less than or equal to 1; alpha is a learning factor, 0<α≤1；Qt+1And QtQ values of the nodes at t +1 and t are respectively;
representing the maximum Q value at which the node takes action a at t + 1.
4. The method according to claim 1, wherein the selecting the action with the highest immediate report and the state at the next time corresponding to the action to update the Q value comprises:
select action a with the highest immediate returntAnd the state S of the next time corresponding to the actiont+1Updating the Q value:
5. the method for controlling topology of power line communication network according to claim 1, wherein said selecting the most reliable node for communication topology selection if Q values of all nodes converge comprises:
acquiring a first action with optimal instant return under the condition that the Q value of a node is converged, and acquiring an optimal neighbor node corresponding to the first action;
and selecting communication topology according to the optimal node corresponding to each node.
6. A power line communication network topology control apparatus, comprising:
the first calculating unit is used for calculating the signal-to-noise ratio of the current link;
the second calculation unit is used for calculating the Q value of each node in the current link when the signal-to-noise ratio belongs to the range of a preset signal-to-noise ratio window;
the first acquisition unit is used for acquiring the current time state and action of the node and observing the state of the next time corresponding to the execution of the current time action, wherein the action comprises the step of sending information to a neighbor node;
the updating unit is used for selecting the action with the highest instant report and the state of the next moment corresponding to the action, and updating the Q value;
the iteration unit is used for returning to the first acquisition unit when the updated Q value is not converged;
and the topology selection unit is used for selecting the most reliable node for communication topology selection when the Q values of all the nodes are converged.
7. The power line communication network topology control device according to claim 6, further comprising:
the third calculating unit is used for calculating the communication success rate of the current link when the signal-to-noise ratio does not belong to the range of the preset signal-to-noise ratio window;
and the judging unit is used for judging the reliability of the link according to the communication success rate.
8. The topology control device of power line communication network according to claim 6, wherein the topology selection unit comprises:
the second acquisition unit is used for acquiring a first action with optimal instant return under the condition that the Q value of the node is converged and acquiring an optimal neighbor node corresponding to the first action;
and the second topology selection unit is used for carrying out communication topology selection according to the optimal node corresponding to each node.
9. A power line communication network topology control device, the device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the power line communication network topology control method according to any one of claims 1 to 5 according to instructions in the program code.
10. A computer-readable storage medium characterized in that the computer-readable storage medium is configured to store a program code for executing the power line communication network topology control method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011603518.6A CN112714064A (en) | 2020-12-29 | 2020-12-29 | Power line communication network topology control method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011603518.6A CN112714064A (en) | 2020-12-29 | 2020-12-29 | Power line communication network topology control method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112714064A true CN112714064A (en) | 2021-04-27 |
Family
ID=75546989
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011603518.6A Pending CN112714064A (en) | 2020-12-29 | 2020-12-29 | Power line communication network topology control method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112714064A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114449374A (en) * | 2022-01-25 | 2022-05-06 | 国网浙江省电力有限公司双创中心 | High-density distributed power supply communication method, device and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019211134A1 (en) * | 2018-05-02 | 2019-11-07 | Telefonaktiebolaget Lm Ericsson (Publ) | First network node, third network node, and methods performed thereby, for handling a performance of a radio access network |
| CN110601973A (en) * | 2019-08-26 | 2019-12-20 | 中移（杭州）信息技术有限公司 | Route planning method, system, server and storage medium |
| CN110719201A (en) * | 2019-10-12 | 2020-01-21 | 西安邮电大学 | Distributed self-adaptive stable topology generation method based on reinforcement learning |
| CN110753384A (en) * | 2019-10-12 | 2020-02-04 | 西安邮电大学 | Distributed reinforcement learning stable topology generation method based on self-adaptive boundary |
| US20200113017A1 (en) * | 2018-10-05 | 2020-04-09 | Airspan Networks Inc. | Apparatus and method for configuring a communication link |
| CN111065145A (en) * | 2020-01-13 | 2020-04-24 | 清华大学 | Q learning ant colony routing method for underwater multi-agent |
| CN111479306A (en) * | 2020-04-02 | 2020-07-31 | 中国科学院上海微系统与信息技术研究所 | Q-learning-based QoS (quality of service) routing method for self-organizing network |
-
2020
- 2020-12-29 CN CN202011603518.6A patent/CN112714064A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019211134A1 (en) * | 2018-05-02 | 2019-11-07 | Telefonaktiebolaget Lm Ericsson (Publ) | First network node, third network node, and methods performed thereby, for handling a performance of a radio access network |
| US20200113017A1 (en) * | 2018-10-05 | 2020-04-09 | Airspan Networks Inc. | Apparatus and method for configuring a communication link |
| CN110601973A (en) * | 2019-08-26 | 2019-12-20 | 中移（杭州）信息技术有限公司 | Route planning method, system, server and storage medium |
| CN110719201A (en) * | 2019-10-12 | 2020-01-21 | 西安邮电大学 | Distributed self-adaptive stable topology generation method based on reinforcement learning |
| CN110753384A (en) * | 2019-10-12 | 2020-02-04 | 西安邮电大学 | Distributed reinforcement learning stable topology generation method based on self-adaptive boundary |
| CN111065145A (en) * | 2020-01-13 | 2020-04-24 | 清华大学 | Q learning ant colony routing method for underwater multi-agent |
| CN111479306A (en) * | 2020-04-02 | 2020-07-31 | 中国科学院上海微系统与信息技术研究所 | Q-learning-based QoS (quality of service) routing method for self-organizing network |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114449374A (en) * | 2022-01-25 | 2022-05-06 | 国网浙江省电力有限公司双创中心 | High-density distributed power supply communication method, device and medium |
| CN114449374B (en) * | 2022-01-25 | 2023-06-20 | 国网浙江省电力有限公司双创中心 | High-density distributed power supply communication method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hasan et al. | Optimizing multipath routing with guaranteed fault tolerance in Internet of Things | |
| US9191304B1 (en) | Reinforcement learning-based distributed network routing method utilizing integrated tracking and selective sweeping | |
| JP5021769B2 (en) | Radio and bandwidth aware routing metrics for multi-radio, multi-channel and multi-hop wireless networks | |
| Stojmenovic | Localized network layer protocols in wireless sensor networks based on optimizing cost over progress ratio | |
| US8582470B2 (en) | Arrangement and method relating to network management | |
| US20020186665A1 (en) | Efficient path learning in network | |
| CN105591937B (en) | A kind of acquisition method and equipment of network topological information | |
| Li et al. | A multi-agent reinforcement learning routing protocol for underwater optical sensor networks | |
| CN108174412A (en) | A kind of RPL multi-path data transmission mechanisms of load balancing | |
| Pinyoanuntapong et al. | Delay-optimal traffic engineering through multi-agent reinforcement learning | |
| Shiang et al. | Online learning in autonomic multi-hop wireless networks for transmitting mission-critical applications | |
| CN105636062A (en) | Cognitive radio network transmission learning method for moderate business services | |
| Arroyo-Valles et al. | A censoring strategy for decentralized estimation in energy-constrained adaptive diffusion networks | |
| CN110932969B (en) | Advanced metering system AMI network anti-interference attack routing algorithm for smart grid | |
| Hu et al. | An adaptive routing protocol based on connectivity prediction for underwater disruption tolerant networks | |
| US11102148B2 (en) | Method to generate stable and reliable 6TiSCH routing paths | |
| CN112714064A (en) | Power line communication network topology control method, device, equipment and medium | |
| Le et al. | An energy-efficient topology control algorithm based on reinforcement learning for wireless sensor networks | |
| WO2012137501A1 (en) | Control server, network control method, and program | |
| Ashour et al. | A survey of applying reinforcement learning techniques to multicast routing | |
| CN116155805A (en) | Distributed intelligent routing method, system, electronic equipment and storage medium | |
| Di Caro et al. | Theory and Practice of Ant-Based Routing in Dynamic Telecommunication Networks | |
| Regilan et al. | A Comprehensive Study on Routing Protocol for IoT-enabled WSNs using Deep Reinforcement Learning Strategy | |
| Lemercier et al. | A new objective function for hybrid network in the smart grid | |
| Patel et al. | Energy aware and link quality based routing in wireless sensor networks under TinyOS-2. x |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210427 |