CN113747384B - Energy Sustainability Decision-Making Mechanism for Industrial Internet of Things Based on Deep Reinforcement Learning - Google Patents

Abstract

The invention discloses a decision-making mechanism for energy sustainability of the industrial Internet of Things based on deep reinforcement learning, including: establishing a sensor wireless local area network; establishing a state transition model based on a Markov chain according to the data transmission collision probability of the sensor, and obtaining the data transmission of the sensor Probability; establish an energy consumption model based on the sensor's data receiving power; establish a throughput optimization model based on the sensor's throughput; optimize the sensor wireless local area network according to the sensor's data transmission probability, energy consumption model and throughput optimization model, and get the energy available Persistent Networks; Competitive windows for smart sensors through energy sustainable network output. The embodiment of the present invention improves the throughput of the system through the throughput optimization model, and can be widely used in the technical field of the Internet of Things.

Description

Energy sustainability decision-making mechanism for industrial Internet of Things based on deep reinforcement learning

Technical Field

The present invention relates to the technical field of Internet of Things, and in particular to an energy sustainability decision-making mechanism for industrial Internet of Things based on deep reinforcement learning.

Background Art

A large number of sensors are deployed in the Industrial Internet of Things. They form a wireless sensor network through the IEEE 802.11ax protocol to monitor the data of various smart devices in real time. Most of these sensors are powered by batteries and deployed in difficult-to-reach locations. Some sensors have a certain degree of mobility, so it is unrealistic to replace the batteries for these sensors. Such sensors can obtain energy from the charging base or the external environment through wireless charging or energy harvesting. Since the energy of a single sensor is limited, we can optimize the energy consumption of the sensor by controlling the frequency of sending data to maximize the transmission throughput in the local area network.

Considering the possible conflicts that may occur during the transmission process, the IEEE 802.11ax protocol often uses the traditional binary backoff algorithm. When a conflict occurs, the user will randomly wait for a period of time before resending. The selection of this random time is related to the contention window (CW) value. A large CW can avoid conflicts, but it will delay the data transmission time and reduce the throughput; a small CW allows users to quickly resend data, but it will increase the probability of conflicts.

In summary, how to adjust the sensor conflict window size to maximize the transmission throughput in the local area network is a technical problem that technicians in this field need to solve.

Summary of the invention

In view of this, an embodiment of the present invention provides an industrial Internet of Things energy sustainability decision-making mechanism based on deep reinforcement learning to improve the transmission throughput of the system.

On the one hand, the present invention provides an industrial Internet of Things energy sustainability decision-making mechanism based on deep reinforcement learning, including:

Establishing a sensor wireless local area network, wherein the sensor wireless local area network includes a gateway and a sensor module, and the sensor module includes a common sensor and an intelligent sensor;

Establishing a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module;

Establishing an energy consumption model according to the data receiving power of the sensor module;

Establishing a throughput optimization model according to the throughput of the sensor module, wherein the throughput is used to characterize the amount of data packets sent by the sensor module within a certain period of time;

Optimizing the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network;

The contention window of the smart sensor is obtained through the energy sustainable network output.

Optionally, establishing a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module includes:

Determine the probability of data transmission collision that occurs when the sensor module performs data transmission in the sensor wireless local area network;

Combining the data transmission collision probability and the discrete-time Markov chain to simulate the data transmission collision process of the sensor module, and determining a state transition model based on the Markov chain;

The state transition model of the Markov chain is subjected to normalization condition processing to obtain the data transmission probability of the sensor module.

Optionally, establishing an energy consumption model according to the data receiving power of the sensor module includes:

Calculate the signal-to-noise ratio threshold of the sensor module according to the received power of the sensor module;

Calculating the energy consumption required for successful transmission of the sensor module according to the signal-to-noise ratio threshold;

The energy consumption model is established according to the energy consumption.

Optionally, the optimizing the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainability system includes:

Building an optimized neural network for the smart sensor;

Calculating the remaining energy of the smart sensor according to the energy consumption model;

The data transmission probability and the remaining energy are input into the optimization neural network to obtain an energy sustainable network.

Optionally, obtaining the contention window of the smart sensor through the energy sustainable network output includes:

Initializing the energy sustainable network and determining an initialization system;

Randomly generating an initial contention window for the smart sensors in the initialization system;

The initial contention window is updated with an environmental reward, and a target contention window is output.

Optionally, establishing a throughput optimization model according to the throughput of the sensor module includes:

The optimization model is:

st C1: n _dead ≤ 0,

Among them, d represents distance, t represents time, α _t represents discount factor, η _t represents throughput of the smart sensor, n _dead represents the number of energy interruptions of the smart sensor, CE _min represents the minimum replenishment energy of the sensor module, CE _j represents the replenishment energy of the j-th sensor in the sensor module, CE _max represents the maximum replenishment energy of the sensor module, d _min represents the minimum distance between the sensor module and the gateway, d _j represents the distance between the j-th sensor in the sensor module and the gateway, d _max represents the maximum distance between the sensor module and the gateway, j represents a variable, and n represents the number of sensors in the sensor module.

Optionally, updating the initial contention window with an environmental reward and outputting a target contention window includes:

The environmental reward r ^t is expressed as:

Wherein, η _t represents the throughput of the smart sensor, and n _dead represents the number of energy interruptions of the smart sensor.

On the other hand, an embodiment of the present invention further discloses an industrial Internet of Things energy sustainability decision system based on deep reinforcement learning, including:

The first unit is used to establish a sensor wireless local area network, wherein the sensor wireless local area network includes a gateway and a sensor module, and the sensor module includes a common sensor and an intelligent sensor;

The second unit is used to establish a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module;

A third unit is used to establish an energy consumption model according to the data receiving power of the sensor module;

A fourth unit is used to establish a throughput optimization model according to the throughput of the sensor module, wherein the throughput is used to characterize the amount of data packets sent by the sensor module within a certain period of time;

A fifth unit, configured to optimize the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network;

The sixth unit is configured to obtain the contention window of the smart sensor through the energy sustainable network output.

On the other hand, an embodiment of the present invention further discloses an electronic device, including a processor and a memory;

The memory is used to store programs;

The processor executes the program to implement the method described above.

On the other hand, an embodiment of the present invention further discloses a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

On the other hand, an embodiment of the present invention further discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects: the present invention establishes a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module; establishes an energy consumption model according to the data receiving power of the sensor module; establishes a throughput optimization model according to the throughput of the sensor module; optimizes the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network; and can improve the throughput of the intelligent sensor without energy interruption.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a specific flow chart of an embodiment of the present invention;

FIG. 2 is a topology diagram of a wireless local area network of sensors according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

The present invention implements an industrial Internet of Things energy sustainability decision-making mechanism based on deep reinforcement learning, including:

S1. Establishing a sensor wireless local area network, wherein the sensor wireless local area network includes a gateway and a sensor module, and the sensor module includes a common sensor and an intelligent sensor;

S2. Establishing a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module;

S3, establishing an energy consumption model according to the data receiving power of the sensor module;

S4, establishing a throughput optimization model according to the throughput of the sensor module, wherein the throughput is used to characterize the amount of data packets sent by the sensor module within a certain period of time;

S5, optimizing the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network;

S6. Obtaining a contention window of the smart sensor through the energy sustainable network output.

Referring to FIG2 , a sensor wireless local area network is established by a gateway 1 and multiple wireless sensors connected thereto under the IEEE 802.11ax protocol. The wireless sensors include a smart sensor 3 and multiple ordinary sensors 2. The signal transmission of the wireless sensors obtains electrical energy from the surrounding environment to supplement energy. All sensors can only communicate through the gateway 1. They cannot communicate directly with each other, and only one data packet is processed at a time. There is a time-varying wireless channel between the gateway 1 and all sensors. The channel coefficient is expressed as h={h _i |i∈n}, where the channel coefficient between the gateway 1 and the smart sensor 3 in the tth period is expressed as h _n,t , and the channel is constant in each period of length T. In addition, b(t) (0≤b(t)≤m) is used to represent the backoff number at time t, where m is the maximum backoff number minus one; s(t) is used to represent the random process of the backoff stage (0, 1, ..., m) of the sensor at a certain time t. Ordinary sensor 2 updates the size of its competition window in a random manner, while smart sensor 3 dynamically selects the best competition window through deep reinforcement learning and interaction with the environment.

As a further preferred implementation, in the above step S2, a state transition model based on a Markov chain is established according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module, including:

Determine the probability of data transmission collision that occurs when the sensor module performs data transmission in the sensor wireless local area network:

Combining the data transmission collision probability and the discrete-time Markov chain to simulate the data transmission collision process of the sensor module, and determining a state transition model based on the Markov chain;

The state transition model of the Markov chain is subjected to normalization condition processing to obtain the data transmission probability of the sensor module.

Among them, at the beginning of each stage of data transmission in the sensor wireless network, the sensor randomly selects P within a certain range as the data transmission collision probability, which is used to characterize the collision probability of data packets transmitted on the channel. In each data transmission attempt, the data packets sent from the sensor collide with a constant and independent probability P. The discrete time Markov chain is used to simulate the two-dimensional process {s(t), b(t)}. In this Markov chain, the non-zero one-step transition probability P can be expressed as follows:

In the formula, i is the variable s(t), k is the variable b(t), m is the maximum backoff number minus one, _W0 is the initialization competition window of the smart sensor, _Wi is the competition window of the i-th ordinary sensor, and _Wm is the competition window of the smart sensor.

When the system tends to be stable, use bi _{, k} = lim _t→∞ P{s(t) = i, b(t) = k} k∈[0, _Wi -1], i∈[0, m] to represent the stationary distribution of the Markov chain. The closed solution of the Markov chain is shown as follows:

b _i-1, 0 · p = b _{i, 0} → b _{i, 0} = p ⁱ b _0, 0 0＜i＜m;

In the formula, bi _-1,0 means {s(t)=i-1, b(t)=0}, p means the conditional collision probability, bi _,0 means {s(t)=i, b(t)=0}, bm _-1,0 means {s(t)=m-1, b(t)=0}, bm _,0 means {s(t)=m, b(t)=0}, _b0,0 means {s(t)=0, b(t)=0}, i means a variable, and m means the maximum backoff number minus one.

Then we can get:

Simplifying, we get:

Applying normalization conditions to the Markov chain, it can be simplified as follows:

Right now:

Let τ represent the probability of a sensor transmitting in a randomly selected time slot. When the backoff time counter is equal to zero, any transmission occurs. The data transmission probability τ of the sensor is:

As a further preferred implementation, in the above step S3, establishing an energy consumption model according to the data receiving power of the sensor module includes:

Calculate the signal-to-noise ratio threshold of the sensor module according to the received power of the sensor module;

Calculating the energy consumption required for successful transmission of the sensor module according to the signal-to-noise ratio threshold;

The energy consumption model is established according to the energy consumption.

Among them, the sensor stores the collected energy in the supercapacitor for data transmission, and the energy capacity of the supercapacitor is expressed as C _max . Considering the general energy harvesting model, we assume that in time period t, the sensor replenishes the energy of the supercapacitor through energy recovery as CE _t joules/millisecond. Due to the mobility of the sensor and the changes in the transmission environment, the energy collected by the sensor in each stage is time-varying. In addition, we assume that there is additive Gaussian white noise with zero mean and equal variance σ ² at the receiving end of all devices. In order to ensure that the signal sent by the sensor can be successfully captured by the gateway, the signal-to-noise ratio (SNR) received at the gateway should be greater than the capture threshold. Therefore, under the condition that the probability of sensor data transmission is τ, the minimum received power is P ₀ , and the minimum threshold of the signal-to-noise ratio received at the gateway is:

In the formula, h is the channel coefficient, _P0 is the minimum received power, and ^σ2 is Gaussian white noise.

Therefore, within the time period t, the minimum energy _E0 consumed by the sensor for successful transmission is:

Among them, ζ is the minimum threshold of signal-to-noise ratio, σ ² is Gaussian white noise, ΔT represents the time required for data packet transmission, and h _t is the channel coefficient in time period t.

为：In order to achieve sustainable use of energy, the energy consumption of all sensors during uplink transmission is E ₀ . Assuming that in a period of t, the sensor transmits a total of z _t data packets, then after the end of the period of t, the energy consumption model

for:

In the formula, _Et is the charge of the supercapacitor before the start of period t, _zt is the number of data packets sent in period t, _E0 is the minimum energy consumption, _CEt is the energy replenished to the capacitor by energy recovery in period t, and T is the time interval of period t.

As a further preferred implementation, in the above step S5, the sensor wireless local area network is optimized according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainability system, including:

Building an optimized neural network for the smart sensor;

Calculating the remaining energy of the smart sensor according to the energy consumption model;

The data transmission probability and the remaining energy are input into the optimization neural network to obtain an energy sustainable network.

Among them, four neural networks are built for smart sensors, namely, the execution strategy network for selecting the contention window value; the execution evaluation network for evaluating the contention window value; the target strategy network for stabilizing the training and providing the contention window value for the update of the execution value network; and the target evaluation network for providing the next value for the update of the execution evaluation network. Under the condition of the data transmission probability being determined, the remaining energy of the smart sensor is calculated through the energy consumption model, and the channel state information, distance movement, energy recovery, and remaining energy are input into the execution strategy network as observations to obtain an optimized wireless local area network, namely, an energy sustainable network.

As a further preferred implementation, in the above step S6, obtaining the contention window of the smart sensor through the energy sustainable network output includes:

Initializing the energy sustainable network and determining an initialization system;

Randomly generating an initial contention window for the smart sensors in the initialization system;

The initial contention window is updated with an environmental reward, and a target contention window is output.

Among them, the experience return pool, the contention window of ordinary sensors, and the systematic environment in the energy sustainable network are initialized. Ordinary sensors independently and randomly set the initial contention window size within a certain range, and randomly generate the contention window value of the smart sensors in the initialization system. According to the energy consumption model, it can be obtained that when the energy consumption of all sensors uplinks a single data packet is E ₀ , energy can be used sustainably. At this time, all sensors send data packets according to the SEH-CSMA/CA protocol. After a certain period of time, the number of successful data packets transmitted by the smart sensor and the energy interruption are counted, and the throughput size of this stage is calculated. According to the reward return formula, the current reward r is calculated. At the same time, the smart sensor uses the mean square error to calculate the loss function of the execution evaluation network, and then updates the execution evaluation network by reverse gradient transfer; at the same time, the state of all sensors at the current moment and the contention window value of the smart sensor are input into the execution evaluation network to obtain the state-action value, and the value is used to update the execution strategy network by reverse gradient transfer. In addition, the target strategy network and the target evaluation network are gradually updated by copying a certain proportion at each step. By repeatedly iterating the above process, the smart sensor will make adaptive adjustments according to changes in dynamic conditions to obtain the competition window size of the smart sensor.

As a further preferred implementation, a throughput optimization model is established according to the throughput of the sensor module, including:

The optimization model is:

st C1: n _dead ≤ 0,

Among them, d represents distance, t represents time, α _t represents discount factor, η _t represents throughput of the smart sensor, n _dead represents the number of energy interruptions of the smart sensor, CE _min represents the minimum replenishment energy of the sensor module, CE _j represents the replenishment energy of the j-th sensor in the sensor module, CE _max represents the maximum replenishment energy of the sensor module, d _min represents the minimum distance between the sensor module and the gateway, d _j represents the distance between the j-th sensor in the sensor module and the gateway, d _max represents the maximum distance between the sensor module and the gateway, j represents a variable, and n represents the number of sensors in the sensor module.

As a further preferred implementation, the initial contention window is updated with an environmental reward, and a target contention window is output, including:

The environmental reward r ^t is expressed as:

Wherein, η _t represents the throughput of the smart sensor, and n _dead represents the number of energy interruptions of the smart sensor.

In conjunction with Figure 1, the process of the present invention specifically includes:

A sensor wireless local area network consisting of a gateway, common sensors and smart sensors is established. The sensors transmit data in the wireless local area network. Data collision occurs during the transmission process. The probability of each sensor transmitting data in a time-varying environment is obtained through mathematical analysis based on the state transition model of the Markov chain. An energy consumption model is established under the condition that the data receiving power of the sensor is determined to be P ₀ , and a throughput optimization model is established based on the throughput of the sensor. Under the condition that the data transmission probability of the sensor is determined, the sensor wireless local area network is optimized according to the energy consumption model and the throughput optimization model to obtain an energy sustainable network. The energy sustainable network is calculated and updated to obtain the contention window of the smart sensor.

The embodiment of the present invention also provides an industrial Internet of Things energy sustainability decision system based on deep reinforcement learning, including:

The first unit is used to establish a sensor wireless local area network, wherein the sensor wireless local area network includes a gateway and a sensor module, and the sensor module includes a common sensor and an intelligent sensor;

The second unit is used to establish a state transition model based on a Markov chain according to the data transmission collision probability of the sensor module to obtain the data transmission probability of the sensor module;

A third unit is used to establish an energy consumption model according to the data receiving power of the sensor module;

A fourth unit is used to establish a throughput optimization model according to the throughput of the sensor module, wherein the throughput is used to characterize the amount of data packets sent by the sensor module within a certain period of time;

A fifth unit, configured to optimize the sensor wireless local area network according to the data transmission probability of the sensor module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network;

The sixth unit is configured to obtain the contention window of the smart sensor through the energy sustainable network output.

Corresponding to the method of FIG. 1 , an embodiment of the present invention further provides an electronic device, including a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method described above.

Corresponding to the method of FIG. 1 , an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method shown in FIG1.

In summary, the embodiments of the present invention have the following advantages:

(1) The embodiment of the present invention can improve the accuracy of the system by analyzing the data transmission probability of each sensor based on the state transition model of the Markov chain.

(2) The embodiment of the present invention optimizes the sensor wireless local area network through a throughput optimization model, thereby improving the throughput of the system.

In some selectable embodiments, the function/operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function/operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and/or features described may be integrated into a single physical device and/or software module, or one or more functions and/or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims (7)

1. Deep reinforcement learning-based industrial internet of things energy sustainability decision method is characterized by comprising the following steps:

establishing a sensor wireless local area network, wherein the sensor wireless local area network comprises a gateway and a sensing module, and the sensing module comprises a common sensor and an intelligent sensor;

establishing a Markov chain-based state transition model according to the data transmission collision probability of the sensing module to obtain the data transmission probability of the sensing module;

establishing an energy consumption model according to the data receiving power of the sensing module;

establishing a throughput optimization model according to the throughput of the sensing module, wherein the throughput is used for representing the data volume of a data packet sent by the sensing module within a certain time;

optimizing the sensor wireless local area network according to the data transmission probability of the sensing module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network;

obtaining a competition window of the intelligent sensor through the energy sustainable network output;

wherein, the optimizing the sensor wireless local area network according to the data transmission probability of the sensing module, the energy consumption model and the throughput optimization model to obtain an energy sustainable network comprises:

building an optimized neural network for the intelligent sensor;

calculating to obtain the residual energy of the intelligent sensor according to the energy consumption model;

inputting the data transmission probability and the residual energy into the optimized neural network to obtain a sustainable energy network;

the obtaining of the contention window of the smart sensor through the energy sustainable network output includes:

initializing the energy sustainable network, and determining an initialization system;

randomly generating an initial competition window for the intelligent sensor in the initialization system;

updating the environment reward of the initial competition window and outputting a target competition window;

the updating of the environment reward to the initial competition window and the outputting of the target competition window comprise:

the environment award r ^t Expressed as:

wherein eta is _t Expressed as the throughput of the smart sensor, n _dead Expressed as the number of energy interruptions of the smart sensor.

2. The deep reinforcement learning-based industrial internet of things energy sustainability decision method as claimed in claim 1, wherein the establishing a state transition model based on a markov chain according to the data transmission collision probability of the sensing module to obtain the data transmission probability of the sensing module comprises:

determining the data transmission collision probability of the sensing module when the sensing module transmits data in the sensor wireless local area network;

simulating a data transmission collision process of the sensing module by combining the data transmission collision probability and the discrete time Markov chain, and determining a state transition model based on the Markov chain;

and carrying out normalization condition processing on the state transition model of the Markov chain to obtain the data transmission probability of the sensing module.

3. The deep reinforcement learning-based industrial internet of things energy sustainability decision method according to claim 1, wherein the building of the energy consumption model according to the data receiving power of the sensing module comprises:

calculating to obtain a signal-to-noise ratio threshold value of the sensing module according to the receiving power of the sensing module;

calculating energy consumption required by the sensing module for successful transmission according to the signal-to-noise ratio threshold;

and establishing the energy consumption model according to the energy consumption.

4. The deep reinforcement learning-based industrial internet of things energy sustainability decision method according to claim 1, wherein the establishing of a throughput optimization model according to the throughput of the sensing module comprises:

the optimization model is as follows:

s.t.C1:n _dead ≤0,

wherein d is distance, t is time, and alpha is _t Expressed as a discount factor, eta _t Expressed as the throughput of the smart sensor, n _dead Expressed as the number of energy interruptions, CE, of the smart sensor _min Expressed as minimum supplemental energy, CE, of the sensing module _j Expressed as the supplemental energy, CE, of the jth sensor in the sensing module _max Maximum supplementary energy for the sensing module, d _min Indicating the minimum distance between the sensing module and the gateway, d _j Represents the distance between the jth sensor in the sensing module and the gateway, d _max The maximum distance between the sensing module and the gateway is represented, j represents a variable, and n represents the number of sensors in the sensing module.

5. An industrial internet of things energy sustainability decision system based on deep reinforcement learning, comprising:

the system comprises a first unit and a second unit, wherein the first unit is used for establishing a sensor wireless local area network, the sensor wireless local area network comprises a gateway and a sensing module, and the sensing module comprises a common sensor and an intelligent sensor;

the second unit is used for establishing a Markov chain-based state transition model according to the data transmission collision probability of the sensing module to obtain the data transmission probability of the sensing module;

the third unit is used for establishing an energy consumption model according to the data receiving power of the sensing module;

a fourth unit, configured to establish a throughput optimization model according to throughput of the sensing module, where the throughput is used to characterize a data volume of a data packet sent by the sensing module within a certain time;

a fifth unit, configured to optimize the sensor wireless local area network according to the data transmission probability of the sensing module, the energy consumption model, and the throughput optimization model, so as to obtain a sustainable energy network;

a sixth unit, configured to obtain a contention window of the smart sensor through the energy sustainable network output;

the fifth unit is configured to optimize the sensor wireless local area network according to the data transmission probability of the sensing module, the energy consumption model, and the throughput optimization model to obtain an energy sustainable network, and includes:

building an optimized neural network for the intelligent sensor;

calculating to obtain the residual energy of the intelligent sensor according to the energy consumption model;

inputting the data transmission probability and the residual energy into the optimized neural network to obtain an energy sustainable network;

the sixth unit is configured to obtain a contention window of the smart sensor through the energy sustainable network output, and includes:

initializing the energy sustainable network, and determining an initialization system;

randomly generating an initial competition window for the intelligent sensor in the initialization system;

updating the environment reward of the initial competition window and outputting a target competition window;

the updating of the environment reward to the initial competition window and the outputting of the target competition window comprise:

the environment award r ^t Expressed as:

wherein eta _t Expressed as the throughput of the smart sensor, n _dead Expressed as the number of energy interruptions of the smart sensor.

6. An electronic device comprising a processor and a memory;

the memory is used for storing programs;

the processor executing the program realizes the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that the storage medium stores a program, which is executed by a processor to implement the method according to any one of claims 1-4.

Images