CN116581816A - Ship comprehensive energy system energy management strategy for dealing with unexpected situations - Google Patents
Ship comprehensive energy system energy management strategy for dealing with unexpected situations Download PDFInfo
- Publication number
- CN116581816A CN116581816A CN202310575994.9A CN202310575994A CN116581816A CN 116581816 A CN116581816 A CN 116581816A CN 202310575994 A CN202310575994 A CN 202310575994A CN 116581816 A CN116581816 A CN 116581816A
- Authority
- CN
- China
- Prior art keywords
- energy
- ship
- strategy
- network
- unexpected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 29
- 238000005070 sampling Methods 0.000 claims abstract description 16
- 238000011217 control strategy Methods 0.000 claims abstract description 11
- 238000010248 power generation Methods 0.000 claims description 57
- 238000003860 storage Methods 0.000 claims description 50
- UFHFLCQGNIYNRP-UHFFFAOYSA-N Hydrogen Chemical compound [H][H] UFHFLCQGNIYNRP-UHFFFAOYSA-N 0.000 claims description 45
- 239000001257 hydrogen Substances 0.000 claims description 45
- 229910052739 hydrogen Inorganic materials 0.000 claims description 45
- 238000000034 method Methods 0.000 claims description 41
- 238000004146 energy storage Methods 0.000 claims description 33
- 239000000446 fuel Substances 0.000 claims description 33
- 230000009471 action Effects 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 27
- 239000003795 chemical substances by application Substances 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 21
- 230000006399 behavior Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 13
- 230000004044 response Effects 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 230000003542 behavioural effect Effects 0.000 claims description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 3
- 238000012360 testing method Methods 0.000 abstract description 12
- 238000009826 distribution Methods 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 38
- 210000004027 cell Anatomy 0.000 description 24
- 230000008859 change Effects 0.000 description 12
- 230000002787 reinforcement Effects 0.000 description 11
- 125000003275 alpha amino acid group Chemical group 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- MWUXSHHQAYIFBG-UHFFFAOYSA-N Nitric oxide Chemical compound O=[N] MWUXSHHQAYIFBG-UHFFFAOYSA-N 0.000 description 6
- 238000013459 approach Methods 0.000 description 6
- 150000001875 compounds Chemical class 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 238000010521 absorption reaction Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 230000001965 increasing effect Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 239000012528 membrane Substances 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 229910002090 carbon oxide Inorganic materials 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000002283 diesel fuel Substances 0.000 description 2
- 238000007599 discharging Methods 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 239000003344 environmental pollutant Substances 0.000 description 2
- 239000007789 gas Substances 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 231100000719 pollutant Toxicity 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- XTQHKBHJIVJGKJ-UHFFFAOYSA-N sulfur monoxide Chemical compound S=O XTQHKBHJIVJGKJ-UHFFFAOYSA-N 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 1
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical class [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 1
- 101000604223 Homo sapiens Nocturnin Proteins 0.000 description 1
- 102100038815 Nocturnin Human genes 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 150000001413 amino acids Chemical class 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 229910052799 carbon Inorganic materials 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- TXKMVPPZCYKFAC-UHFFFAOYSA-N disulfur monoxide Inorganic materials O=S=S TXKMVPPZCYKFAC-UHFFFAOYSA-N 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005868 electrolysis reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 229910052757 nitrogen Inorganic materials 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 229910052815 sulfur oxide Inorganic materials 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
- H02J3/381—Dispersed generators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N7/00—Computing arrangements based on specific mathematical models
- G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J15/00—Systems for storing electric energy
- H02J15/008—Systems for storing electric energy using hydrogen as energy vector
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/04—Circuit arrangements for ac mains or ac distribution networks for connecting networks of the same frequency but supplied from different sources
- H02J3/06—Controlling transfer of power between connected networks; Controlling sharing of load between connected networks
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/28—Arrangements for balancing of the load in a network by storage of energy
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/28—Arrangements for balancing of the load in a network by storage of energy
- H02J3/32—Arrangements for balancing of the load in a network by storage of energy using batteries with converting means
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/36—Arrangements for transfer of electric power between ac networks via a high-tension dc link
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
- H02J3/388—Islanding, i.e. disconnection of local power supply from the network
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/10—The dispersed energy generation being of fossil origin, e.g. diesel generators
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/20—The dispersed energy generation being of renewable origin
- H02J2300/22—The renewable source being solar energy
- H02J2300/24—The renewable source being solar energy of photovoltaic origin
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/20—The dispersed energy generation being of renewable origin
- H02J2300/28—The renewable source being wind energy
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/30—The power source being a fuel cell
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2310/00—The network for supplying or distributing electric power characterised by its spatial reach or by the load
- H02J2310/40—The network being an on-board power network, i.e. within a vehicle
- H02J2310/42—The network being an on-board power network, i.e. within a vehicle for ships or vessels
Landscapes
- Engineering & Computer Science (AREA)
- Power Engineering (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Algebra (AREA)
- Probability & Statistics with Applications (AREA)
- Supply And Distribution Of Alternating Current (AREA)
Abstract
Aiming at the problem of energy management of a ship comprehensive energy system under unexpected conditions, the invention provides a ship comprehensive energy system energy management strategy for dealing with unexpected conditions, which is also a ship comprehensive energy system elasticity enhancement control strategy based on a depth deterministic strategy gradient algorithm of priority sampling. The strategy aims at providing energy optimized scheduling under normal operation conditions and elastic control under unexpected conditions for the ship comprehensive energy system. The simulated ship navigation test results of different scenes show that the strategy can fully utilize the operation characteristics of various devices, reasonable energy distribution can be realized by taking economy as a guide under a steady-state working condition, and safe operation of the system can be ensured to the maximum extent by taking reliability as a guide under an unexpected working condition.
Description
Technical Field
The invention belongs to the technical field of new energy ship energy management, and particularly relates to a ship comprehensive energy system energy management strategy for dealing with unexpected situations.
Background
With the increasing of the duty ratio of new energy power generation equipment in a ship power system, the reliability and vulnerability problems of the ship comprehensive energy system are increasingly serious. Although marine integrated energy systems are designed with some resistance to potential component failure (commonly referred to as N-1 safety principles), many uncertainty factors may cause more serious damage to the marine power system, which exceeds the self-repair capability of the system. Although a certain risk resistance is considered in the design of the ship, unexpected events such as severe working environment, extreme weather events, improper operation of personnel and the like are difficult to avoid, and the power system of the new energy ship is seriously damaged and exceeds the self-repairing capability of the system. Therefore, measures must be taken to ensure stable operation of the ship's integrated energy system under unexpected conditions.
The energy management strategy of a power system in unexpected situations can be summarized in two aspects: redundancy design and system resiliency enhancement control. The redundancy design aims to improve the overall safety level of the ship comprehensive energy system so as to reduce the systemThe possibility that various devices are affected by extreme events is unified. However, a redundancy design may only be valid for a certain burst scenario, and thus the method is poorly applicable. In contrast, elasticity-enhanced control strategy for power systems [1] The system aims to improve the adaptability of the system to uncertainty and dynamics so as to better meet the power requirements in different scenes, and the system is attracting more and more attention of researchers.
In the existing studies, literature: the design of the automatic regulation and control system for the ship power load in extreme weather is provided, so that the detection efficiency of the load power demand is improved, and the normal operation of the ship power system in extreme weather is ensured. The literature Adecision-making approach for the health-aware energy management ofship hybridpower plants [ J ]. Reliability Engineering & System Safety,2023,235:109263. An energy management method with health awareness has been developed based on equivalent minimization strategies and dynamic Bayesian networks that combines System health monitoring information with energy management to circumvent operational control intervals that may reduce component and System reliability. The document Enhancing the transmission grid resilience in ice storms by optimal coordination of power system schedule withpre-positioning and routing ofmobile DC De-imaging devices [ J ]. IEEE Transactions on Power Systems,2019,34 (4): 2663-2674. Proposes a two-stage energy optimized scheduling method based on Mixed Integer Linear Programming (MILP). The method enhances the elasticity of the power system by changing the network topology structure so as to cope with the power demand of the ice storm weather on the key load of the micro-grid. The patent documents Microgrid resilience-oriented scheduling: A robust misocp model [ J ]. IEEE Transactions on Smart Grid,2021,12 (3): 1867-1879. The problem of energy optimization scheduling of a micro-grid operated in a island mode during power failure is studied, and uncertainty of the micro-grid is modeled by adopting a mixed integer second order cone robust optimization method, so that elastic redundancy and operation performance of the micro-grid are balanced, control accuracy is improved, load rejection rate is reduced, and operation stability of the micro-grid is improved. The document Power system resilience enhancement in typhoons using a three-stage day-aheadunit commitment [ J ]. IEEE Transactions on Smart Grid,2021,12 (3): 2153-2164. A three-order robust combination model consisting of preventive control, emergency control and recovery is proposed taking into account the uncertainty of typhoon path and line faults. The fault characteristics of asymmetric operation are analyzed in detail by a document modularized multi-level converter submodule fault characteristic analysis and decoupling control strategy, a mathematical model of an unbalanced condition is established, an asymmetric component is counteracted by compensating impedance, and a decoupling control strategy is provided.
While the above work has been shown to be effective in enhancing the resiliency of electrical power systems, these methods require the creation of accurate system models. In addition, the algorithm-based strategy can effectively improve the calculation performance of the algorithm, and is particularly important in an energy system running in an island mode. Therefore, data-driven approaches that do not rely on accurate system parameters are of increasing interest.
Reinforcement learning is a model-free data driving method, and effectively avoids dependence on models. The unmanned ship hybrid power system energy management strategy based on DRL is researched by taking a light diesel storage hybrid unmanned ship as a research object, and the unmanned ship hybrid power system energy management strategy based on deep reinforcement learning is provided, has certain self-learning capability, can effectively cope with emergency conditions faced by the unmanned ship, and promotes the development of unmanned ship technology. The document Model-free real-time autonomous control for a residential multi-energy system using deep reinforcement learning [ J ]. IEEE Transactions on Smart Grid,2020,11 (4): 3068-3082. An energy optimized scheduling method based on an improved DDPG algorithm is proposed, and the training process is accelerated by using the priority sampling, so that the response speed of the strategy is improved. Therefore, the reinforcement learning avoids the accurate modeling of the energy system, and effectively improves the calculation performance of the control strategy, so that the reinforcement learning has great potential in the aspects of real-time system control and decision. In addition, researchers use reinforcement learning methods to improve the anti-interference capability of the system. Adeep reinforcement learning-basedmuli-agent frameworkto enhance power system resilience using shunt resources [ J ]. IEEE Transactions on Power Systems,2021,36 (6): 5525-5536. Deep reinforcement learning is used to program control sequences for shunts to enhance the resistance of power systems to multiple line faults. Document Optimizing the post-disaster control ofislandedmicrogrid Amulti-agent deep reinforcement learning approach [ J ]. IEEEAccess,2020,8:153455-153469. Multi-agent elasticity enhancement control strategies were investigated when the microgrid was operated in island mode. Therefore, the reinforcement learning method has wide application prospect in improving the performance and the anti-interference capability of the comprehensive energy system.
In summary, most of the existing researches are based on a definite model, lack the capability of capturing uncertainty, and are relatively limited in research on the comprehensive energy system of the offshore mobile ship. In consideration of the complexity and the dynamics of the offshore environment, the operation condition of the ship comprehensive energy system has obvious uncertainty. Therefore, it is necessary to develop an elasticity enhancement control strategy of the ship comprehensive energy system based on data driving so as to cope with unexpected situations such as extreme weather at sea, system equipment faults and the like and improve the reliability and stability of the ship comprehensive energy system.
Disclosure of Invention
Aiming at the problem of energy management of a ship comprehensive energy system under unexpected conditions, the invention provides a ship comprehensive energy system energy management strategy for dealing with unexpected conditions, which is also a ship comprehensive energy system elasticity enhancement control strategy based on a depth deterministic strategy gradient algorithm of priority sampling. The strategy aims at providing energy optimized scheduling under normal operation conditions and elastic control under unexpected conditions for the ship comprehensive energy system. The simulated ship navigation test results of different scenes show that the strategy can fully utilize the operation characteristics of various devices, reasonable energy distribution can be realized by taking economy as a guide under a steady-state working condition, and safe operation of the system can be ensured to the maximum extent by taking reliability as a guide under an unexpected working condition.
The invention adopts the following technical scheme:
an energy management strategy of a ship integrated energy system for dealing with unexpected situations, which is characterized in that: based on a marine integrated energy system comprising a distributed power generation unit, a load and an energy storage system, the distributed power generation unit comprising: a photovoltaic power generation unit, a wind power generation unit, a fuel cell and a diesel generator;
modeling the energy management problem under unexpected working conditions as a Markov decision process, and based on an elasticity enhancement control strategy of a PDDPG algorithm, by capturing uncertainty in the operation process of the system, the energy management problem can be used for providing reliable power supply for necessary loads by an assistance system during unexpected events, and under normal conditions, near-optimal control can be realized with the aim of minimum operation cost.
Further, the new energy ship comprehensive energy system operates in an island mode, wherein a wind power generation module and a photovoltaic power generation module are used as main power supply units, a fuel cell and a diesel engine are used as auxiliary power supply units, and an energy storage module and a hydrogen storage module are used as energy storage units; when wind power generation and photovoltaic power generation cannot meet load requirements, the energy storage unit is used for making up for the shortage of electric power; when the output power of the clean energy source is higher than the load demand, the redundant energy is stored in a storage battery or used for electrolyzing water to produce hydrogen for subsequent use; during periods of low power, energy is stored for meeting load demands.
Further, the implementation flow of the PDDPG algorithm in the invention is as follows:
step S1: initializing a Q network of a commentator and a behavior strategy network of an actor, wherein network parameters are respectively theta and phi;
step S2: initializing a target Q network of a commentator and a target strategy network of an actor, wherein network parameters are respectively theta '=theta and phi' =phi;
step S3: randomly selecting voyage from the training data set, and obtaining the initial state s of the voyage ship comprehensive energy system 1 ；
Step S4: initializing random gaussian exploration noise N t ；
Step S5: will s t The optimal scheduling decision a is output as an input to the Actor behavior policy network μ (s|φ) t And increase the action exploration ability;
step S6: root of Chinese characterAccording to scheduling scheme a t The actions of each controllable device in the system are used for calculating the state s at the next moment on the premise that the operation conditions of each device are met t+1 And calculates a return value r t+1 ；
Step S9: will experience { s ] t ,a t ,r t ,s t+1 Store in experience playback pool SumTree;
step S10: sampling n samples from SumPree, calculating the probability of each sample, calculating absolute TD error, calculating importance sampling weight, and updating experience level according to TD error;
step S11: calculating a loss function and updating the Q network parameters of the commentators;
Step S12: updating actor behavior strategy network parameters;
step S13: updating parameters of two target networks;
step S14: if the voyage ending time is reached, returning to the step S4 if the voyage ending time is not reached;
step S15: and (3) whether the cycle termination condition is met, if not, returning to the step (S3), otherwise, terminating.
Further, in step S5, random gaussian white noise is added to the output of the behavior policy network μ to increase the action exploration ability.
Further, in step S10, the priority experience playback method is adopted, and the experience level is updated according to the TD error.
Further, in step S11, a loss function is calculated based on the rule of thumb playback mechanism by employing the target network and the priority.
Compared with the prior art, the invention and the preferable scheme thereof provide energy management and elastic control for the multi-energy ship power system. Firstly, different unexpected scenes of the ship comprehensive energy system are described, then the start-stop cost and the start-stop constraint of the controllable diesel generator are increased in consideration of the continuity of the control action, and an optimal scheduling model of the ship comprehensive energy system under the unexpected scenes is established based on a Markov decision process. Next, a ship comprehensive energy system energy management implementation flow based on the PDDPG algorithm is described. Finally, the running condition of the ship comprehensive energy system under the steady-state working condition and the adaptability to unexpected scenes are analyzed, and the method can capture the uncertainty of various types of equipment, and makes optimal energy scheduling for each energy supply unit of the ship when unexpected events (namely extreme events occurring outside or inside the ship comprehensive energy system) occur, so that the safe and reliable running of the ship is ensured.
Drawings
The invention is described in further detail below with reference to the attached drawings and detailed description:
FIG. 1 is a schematic diagram of a ship integrated energy system according to an embodiment of the present invention.
FIG. 2 is a diagram of an Actor-Critic framework in accordance with an embodiment of the present invention.
Fig. 3 illustrates an energy management strategy diagram based on PDDPG in accordance with an embodiment of the present invention.
FIG. 4 is a graph showing the output of each power supply under steady-state conditions in accordance with an embodiment of the present invention.
Fig. 5 is a graph showing the state of charge of the storage battery and the charge/discharge power change of the storage battery according to the embodiment of the present invention.
FIG. 6 is a graph showing changes in the amount of hydrogen in the hydrogen storage tank, the output power of the fuel cell, and the absorption power of the electrolyzer according to the embodiment of the invention.
FIG. 7 is a graph of the power output of a clean energy power generation unit during an airliner process in accordance with an embodiment of the invention.
FIG. 8 is an energy optimized schedule for scenario one of the present invention.
FIG. 9 is an energy optimized schedule for scenario two of the present invention.
Fig. 10 is a diagram illustrating a state of charge and a power change of a battery in a second unexpected scenario according to an embodiment of the present invention.
Fig. 11 is a graph showing changes in the amount of hydrogen in the hydrogen storage tank, the output power of the fuel cell, and the absorption power of the electrolyzer in a second unexpected scenario of the embodiment of the invention.
FIG. 12 is an energy optimized schedule for scenario three of the present invention.
Fig. 13 is a diagram of the state of charge and power change of the battery in a third scenario, which is not contemplated by the present embodiment.
Fig. 14 is a graph showing changes in the amount of hydrogen in the hydrogen storage tank, the output power of the fuel cell, and the absorption power of the electrolyzer in a third unexpected scenario of the embodiment of the present application.
Detailed Description
In order to make the features and advantages of the present patent more comprehensible, embodiments accompanied with figures are described in detail below:
it should be noted that the following detailed description is illustrative and is intended to provide further explanation of the application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
1 theoretical model of comprehensive energy system of ship
The basic framework of the ship comprehensive energy system studied by the invention is shown in fig. 1, and comprises a distributed power generation unit (photovoltaic power generation unit, wind power generation unit, fuel cell and diesel generator), related loads and an energy storage system.
The new energy ship comprehensive energy system operates in an island mode, wherein a wind power generation module and a photovoltaic power generation module are used as main power supply units, a fuel cell and a diesel engine are used as auxiliary power supply units, and an energy storage module and a hydrogen storage module are used as energy storage units. When wind power generation and photovoltaic power generation cannot meet load requirements, the energy storage unit can make up for the shortage of electric power. Meanwhile, when the output power of the clean energy source is higher than the load demand, the redundant energy is stored in a storage battery or used for electrolyzing water to produce hydrogen for subsequent use. During periods of low power, the stored energy will be used to meet load demands, improving the reliability of the system and the sustainability of the energy source.
1.1 clean energy Power Generation Module
1. Photovoltaic power generation module
The output power of a single photovoltaic power generation panel at the time t is described by the formula (1):
in the formula (1), p pv (t) is the output power of a single photovoltaic power generation panel at the moment t; p (P) max Maximum output power for a single photovoltaic power generation panel; i (t) is the radiation intensity of the sun at the moment t; t (T) c (t) is the operating temperature of the photovoltaic module in the period t; i r A reference value of the radiation intensity of the photovoltaic module; t (T) c (t) is the power temperature coefficient; t (T) r Is the operating temperature of the photovoltaic module under standard test conditions.
The operating temperature at time t of a single photovoltaic power generation panel is described by formula (2):
in the formula (2), T e (t) is the ambient temperature at time t; NOCT is the rated operating temperature of the photovoltaic panel.
The total output power of the photovoltaic power generation unit at time t is described by formula (3):
P PV (t)＝N PV ×p PV (t) (3)
in the formula (3), P PV (t) the photovoltaic power generation unit outputs total power in a t period; n (N) PV Is the total number of photovoltaic power generation panels in the system.
2. Wind power generation module
The output power of the wind power generation module is mainly dependent on the wind speed at the height of the hub of the wind turbine. After the wind energy passes through the wind generating set, a part of the mechanical energy is converted into electric energy. The secondary power output model of the wind driven generator is described by formula (4):
in the formula (4), v t The wind speed at time t; v r Is the rated wind speed of the fan; p (P) r_WT The rated power of the fan corresponding to the rated wind speed is set; v min The starting wind speed of the fan; v max The wind speed of the fan is pre-warned. When the wind speed is smaller than the cutting-in speed v min And no electric energy is output. At the cut-in speed and rated wind speed (v min ～v r ) In between, the output power of the wind turbine is directly related to the cube of the wind speed. When the wind speed exceeds the rated value, the output power needs to be limited to a fixed value. When the wind speed is higher than the early warning speed v max When the wind power system is shut down, the system components are protected so that the generator and the corresponding power electronics are not damaged.
The total power output by the wind power generation module is described by formula (5):
P WT (t)＝N WT ×p WT (t) (5)
in the formula (5), P WT (t) outputting total power at the moment of t by the wind power generation module; n (N) WT Is the total number of fans in the system.
1.2 proton exchange Membrane Fuel cell
The output power of a proton exchange membrane fuel cell using hydrogen as the primary energy source is described by equation (6):
in the formula (6), the amino acid sequence of the compound,is the low calorific value of hydrogen> The hydrogen amount in the hydrogen storage tank at the time t; η (eta) T Is thermodynamic effectRate (0.98 at 25 ℃); u (U) f Is the fuel utilization rate; η (eta) FC Is the efficiency of the fuel cell; />Is the capacity of the hydrogen storage tank; />Is the minimum hydrogen storage amount in the hydrogen storage tank.
The consumption amount of hydrogen per unit time of the fuel cell is described by formulas (7) and (8):
F FC (t)＝a FC ×p FC (t)+b FC (7)
in the formulas (7) and (8), F FC (t) is the consumption of hydrogen at time t; a, a FC And b FC Is the linearization factor of the fuel cell stack when in operation;and the amount of hydrogen released by the hydrogen storage tank at the time t.
The efficiency of a proton exchange membrane fuel cell is described by formula (9):
in the formula (9), the amino acid sequence of the compound,high heating value (120-140 MJ/kg) of hydrogen; />Is the flow of hydrogen.
1.3 energy storage Module
As one of the controllable units of the new energy ship, the energy storage device needs to consider the constraint of power output, and meanwhile, the energy storage charge state is also influenced by the charge and discharge power at the previous moment, so that the energy storage device has the constraint of time sequence. The energy storage device needs to select proper charge and discharge time according to load and renewable energy resource shortage conditions so as to realize the optimization targets of economic operation and peak clipping and valley filling. Optimizing the energy storage charging and discharging strategy can achieve more efficient energy utilization and economical operation. Meanwhile, when the new energy is insufficient, the energy storage device can provide standby energy, so that safe and reliable operation of the ship is ensured. Therefore, the whole performance and the running efficiency of the new energy ship can be improved by fully utilizing the energy storage device.
1. Electrolysis tank and hydrogen storage tank
The relation between the amount of hydrogen produced in the electrolyzer and the required power is described by the formula (10):
in the formula (10), P E (t) is the power consumed by the electrolyzer at time t;the amount of hydrogen generated by the electrolyzer at the moment t; η (eta) E Is the efficiency of the electrolytic cell.
The amount of hydrogen in the hydrogen storage tank is described by formula (11):
in the formula (11), the amino acid sequence of the compound,and the residual hydrogen quantity in the hydrogen storage tank at the time t.
2. Storage battery
The power output of clean energy power generation modules such as photovoltaic power generation and wind power generation in the hybrid energy system changes along with the changes of environmental conditions such as illumination, wind speed and the like, and the power load has variability. Therefore, in order to improve the stability of the entire system, a battery having a high response speed is provided in the system. Maximum stored energy e of battery max_batt And the most of themSmall stored energy e min_batt The relationship of (2) is described by the formula (12):
e min_batt ＝(1-DOD)×e max_batt (12)
in the formula (12), DOD is the maximum depth of discharge of the battery; if the number of the storage batteries equipped in the meter system is N batt The method comprises the steps of carrying out a first treatment on the surface of the The maximum electric energy capacity of the storage battery pack is E max_batt ＝N batt ×e max_batt The method comprises the steps of carrying out a first treatment on the surface of the Minimum electric energy capacity E min_batt ＝N batt× e min_batt 。
The storage battery is generally used as an auxiliary power supply unit, when the sum of the electric energy generated by wind power generation and photovoltaic power generation is larger than the load demand, the storage battery is charged, and the electric energy change of the storage battery is described by a formula (13); if the total electric energy generated by wind power generation and photovoltaic power generation cannot meet the load requirement, the storage battery is discharged, and the electric energy change of the storage battery is described by the formula (14):
In the formulae (13) and (14), E batt (t) is the total electric energy stored by the storage battery at the time t; p (P) load (t) is the electric energy demand of the storage battery by the load at the moment t; η (eta) inv Is the inverter conversion rate; η (eta) batt_c Charging efficiency for the battery pack; η (eta) batt_d And is the discharge efficiency of the battery pack.
The state of charge of the battery is described by formula (15):
in the equation (15), SOC (t) is the state of charge of the battery in the period t.
The discharge power of the battery t period is described by the formula (16):
in the formula (16), P batt And (t) is the discharge power of the storage battery pack at the moment t.
The discharge power of the energy storage module t period is described by equation (17):
P s (t)＝P FC (t)+P batt (t) (17)
1.4 Diesel generator Module
If the sum of the wind power generation, the photovoltaic power generation and the electric energy of the energy storage module at the moment t cannot meet the load requirement, starting the diesel engine, wherein the output power of the required diesel engine is described by the formula (18):
in the formula (18), P D (t) is the output power of the diesel engine at the moment t; p (P) load And (t) is the load at time t.
The output power of the diesel generator is determined by the fuel consumption, so the relation between the power output and the fuel consumption rate of the diesel engine is described by the formula (19):
f D (t)＝μ D ×P rate +v D ×P D (t) (19)
in the formula (19), f D (t) is the fuel consumption of the diesel engine during period t; p (P) rate Rated output power of the diesel engine; parameter mu D The fuel curve intercept coefficient; v D Is the slope of the fuel curve.
1.5 energy management Markov decision process modeling for comprehensive energy systems of ships
In order to cope with the challenge of unexpected events to the safe and stable operation of the ship comprehensive energy system, the invention provides an elasticity enhancement control strategy based on a PDDPG algorithm, which can help the system to provide reliable power supply for necessary loads during unexpected events by capturing the uncertainty in the operation process of the system, and can realize near-optimal control with minimum operation cost as a target under normal conditions. The energy management problem under unexpected conditions is modeled as a markov decision process.
Status: state vector s t Is to provide an ambient feedback signal to the agent. In this operation, the state vector is described by equation (20):
in the formula (20), the amino acid sequence of the compound,the start-stop state of the ith diesel generator is represented by 0 or 1; />And->The continuous start time and the continuous stop time of the ith diesel generator are respectively indicated.
The actions are as follows: in order to improve the ability of the agent to cope with unexpected situations, corresponding regulation and control are made to the storage battery and the fuel cell system. Given state s t Action a of the agent at time step t t Is defined as:
In the formula (21), the amino acid sequence of the amino acid,representing the percentage of the charge and discharge energy of the storage battery to the total energy storage at the moment t; />The percentage of the total capacity of the hydrogen storage tank is represented by the charging and discharging gas of the hydrogen storage pipe in the t period.
Reporting: the return function formulated by the present invention is conventionally composed of four parts. The first part penalizes the energy storage system for exceeding the relevant constraint conditions, and the penalty functions are described by equations (22), (23) and (24):
in the formulas (22), (23) and (24),and->Respectively representing out-of-limit penalties of an energy storage system, a storage battery and a hydrogen storage tank in the current period t; />The amount of hydrogen consumed by the fuel cell for the current period t; p (P) batt (t) represents the output power of the battery for the current period t; />And C batt And respectively representing out-of-limit penalty coefficients of the hydrogen storage tank and the storage battery.
In order to improve the electric energy utilization rate of the clean energy power generation system and reduce the phenomenon of 'power abandoning unbalance', a 'power abandoning' punishment and a 'unbalance' punishment are introduced, and are described by a formula (25) and a formula (26):
in the formulas (25) and (26),indicating the power-off penalty at the current time t +.>Represents the unbalance penalty of the current time t, P exc (t) represents the power of the current time t, P ub (t) represents the unbalanced power at the current time t, C exc Represents the "power-off" penalty coefficient, C ub Representing the "imbalance" penalty factor.
The third part is used for reducing the energy consumption of the ship comprehensive energy system. Introducing fuel consumption costs, including the fuel cost of a diesel engine, the fuel cell hydrogen cost, and the electrolyzer hydrogen production benefits, described by formula (27):
in the formula (27), the amino acid sequence of the compound,representing the fuel consumption cost at the current time t, C d And->Fu, which respectively represent the unit price of diesel oil and hydrogen d (t) represents the fuel consumption of the diesel engine at the current moment t +.>And->The amount of hydrogen consumed and produced by the system at the current time t is shown.
The fourth section introduces a pollution abatement cost penalty for reducing the pollutant gas emissions from the vessel, described by formula (28):
in the formula (28), the amino acid sequence of the compound,representing the pollutant treatment cost of the system in the current period t, C c 、C N And C S The treatment cost unit price of the carbon oxide, the nitrogen oxide and the sulfur oxide is respectively represented by O C 、O N And O S The emission coefficients of carbon oxides, nitrogen oxides and sulfur oxides produced by the combustion of diesel fuel are shown respectively.
To sum up, the return function at the current time t is described by equation (29):
in the formula (29), r e And (t) represents a return function of the current time t environment.
The goal of the energy management agent in unexpected situations is to optimally manage the energy schedule of each controllable power supply unit to avoid load drops when unexpected events occur and to minimize system energy consumption. Thus, the return function can be increased on the basis of considering the operation cost of the ship integrated energy system in the following aspects:
1. The penalty in violating the power balance is described by equation (30):
r ub (t)＝α|O PV (t)+O WT (t)+O batt (t)+O fc (t)+O d (t)-P l (t)-δ| (30)
in equation (30), α is used as a bias constant to help balance the punishment and punishment ratio during training, and δ represents a tolerable slack margin to enhance system flexibility.
2. The penalty in the event that the system fails to support the critical power load during the occurrence of an extreme event is described by equation (31):
r a (t)＝β×P aload (t) (31)
in the formula (31), beta is a bias constant and is used for balancing the proportion of rewards and punishments in the training process; p (P) aload (t) represents the electrical load power that is not satisfied during the t period.
3. The rewards when the ship integrated energy system meets the power balance are described by the formula (32):
in the formula (32), P t And the total power output by each power supply unit t of the ship comprehensive energy system at the moment is shown.
4. The running cost of the system adds to the start-stop cost of the diesel engine. The running cost penalty is described by equation (33):
thus, the overall payback function can be described by equation (34):
r all (t)＝r s (t)+r o (t)+r ub (t)+r a (t)-r bal (t) (34)
value function: the invention aims at the minimum expected total operation cost, searches the most suitable energy management strategy pi, and realizes the energy optimization scheduling of the ship comprehensive energy system. The objective function is described by equation (35):
1.6 Ship comprehensive energy System operation constraint
The ship integrated energy system operates in an island mode without auxiliary power supply of an external power supply, so that in order to ensure safe and reliable operation of the ship integrated energy system, real-time supply and demand balance should be kept in the operation process of the ship integrated energy system, and the ship integrated energy system is described by a formula (36):
P avail (t)+P d (t)+P batt (t)+P Fc (t)＝P load (t) (36)
The diesel generator real-time energy output related constraints are described by equation (37) and equation (38):
-D d Δt≤P d (t+Δt)-P d (t)≤D u Δt (38)
in the formulas (37) and (38),and->Respectively represent the maximum and minimum output power of the diesel engine, D d And D u Indicating the ramp down and ramp up rates of the diesel generator, respectively.
For the energy storage system, the service life of the equipment is influenced by overcharge and overdischarge, so that the service life of the energy storage system is prolonged on the premise of ensuring safe and reliable operation of a ship power system, the real-time output power constraint of a storage battery and a fuel cell is described by a formula (39), the state of charge of the storage battery is constrained, and the constraint is described by a formula (40):
in the formulas (39) and (40),and->The upper limit and the lower limit of the charge state of the storage battery are respectively 0.1 and 0.9; />And->Respectively representing the maximum and minimum output power of the storage battery, +.>And->Respectively, the maximum and minimum output powers of the fuel cells.
Considering the start-stop state of the marine diesel generator, the running cost should also increase the start-stop cost of the diesel engine, described by formula (41):
in formula (41), c sd Indicating the start-stop costs of a single diesel engine,the start-stop state of the ith diesel engine at time t is represented by 0 and 1 (0 represents the stop state and 1 represents the start-up state).
The constraints of the diesel-electric set will also change at the same time, described by the formulae (42) and (43):
In the formulas (42) and (43),and->Respectively represent the upper limit and the lower limit of the output of the diesel generator, < + >>And->Indicating the upward and downward ramp rates of the diesel generator, respectively,/->And->The lowest on-time and the minimum off-time are indicated as 3h and 2h, respectively.
2 energy management strategy of ship comprehensive energy system based on PDDPG algorithm
The depth deterministic strategy gradient algorithm is based on a DPG algorithm, adopts an Actor-Critic algorithm framework, and utilizes a depth neural network to learn an approximation action value function Q and determine a strategy mu. Wherein the Actor network models the policy function in terms of state s t As input and based on the estimated Q value update strategy, outputting continuous action a t The method comprises the steps of carrying out a first treatment on the surface of the Critic network models a value function in terms of state s t And action a t As input, a scalar estimate of the Q-value function is output. The flow of Actor-Critic is shown in FIG. 2.
The Actor-Critic algorithm is essentially a hybrid approach that combines a strategy gradient approach with a cost function approach. The policy function is called an actor and the cost function is called a reviewer. Essentially, the Actor acts according to the current environmental state, while Critic is used to evaluate the action taken by the Actor. The DDPG algorithm generates a twin network for each of the actor and reviewer networks for its stability. Thus, the DDPG algorithm includes four deep neural networks, two critique networks Q (s, a|θ) and target Q network Q '(s, a|θ') respectively, and two actor networks behavior policy network μ (s|Φ) and target policy network μ '(s|Φ') respectively. The reviewer Q network updates its parameters by gradient descent to minimize the loss function.
δ t ＝r t +γQ'(s t+1 ,μ'(s t+1 |φ')|θ')-Q(s t ,a t |θ) (45)
In the formulas (44) and (45), μ represents an exploration strategy; ρ μ Representing the state of behavior policy μ generation; delta t Representing TD error, and theta' respectively represent network parameters corresponding to the Q network and the target Q network; phi' represents a parameter of the target policy network; r is (r) t +γQ'(s t+1 ,μ'(s t+1 The |φ ') | θ') represents the target action Q value for the t period.
And compared with the commentator network, the actor network updates the parameters of the commentator network in a gradient direction of the Q value of the action of the commentator in a gradient rising mode. The differential chain law is applied to obtain the desired return, described by equation (46):
in the formula (46), phi represents a parameter of the behavior policy network;representing the gradient of action a, +.>Representing the gradient of the behavioural policy network parameter phi.
The weight updates of the actor's Q network and the reviewer's behavior policy network are described by equations (47) and (48):
θ←θ+α θ Δ θ L θ (47)
in the formulas (47) and (48), alpha θ And alpha φ Representing the learning rate in the gradient descent algorithm.
The reviewer's target Q network and the actor's target policy network perform parameter updates in a "soft" update manner, respectively, described by equations (49) and (50):
θ'←τθ+(1-τ)θ' (49)
φ'←τφ+(1-τ)φ' (50)
in equations (49) and (50), τ is the target smoothing factor, τ is much smaller than 1.
The agent must maintain a proper balance for exploration and utilization when selecting actions. During the exploration process, the intelligent agent collects more information by continuously trying actions in the action space; during the utilization process, the intelligent agent can make the best decision given the available information. A significant advantage of the DDPG algorithm is that the exploration problem is decoupled from the learning algorithm. To ensure that the agent performs a better exploration in the environment, random gaussian white noise is added to the output of the behavioural policy network μ, described by equation (51):
In the formula (51), N t (0,σ 2 I) Representing random gaussian white noise.
DDQN is able to learn Q-value functions using DNN in a stable and robust manner, benefiting from empirical playback buffer pools and standardized applications. The experience playback pool in the DDPG algorithm is the same as the experience playback mechanism in the DDQN algorithm, which in the training process will have a priori experience (s t ,a t ,r t ,s t+1 ) Stored in an experience playback pool for subsequent update training of the network model, whereby, at each time step, the agent randomly selects a small batch of independent co-distributed samples from the experience playback pool for network trainingAnd (5) training. Batch normalization is based on the deep learning idea, where each dimension of a sample is normalized in a small batch of samples to have the same mean and variance, while the running average of the mean and variance is stored for use during the test phase.
In order to further improve the sampling efficiency of the original experience playback mechanism and thereby accelerate the learning process, the present embodiment employs a priority experience playback method. In this method, the magnitude of the TD error is used as a correction metric for the Q value estimation. Experience with smaller TD errors is more relevant to successful experience, while experience with larger TD errors indicates that the actions of the agent are not ideal because the samples are sampled less often, and preferential playback of these experiences during training can allow the agent to utilize the successful experience to more quickly refine the strategy, as well as prevent the agent from selecting adverse actions in certain situations, thereby improving the performance of the learned strategy.
If the size of the experience playback pool is set to N PR Probability P of sampling empirical sample n based on absolute TD error n Described by formula (52) and formula (53):
beta in the formula (52) and the formula (53) 1 Determining the priority of sampling, p n Representing the priority of the empirical sample n, rank n Representing absolute error in TD delta n Class of experience n in ranking, absolute error |delta n The formula of i is described by equation (54):
|δ n |＝|r n +γQ' θ' (s n+1 ,μ φ' (s n+1 ))-Q θ (s n ,a n )| (54)
wherein μ represents an exploration strategy; θ and θ' respectively represent the correspondence of the Q network and the target Q networkNetwork parameters of (a); r is (r) n +γQ′ θ′ (s n+1 ，μ φ′ (s n+1 ) A) represents a target action Q value corresponding to the experience n, s represents a state, and a represents an action;
due to absolute error delta n The larger the i the more times the empirical sample is sampled, thereby changing the frequency with which certain states are sampled, and thus resulting in learning with bias. To correct for this biased sampling, importance sampling weights are introduced, described by equation (55):
in the formula (55), beta 2 The degree of absolute correction is controlled. And W is taken as n δ n Instead of delta n For the calculation of Critic loss functions.
Finally, by employing the target network and the preferential empirical playback mechanism, the Critic network loss function is described again by equation (56):
3 energy management implementation flow of ship comprehensive energy system based on PDDPG algorithm
The implementation flow of the PDDPG algorithm in the energy management of the present invention is shown in fig. 3:
step1: initializing a Q network of a commentator and a behavior strategy network of an actor, wherein network parameters are respectively theta and phi.
Step2: initializing a target Q network of a commentator and a target strategy network of an actor, wherein network parameters are respectively theta '=theta and phi' =phi.
Step3: randomly selecting voyage from the training data set, and obtaining the initial state s of the voyage ship comprehensive energy system 1 。
Step4: initializing random gaussian exploration noise N t 。
Step5: will s t Input to the Actor behavior policy network μ (s|φ)Outputting the optimal scheduling decision a t And increases the action exploring ability based on the expression (51).
Step6: according to scheduling scheme a t The actions of each controllable device in the system are used for calculating the state s at the next moment on the premise that the operation conditions of each device are met t+1 And calculates a return value r based on equation (34) t+1 。
Step9: will experience { s ] t ,a t ,r t ,s t+1 Store in experience playback pool SumTree.
Step10: n samples are sampled from SumTree, the probability of each sample IS calculated based on equation (52) and equation (53), the absolute TD error IS calculated based on equation (54), IS weight IS calculated based on equation (55), and the experience level IS updated based on equation TD error.
Step11: a loss function is calculated based on equation (56) and reviewer Q network parameters are updated based on equation (47).
Step12: actor behavior policy network parameters are updated based on equations (46) and (48).
Step13: updating parameters of two target networks based on equation (49) and equation (50)
Step14: if the voyage ending time is reached, returning to Step4 if the voyage ending time is not reached.
Step15: if the cycle termination condition is reached, if not, returning to Step3, otherwise, terminating.
4 example simulation and analysis
In this section, four case studies were designed to verify the superior performance of the proposed PDDPG algorithm-based energy management strategy in marine integrated energy systems. The embodiment firstly trains the normal and steady-state ship comprehensive energy system operation data, which is a conventional energy management problem. And then analyzing the elastic control effect of the ship comprehensive system in an unexpected scene by using the trained PDDPG agent.
4.1 example settings
The strategy network of the PDDPG decision agent is built based on a deep neural network, namely an action strategy network, a target strategy network, an action value network and a target value network, wherein all the deep neural networks comprise three hidden layers, the number of neurons in each layer is 64-128-64, the activation function of each layer is a Relu function, and an Adam optimizer is adopted for training an Actor and Critic network model. Table 1 gives the detailed parameters of the energy management policy network model.
The programming language used by the energy management strategy of the ship comprehensive energy system based on the PDDPG algorithm is written based on deep learning packages of Tensorflow 1.15.0 and keras, and the language environment is Python 3.7. The deep reinforcement learning tool bag OpenAI Gym is used for realizing the simulation of the ship comprehensive energy system. The computer of the simulation environment was equipped with Intel Core i7-11700 CPU,16GB RAM, injeida GeForce GTX 3060GPU.
Table 1PDDPG energy management policy network model superparameter
4.2 description of unexpected scenarios
Most of the current research is limited to solving the energy management problem under dynamic stable conditions. And the ship often encounters unexpected events such as extreme weather, equipment faults and the like during the voyage, which can seriously affect the performance of the comprehensive energy system of the ship. For more accurate description of the common unexpected scene features during sailing, the following definitions are made:
high power consumption requirements: the load demand of the ship is greater than 1000kW over 48 continuous hours.
Low power requirements: the continuous ship for more than 48 hours only meets the necessary requirement, and the load requirement does not exceed 600kW.
Scene one: during the navigation process of the ship, the wind speed exceeds the maximum rotating speed under the influence of typhoon weather, and the wind driven generator is stopped; meanwhile, part of photovoltaic power generation panels are seriously damaged and cannot work.
Scene II: in the sailing process of the ship, the ship load is subjected to continuous high-power requirement scenes due to interference of external factors.
Scene III: in the sailing process of the ship, partial load suddenly breaks down, only necessary equipment runs, and the ship load has a continuous low-power demand scene.
4.3 simulation test and result analysis
The trained energy management strategy is verified by using the data of the first course so as to evaluate the performance of the energy management strategy under the unknown future sailing working condition, and the power demand balance and the output condition of each power supply source of the ship comprehensive energy system in the course are shown in fig. 4. Fig. 5 shows the state of charge change curve of the storage battery and the charge/discharge power change of the storage battery in the voyage, and fig. 6 shows the change curve of the hydrogen amount in the hydrogen storage tank in the voyage and the output power of the fuel cell and the load power change of the electrolytic cell.
The box diagram in fig. 4 shows the elastic section of the ship electrical load demand. As can be seen from the figure, the strategy can flexibly utilize the operation characteristics of a plurality of power supplies, and maintain the balance of the supply and the demand within the bearable range of the power system. It is worth noting that no power loss occurs under steady state conditions. As can be seen from fig. 5 and fig. 6, the PDDPG intelligent agent can well track the output power of the clean energy power generation device, when the output power of the clean energy power generation device is abundant, the energy storage device is charged, and when the load gradually increases, the renewable energy source cannot meet the load demand, the storage battery and the fuel cell are fully utilized to participate in dispatching and energy supply.
In order to test the performance of the energy management strategy based on the PDDPG method provided by the invention facing unexpected situations, it is assumed that a ship generates unexpected situations described in a first scene in the sailing process of a test route, and a comparison graph of clean energy output power of the normal situations of the test route and the unexpected situations is shown in fig. 7. The purpose of this test is therefore to provide energy balance in normal mode at minimal cost and to guarantee critical load requirements in the event of unexpected events. Figure 8 shows the output of each distributed power supply.
As can be seen from fig. 7 and 8, during the 1 st to 53 th hours and the 105 th to 149 th hours, the comprehensive energy system of the ship is in a normal operation state, the clean energy generator can normally supply power to the comprehensive energy system of the ship, and the energy storage device can often obtain more energy in a normal period, which can increase the standby capacity when unexpected events occur, and plays an important role in reducing the operation cost of the comprehensive energy system of the ship. When the unexpected event scene 54-104 is entered, the wind driven generator is stopped due to an extreme event, the photovoltaic power generation equipment is seriously damaged, the output power of the clean energy power generation unit is greatly reduced, and the standby diesel generator is started to furthest maintain the power requirement of important loads of the ship, so that the stable operation of the system is furthest ensured. Thus, in this test, the power generation capacity of the system may be lower than the total demand due to the extreme weather affecting part of the power generation equipment not being able to generate power normally, which means that in this case emergency power generation equipment has to be started up to maintain conservation of energy of the system.
In addition, the test sample is realized by adding a random variable into a data sample of a first route, and navigation features contained in the sample never appear in a training sample, which means that PDDPG (packet data processing G) intelligent agent can make reasonable energy optimization scheduling decisions according to different unexpected scenes. Thus, the proposed PDDPG energy management intelligent agent can formulate a cost-effective scheduling strategy under normal conditions and can also adapt to unexpected conditions by capturing the uncertainty of the training phase.
In order to further demonstrate the superior performance of the proposed energy management strategy against unexpected situations, the present embodiment further verifies that the strategy operates under another unexpected situation, in which case when an unexpected event occurs, the high power continuation continues to occur on the demand side of the ship's integrated energy system. In other words, the above scenario considers the case where the unexpected event occurs outside the integrated energy system, whereas the present scenario study considers the influence of the unexpected event occurring inside the integrated energy system on the system. Therefore, in the present research scenario, it is difficult to effectively utilize limited power generation resources to satisfy the cost-effective normal operation and the reliability-oriented elastic operation. Fig. 9 shows the optimal scheduling conditions of each power supply in the unexpected scenario two, fig. 10 shows the state of charge and power change conditions of the storage battery, and fig. 11 shows the hydrogen amount of the hydrogen storage tank, the output power of the fuel cell and the absorption power change conditions of the electrolytic cell.
Similar to the above cases, the system is in a normal state in the 1-52 hours and 105-149 hours, the strategy can fully mobilize each power supply, reasonably plan the discharge power of each power supply, start the emergency generator before unexpected scene occurs, and the storage battery absorbs surplus energy, thereby being beneficial to enhancing the tolerance of the system to high-power pulse load, and the system is in a high-power demand scene in the 53-105 hours, and can fully utilize the performances of the energy storage unit and the diesel generator, make reasonable planning in advance and furthest meet the load power demand.
In this test, the present embodiment further verifies another unexpected situation of the comprehensive energy system of the ship, and in this scenario, when an unexpected event occurs, the system part load unit fails, and the demand side of the comprehensive energy system of the ship continues to develop a low power continuation. Therefore, in the present research scenario, it is difficult to fully utilize the "peak clipping and valley filling" capability of the energy storage unit, so as to meet the normal operation with cost saving and the elastic operation with reduced waste electric quantity as a guide.
The box diagram in fig. 12 represents the power demand per moment for the voyage under normal conditions, with the pentagram representing the power demand per moment for the voyage when the scene three unexpected conditions occur. Similar to the above case, the comprehensive energy system of the ship is in a normal operation state during the 1-53 hours and 105-149 hours, and the energy storage device fully releases the stored energy, which increases the energy absorption capacity when unexpected events occur, and plays an important role in reducing the operation cost of the comprehensive energy system of the ship. As shown in fig. 13 and 14, when the unexpected event scenario 54-104 is entered, the partial load is shut down due to the fault and the ship's integrated energy system is operated in island mode, the battery and electrolyzer will fully absorb the excess energy in order to minimize the occurrence of system power rejection. In a word, when unexpected low-load scene appears in this scene, this tactics can make full use of energy storage unit and diesel generator's performance, makes reasonable planning in advance, furthest reduces the emergence of abandoning the electric phenomenon.
5 phrase
The embodiment provides a ship comprehensive energy system elasticity enhancement control method based on a PDDPG algorithm, which provides energy management and elasticity control for a multi-energy ship power system. Firstly, different unexpected scenes of the ship comprehensive energy system are described, then the start-stop cost and the start-stop constraint of the controllable diesel generator are increased in consideration of the continuity of the control action, and an optimal scheduling model of the ship comprehensive energy system under the unexpected scenes is established based on a Markov decision process. Next, a ship comprehensive energy system energy management implementation flow based on the PDDPG algorithm is described. Finally, the running condition of the ship comprehensive energy system under the steady-state working condition and the adaptability to unexpected scenes are analyzed, and the method can capture the uncertainty of various types of equipment, and makes optimal energy scheduling for each energy supply unit of the ship when unexpected events (namely extreme events occurring outside or inside the ship comprehensive energy system) occur, so that the safe and reliable running of the ship is ensured.
The present patent is not limited to the above-mentioned best mode, any person can obtain other various energy management strategies of the ship comprehensive energy system for dealing with unexpected situations under the teaching of the present patent, and all equivalent changes and modifications made according to the scope of the present patent should be covered by the present patent.
Claims (6)
1. An energy management strategy of a ship integrated energy system for dealing with unexpected situations, which is characterized in that: based on a marine integrated energy system comprising a distributed power generation unit, a load and an energy storage system, the distributed power generation unit comprising: a photovoltaic power generation unit, a wind power generation unit, a fuel cell and a diesel generator;
modeling the energy management problem under unexpected working conditions as a Markov decision process, and based on an elasticity enhancement control strategy of a PDDPG algorithm, by capturing uncertainty in the operation process of the system, the energy management problem can be used for providing reliable power supply for necessary loads by an assistance system during unexpected events, and under normal conditions, near-optimal control can be realized with the aim of minimum operation cost.
2. The comprehensive energy system energy management strategy for a ship in response to an unexpected situation according to claim 1, wherein: the new energy ship comprehensive energy system operates in an island mode, wherein a wind power generation module and a photovoltaic power generation module are used as main power supply units, a fuel cell and a diesel engine are used as auxiliary power supply units, and an energy storage module and a hydrogen storage module are used as energy storage units; when wind power generation and photovoltaic power generation cannot meet load requirements, the energy storage unit is used for making up for the shortage of electric power; when the output power of the clean energy source is higher than the load demand, the redundant energy is stored in a storage battery or used for electrolyzing water to produce hydrogen for subsequent use; during periods of low power, energy is stored for meeting load demands.
3. The comprehensive energy system energy management strategy for a ship in response to an unexpected situation according to claim 2, wherein:
the implementation flow of the PDDPG algorithm for energy management in the invention is as follows:
step S1: initializing a Q network of a commentator and a behavior strategy network of an actor, wherein network parameters are respectively theta and phi;
step S2: initializing a target Q network of a commentator and a target strategy network of an actor, wherein network parameters are respectively theta '=theta and phi' =phi;
step S3: randomly selecting voyage from the training data set, and obtaining the initial state s of the voyage ship comprehensive energy system 1 ；
Step S4: initializing random gaussian exploration noise N t ；
Step S5: will s t The optimal scheduling decision a is output as an input to the Actor behavior policy network μ (s|φ) t And increase the action exploration ability;
step S6: according to scheduling scheme a t The actions of each controllable device in the system are used for calculating the state s at the next moment on the premise that the operation conditions of each device are met t+1 And calculates a return value r t+1 ；
Step S9: will experience { s ] t ,a t ,r t ,s t+1 Store in experience playback pool SumTree;
step S10: sampling n samples from SumPree, calculating the probability of each sample, calculating absolute TD error, calculating importance sampling weight, and updating experience level according to TD error;
Step S11: calculating a loss function and updating the Q network parameters of the commentators;
step S12: updating actor behavior strategy network parameters;
step S13: updating parameters of two target networks;
step S14: if the voyage ending time is reached, returning to the step S4 if the voyage ending time is not reached;
step S15: and (3) whether the cycle termination condition is met, if not, returning to the step (S3), otherwise, terminating.
4. The marine integrated energy system energy management strategy of claim 3, wherein: in step S5, random gaussian white noise is added to the output of the behavioural policy network μ to increase the action exploration ability.
5. The comprehensive energy system energy management strategy for a ship in response to an unexpected situation according to claim 4, wherein:
in step S10, the priority experience playback method is adopted, and the experience level is updated according to the TD error:
the magnitude of the TD error is used as a correction measure for the Q value estimate: experience with smaller TD errors is more relevant to successful experience, while experience with larger TD errors indicates that the actions of the agent are not ideal;
if the size of the experience playback pool is set to N PR Probability P of sampling empirical sample n based on absolute TD error n Described by the following two formulas:
wherein beta is 1 Determining the priority of sampling, p n Representing the priority of the empirical sample n, rank n Representing absolute error in TD delta n Class of experience n in ranking, absolute error |delta n The formula of i is described by the following formula:
|δ n |＝|r n +γQ' θ' (s n+1 ,μ φ' (s n+1 ))-Q θ (s n ,a n )|
μ represents an exploration strategy; θ and θ' represent network parameters corresponding to the Q network and the target Q network, respectively; r is (r) n +γQ′ θ′ (s n+1 ，μ φ′ (s n+1 ) A) represents a target action Q value corresponding to the experience n, s represents a state, and a represents an action;
due to absolute error delta n The larger the i the more times an empirical sample is sampled, thereby changing the frequency with which certain states are sampled, thereby resulting in learning biased, to correct for such biased sampling, importance sampling weights are introduced, described by:
wherein beta is 2 Control the degree of absolute correction and let W n δ n Instead of delta n For the calculation of Critic loss functions.
6. The comprehensive energy system energy management strategy for a ship in response to an unexpected situation according to claim 5, wherein: in step S11, a loss function is calculated based on the rule of thumb playback mechanism by employing the target network and the priority.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310575994.9A CN116581816A (en) | 2023-05-22 | 2023-05-22 | Ship comprehensive energy system energy management strategy for dealing with unexpected situations |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310575994.9A CN116581816A (en) | 2023-05-22 | 2023-05-22 | Ship comprehensive energy system energy management strategy for dealing with unexpected situations |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116581816A true CN116581816A (en) | 2023-08-11 |
Family
ID=87542869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310575994.9A Pending CN116581816A (en) | 2023-05-22 | 2023-05-22 | Ship comprehensive energy system energy management strategy for dealing with unexpected situations |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116581816A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350496A (en) * | 2023-10-17 | 2024-01-05 | 安徽大学 | Ocean island group energy management method based on hybrid action space reinforcement learning |
| CN118281873A (en) * | 2024-05-31 | 2024-07-02 | 华南理工大学 | Marine wind power probability prediction method based on Bayesian and personalized federal learning |
-
2023
- 2023-05-22 CN CN202310575994.9A patent/CN116581816A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350496A (en) * | 2023-10-17 | 2024-01-05 | 安徽大学 | Ocean island group energy management method based on hybrid action space reinforcement learning |
| CN117350496B (en) * | 2023-10-17 | 2024-05-24 | 安徽大学 | Ocean island group energy management method based on hybrid action space reinforcement learning |
| CN118281873A (en) * | 2024-05-31 | 2024-07-02 | 华南理工大学 | Marine wind power probability prediction method based on Bayesian and personalized federal learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116581816A (en) | Ship comprehensive energy system energy management strategy for dealing with unexpected situations | |
| CN105048516A (en) | Wind-light-water-fire multi-source complementary optimization scheduling method | |
| Vasallo et al. | A methodology for sizing backup fuel-cell/battery hybrid power systems | |
| Goh et al. | An assessment of multistage reward function design for deep reinforcement learning-based microgrid energy management | |
| CN107453402B (en) | Scheduling method for reducing operation risk of wind-storage-containing integrated power station power system | |
| CN110535132B (en) | Robust optimization-based power system construction planning method | |
| CN110350589B (en) | Renewable energy and energy storage scheduling model and scheduling method | |
| CN114676991B (en) | Multi-energy complementary system optimal scheduling method based on source-load double-side uncertainty | |
| CN113541205B (en) | Cluster learning-based low-carbon CSP system collaborative optimization method and device | |
| CN111641233A (en) | Electric power system day-based flexible peak regulation method considering new energy and load uncertainty | |
| CN111293718A (en) | AC/DC hybrid microgrid partition two-layer optimized operation method based on scene analysis | |
| Xie et al. | A distributed real-time power management scheme for shipboard zonal multi-microgrid system | |
| CN114741960A (en) | Comprehensive energy resource economic environment scheduling optimization method and system | |
| CN116596340A (en) | Ship comprehensive energy system energy management strategy with working condition adaptability | |
| CN115051388A (en) | Distribution robustness-based 'source-network-load-storage' two-stage scheduling optimization method | |
| CN110336308B (en) | Opportunity constraint-based active power distribution network economic dispatching method | |
| CN116436094A (en) | Multi-time-scale comprehensive energy management method for full-direct-current ship micro-grid system | |
| CN116611628A (en) | Two-stage capacity configuration method of comprehensive energy production unit considering dynamic performance | |
| CN116805192A (en) | Comprehensive energy system double-layer planning optimization method considering optimal energy rejection rate and application thereof | |
| CN116384655A (en) | New energy consumption-oriented source network load side energy storage system optimization planning method and system | |
| CN114398777B (en) | Power system flexible resource allocation method based on Yu Bashen game theory | |
| Song et al. | Unit commitment optimization model of wind storage combined system considering peak load regulation of energy storage system | |
| CN113224751A (en) | Method for optimizing line capacity in high wind power ratio system | |
| CN111740443A (en) | Multi-time scale collaborative optimization scheduling method for independent micro-grid with multiple distributed power supplies | |
| CN106972548B (en) | Wind-powered electricity generation participates in the dispatching method of spare pool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |