CN111637435B - Nuclear power system steam generator water level control method based on SARSA - Google Patents

Nuclear power system steam generator water level control method based on SARSA Download PDF

Info

Publication number
CN111637435B
CN111637435B CN202010502732.6A CN202010502732A CN111637435B CN 111637435 B CN111637435 B CN 111637435B CN 202010502732 A CN202010502732 A CN 202010502732A CN 111637435 B CN111637435 B CN 111637435B
Authority
CN
China
Prior art keywords
sarsa
controller
water level
steam generator
nuclear power
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010502732.6A
Other languages
Chinese (zh)
Other versions
CN111637435A (en
Inventor
齐义文
马云龙
李献领
于洋
张弛
赵秀娟
陈禹西
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerospace Promotion Suzhou Aerospace Technology Co ltd
Original Assignee
Shenyang Aerospace University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenyang Aerospace University filed Critical Shenyang Aerospace University
Priority to CN202010502732.6A priority Critical patent/CN111637435B/en
Publication of CN111637435A publication Critical patent/CN111637435A/en
Application granted granted Critical
Publication of CN111637435B publication Critical patent/CN111637435B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F22—STEAM GENERATION
    • F22B—METHODS OF STEAM GENERATION; STEAM BOILERS
    • F22B35/00—Control systems for steam boilers
    • F22B35/004—Control systems for steam generators of nuclear power plants
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F22—STEAM GENERATION
    • F22B—METHODS OF STEAM GENERATION; STEAM BOILERS
    • F22B37/00—Component parts or details of steam boilers
    • F22B37/02—Component parts or details of steam boilers applicable to more than one kind or type of steam boiler
    • F22B37/38—Determining or indicating operating conditions in steam boilers, e.g. monitoring direction or rate of water flow through water tubes
    • G—PHYSICS
    • G21—NUCLEAR PHYSICS; NUCLEAR ENGINEERING
    • G21D—NUCLEAR POWER PLANT
    • G21D3/00—Control of nuclear power plant
    • G21D3/001—Computer implemented control
    • G21D3/005—Thermo-hydraulic simulations
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E30/00—Energy generation of nuclear origin

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Thermal Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Plasma & Fusion (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Feedback Control In General (AREA)

Abstract

The invention relates to the technical field of nuclear power system control, and provides a nuclear power system steam generator water level control method based on SARSA, which comprises the following steps: step 1: SARSA controller designed for nuclear system steam generator level control: selecting partial influence factors and partial control quantity of the water level as input and output of the SARSA controller respectively, and setting a control target; step 2: respectively carrying out interval division on input and output parameters of the SARSA controller, and designing an incentive rule; and step 3: training an SARSA controller, and updating a Q table; and 4, step 4: if the training result does not meet the control target, adjusting the parameters of the SARSA controller according to the training result, and turning to the step 3; otherwise, the Q table is saved. The invention can simultaneously realize the rapid, accurate and stable control of the water level of the steam generator and realize the autonomous learning capability of the water level controller of the steam generator under the condition of variable working conditions and the decline of system characteristics.

Description

Nuclear power system steam generator water level control method based on SARSA
Technical Field
The invention relates to the technical field of nuclear power system control, in particular to a nuclear power system steam generator water level control method based on SARSA.
Background
The nuclear power system is used as a power source of a ship and needs to meet stable operation under various complex working conditions. The steam generator is used as a pivotal part of a secondary loop, and the stable operation of the steam generator is an important guarantee that the whole nuclear power system is in a normal operation state. The water level is an important index for measuring the operation state and safety of the steam generator system and must be kept within a safe range. High water level will cause damage to the blades of the two-circuit turbine, and low water level will cause damage to the U-tubes that complete the heat exchange or failure of the reactor heat dissipation. Therefore, it is necessary to design a strong robustness, high stability, fast response steam generator level control system to ensure that the steam generator level is within a safe range.
The steam generator water level control problem lies in false water level, small controllable range, requirement of quick and accurate response of a controller, complex physical process, linkage change of other parameters caused by change of a certain parameter and the like. The existing steam generator water level control method mostly adopts the technologies of PID control, active disturbance rejection control and the like, the technologies are difficult to properly solve the problems, three factors of rapidness, accuracy and stability of a control system cannot be simultaneously met, and the self-adaptive learning capability under variable working conditions and system characteristic decline is not realized.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a nuclear power system steam generator water level control method based on SARSA, which can simultaneously realize the fast, accurate and stable control of the water level of a steam generator and realize the autonomous learning capability of the steam generator water level controller under the condition of variable working conditions and system characteristic decline.
The technical scheme of the invention is as follows:
a nuclear power system steam generator water level control method based on SARSA is characterized by comprising the following steps:
step 1: SARSA controller designed for nuclear system steam generator level control: selecting part of influence factors of the water level from the nuclear power steam generator system as the input of the SARSA controller, selecting part of control quantity from the nuclear power steam generator system as the output of the SARSA controller, and setting a control target;
step 2: respectively carrying out interval division on input parameters, namely state parameters, and output parameters, namely action instructions of the SARSA controller, and designing an incentive rule;
and step 3: training the SARSA controller, and updating a Q table in the SARSA controller;
and 4, step 4: judging whether the training result of the SARSA controller meets the control target or not, if not, adjusting the parameters of the SARSA controller according to the training result, and turning to the step 3; otherwise, the Q table is saved.
Further, the step 1 comprises the following steps:
step 1.1: determining five state parameters of the input of the SARSA controller, including last-time opening degree of a main water supply valve, water level deviation, a water level deviation derivative, flow mismatch and a flow mismatch derivative; wherein, the water level deviation is the deviation between the actual water level and the target water level, and the flow mismatch is the deviation between the feed water flow and the steam flow of the steam generator;
step 1.2: determining the output of the SARSA controller as the opening of a main water supply valve;
step 1.3: setting the control target as the absolute value of the product E of the water level deviation and the water level deviation derivative is less than or equal to a preset product threshold.
Further, the step 2 comprises the following steps:
step 2.1: and (3) carrying out interval division on input parameters of the SARSA controller: dividing the last opening degree of the main water supply valve into a1Individual interval, water level deviation divided into a2The interval, water level deviation derivative is divided into a3Segment, flow mismatch is divided into4Fractional, fractional flow derivative division into a5An interval;
step 2.2: and (3) carrying out interval division on output parameters of the SARSA controller: dispersing the opening degree of a main water supply valve into b action instructions;
step 2.3: designing reward rules: and dividing the product E into c intervals by taking the product E of the water level deviation and the water level deviation derivative as a reward basis, and setting corresponding reward for each interval of the product E.
Further, in step 2, before the interval division and the reward rule design, threshold calibration is performed on all input parameters and output parameters.
Further, in the step 3, during the training process of the SARSA controller,
in a state StThe basis of the next selection execution action is
Figure BDA0002525367560000021
Wherein randomA means randomly selecting an action a from the action set,
Figure BDA0002525367560000022
indicates that Q (S) is selectedtThe action A, Q (S) with the largest value of A)tA) is in state StThe action value function of the action A is executed, rand is a random number, and rand belongs to [0, 1]]Epsilon is a random factor, epsilon is epsilon [0,1 ∈]；
The update rule of the Q table is
Figure BDA0002525367560000031
Wherein S istIndicating the state of the steam generator input to the controller at time t, AtRepresenting the action of the SARSA controller output at time t, RtIndicating that the SARSA controller is in state StDown execution action AtThe reward obtained; α represents the learning rate of the SARSA controller, α ∈ (0, 1)](ii) a γ represents forgetting speed, γ ∈ (0, 1)](ii) a The isdone is a termination mark, the isdone-1 represents that the t moment reaches the preset maximum simulation time, and the isdone-0 represents that the t moment does not reach the preset maximum simulation time.
Further, in step 3, the training environment of the SARSA controller is a nuclear power system steam generator simulation model, the input of the nuclear power system steam generator simulation model includes a feedwater flow, a feedwater temperature, an outflow steam flow, a primary side inlet specific enthalpy, a primary side inlet flow, and a primary side inlet temperature, and the output of the nuclear power system steam generator simulation model includes a steam generator water level, a steam chamber pressure, a primary side outlet specific enthalpy, and a primary side outlet temperature.
Further, in the step 4, the method for adjusting the parameters of the SARSA controller according to the training result includes:
calculating the fluctuation of a training result, namely the fluctuation of the absolute value of the product E of the water level deviation and the water level deviation derivative relative to a preset product threshold, and adjusting one or more of the learning speed alpha, the forgetting speed gamma and the random factor epsilon when the fluctuation exceeds an allowable oscillation range; the learning speed alpha is adjusted to be increased or decreased, the forgetting speed gamma is adjusted to be increased or decreased, the random factor epsilon is adjusted to be decreased, the adjusting range of the learning speed alpha and the forgetting speed gamma is within 0.01-0.05, and the adjusting range of the random factor epsilon is within 0.01-0.08.
The invention has the beneficial effects that:
the invention designs the SARSA controller for the water level self-adaptive control of the steam generator of the nuclear power system, utilizes the online self-learning capability of the SARSA reinforcement learning algorithm and the 'trial and error' principle, enables the steam generator control system to optimize control decisions based on the evaluation feedback signal dependent control performance during interaction with the environment (including controlled objects, signal transmission, internal uncertainties and upstream and downstream disturbances), thereby solving the problems of self-learning and self-optimizing control which are difficult to realize by the traditional water level control method of the steam generator, solves the control problems caused by false water level, weak robustness of a complex system and the like, can simultaneously realize the fast, accurate and stable control of the water level of the steam generator, and the water level control of the steam generator has the autonomous learning ability under the variable working conditions and the system characteristic decline, and the overall performance of the steam generator system is obviously improved.
Drawings
FIG. 1 is a schematic diagram of reinforcement learning;
FIG. 2 is a control structure diagram of the water level control method of the nuclear power system steam generator based on SARSA according to the present invention;
FIG. 3 is a flow chart of the SARSA-based method for controlling the water level of the steam generator of the nuclear power system according to the present invention;
FIG. 4 is a diagram illustrating the steady state control effect of the SARSA-based steam generator level control method for a nuclear power system according to the present invention;
FIG. 5 is a diagram illustrating the effect of variable command control on the SARSA-based steam generator level control method for a nuclear power system according to the present invention;
fig. 6 is a comparison graph of the variable command control effect of the nuclear power system steam generator water level control method based on SARSA according to the present invention and the conventional nuclear power system steam generator water level control method based on PID controller according to the embodiment.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific embodiments.
The existing steam generator water level control method mostly adopts the technologies of PID control, active disturbance rejection control and the like, the technologies are difficult to properly solve the control problems brought by false water level, weak robustness of a complex system and the like, the three factors of rapidness, accuracy and stability of the control system cannot be simultaneously met, and the self-adaptive learning capability under variable working conditions and system characteristic decline is not provided.
In recent years, because the alpha dog defeats the symbolic events such as world go champions and the like, artificial intelligence becomes a hot topic again, thereby drawing wide attention of scientific researchers around the world and leading the field of artificial intelligence to have unprecedented development and technological breakthrough. The reinforcement learning method is one of the important branches of machine learning, and is widely applied to a plurality of fields, and is related to various aspects such as path planning, intelligent robots, medical equipment and the like. Fig. 1 is a schematic diagram of reinforcement learning. As can be seen from fig. 1, the agent is in a certain environment state (S), takes a certain action (a), gets environment feedback (reward R), measures the benefit and disadvantage of executing the action through reward, and continuously iterates and reciprocates the process until the specific goal is completed with high reward action.
In order to realize the rapid, accurate and stable control with certain autonomous learning capability on the water level of the system, the invention applies a typical algorithm SARSA in reinforcement learning, namely online multi-step time sequence difference learning, to the water level control of the steam generator of the nuclear power system. The control structure of the nuclear power system steam generator water level control method based on SARSA is shown in figure 2. Wherein, the whole system consists of three parts of an SARSA controller, an actuating mechanism and a steam generator. In most cases, steam generators are in extremely complex operating conditions and are subject to disturbances from various factors, such as the generation of false water levels, the amount of heat transferred by the various circuits, variations in steam quality, etc. The above problems not only increase the complexity of the working conditions, but also bring great challenges to the conventional PID-based control method, making it difficult for the conventional control method to achieve the desired control target. Therefore, the invention adopts the SARSA control method, and the controller continuously obtains learning experience through the trial and error principle, so that the rapidity, the accuracy and the stability of the controller are gradually improved, the overall performance of the steam generator is obviously improved, and the safe and efficient operation of the steam generator is ensured.
As shown in fig. 3, the water level control method of the nuclear power system steam generator based on SARSA according to the present invention comprises the following steps:
step 1: SARSA controller designed for nuclear system steam generator level control: part of the influencing factors of the water level are selected from the nuclear power steam generator system as the input of the SARSA controller, part of the control quantity are selected from the nuclear power steam generator system as the output of the SARSA controller, and a control target is set.
In this embodiment, the step 1 includes the following steps:
step 1.1: determining five state parameters of the input of the SARSA controller, including last-time opening degree of a main water supply valve, water level deviation, a water level deviation derivative, flow mismatch and a flow mismatch derivative; wherein, the water level deviation is the deviation between the actual water level and the target water level, and the flow mismatch is the deviation between the feed water flow and the steam flow of the steam generator;
step 1.2: determining the output of the SARSA controller as the opening of a main water supply valve;
step 1.3: setting the control target as the absolute value of the product E of the water level deviation and the water level deviation derivative is less than or equal to a preset product threshold.
Step 2: and respectively carrying out interval division on input parameters, namely state parameters, and output parameters, namely action instructions of the SARSA controller, and designing an incentive rule.
In order to prevent the dimension disaster problem caused by the continuous state space, before the interval division and the reward rule design, threshold value calibration is carried out on all input parameters and output parameters, and appropriate discretization processing is respectively carried out. The specific dispersion method is as follows:
step 2.1: and (3) carrying out interval division on input parameters of the SARSA controller:
dividing the last time opening degree of the main water supply valve (after per unit processing) into a15 intervals: (0,0.2]、(0.2，0.4]、(0.4，0.6]、(0.6，0.8]、(0.8，1]；
Divide the water level deviation (after per unit processing) into a29 intervals: (∞ -0.1)]、(-0.1，-0.08]、(-0.08，-0.06]、(-0.06，-0.02]、(-0.02，0]，(0，0.02]、(0.02，0.06]、(0.06，0.1]、(0.1，+∞)；
Dividing the water level deviation derivative (after per unit processing) into a36 intervals: (∞ -0.04)]、(-0.04，-0.02]、(-0.02，0]、(0，0.02]、(0.02，0.04]、(0.04，+∞)；
Divide the traffic lost (after per unit processing) into a48 intervals: (∞ -4)]、(-4，-1.6]、(-1.6，-0.8]、(-0.8，0]、(0，0.8]、(0.8，1.6]、(1.6，4]、(4，+∞)；
Dividing the flow mismatch derivative (after per unit processing) into a58 intervals: (∞ -1)]、(-1，-0.6]、(-0.6，-0.2]、(-0.2，0]、(0，0.2]、(0.2，0.6]、(0.6，1]、(1，+∞)。
Step 2.2: and (3) carrying out interval division on output parameters of the SARSA controller:
the opening degree of the main water supply valve is dispersed into b-7 action commands (after per unit processing): [ -0.2, -0.1, -0.05,0,0.05,0.1,0.2].
Through the interval division, a space with the total capacity of 17280 × 7 is obtained, and the space is the Q table.
Step 2.3: designing reward rules:
the core idea of reinforcement learning is "trial and error" in that high reward actions are continuously explored according to the environmental state. For the water level control of the steam generator of the nuclear power system, the states and actions are contained in the states and action spaces, and the reward for executing certain action in different states is set artificially.
Considering from the two aspects of control effect and response speed, taking the product E of the water level deviation and the water level deviation derivative as a reward basis, dividing the product E into 9 sections, setting corresponding rewards for each section of the product E, wherein the smaller the absolute value of the product E is, the higher the reward is, and both positive rewards and negative rewards exist:
e ∈ (— infinity, -0.004], R ═ 10;
when E belongs to (-0.004, -0.002), R is-8;
when E belongs to (-0.002, -0.001), R is-5;
when E belongs to (-0.001, -0.0003), R is 1;
when E belongs to (-0.0003, 0), R is 2;
when E is 0, R is 3;
when E belongs to (0, 0.002), R is 2;
e ∈ [0.002, 0.004), R ═ 1;
e ∈ [0.004, 0.006 ]), R ═ 1;
e ∈ [0.006, 0.008)), R ═ 2;
e ∈ [0.008, + ∞), R ═ 5.
And step 3: training the SARSA controller, and updating a Q table in the SARSA controller;
wherein, in the training process of the SARSA controller,
in a state StThe basis of the next selection execution action is
Figure BDA0002525367560000061
Wherein randomA means randomly selecting an action a from the action set,
Figure BDA0002525367560000062
indicates that Q (S) is selectedtThe action A, Q (S) with the largest value of A)tA) is in state StThe action value function of the action A is executed, rand is a random number, and rand belongs to [0, 1]]Epsilon is a random factor, epsilon is epsilon [0,1 ∈]；
The update rule of the Q table is
Figure BDA0002525367560000071
Wherein S istIndicating the state of the steam generator input to the controller at time t, AtRepresenting the action of the SARSA controller output at time t, RtIndicating that the SARSA controller is in state StDown execution action AtThe reward obtained; α represents the learning rate of the SARSA controller, α ∈ (0, 1)]α determines Q (S) to be updatedt,At) Degree of retention on itself; γ represents forgetting speed, γ ∈ (0, 1)](ii) a Alpha and gamma jointly determine Q (S)t,At) Learning degree of state and action Q value at next time; the isdone is a termination mark, the isdone-1 represents that the t moment reaches the preset maximum simulation time, and the isdone-0 represents that the t moment does not reach the preset maximum simulation time.
In this embodiment, the training environment of the SARSA controller is a simulation model of a steam generator of a nuclear power system. The simulation model of the steam generator of the nuclear power system is constructed by adopting a lumped parameter method according to a thermodynamic hydraulic process in the steam generator under reasonable assumption, comprises important parameters of the steam generator such as a primary heat transfer pipe, a working medium in a water chamber, a secondary side working medium liquid phase part, a secondary side working medium steam part and the like, can reflect certain thermodynamic characteristics and is convenient to call in an MATLAB environment to design a control system.
The input of the nuclear power system steam generator simulation model comprises feed water flow, feed water temperature, outflow steam flow, primary side inlet specific enthalpy, primary side inlet flow and primary side inlet temperature, and the output of the nuclear power system steam generator simulation model comprises steam generator water level, steam chamber pressure, primary side outlet specific enthalpy and primary side outlet temperature. The calculation of the input and output of the simulation model of the steam generator of the nuclear power system adopts technical means which are well known to those skilled in the art, and the technical means corresponding to the formulas (1) to (22) described in 'the nonlinear mechanism model and dynamic characteristic analysis of the steam generator' by weishiwei et al (2018) are adopted in the embodiment, and the details are as follows: wenzhiwei, Wangmingchun, Zhang Yufei, Wang strongly, Gunn Key. non-linear mechanism model and dynamic characteristic analysis of steam generator [ J ] generating equipment, 2018,32(04): 261-. The feed water flow, the feed water temperature, the outflow steam flow, the primary side inlet specific enthalpy, the primary side inlet flow, the primary side inlet temperature, the steam generator water level, the steam chamber pressure, the primary side outlet specific enthalpy and the primary side outlet temperature are respectively the mass flow of the inlet of a two-loop descending section, the outlet temperature of a hot water section, the mass flow of steam at the outlet of a steam-water separator, the inlet specific enthalpy of a primary loop working medium, the inlet mass flow of the primary loop working medium, the inlet temperature of the primary loop working medium, the SG water level, the SG steam chamber pressure, the outlet specific enthalpy of the primary loop working medium and the outlet temperature of the primary loop working medium in a steam generator nonlinear mechanism model and dynamic characteristic analysis.
And 4, step 4: judging whether the training result of the SARSA controller meets the control target or not, if not, adjusting the parameters of the SARSA controller according to the training result, and turning to the step 3; otherwise, the Q table is saved.
The control performance of the SARSA controller is improved by training the SARSA controller and gradually updating a controller core Q table. In the training process, the convergence speed can be accelerated and the control effect can be improved by adjusting alpha, gamma and epsilon. In this embodiment, the method for adjusting the parameters of the SARSA controller according to the training result includes:
calculating the fluctuation of a training result, namely the fluctuation of the absolute value of the product E of the water level deviation and the water level deviation derivative relative to a preset product threshold, and adjusting one or more of the learning speed alpha, the forgetting speed gamma and the random factor epsilon when the fluctuation exceeds an allowable oscillation range; the learning speed alpha is adjusted to be increased or decreased, the forgetting speed gamma is adjusted to be increased or decreased, the random factor epsilon is adjusted to be decreased, the adjusting range of the learning speed alpha and the forgetting speed gamma is within 0.01-0.05, and the adjusting range of the random factor epsilon is within 0.01-0.08.
And (3) carrying out simulation test on the SARSA controller which is updated in a large quantity, and gradually standardizing reward conditions according to a test result and a control target, so that the SARSA controller can be converged quickly and the control precision is ensured to meet the control requirement. And further carrying out variable instruction and variable disturbance tests on the controller, and adjusting parameters for multiple times until the control requirements are met.
In this embodiment, the stable state control and the variable instruction control are respectively performed on the water level of the steam generator by using the SARSA controller which achieves the control target, so as to obtain the stable state control effect diagram and the variable instruction control effect diagram of the nuclear power system steam generator water level control method based on the SARSA according to the present invention, which are respectively shown in fig. 4 and 5. Under the variable instruction, the water level of the steam generator is controlled by using the traditional nuclear power system steam generator water level control method based on the PID controller, and the comparison graph of the variable instruction control effect of the nuclear power system steam generator water level control method based on the SARSA and the traditional nuclear power system steam generator water level control method based on the PID controller is shown in FIG. 6. As can be seen from fig. 4, in the steady state, the SARSA controller of the present invention has a slight oscillation in controlling the water level, but still can meet the control requirement. As can be seen from fig. 5, the SARSA controller of the present invention is able to respond quickly and reach a new control target quickly when changing commands. As can be seen from fig. 6, the SARSA controller of the present invention has great advantages over the PID controller, which is embodied as follows: small overshoot, quick response and short regulation time. Therefore, compared with the traditional PID controller, the SARSA controller has stronger environmental adaptability, and is superior to the traditional PID controller in three aspects of rapidness, accuracy and stability. In addition, the SARSA controller can learn online and has good self-learning and self-adaptive control capabilities for environment changes (parameter changes, characteristic decline, external disturbance, instruction changes and mode switching) of a more complex control system.
It is to be understood that the above-described embodiments are only a few embodiments of the present invention, and not all embodiments. The above examples are only for explaining the present invention and do not constitute a limitation to the scope of protection of the present invention. All other embodiments, which can be derived by those skilled in the art from the above-described embodiments without any creative effort, namely all modifications, equivalents, improvements and the like made within the spirit and principle of the present application, fall within the protection scope of the present invention claimed.

Claims (6)

1. A nuclear power system steam generator water level control method based on SARSA is characterized by comprising the following steps:
step 1: SARSA controller designed for nuclear system steam generator level control: selecting part of influence factors of the water level from the nuclear power steam generator system as the input of the SARSA controller, selecting part of control quantity from the nuclear power steam generator system as the output of the SARSA controller, and setting a control target;
step 2: respectively carrying out interval division on input parameters, namely state parameters, and output parameters, namely action instructions of the SARSA controller, and designing an incentive rule;
the step 2 comprises the following steps:
step 2.1: and (3) carrying out interval division on input parameters of the SARSA controller: dividing the last opening degree of the main water supply valve into a1Individual interval, water level deviation divided into a2The interval, water level deviation derivative is divided into a3Segment, flow mismatch is divided into4Fractional, fractional flow derivative division into a5An interval;
step 2.2: and (3) carrying out interval division on output parameters of the SARSA controller: dispersing the opening degree of a main water supply valve into b action instructions;
step 2.3: establishing a two-dimensional table representing the relationship between the input parameters and the output parameters of the controller by taking the input parameters as columns and the output parameters as rows to obtain a Q table;
step 2.4: designing reward rules: dividing the product E into c intervals by taking the product E of the water level deviation and the water level deviation derivative as a reward basis, and setting corresponding reward for each interval of the product E;
and step 3: training the SARSA controller, and updating a Q table in the SARSA controller;
and 4, step 4: judging whether the training result of the SARSA controller meets the control target or not, if not, adjusting the parameters of the SARSA controller according to the training result, and turning to the step 3; otherwise, the Q table is saved.
2. A SARSA-based nuclear power system steam generator level control method as claimed in claim 1 wherein said step 1 comprises the steps of:
step 1.1: determining five state parameters of the input of the SARSA controller, including last-time opening degree of a main water supply valve, water level deviation, a water level deviation derivative, flow mismatch and a flow mismatch derivative; wherein, the water level deviation is the deviation between the actual water level and the target water level, and the flow mismatch is the deviation between the feed water flow and the steam flow of the steam generator;
step 1.2: determining the output of the SARSA controller as the opening of a main water supply valve;
step 1.3: setting the control target as the absolute value of the product E of the water level deviation and the water level deviation derivative is less than or equal to a preset product threshold.
3. The SARSA-based nuclear power system steam generator level control method as claimed in claim 1, wherein in step 2, all input parameters and output parameters are calibrated with thresholds before interval division and reward rule design.
4. The SARSA-based nuclear power system steam generator level control method of claim 1, wherein in step 3, during training of the SARSA controller,
in a state StThe basis of the next selection execution action is
Figure FDA0003199190590000021
Wherein randomA means randomly selecting an action a from the action set,
Figure FDA0003199190590000022
indicates that Q (S) is selectedtThe action A, Q (S) with the largest value of A)tA) is in stateStThe action value function of the action A is executed, rand is a random number, and rand belongs to [0, 1]]Epsilon is a random factor, epsilon is epsilon [0,1 ∈]；
The update rule of the Q table is
Figure FDA0003199190590000023
Wherein S istIndicating the state of the steam generator input to the controller at time t, AtRepresenting the action of the SARSA controller output at time t, RtIndicating that the SARSA controller is in state StDown execution action AtThe reward obtained; α represents the learning rate of the SARSA controller, α ∈ (0, 1)](ii) a γ represents forgetting speed, γ ∈ (0, 1)](ii) a The isdone is a termination mark, the isdone-1 represents that the t moment reaches the preset maximum simulation time, and the isdone-0 represents that the t moment does not reach the preset maximum simulation time.
5. The method according to claim 1, wherein in step 3, the training environment of the SARSA controller is a nuclear power system steam generator simulation model, the inputs of the nuclear power system steam generator simulation model include a feedwater flow, a feedwater temperature, an effluent steam flow, a primary side inlet specific enthalpy, a primary side inlet flow, and a primary side inlet temperature, and the outputs of the nuclear power system steam generator simulation model include a steam generator water level, a steam chamber pressure, a primary side outlet specific enthalpy, and a primary side outlet temperature.
6. The SARSA-based nuclear power system steam generator water level control method of claim 4, wherein in step 4, the method for adjusting the parameters of the SARSA controller according to the training result comprises:
calculating the fluctuation of a training result, namely the fluctuation of the absolute value of the product E of the water level deviation and the water level deviation derivative relative to a preset product threshold, and adjusting one or more of the learning speed alpha, the forgetting speed gamma and the random factor epsilon when the fluctuation exceeds an allowable oscillation range; the learning speed alpha is adjusted to be increased or decreased, the forgetting speed gamma is adjusted to be increased or decreased, the random factor epsilon is adjusted to be decreased, the adjusting range of the learning speed alpha and the forgetting speed gamma is within 0.01-0.05, and the adjusting range of the random factor epsilon is within 0.01-0.08.
CN202010502732.6A 2020-06-05 2020-06-05 Nuclear power system steam generator water level control method based on SARSA Active CN111637435B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010502732.6A CN111637435B (en) 2020-06-05 2020-06-05 Nuclear power system steam generator water level control method based on SARSA

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010502732.6A CN111637435B (en) 2020-06-05 2020-06-05 Nuclear power system steam generator water level control method based on SARSA

Publications (2)

Publication Number Publication Date
CN111637435A CN111637435A (en) 2020-09-08
CN111637435B true CN111637435B (en) 2021-09-28

Family

ID=72330663

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010502732.6A Active CN111637435B (en) 2020-06-05 2020-06-05 Nuclear power system steam generator water level control method based on SARSA

Country Status (1)

Country Link
CN (1) CN111637435B (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100541360C (en) * 2007-11-29 2009-09-16 哈尔滨工程大学 Nuclear power device two-loop multi-variable integrated model fuzzy predication control method
CN104965410A (en) * 2015-04-17 2015-10-07 华南理工大学 Dynamic sliding mode variable structure controller of steam generator water level system and control method
CN105137957A (en) * 2015-06-23 2015-12-09 黄红林 Control method of nuclear power steam generator
CN109143870B (en) * 2018-10-23 2021-08-06 宁波溪棠信息科技有限公司 Multi-target task control method
CN110880048B (en) * 2019-11-06 2022-06-21 国网湖北省电力有限公司宜昌供电公司 Cascade reservoir ecological random optimization scheduling model and solving method

Also Published As

Publication number Publication date
CN111637435A (en) 2020-09-08

Similar Documents

Publication Publication Date Title
Esfandyari et al. Adaptive fuzzy tuning of PID controllers
Coban A fuzzy controller design for nuclear research reactors using the particle swarm optimization algorithm
CN111637444B (en) Nuclear power steam generator water level control method based on Q learning
CN110879620B (en) Liquid level control method and system for vertical steam generator of nuclear power station
Hosseini et al. Design and application of supervisory control based on neural network PID controllers for pressurizer system
CN104102782A (en) Method for controlling temperatures of reactors of pressurized water reactor nuclear power stations by aid of RBF (radial basis function) neural networks
CN108121215A (en) Process control loops method of evaluating performance and device based on full loop reconstruct emulation
Liu et al. Multiobjective robust H2/H∞ fuzzy tracking control for thermal system of power plant
CN110515304A (en) Overheating steam temperature PID forecast Control Algorithm based on ARX-Laguerre function model
Qiu et al. Research on fuzzy weighted gain scheduling water level control system of U-tube steam generator
CN105720574A (en) SPSA-based data drive control method for single region load frequency of power system
CN111637435B (en) Nuclear power system steam generator water level control method based on SARSA
Kavaklioglu Support vector regression model based predictive control of water level of U-tube steam generators
CN113885328A (en) Nuclear power tracking control method based on integral reinforcement learning
CN108427271A (en) Pressurized-water reactor nuclear power plant primary Ioops coolant temperature control method
Nie et al. A tuning method for PID controller parameters based on particle swarm optimization (PSO)
Lu et al. Evaluation of an FPGA-based fuzzy logic control of feed-water for ABWR under automatic power regulating
Shen et al. Design and development of the power regulation system for lead-cooled fast reactor based on system modeling and fuzzy algorithm
Wu et al. Parameter optimization for AP1000 steam generator feedwater control system using particle swarm optimization algorithm
Acharya et al. Optimal rule based fuzzy-PI controller for core power control of nuclear reactor
Sheng et al. Sliding mode controller with AGA for drum water level of ship boiler
Zhou et al. The preliminary research of pressure control system danymic simulation for Ap1000 pressurizer based on parameter adaptive fuzzy pid control algorithm
Xiao et al. Research on the robustness of the roots waste heat power generation system
Huang et al. Design of a FPGA based ABWR feedwater controller
Dai et al. Simulation of Main Steam Temperature Control System Based on Neural Network.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20231005

Address after: 215200 No. 158, Chengsi Road, Lili Town, Wujiang District, Suzhou City, Jiangsu Province

Patentee after: Aerospace Promotion (Suzhou) Aerospace Technology Co.,Ltd.

Address before: 110136, Liaoning, Shenyang moral and Economic Development Zone, No. 37 South Avenue moral

Patentee before: SHENYANG AEROSPACE University