CN114507881A - Model-free self-learning and stable control method for electrolyte temperature in zinc electrolysis process - Google Patents

Abstract

The embodiments of the present disclosure provide a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process, which belongs to the field of chemical technology, and specifically includes: establishing an environmental interaction model corresponding to a Q-learning algorithm, a reward mechanism and a Q-table, setting an electrolysis The target interval that the liquid temperature needs to be controlled, and initialize the parameters required for the Q table update; define the state space and action space of the electrolyte in the zinc electrolysis process; define the Q table, the horizontal axis represents the optional action, and the vertical axis represents the type of state space; The data generated by the interaction between the agent and the environment interaction model updates the Q table; according to the updated Q table, the stable control model corresponding to the electrolyte temperature in the zinc electrolysis process is obtained, and the optimal cooling corresponding to the current electrolyte state is output according to the stable control model. Tower fan frequency. Through the solution of the present disclosure, it is automatically ensured that the temperature of the zinc electrolyte is always within the required range of the process, and the control efficiency, adaptability and stability of the production of the zinc electrolysis process are improved.

Description

Model-free self-learning and stable control method for electrolyte temperature in zinc electrolysis process

technical field

The embodiments of the present disclosure relate to the field of chemical technology, and in particular, to a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process.

Background technique

At present, the important process of zinc electrolysis hydrometallurgy process refers to the process in which zinc ions in the electrolyte flow from the anode plate to the cathode plate under the action of direct current, and then precipitate on the cathode plate to form zinc element. Because it consumes a lot of electricity, it greatly affects the production cost of zinc smelters. As one of the key parameters of electrolysis process control, the temperature of the electrolyte has an important influence on the efficiency and quality of zinc precipitation. In order to ensure the efficient precipitation of zinc, the outlet temperature of the electrolyte is required to be within a suitable range. Too high electrolyte temperature will reduce the overvoltage of hydrogen precipitation, which will intensify the inversion of precipitation zinc and reduce the current efficiency.

The electrolyte solution circulation system has the characteristics of large circulation flow and uninterrupted circulation. Taking a zinc smelting enterprise with an annual output of 300,000 tons as an example, the volume of electrolyte solution in the electrolytic cell alone reaches thousands of cubic meters, and the solution circulates for 24 hours. Uninterrupted, is a typical large-scale circulation process system. Not only that, because the electrolyte is stored in the open air, the ambient temperature has a great influence on the temperature of the electrolyte. Due to the obvious changes in ambient temperature during the day and night and the four seasons, it is very difficult to accurately control the temperature of the electrolyte in different time periods.

In order to keep the electrolyte temperature within a suitable range, the current practice of enterprises is to install a mechanical ventilation cooling tower in the electrolyte circulation system, and cool the electrolyte temperature by blowing low-temperature air from outside the workshop into the electrolyte circulation pipeline. According to whether the cooling tower fan is installed with a frequency conversion device, the control strategy of the cooling tower fan can be divided into two types: frequency conversion and non-frequency conversion. For frequency conversion cooling towers, the cooling performance of the cooling tower can be adjusted by adjusting the fan frequency; for non-frequency conversion cooling towers, the cooling performance of the cooling tower can be adjusted by adjusting the opening time of the cooling tower fans.

According to industry standards, the suitable range of electrolyte temperature is 35-40°C. Taking the variable frequency cooling tower as an example, in a high temperature environment, the frequency of the cooling tower fan needs to be increased to improve the cooling performance of the cooling tower; in a low temperature environment, the frequency of the cooling tower fan needs to be reduced to reduce the cooling tower cooling performance. If the cooling tower is not installed with a frequency conversion device, the same as the frequency conversion cooling tower, increase the operating time of the cooling tower in the high temperature season, and reduce the operating time of the cooling tower in the low temperature season. This ensures that the electrolyte temperature is within a suitable range. The method aims at the frequency conversion cooling tower, realizes the adjustment of the cooling performance of the cooling tower by changing the frequency of the cooling tower fan, and finally realizes the adjustment of the temperature of the electrolyte.

At present, the control strategy of zinc electrolysis enterprises for cooling towers relies on manual experience. When the temperature difference between day and night is large, the frequency of cooling tower fans needs to be adjusted frequently, which is labor-intensive. The feedback information lags, resulting in serious lag and instability in the electrolyte temperature control based on artificial experience.

It can be seen that there is an urgent need for a stable control method that can automatically and effectively realize the temperature of the electrolyte in the zinc electrolysis process, to ensure that the temperature of the electrolyte is always within the range of process requirements, to ensure that the zinc ions in the electrolyte can be efficiently precipitated at the cathode, and to reduce the labor intensity of workers. A model-free self-learning stable control method for electrolyte temperature during zinc electrolysis.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present disclosure provide a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process, which at least partially solves the problems of poor automation, adaptability, control efficiency and stability in the prior art.

The embodiments of the present disclosure provide a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process, including:

Step 1, establish the environmental interaction model, reward mechanism and Q table corresponding to the Q learning algorithm, set the target interval that the electrolyte temperature needs to be controlled, and initialize the parameters required for the Q table update, wherein the parameters include a discount factor, learning rate and random factor;

Step 2, define the state space and action space of the electrolyte in the zinc electrolysis process, wherein, the action space is the cooling tower fan frequency;

Step 3, define the Q table, the horizontal axis represents optional actions, and the vertical axis represents the type of state space, wherein the state space includes ambient dry bulb temperature, ambient air wet bulb temperature, ambient relative humidity and electrolyte temperature. variables, and the number of the categories is the number of combinations of the four variables permutation and combination;

Step 4, update the Q table according to the data generated by the interaction between the agent and the environment interaction model;

Step 5, according to the updated Q table, obtain the stable control model corresponding to the electrolyte temperature in the zinc electrolysis process, and output the optimal cooling tower fan frequency corresponding to the current electrolyte state according to the stable control model.

According to a specific implementation of the embodiment of the present disclosure, the environment interaction model is built by using a BP neural network, and the input parameters are the fan frequency f _tower at time t, the ambient dry bulb temperature at time t T _dry , and the ambient wet bulb at time t The temperature T _wet , the ambient relative humidity RH at time t, the temperature of the electrolyte solution at time t T _elec , and the output parameter is the temperature of the electrolyte solution at time t+1 T _elec .

According to a specific implementation manner of the embodiment of the present disclosure, before the step 4, the method further includes:

Define the upper and lower limits of the controlled temperature as the preset interval;

The control target of the electrolyte temperature during the interaction between the agent and the environment interaction model is set within the preset interval.

According to a specific implementation of the embodiment of the present disclosure, the calculation method of the reward mechanism is as follows

According to a specific implementation manner of the embodiment of the present disclosure, the step 4 specifically includes:

Step 4.1, set the initial state s, step=0;

Step 4.2, action selection: random factor rand takes a random value, if rand>0, select the action a with the largest Q value in state s; if rand=0, select a state randomly from all states, and select the Action a with the largest Q value;

Step 4.3, the agent enters the action a into the environment interaction model env, and obtains the new state s' at the next moment;

Step 4.4, check all actions in s', and take the action with the largest Q value in s' as a';

Step 4.5, determine whether this iteration process is qualified: obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment, if T _min ≤ T' ≤ T _max , it means that this iteration process is qualified, The reward is 0, go to step 4.6, if T _min ≤ T'≤T _max is not satisfied, it means that this iteration process failed, the reward is -1, and go back to step 4.1;

Step 4.6, use the following formula to update the Q value of the current action a:

Step 4.7, let s=s', a=a', step=step+1, go back to step 4.1, continue the cycle, define 4.3 to 4.7 as a step, the process of interaction between the agent and the environment interaction model env is step The process of updating the Q table for _max times;

Step 4.8, the step is accumulated to the value step _max specified by the user, and the step of updating the Q table ends.

The model-free self-learning and stable control scheme for the electrolyte temperature in the zinc electrolysis process in the embodiments of the present disclosure includes: Step 1, establishing an environmental interaction model, a reward mechanism and a Q table corresponding to the Q-learning algorithm, and setting the target to be controlled by the electrolyte temperature interval, and initialize the parameters required for the Q table update, wherein the parameters include discount factor, learning rate and random factor; Step 2, define the state space and action space of the electrolyte in the zinc electrolysis process, wherein the action space is the cooling tower fan frequency; step 3, define the Q table, the horizontal axis represents optional actions, and the vertical axis represents the type of state space, wherein the state space includes ambient dry bulb temperature, ambient air wet bulb temperature, ambient relative There are four variables of humidity and electrolyte temperature, and the number of the types is the number of combinations of the four variables; step 4, update the Q table according to the data generated by the interaction between the agent and the environment interaction model; step 5, according to the update After completing the Q table, the stable control model corresponding to the electrolyte temperature in the zinc electrolysis process is obtained, and the optimal cooling tower fan frequency corresponding to the current electrolyte state is output according to the stable control model.

The beneficial effects of the embodiments of the present disclosure are: through the solution of the present disclosure, the temperature of the electrolyte, the ambient dry bulb temperature, the ambient wet bulb temperature, and the ambient relative humidity are used as the electrolyte temperature state space in the zinc electrolysis process, and the cooling tower fan frequency is set It is the action space, and the Q table is updated according to the data generated by the interaction process between the agent and the environment interaction model, and finally the electrolyte temperature stability control model of the zinc electrolysis process is obtained. The unstable temperature control of the electrolyte caused by the changing environmental temperature parameters ensures that the temperature of the zinc electrolyte is always within the required range of the process and improves the stability of the production of the zinc electrolysis process.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process provided by an embodiment of the present disclosure;

2 is a schematic diagram of an electrolyte environment interaction model based on BP network fitting provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of an interaction relationship between another agent and an environment interaction model provided by an embodiment of the present disclosure;

4 is a flowchart of a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process provided by the embodiments of the present disclosure;

5 is a flowchart of a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process according to Embodiment 2 provided by the embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a comparison diagram of the effects of a method using Embodiment 2 of the present invention and artificial experience provided by an embodiment of the present disclosure.

Detailed ways

The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

The embodiments of the present disclosure are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only some, but not all, embodiments of the present disclosure. The present disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

It is noted that various aspects of embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is illustrative only. Based on this disclosure, those skilled in the art should appreciate that an aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method may be practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.

It should also be noted that the drawings provided in the following embodiments are only illustrative of the basic concept of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and the number of components in actual implementation. For dimension drawing, the type, quantity and proportion of each component can be arbitrarily changed in actual implementation, and the component layout may also be more complicated.

Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, one skilled in the art will understand that the described aspects may be practiced without these specific details.

Embodiments of the present disclosure provide a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process, and the method can be applied to a zinc electrolysis process electrolyte temperature stability control process in chemical and metallurgical scenarios.

Referring to FIG. 1 , a schematic flowchart of a model-free self-learning stabilization method for electrolyte temperature in a zinc electrolysis process provided by an embodiment of the present disclosure. As shown in Figure 1, the method mainly includes the following steps:

Step 1, establish the environmental interaction model, reward mechanism and Q table corresponding to the Q learning algorithm, set the target interval that the electrolyte temperature needs to be controlled, and initialize the parameters required for the Q table update, wherein the parameters include a discount factor, learning rate and random factor;

Optionally, as shown in FIG. 2 , the environment interaction model can be built using a BP neural network, and the input parameters are the fan frequency f _tower at time t , the ambient dry bulb temperature at time t T _dry , and the ambient wet bulb temperature at time t . T _wet , the ambient relative humidity RH at time t, the temperature of the electrolyte solution at time t T _elec , and the output parameter is the temperature of the electrolyte solution at time t+1 T _elec .

Optionally, the calculation method of the reward mechanism is:

In the specific implementation, the environment interaction model env and the reward mechanism r used by the Q-learning control algorithm can be established, the target interval C={T _min , T _max } that needs to be controlled for the temperature of the electrolyte can be set, and the parameters required for the update of the Q table can be initialized, such as Discount factor γ, learning rate α, random factor rand, etc. Specifically, the environment interaction model env is built by using a BP neural network, and the input parameters are: the fan frequency f _tower at time t , the ambient dry bulb temperature at time t T _dry , the ambient wet bulb temperature at time t T _wet , and the relative environment at time t Humidity RH, electrolyte temperature _Telec at time t. The output parameter is: electrolyte temperature T _elec at time t+1.

The calculation method of the reward mechanism r is:

Step 2, define the state space and action space of the electrolyte in the zinc electrolysis process, wherein, the action space is the cooling tower fan frequency;

During the specific implementation, it is necessary to define the state space and action space of the electrolyte in the zinc electrolysis process, wherein the state space includes the ambient dry bulb temperature T _dry , the ambient air wet bulb temperature T _wet , the ambient relative humidity RH, and the electrolyte temperature T _elec Four variables, the action space is the cooling tower fan frequency f _tower .

Step 3, define the Q table, the horizontal axis represents optional actions, and the vertical axis represents the type of state space, wherein the state space includes ambient dry bulb temperature, ambient air wet bulb temperature, ambient relative humidity and electrolyte temperature. variables, and the number of the categories is the number of combinations of the four variables permutation and combination;

In the specific implementation, the Q table is an m×n matrix, where the horizontal direction represents the optional action (cooling tower fan frequency) [a ₁ , a ₂ , a ₃ ,..., a _n ], where n is the optional action The number of types of , the vertical axis is the different state spaces [S ₁ , S ₂ , S ₃ ,..., S _m ], the number m depends on the ambient dry bulb temperature T _dry , the ambient air wet bulb temperature T _wet , Ambient relative humidity RH, electrolyte temperature T _elec The number of combinations of four variable permutations: divide the ambient dry bulb temperature as m ₁ case, divide the ambient wet bulb temperature into m ₂ cases, and divide the relative humidity into m ₃ cases The temperature of the electrolyte is divided into m ₄ cases, and the last state is that the electrolyte exceeds the standard, so the number of states is m=m ₁ ×m ₂ ×m ₃ ×m ₄ +1.

Step 4, update the Q table according to the data generated by the interaction between the agent and the environment interaction model;

Optionally, before the step 4, the method further includes:

Define the upper and lower limits of the controlled temperature as the preset interval;

The control target of the electrolyte temperature during the interaction between the agent and the environment interaction model is set within the preset interval.

Further, the step 4 specifically includes:

Step 4.1, set the initial state s, step=0;

Step 4.2, action selection: random factor rand takes a random value, if rand>0, select the action a with the largest Q value in state s; if rand=0, select a state randomly from all states, and select the Action a with the largest Q value;

Step 4.3, the agent enters the action a into the environment interaction model env, and obtains the new state s' at the next moment;

Step 4.4, check all actions in s', and take the action with the largest Q value in s' as a';

Step 4.5, determine whether this iteration process is qualified: obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment, if T _min ≤ T' ≤ T _max , it means that this iteration process is qualified, The reward is 0, go to step 4.6, if T _min ≤ T'≤T _max is not satisfied, it means that this iteration process failed, the reward is -1, and go back to step 4.1;

Step 4.6, use the following formula to update the Q value of the current action a:

Step 4.7, let s=s', a=a', step=step+1, go back to step 4.1, continue the cycle, define 4.3 to 4.7 as a step, the process of interaction between the agent and the environment interaction model env is step The process of updating the Q table for _max times;

Step 4.8, the step is accumulated to the value step _max specified by the user, and the step of updating the Q table ends.

During specific implementation, as shown in Figure 3, the data [s _t , at , r _t , s _{t +1} ] generated by the constant interaction between the agent and the zinc electrolyte environment interaction model env. where s _t is the zinc electrolysis state at time t, that is, the above state space, at is the frequency of the tower fan at time t, that is, the above action space, _rt is the reward at time t, and s _t+1 is the time at t+1. In the zinc electrolysis state, the agent outputs the action a _t+1 at the next moment according to the reward r _t input at the current moment t and the state s _t , and the environment interaction model env outputs the action at the current moment t according to the input action a _t and the state s _t In the state s _t+1 at a moment, the training process of the agent is actually the continuous interaction between the agent and the environment interaction model to generate data [s _t , a _t , r _t , s _t+1 ], thereby updating the content of the Q table. . When the training requirements are met, the content of the Q table is fixed, and the agent has learned the electrolyte temperature control method that meets the requirements. Specifically, there is no need to specifically model the agent, and the training process is actually the agent and the environment. The interaction model env constantly interacts, and the process of revising the Q table does not require human control.

At the same time, when the agent interacts with the zinc electrolyte environment interaction model env to generate data [s _t , at , r _t , s _{t +1} ], the following content needs to be defined before training the Q table: Define the upper and lower limits of the controlled temperature , that is, it is required that the temperature of the electrolyte must be within a specified interval, and it is set to C={T _min , T _max }, where Tmin is the minimum value of the specified electrolyte temperature, and Tmax is the specified maximum value of the electrolyte temperature. During the interaction with the zinc electrolyte environment interaction model env, the control target is that the temperature of the controlled electrolyte is always within the interval C, that is, the controlled temperature is not lower than Tmin and not higher than Tmax.

The goal of the Q-learning algorithm is to obtain the value function corresponding to the state-action, which is represented by Q(s, a), where s is the state and a is the action. The learning goal of the Q-learning algorithm is to learn m×n Q-tables Q value, the learning process is to continuously interact with the environment model env to obtain the reward r of the environment, thereby forming the Q value corresponding to the state-action pair in the Q table, and iteratively modify the Q value in the Q table through the Q value update rule. value. The update rule for the Q value is:

(1) Initialize the Q table. The horizontal axis of Q is n alternative actions, and the vertical axis is m states. Therefore, there are m×n Q values in the table. In the initial stage, all Q values are 0.

(2) Define a random factor Rand, whose value is a random number between 0 and 1. Every time step (7) is executed, rand will be updated.

(3) Set the discount factor γ, the value is between 0 and 1, and the specific value is specified by the user.

(4) Set the learning rate α, the value is between 0 and 1, and the specific value is specified by the user.

(5) Set the maximum value step _max of the iterative step size step, and specify the specific value by the user.

(6) Set the calculation method of the reward r.

(7) Set the initial state s, step=0.

(8) Action selection: if rand>0, select the action a with the largest Q value in the state s; if rand=0, randomly select a state from all states, and select the action a with the largest Q value in this state.

(9) The agent inputs the action a into the environment interaction model env, and obtains the new state s' at the next moment.

(10) Check all actions in s' to see which action has the largest Q value under s', assuming that the action is a'.

(11) Judging whether this iteration process is qualified: Obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment. If T _min ≤ T' ≤ T _max , it means that this iteration process is qualified, If the reward is 0, go to step (12); otherwise, if T _min ≤ T'≤T _max is not satisfied, it means that this iteration process fails, the reward is -1, and go back to step (7).

(12) Use the following formula to update the Q value of the current action a:

(13)

(14) Let s=s', a=a', step=step+1, return to step (7), continue the loop, define (9) to (14) as a step, the agent interacts with the environment model env The process is the process of updating the Q table step _max times.

(15) step is accumulated to the value step _max specified by the user, and the Q table update step ends.

Step 5, according to the updated Q table, obtain the stable control model corresponding to the electrolyte temperature in the zinc electrolysis process, and output the optimal cooling tower fan frequency corresponding to the current electrolyte state according to the stable control model.

During specific implementation, after the Q table is updated, a stable control model corresponding to the electrolyte temperature in the zinc electrolysis process can be obtained according to the updated Q table, and the optimal corresponding to the current electrolyte state can be output according to the stable control model. Cooling tower fan frequency to complete the stable control process.

The model-free self-learning and stable control method for the electrolyte temperature in the zinc electrolysis process provided in this embodiment, by using the electrolyte temperature, the ambient dry bulb temperature, the ambient wet bulb temperature, and the ambient relative humidity as the electrolyte temperature state space in the zinc electrolysis process, the cooling The tower fan frequency is set as the action space, and the Q table is updated according to the data generated by the interaction process between the agent and the environment interaction model, and finally the electrolyte temperature stability control model in the zinc electrolysis process is obtained, which solves the problem of the existing technology due to the large electrolyte circulation flow. , Uninterrupted cycle, unstable temperature control of electrolyte caused by changing environmental temperature parameters, so as to ensure that the temperature of zinc electrolyte is always within the required range of the process, and improve the stability of zinc electrolysis process production.

The present solution will be described below with reference to two embodiments.

Example 1

Referring to Fig. 4, the zinc electrolysis process electrolyte temperature model-free self-learning and stable control method provided by Example 1 of the present invention includes:

Step S101: establish an environmental interaction model env and a reward r mechanism used by the Q-learning control algorithm, set a target interval C={T _min , T _max } that needs to be controlled for the temperature of the electrolyte, and initialize the parameters required for the Q table update, such as a discount factor γ, learning rate α, random factor rand, etc.

Step S102 : define the state space and action space of the electrolyte in the zinc electrolysis process, wherein the state space includes four variables: the electrolyte temperature T _elec , the ambient dry bulb temperature T _dry , the ambient wet bulb temperature T _wet , and the relative humidity RH , that is, S=[ T _dry , T _wet , RH, _Telec ], the action space is the cooling tower fan frequency f _tower .

Step S103: define a Q table, the horizontal axis represents optional actions, the vertical axis represents the types of state space, and the number of types m depends on four variables: ambient dry bulb temperature, ambient air wet bulb temperature, ambient relative humidity, and electrolyte temperature The number _of combinations of permutations: divide the ambient dry bulb temperature into m1 cases, divide the ambient wet bulb temperature into _m2 cases, divide the relative humidity into _m3 cases, and divide the electrolyte temperature into _m4 cases, The last state is that the electrolyte exceeds the standard, so the number of states is m=m ₁ ×m ₂ ×m ₃ ×m ₄ +1.

Step S104: Update the Q table according to the data generated by the interaction model between the agent and the environment, so as to obtain a temperature stability control model of the electrolyte in the zinc electrolysis process. The update process is:

(1) Set the initial state s, step=0.

(2) Action selection: the random factor rand takes a random value. If rand>0, select the action a with the largest Q value in state s; if rand=0, randomly select a state from all states, and select the Action a with the largest Q value.

(3) The agent inputs the action a into the environment interaction model env, and obtains the new state s' at the next moment.

(4) Check all the actions in s' to see which action has the largest Q value under s', assuming that the action is a'.

(5) Judging whether this iteration process is qualified: Obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment. If T _min ≤ T' ≤ T _max , it means that this iteration process is qualified If the reward is 0, go to step (6); on the contrary, if T _min ≤ T'≤T _max is not satisfied, it means that the iterative process fails, the reward is -1, and return to step (1).

(6) Use the following formula to update the Q value of the current action a:

(7)

(8) Let s=s', a=a', step=step+1, return to step (3), continue the cycle, define (3) to (8) as a step, the agent interacts with the environment model env The process is the process of updating the Q table step _max times.

(9) step is accumulated to the value step _max specified by the user, and the Q table update step ends.

Step S105: According to the updated Q table, obtain the electrolyte temperature stability control model in the zinc electrolysis process, and output the optimal cooling tower fan frequency corresponding to the current electrolyte state.

The present invention provides a model-free self-learning and stable control method for electrolyte temperature in zinc electrolysis process. The fan frequency is set as the action space, and the Q table is updated according to the data generated by the interaction process between the agent and the environment interaction model, and finally the electrolyte temperature stability control model in the zinc electrolysis process is obtained, which solves the problem of the existing technology due to the large circulating flow of electrolyte, The uninterrupted cycle and the unstable temperature control of the electrolyte caused by the changing environmental temperature parameters ensure that the temperature of the zinc electrolyte is always within the required range of the process, improve the production stability of the zinc electrolysis process, and help optimize the zinc hydrometallurgy process. significant.

Specifically, the embodiment of the present invention innovatively introduces the idea of Q-learning reinforcement learning algorithm, the self-defined zinc electrolyte temperature space is the electrolyte temperature, the environmental dry bulb temperature, the environmental wet bulb temperature, and the relative humidity, and the self-defined action space is the cooling tower Fan frequency, establish a chemical environment interaction model based on BP network, and obtain data through continuous interaction between the agent and the environment interaction model, thereby updating the Q table, so that the agent can realize model-free autonomous learning of new electrolyte temperature stability control strategies, ensuring zinc The temperature of the electrolyte is always within the range of process requirements, which improves the stability of the zinc electrolysis process.

Embodiment 2

5 , a model-free self-learning and stable control method for electrolyte temperature in a zinc electrolysis process provided by Embodiment 2 of the present invention includes:

Step S101: establish an environmental interaction model, a Q table, and a reward mechanism used by the Q-learning control algorithm, set a target interval C={T _min , T _max } that needs to be controlled for the temperature of the electrolyte, and initialize the parameters required for updating the Q table, such as discounts Factor γ, learning rate α, random factor rand, etc. The specific contents are:

The environment interaction model env used by the Q-learning algorithm is established. The model is built using a BP network. The input parameters are the electrolyte temperature at time t, the ambient dry bulb temperature at time t, the ambient wet bulb temperature at time t, and the relative environment at time t. Humidity, the cooling tower fan frequency at time t (this value is given by the agent). The output parameter is the electrolyte temperature at time t+1. The number of hidden layers of the network is 20.

Define the reward mechanism, and determine the value of the reward r according to the state s _t+1 output by the environment interaction model env, specifically:

Define the target interval C={T _min , T _max } that the electrolyte temperature needs to be controlled, wherein T _min =37, T _max =40.

Step S102 : define the state space and action space of the electrolyte in the zinc electrolysis process, wherein the state space includes four variables: the electrolyte temperature T _elec , the ambient dry bulb temperature T _dry , the ambient wet bulb temperature T _wet , and the relative humidity RH , that is, S=[ T _dry , T _wet , RH, _Telec ], the action space is the cooling tower fan frequency f _tower .

Step S103: establish a Q table, in which the horizontal axis is the optional frequency of the cooling tower fan, which is divided into 14 types, namely 24Hz, 26Hz, 28Hz, 30Hz, 32Hz, 34Hz, 36Hz, 38Hz, 40Hz, 42Hz, 44Hz, 46Hz, 48Hz , 50Hz; the vertical axis is the 109 states of the electrolyte space, according to the different numerical arrangement and combination of the four parameters of electrolyte temperature, ambient dry bulb temperature, ambient wet bulb temperature, and relative humidity, a total of 3×3×3×4+1 = 109 states. They are:

(1) If the electrolyte temperature exceeds 40 °C or is lower than 37 °C, it will enter the 109th state, and it means that this step has failed. The agent needs to clear all the contents of the Q table and restart the update of the Q value. Where T _elec is the electrolyte temperature, box is the state name of the 109th state

if(T _elec <37||T _elec >40)

box=109;

else

(2) The temperature of the electrolyte is changed into three grades, and there are 3 situations in total, wherein T _elec is the temperature of the electrolyte, and T _{elec_Bucket} is the sign used to divide the temperature state of the electrolyte

(3) Convert the ambient dry bulb temperature into three grades, with 3 cases in total, where T _dry is the ambient dry bulb temperature, and T _dry _Bucket is the identifier used to divide the ambient dry bulb temperature state

(4) Convert the relative humidity into three grades, a total of 3 situations, where RH is the relative humidity of the environment, and RH_Bucket is the identifier used to divide the relative humidity state of the environment

(5) Convert the ambient wet bulb temperature into four grades, a total of 4 cases, in which T _wet is the ambient wet bulb temperature, and T _wet _Bucket is the identifier used to divide the ambient wet bulb temperature state

Initialize the parameters required to update the Q table, such as discount factor γ=0.9, learning rate α=0.5, and random factor rand will be updated at each step, ranging from 0 to 1.

Step S104: Update the Q table according to the data generated by the interaction model between the agent and the environment, so as to obtain a temperature stability control model of the electrolyte in the zinc electrolysis process. The update process is:

(1) Set the initial state s, step=0.

(2) Action selection: the random factor rand takes a random value. If rand>0, select the action a with the largest Q value in state s; if rand=0, randomly select a state from all states, and select the Action a with the largest Q value.

(3) The agent inputs the action a into the environment interaction model env, and obtains the new state s' at the next moment.

(4) Check all the actions in s' to see which action has the largest Q value under s', assuming that the action is a'.

(5) Judging whether this iteration process is qualified: Obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment. If T _min ≤ T' ≤ T _max , it means that this iteration process is qualified If the reward is 0, go to step (12); on the contrary, if T _min ≤ T'≤T _max is not satisfied, it means that the iterative process failed, the reward is -1, and go back to step (7)

(6) Use the following formula to update the Q value of the current action a:

(7)

(8) Let s=s', a=a', step=step+1, return to step (3), continue the cycle, define (3) to (8) as a step, the agent interacts with the environment model env The process is the process of updating the Q table step _max times.

(9) step is accumulated to the value step _max specified by the user, and the Q table update step ends.

Step S105: According to the updated Q table, obtain the electrolyte temperature stability control model in the zinc electrolysis process, and output the optimal cooling tower fan frequency corresponding to the current electrolyte state.

The existing electrolyte temperature relies on manual experience, and the worker sets the frequency of the cooling tower fan according to the ambient temperature of the day, which is labor-intensive. In addition, the temperature of the electrolyte in the electrolysis workshop is still measured manually, that is, the temperature of the electrolyte is offline, resulting in a certain hysteresis in the manual temperature control method.

In contrast, the model-free self-learning electrolyte temperature stability control method proposed by this method does not require special modeling of the electrolyte temperature control process, but only needs to collect on-site ambient temperature data and electrolyte temperature data. The environment interaction model of the network, through the continuous interaction between the agent and the environment interaction model, obtains the updated Q table, and the model-free adaptive and stable control of the electrolyte temperature can be realized. The modeling link is omitted, and a good electrolyte can be realized. Temperature stable control effect.

Fig. 6 is the temperature control effect comparison between the on-site manual control method and this method, wherein, (a) of Fig. 6 is the electrolyte temperature curve controlled by manual experience in the electrolysis workshop, wherein the horizontal axis is time, the interval is 2 hours, and the vertical axis For the electrolyte temperature, the two horizontal lines in the figure are the interval limits of the lowest temperature 37 and the highest temperature 40 respectively; (b) of Fig. 6 adopts the electrolyte temperature curve controlled by this method, wherein the horizontal axis is time, and the interval is 2 Hour, the vertical axis is the temperature of the electrolyte, and the two horizontal lines in the figure are the interval limits of the minimum temperature of 37 and the maximum temperature of 40. After calculation, the average temperature of the artificially controlled temperature is 37.797, and the standard deviation is 1.16. The temperature controlled by this method is The mean value is 38.584, and the standard deviation is 0.545. It can be seen that the temperature curve under the control of this method meets the process requirements of 37-40. At the same time, compared with manual control, the standard deviation is smaller, indicating that the temperature control is smoother and more stable.

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.

The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited to this. Any person skilled in the art who is familiar with the technical scope of the present disclosure can easily think of changes or substitutions. All should be included within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims (5)

1. a zinc electrolysis process electrolyte temperature model-free self-learning stable control method, is characterized in that, comprises: Step 1, establish the environmental interaction model, reward mechanism and Q table corresponding to the Q learning algorithm, set the target interval that the electrolyte temperature needs to be controlled, and initialize the parameters required for the Q table update, wherein the parameters include a discount factor, learning rate and random factor; Step 2, define the state space and action space of the electrolyte in the zinc electrolysis process, wherein the action space is the cooling tower fan frequency; Step 3, define the Q table, the horizontal axis represents optional actions, and the vertical axis represents the type of state space, wherein the state space includes ambient dry bulb temperature, ambient air wet bulb temperature, ambient relative humidity and electrolyte temperature. variables, and the number of the categories is the number of combinations of the four variables permutation and combination; Step 4, update the Q table according to the data generated by the interaction between the agent and the environment interaction model; Step 5, according to the updated Q table, obtain the stable control model corresponding to the electrolyte temperature in the zinc electrolysis process, and output the optimal cooling tower fan frequency corresponding to the current electrolyte state according to the stable control model. 2. method according to claim 1 is characterized in that, described environment interaction model adopts BP neural network to build, input parameter is the fan frequency f _tower at time t, the ambient dry bulb temperature T _dry at time t, the The ambient wet bulb temperature T _wet , the ambient relative humidity RH at time t, the electrolyte temperature T _elec at time t, and the output parameter is the electrolyte temperature T _elec at time t+1. 3. The method according to claim 1, wherein, before the step 4, the method further comprises: Define the upper and lower limits of the controlled temperature as the preset interval; The control target of the electrolyte temperature during the interaction between the agent and the environment interaction model is set within the preset interval. 4. method according to claim 1, is characterized in that, the calculation mode of described reward mechanism is

5. method according to claim 1, is characterized in that, described step 4 specifically comprises: Step 4.1, set the initial state s, step=0; Step 4.2, action selection: random factor rand takes a random value, if rand>0, select the action a with the largest Q value in state s; if rand=0, select a state randomly from all states, and select the Action a with the largest Q value; Step 4.3, the agent enters the action a into the environment interaction model env, and obtains the new state s' at the next moment; Step 4.4, check all actions in s', and take the action with the largest Q value in s' as a'; Step 4.5, determine whether this iteration process is qualified: obtain the electrolyte temperature T' at the next moment according to the new state s' at the next moment, if T _min ≤ T' ≤ T _max , it means that this iteration process is qualified, The reward is 0, go to step 4.6, if T _min ≤ T'≤T _max is not satisfied, it means that this iteration process failed, the reward is -1, and go back to step 4.1; Step 4.6, use the following formula to update the Q value of the current action a:

Step 4.7, let s=s', a=a', step=step+1, go back to step 4.3, continue the cycle, define 4.3 to 4.7 as a step, the process of interaction between the agent and the environment interaction model env is step The process of updating the Q table for _max times; Step 4.8, the step is accumulated to the value step _max specified by the user, and the step of updating the Q table ends.

Images