CN114648178A - Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm - Google Patents

Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm Download PDF

Info

Publication number
CN114648178A
CN114648178A CN202210515703.2A CN202210515703A CN114648178A CN 114648178 A CN114648178 A CN 114648178A CN 202210515703 A CN202210515703 A CN 202210515703A CN 114648178 A CN114648178 A CN 114648178A
Authority
CN
China
Prior art keywords
electric energy
metering device
error
energy metering
maintenance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210515703.2A
Other languages
Chinese (zh)
Other versions
CN114648178B (en
Inventor
饶芳
陈勉舟
陈应林
袁成伟
黄晖
胡文韬
殷晓君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Gelanruo Intelligent Technology Co.,Ltd.
Original Assignee
Wuhan Glory Road Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Glory Road Intelligent Technology Co ltd filed Critical Wuhan Glory Road Intelligent Technology Co ltd
Priority to CN202210515703.2A priority Critical patent/CN114648178B/en
Publication of CN114648178A publication Critical patent/CN114648178A/en
Application granted granted Critical
Publication of CN114648178B publication Critical patent/CN114648178B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0639Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/20Administration of product repair or maintenance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/06Electricity, gas or water supply
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04SSYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

The invention relates to an operation and maintenance strategy optimization method of an electric energy metering device based on a DDPG algorithm, which comprises the following steps: evaluating the error state of the electric energy metering device based on the error state score and the set threshold range; establishing an operation and maintenance model based on a DDFG algorithm, wherein an Actor network of the operation and maintenance model generates operation and maintenance actions for the electric energy metering device; the Critic network of the operation and maintenance model evaluates the performance of the operation and maintenance action and guides the strategy function to generate the operation and maintenance action of the next stage; performing iterative update training on the operation and maintenance model; performing iterative optimization on a threshold range corresponding to the electric energy metering device to be evaluated based on the operation and maintenance model, and determining an operation and maintenance strategy of the electric energy metering device to be evaluated according to the optimized threshold range and the operation and maintenance model; the idea of reinforcement learning is used for solving the optimization problem of the operation and maintenance strategy of the electric energy metering device, so that the constraint of human experience in the traditional operation and maintenance mode is eliminated, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved.

Description

Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm
Technical Field
The invention relates to the field of operation and maintenance management of electric energy metering devices, in particular to an operation and maintenance strategy optimization method of an electric energy metering device based on a DDPG algorithm.
Background
The electric energy metering device comprises a voltage transformer, a current transformer, a secondary circuit and an electric energy meter, is an important tool for carrying out fair and fair transaction and accurate trade settlement among a power generation group, a power grid company, a power selling company and power utilization customers, is also an important basis for carrying out line loss assessment and bus balance calculation inside a power grid enterprise, and the accuracy and the stability of the operation of the electric energy metering device are related to the economic benefits of both sides of a trade and the economic benefits inside the enterprise. The periodic maintenance is the field verification of the electric energy metering device according to the management rule of the electric energy metering device and the periodic time required by the classification of the electric energy metering device. Along with the quantity and the scale of the electric energy metering devices are gradually increased, a large amount of manpower and material resources are wasted by the period verification required by regulations, and the enterprise benefits are damaged.
Aiming at the problems of 'insufficient maintenance' and 'excessive operation and maintenance' caused by periodic maintenance of the electric energy metering device, the invention provides an operation and maintenance strategy optimization method of the electric energy metering device based on a DDPG (Deep Deterministic Policy Gradient) algorithm, wherein the idea of reinforcement learning is used for solving the optimization problem of the operation and maintenance strategy of the electric energy metering device, the artificial experience constraint in the traditional operation and maintenance mode is eliminated, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides the DDPG algorithm-based electric energy metering device operation and maintenance strategy optimization method, which uses the idea of reinforcement learning to solve the optimization problem of the electric energy metering device operation and maintenance strategy, breaks away from the constraint of artificial experience in the traditional operation and maintenance mode, reduces the operation and maintenance cost and improves the operation and maintenance efficiency.
According to a first aspect of the invention, an electric energy metering device operation and maintenance strategy optimization method based on a DDPG algorithm is provided, and comprises the following steps: step 1, selecting each error state parameter of an electric energy metering device in historical data to establish a decision characteristic vector representing the error state of the electric energy metering device, calculating the error state score of the electric energy metering device according to the decision characteristic vector, and evaluating whether the error state of the electric energy metering device is stable, good or early-warning based on the error state score and a set threshold range;
step 2, establishing an operation and maintenance model based on a DDFG algorithm, wherein an Actor network of the operation and maintenance model generates operation and maintenance actions for the electric energy metering device; the Critic network of the operation and maintenance model evaluates the performance of the operation and maintenance action and guides a strategy function to generate the operation and maintenance action of the next stage; performing iterative update training on the operation and maintenance model according to the error state and the corresponding operation and maintenance action obtained in the step 1;
and 3, performing iterative optimization on the threshold range corresponding to the electric energy metering device to be evaluated based on the operation and maintenance model, and determining the operation and maintenance strategy of the electric energy metering device to be evaluated according to the optimized threshold range and the operation and maintenance model.
On the basis of the technical scheme, the invention can be improved as follows.
Optionally, in step 1, the decision feature vector is
Figure 960477DEST_PATH_IMAGE001
Wherein, in the step (A),
Figure 882297DEST_PATH_IMAGE002
for the purpose of real-time error estimation,
Figure 211778DEST_PATH_IMAGE003
for a short-term prediction error estimate sequence,
Figure 540778DEST_PATH_IMAGE004
for the long-term prediction error estimate sequence, W is the stability state.
Optionally, in step 1, the method for obtaining the error state parameter of the electric energy metering device includes:
calculating to obtain the real-time error estimated value by adopting an algorithm based on data driving
Figure 690131DEST_PATH_IMAGE002
Preprocessing error data of the electric energy metering device, stripping the error data into self error and additional error, constructing a trend prediction model by utilizing an ARIMA algorithm, inputting the self error into the trend prediction model to obtain a self error short-term prediction value of the electric energy metering device, calculating the additional error short-term prediction value of the electric energy metering device to be measured according to temperature information and frequency information, and adding the self error prediction value and the additional error prediction value to obtain a short-term prediction error estimation value sequence
Figure 805855DEST_PATH_IMAGE003
Processing the plurality of self errors into a time sequence, and inputting the time sequence into a trained LSTM model to obtain a self error long-term predicted value of the electric energy metering device; calculating an additional error long-term predicted value of the electric energy metering device according to the temperature information and the frequency information; fusing the self error long-term predicted value and the additional error long-term predicted value to obtain the long-term prediction error estimated value sequence
Figure 458684DEST_PATH_IMAGE004
The method comprises the following steps of constructing and obtaining a stability state evaluation index of the electric energy metering device, and establishing a stability state index data model of the electric energy metering device, wherein the stability state evaluation index comprises the following steps: a sudden change error stable frequency function model, a sudden change error unstable frequency function model, a gradual change error monotonous significance function model and a gradual change error standard deviation function model; comparing the importance of each state evaluation index by adopting a hierarchical analysis theory, determining the weight of the state evaluation index, calculating the stability state score of the electric energy metering device according to the result of each state evaluation index of the stability state index data model of the electric energy metering device and the corresponding weight, and evaluating whether the stability state W of the electric energy metering device is stable, slightly stable, moderately stable or severely stable according to the stability state score of the electric energy metering device;
in the step 1, after each error state parameter of the decision feature vector is calculated by a weighted comprehensive scoring method, the error state score is obtained.
Optionally, the step 1 of calculating the error state score of the electric energy metering device according to the decision feature vector includes:
101, estimating the error value based on the electric energy metering device
Figure 204923DEST_PATH_IMAGE005
Standard deviation of error estimate
Figure 435047DEST_PATH_IMAGE006
Establishing an error state scoring model of the electric energy metering device according to the precision k;
102, respectively calculating real-time error estimation values based on the error state scoring model
Figure 587418DEST_PATH_IMAGE002
Short-term prediction error estimation sequence
Figure 547283DEST_PATH_IMAGE003
And long-term prediction error estimate sequence
Figure 74211DEST_PATH_IMAGE004
Respectively is
Figure 916265DEST_PATH_IMAGE007
Figure 115734DEST_PATH_IMAGE008
And
Figure 992423DEST_PATH_IMAGE009
103, setting corresponding weights for the stability W of the electric energy metering device to be stable, slightly stable, moderately stable and heavily stable according to the stability degree respectively
Figure 768881DEST_PATH_IMAGE010
Figure 911280DEST_PATH_IMAGE011
Figure 592404DEST_PATH_IMAGE012
And
Figure 526862DEST_PATH_IMAGE013
to obtain
Figure 926750DEST_PATH_IMAGE014
104, calculating the error state scores of the error state parameters by adopting a weighted comprehensive scoring method to obtain the error state scores of the electric energy metering device
Figure 903583DEST_PATH_IMAGE015
105, setting each threshold value range of corresponding error state scores representing that the error state of the electric energy metering device is stable, good or early-warning, and scoring according to the error states
Figure 860169DEST_PATH_IMAGE015
The threshold value range is used for evaluating the error of the electric energy metering deviceA bad state.
Optionally, a scoring threshold parameter is used in the step 1 and the step 3
Figure 711451DEST_PATH_IMAGE016
Figure 157606DEST_PATH_IMAGE017
And
Figure 461549DEST_PATH_IMAGE018
represents the threshold range:
error state of the electric energy metering device
Figure 546792DEST_PATH_IMAGE019
Wherein the content of the first and second substances,
Figure 49318DEST_PATH_IMAGE020
Figure 400796DEST_PATH_IMAGE021
and
Figure 5083DEST_PATH_IMAGE022
respectively showing stability, good and early warning;
Figure 849279DEST_PATH_IMAGE023
scoring an error state of an electric energy metering device, a scoring threshold parameter
Figure 675152DEST_PATH_IMAGE016
Figure 587745DEST_PATH_IMAGE017
And
Figure 617012DEST_PATH_IMAGE018
satisfy the requirement of
Figure 296255DEST_PATH_IMAGE024
Optionally, step 101 is implementedThe error state scoring model is:
Figure 461788DEST_PATH_IMAGE025
;
wherein the content of the first and second substances,
Figure 732232DEST_PATH_IMAGE026
based on error estimation
Figure 183549DEST_PATH_IMAGE005
Error truth value of electric energy metering device obtained by calculation
Figure 666483DEST_PATH_IMAGE027
Out of range [ -k, k]Probability of (c):
Figure 952102DEST_PATH_IMAGE028
optionally, the error state score in step 102
Figure 393447DEST_PATH_IMAGE007
Figure 866148DEST_PATH_IMAGE008
And
Figure 152773DEST_PATH_IMAGE009
the calculation formulas of (A) and (B) are respectively as follows:
Figure 27319DEST_PATH_IMAGE029
Figure 373987DEST_PATH_IMAGE030
Figure 602493DEST_PATH_IMAGE031
where i and j are constants.
OptionallyCalculating an error state score of the electric energy metering device in step 104
Figure 692808DEST_PATH_IMAGE015
Comprises the following steps:
Figure 953020DEST_PATH_IMAGE032
optionally, the calculating process of the target Q value of the operation and maintenance model in step 2 includes:
step 201, setting the error state of the electric energy metering device at time t as
Figure 549217DEST_PATH_IMAGE033
Operation and maintenance actions as
Figure 245778DEST_PATH_IMAGE034
Figure 624938DEST_PATH_IMAGE035
Figure 661027DEST_PATH_IMAGE036
Wherein the content of the first and second substances,
Figure 159617DEST_PATH_IMAGE037
Figure 609052DEST_PATH_IMAGE038
and
Figure 713275DEST_PATH_IMAGE039
respectively representing forward extending of a verification period, and performing and arranging field verification according to a specified period;
step 202, define the function
Figure 416920DEST_PATH_IMAGE040
Representing a deterministic operation and maintenance action strategy, and the operation and maintenance of the electric energy metering device at any time tMovement of
Figure 276291DEST_PATH_IMAGE034
The calculation formula of (2) is as follows:
Figure 698176DEST_PATH_IMAGE041
step 203, define parameters
Figure 933986DEST_PATH_IMAGE042
And a function J, the parameter
Figure 492137DEST_PATH_IMAGE042
For the function
Figure 787989DEST_PATH_IMAGE043
The parameters of the strategy network for simulation are measured by the function J
Figure 688381DEST_PATH_IMAGE043
A function of the performance of the represented operation and maintenance action policy;
Figure 196723DEST_PATH_IMAGE044
wherein the content of the first and second substances,
Figure 874960DEST_PATH_IMAGE045
is a distribution function;
Figure 76135DEST_PATH_IMAGE046
indicating different error states as a function
Figure 472612DEST_PATH_IMAGE043
Q value which can be generated when the represented operation and maintenance action strategy is adopted;
Figure 784645DEST_PATH_IMAGE047
is an error state
Figure 582967DEST_PATH_IMAGE033
According to
Figure 751781DEST_PATH_IMAGE045
When distributed
Figure 835887DEST_PATH_IMAGE046
The expected value of (d);
step 204, the error state of the electric energy metering device is
Figure 951610DEST_PATH_IMAGE033
Take operation and maintenance actions
Figure 401177DEST_PATH_IMAGE034
Thereafter, and continuously executing the function
Figure 944154DEST_PATH_IMAGE043
Calculating the target under the represented operation and maintenance action strategy
Figure 49645DEST_PATH_IMAGE046
The values are:
Figure 969059DEST_PATH_IMAGE048
wherein the content of the first and second substances,
Figure 476395DEST_PATH_IMAGE049
r is the return value for the discount factor.
Optionally, step 3 includes:
step 301, defining an error state of an electric energy metering device
Figure 924694DEST_PATH_IMAGE033
The operation and maintenance actions
Figure 51568DEST_PATH_IMAGE034
A function of relationship of
Figure 243515DEST_PATH_IMAGE050
Satisfies the following conditions: as electric energy metering devicesError states are respectively
Figure 605358DEST_PATH_IMAGE020
Figure 21295DEST_PATH_IMAGE021
And
Figure 304640DEST_PATH_IMAGE022
the operation and maintenance actions executed are respectively corresponding to
Figure 565857DEST_PATH_IMAGE037
Figure 782206DEST_PATH_IMAGE038
And
Figure 900204DEST_PATH_IMAGE039
step 302, based on the target in the operation and maintenance model in the step 2
Figure 464653DEST_PATH_IMAGE051
Value and function of
Figure 263982DEST_PATH_IMAGE050
Corresponding deterministic action strategy to set the initial scoring threshold parameter
Figure 600416DEST_PATH_IMAGE016
Figure 295840DEST_PATH_IMAGE017
And
Figure 881673DEST_PATH_IMAGE018
performing iterative optimization to obtain the optimized grading threshold parameter
Figure 484693DEST_PATH_IMAGE052
Figure 941213DEST_PATH_IMAGE053
And
Figure 338696DEST_PATH_IMAGE054
Figure 98578DEST_PATH_IMAGE055
indicating the number of optimization iterations.
According to the operation and maintenance strategy optimization method of the electric energy metering device based on the DDPG algorithm, the accuracy of error state evaluation of the electric energy metering device is ensured by adopting a weighted comprehensive scoring method and setting of a dynamic threshold; an operation and maintenance model based on a DDPG algorithm is established, the operation and maintenance strategy of the electric energy metering device is optimized, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved; the idea of reinforcement learning is used for solving the optimization problem of the operation and maintenance strategy of the electric energy metering device, the constraint of artificial experience in the traditional operation and maintenance mode is eliminated, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved.
Drawings
FIG. 1 is a flow chart of an operation and maintenance strategy optimization method of an electric energy metering device based on DDPG algorithm provided by the invention;
FIG. 2 is a schematic diagram of a network structure of DDPG;
fig. 3 is a schematic diagram of a scoring threshold parameter optimization method according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of an operation and maintenance strategy optimization method for an electric energy metering device based on a DDPG algorithm, as shown in fig. 1, the operation and maintenance strategy optimization method includes:
step 1, selecting each error state parameter of the electric energy metering device from historical data to establish a decision characteristic vector representing the error state of the electric energy metering device, calculating the error state score of the electric energy metering device according to the decision characteristic vector, and evaluating whether the error state of the electric energy metering device is stable, good or early-warning based on the error state score and a set threshold range.
Step 2, establishing an operation and maintenance model based on a DDFG algorithm, wherein an Actor network of the operation and maintenance model generates operation and maintenance actions for the electric energy metering device; the Critic network of the operation and maintenance model evaluates the performance of the operation and maintenance action and guides the strategy function to generate the operation and maintenance action of the next stage; and (4) performing iterative update training on the operation and maintenance model according to the error state and the corresponding operation and maintenance action obtained in the step (1).
The DDPG algorithm inherits a deterministic strategy of a DPG (deterministic policy gradient) algorithm, an agent outputs deterministic actions according to state decisions, and the DDPG adopts a deep neural network to enhance the fitting capacity of a decision function. Compared with a random strategy, the DDPG greatly reduces the sampling data volume, improves the algorithm efficiency and is more beneficial to the learning of an intelligent agent in a continuous action space.
The network structure of the DDPG is shown in fig. 2, and as can be seen from fig. 2, the DDPG algorithm takes the form of an Actor-Critic framework, and mainly includes an Actor (Actor) network and a Critic (Critic) network. The Actor network is responsible for generating actions and environment interaction for the electric energy metering device, and the criticic network is responsible for evaluating states and performance of the actions and guiding the strategy function to generate the action of the next stage. Both the Actor and Critic adopt a dual-network structure, and have respective Target (Target) network and estimation (Eval) network. The Actor-eval network is mainly responsible for policy network parameters
Figure 239709DEST_PATH_IMAGE056
The action a is selected according to the current state s, and the state of the next moment is generated in the process of interacting with the environment
Figure 347474DEST_PATH_IMAGE057
And a reward value r resulting from executing the current action. The Actor-target network is responsible for sampling the next state according to the experience pool
Figure 915858DEST_PATH_IMAGE057
Selecting an optimal next moment action
Figure 148387DEST_PATH_IMAGE058
. Network parameters
Figure 93210DEST_PATH_IMAGE059
Periodically from Actor-eval network
Figure 258743DEST_PATH_IMAGE056
And (6) copying. Critic-eval network mainly aims at network parameters
Figure 263608DEST_PATH_IMAGE060
Is updated by iteration, the current Q value is calculated
Figure 105138DEST_PATH_IMAGE061
And target Q value
Figure 604383DEST_PATH_IMAGE062
Wherein
Figure 608111DEST_PATH_IMAGE063
Representing a discount factor that affects how important the future reward is relative to the current reward during the training process. Critic-target network main parameter
Figure 3452DEST_PATH_IMAGE064
From periodic copying of the critical-eval network
Figure 990999DEST_PATH_IMAGE060
Derived from parameters primarily responsible for calculating the target Q value
Figure 28357DEST_PATH_IMAGE065
And 3, performing iterative optimization on the threshold range corresponding to the electric energy metering device to be evaluated based on the operation and maintenance model, and determining the operation and maintenance strategy of the electric energy metering device to be evaluated according to the optimized threshold range and the operation and maintenance model.
The invention provides an operation and maintenance strategy optimization method of an electric energy metering device based on a DDPG algorithm, which uses the idea of reinforcement learning to solve the optimization problem of the operation and maintenance strategy of the electric energy metering device, gets rid of the constraint of artificial experience in the traditional operation and maintenance mode, reduces the operation and maintenance cost and improves the operation and maintenance efficiency.
Example 1
Embodiment 1 provided by the present invention is an embodiment of an operation and maintenance policy optimization method for an electric energy metering device based on a DDPG algorithm, and as can be seen from fig. 1, the embodiment of the operation and maintenance policy optimization method includes:
step 1, selecting each error state parameter of the electric energy metering device from historical data to establish a decision characteristic vector representing the error state of the electric energy metering device, calculating the error state score of the electric energy metering device according to the decision characteristic vector, and evaluating whether the error state of the electric energy metering device is stable, good or early-warning based on the error state score and a set threshold range.
In a possible embodiment, the decision feature vector established in step 1 is
Figure 417750DEST_PATH_IMAGE001
Wherein, in the step (A),
Figure 518079DEST_PATH_IMAGE002
for the purpose of real-time error estimation,
Figure 992923DEST_PATH_IMAGE003
for a short-term prediction error estimate sequence,
Figure 365130DEST_PATH_IMAGE004
for the long-term prediction error estimate sequence, W is the stability state. Namely, the error state parameters of the electric energy metering device comprise: real-time error estimates, short-term prediction error estimate sequences, long-term prediction error estimate sequences, and stability states.
In a possible embodiment mode, in step 1, the method for acquiring the error state parameter of the electric energy metering device includes:
1) real-time error estimation value calculated by data-driven algorithm
Figure 609029DEST_PATH_IMAGE002
2) Preprocessing error data of the electric energy metering device, stripping the error data into self error and additional error, constructing a trend prediction model by utilizing an Auto Regression Integration Moving Average (ARIMA) algorithm, inputting the self error into the trend prediction model to obtain a self error short-term prediction value of the electric energy metering device, calculating the additional error short-term prediction value of the electric energy metering device to be measured according to temperature information and frequency information, and adding the self error short-term prediction value and the additional error short-term prediction value to obtain a short-term prediction error estimation value sequence
Figure 877331DEST_PATH_IMAGE003
As shown in equation (1).
Figure 839470DEST_PATH_IMAGE066
(1)
m is a constant.
3) Processing the plurality of self errors into a time sequence, and inputting the time sequence into a trained LSTM model to obtain a self error long-term predicted value of the electric energy metering device; calculating an additional error long-term predicted value of the electric energy metering device according to the temperature information and the frequency information; fusing the self error long-term predicted value and the additional error long-term predicted value to obtain a long-term prediction error estimated value sequence
Figure 218630DEST_PATH_IMAGE004
As shown in equation (2).
Figure 317036DEST_PATH_IMAGE067
(2)
n is a constant.
4) The method comprises the following steps of constructing and obtaining a stability state evaluation index of the electric energy metering device, and establishing a stability state index data model of the electric energy metering device, wherein the stability state evaluation index comprises the following steps: a sudden change error stable frequency function model, a sudden change error unstable frequency function model, a gradual change error monotonous significance function model and a gradual change error standard deviation function model; and comparing the importance of each state evaluation index by adopting a hierarchical analysis theory, determining the weight of the state evaluation index, calculating the stability state score of the electric energy metering device according to the result of each state evaluation index of the stability state index data model of the electric energy metering device and the corresponding weight, and evaluating whether the stability state W of the electric energy metering device is stable, slightly stable, moderately stable or severely stable according to the stability state score of the electric energy metering device.
Wherein the content of the first and second substances,
Figure 753309DEST_PATH_IMAGE068
Figure 202745DEST_PATH_IMAGE069
and
Figure 120017DEST_PATH_IMAGE070
respectively represent each time, an
Figure 338509DEST_PATH_IMAGE071
In a possible embodiment, in step 1, after each error state parameter of the decision feature vector is calculated by a weighted comprehensive scoring method, an error state score is obtained.
In a possible embodiment, the step 1 of calculating the error state score of the electric energy metering device according to the decision feature vector includes:
101, estimating error value based on electric energy metering device
Figure 948613DEST_PATH_IMAGE005
Standard deviation of error estimate
Figure 619765DEST_PATH_IMAGE006
Figure 340728DEST_PATH_IMAGE006
Is constant) and precision k (k isConstant) to establish an error state grading model of the electric energy metering device.
It can be understood that, for a power metering device to be evaluated, the true error value and the estimated error value obtained by the algorithm are respectively recorded as
Figure 413726DEST_PATH_IMAGE027
And
Figure 185942DEST_PATH_IMAGE005
obeying the following distribution:
Figure 344391DEST_PATH_IMAGE072
based on the error estimate
Figure 524837DEST_PATH_IMAGE005
Error truth value of electric energy metering device is calculated
Figure 796549DEST_PATH_IMAGE027
Out of range [ -k, k]Probability of (2)
Figure 748456DEST_PATH_IMAGE026
As shown in equation (3):
Figure 394201DEST_PATH_IMAGE073
(3)
the specific calculation is as follows:
Figure 456966DEST_PATH_IMAGE028
(4)
based on probability
Figure 504556DEST_PATH_IMAGE026
Calculating error state scores for an electric energy metering device
Figure 890014DEST_PATH_IMAGE023
Figure 23055DEST_PATH_IMAGE025
(5)
102, respectively calculating real-time error estimation values based on the error state scoring model
Figure 889511DEST_PATH_IMAGE002
Short-term prediction error estimation sequence
Figure 791608DEST_PATH_IMAGE003
And long-term prediction error estimate sequence
Figure 85317DEST_PATH_IMAGE004
Respectively is
Figure 440075DEST_PATH_IMAGE007
Figure 110222DEST_PATH_IMAGE008
And
Figure 70088DEST_PATH_IMAGE009
it will be appreciated that error state scoring in step 102
Figure 334365DEST_PATH_IMAGE007
Figure 441999DEST_PATH_IMAGE008
And
Figure 650257DEST_PATH_IMAGE009
the calculation formulas of (A) and (B) are respectively as follows:
Figure 464630DEST_PATH_IMAGE029
(6)
Figure 365721DEST_PATH_IMAGE030
(7)
Figure 695071DEST_PATH_IMAGE031
(8)
where i and j are constants.
103, setting corresponding weights for the stability W of the electric energy metering device to be stable, slightly stable, moderately stable and heavily stable according to the stability degree respectively
Figure 238179DEST_PATH_IMAGE010
Figure 969374DEST_PATH_IMAGE011
Figure 241699DEST_PATH_IMAGE012
And
Figure 792766DEST_PATH_IMAGE013
to obtain
Figure 608407DEST_PATH_IMAGE014
In specific implementation, the weight value can be determined by a method of expert judgment.
104, calculating the error state scores of the error state parameters by adopting a weighted comprehensive scoring method to obtain the error state scores of the electric energy metering device
Figure 194109DEST_PATH_IMAGE015
Calculating an error state score for the electric energy metering device in step 104
Figure 764899DEST_PATH_IMAGE015
Comprises the following steps:
Figure 85153DEST_PATH_IMAGE074
(9)
step 105, setting the representation powerThe error state of the metering device is stable, good or early-warning and corresponding to each threshold range of the error state score according to the error state score
Figure 891435DEST_PATH_IMAGE015
And evaluating the error state of the electric energy metering device within the threshold range.
In specific implementation, each threshold range corresponding to each error state score can be determined by an expert judgment method.
In a possible embodiment, the scoring threshold parameter is used in step 1 and step 3
Figure 97024DEST_PATH_IMAGE016
Figure 760087DEST_PATH_IMAGE017
And
Figure 302058DEST_PATH_IMAGE018
represents a threshold range:
error state of the electric energy metering device:
Figure 787397DEST_PATH_IMAGE019
(10)
wherein the content of the first and second substances,
Figure 19795DEST_PATH_IMAGE020
Figure 322600DEST_PATH_IMAGE021
and
Figure 351867DEST_PATH_IMAGE022
respectively showing stability, good and early warning;
Figure 93427DEST_PATH_IMAGE023
scoring an error state of an electric energy metering device, a scoring threshold parameter
Figure 990452DEST_PATH_IMAGE016
Figure 995317DEST_PATH_IMAGE017
And
Figure 246301DEST_PATH_IMAGE018
satisfy the requirement of
Figure 542284DEST_PATH_IMAGE024
Step 2, establishing an operation and maintenance model based on a DDFG algorithm, wherein an Actor network of the operation and maintenance model generates operation and maintenance actions for the electric energy metering device; the Critic network of the operation and maintenance model evaluates the performance of the operation and maintenance action and guides the strategy function to generate the operation and maintenance action of the next stage; and (4) performing iterative update training on the operation and maintenance model according to the error state and the corresponding operation and maintenance action obtained in the step (1).
In a possible embodiment, the calculation process of the target Q value of the operation and maintenance model in step 2 includes:
step 201, setting the error state of the electric energy metering device at time t as
Figure 811591DEST_PATH_IMAGE033
Operation and maintenance actions as
Figure 738090DEST_PATH_IMAGE034
Figure 725637DEST_PATH_IMAGE035
(11)
Figure 765924DEST_PATH_IMAGE036
(12)
Wherein the content of the first and second substances,
Figure 420897DEST_PATH_IMAGE037
Figure 518297DEST_PATH_IMAGE038
and
Figure 727561DEST_PATH_IMAGE039
respectively showing that the field verification is carried out and arranged according to a specified period along a verification period.
Step 202, define the function
Figure 303030DEST_PATH_IMAGE040
Representing a deterministic operation and maintenance action strategy, and the operation and maintenance actions of the electric energy metering device at any time t
Figure 219034DEST_PATH_IMAGE034
The calculation formula of (2) is as follows:
Figure 471023DEST_PATH_IMAGE041
step 203, define parameters
Figure 980633DEST_PATH_IMAGE042
And function J, parameter
Figure 609061DEST_PATH_IMAGE042
Is a function of pair
Figure 189690DEST_PATH_IMAGE043
The parameters of the strategy network for simulation, function J is a weighing function
Figure 550264DEST_PATH_IMAGE043
A function of the performance of the represented operation and maintenance action policy.
Figure 468542DEST_PATH_IMAGE075
(13)
Wherein the content of the first and second substances,
Figure 448130DEST_PATH_IMAGE045
is a distribution function;
Figure 666622DEST_PATH_IMAGE046
indicating different error states as a function
Figure 11147DEST_PATH_IMAGE043
Q value which can be generated when the represented operation and maintenance action strategy is adopted;
Figure 354404DEST_PATH_IMAGE047
is an error state
Figure 59054DEST_PATH_IMAGE033
According to
Figure 679523DEST_PATH_IMAGE045
When distributed
Figure 709796DEST_PATH_IMAGE046
Is calculated from the expected value of (c).
Step 204, the error state of the electric energy metering device is
Figure 344609DEST_PATH_IMAGE033
Take operation and maintenance actions
Figure 790633DEST_PATH_IMAGE034
Then, and continuously executing the function
Figure 718138DEST_PATH_IMAGE043
Calculating the target under the represented operation and maintenance action strategy
Figure 466782DEST_PATH_IMAGE046
The values are:
Figure 846948DEST_PATH_IMAGE048
(14)
wherein the content of the first and second substances,
Figure 909713DEST_PATH_IMAGE049
r is the return value for the discount factor.
And 3, performing iterative optimization on the threshold range corresponding to the electric energy metering device to be evaluated based on the operation and maintenance model, and determining the operation and maintenance strategy of the electric energy metering device to be evaluated according to the optimized threshold range and the operation and maintenance model.
As shown in fig. 3, a schematic diagram of a scoring threshold parameter optimization method provided in the embodiment of the present invention is shown, and as can be seen from fig. 3, in a possible embodiment, step 3 includes:
step 301, defining an error state of an electric energy metering device
Figure 629408DEST_PATH_IMAGE033
The operation and maintenance actions
Figure 735904DEST_PATH_IMAGE034
A function of relationship of
Figure 150836DEST_PATH_IMAGE050
Satisfies the following conditions: when the error states of the electric energy metering device are respectively
Figure 266559DEST_PATH_IMAGE020
Figure 650880DEST_PATH_IMAGE021
And
Figure 865960DEST_PATH_IMAGE022
the operation and maintenance actions executed are respectively corresponding to
Figure 220718DEST_PATH_IMAGE037
Figure 625286DEST_PATH_IMAGE038
And
Figure 444206DEST_PATH_IMAGE039
step 302, based on the target in the operation and maintenance model in step 2
Figure 908817DEST_PATH_IMAGE051
Value and function of
Figure 485292DEST_PATH_IMAGE050
Corresponding deterministic action strategy, setting initial grading threshold parameter
Figure 614922DEST_PATH_IMAGE016
Figure 976764DEST_PATH_IMAGE017
And
Figure 127123DEST_PATH_IMAGE018
performing iterative optimization to obtain optimized scoring threshold value parameters
Figure 944556DEST_PATH_IMAGE052
Figure 2511DEST_PATH_IMAGE053
And
Figure 218859DEST_PATH_IMAGE054
Figure 274540DEST_PATH_IMAGE055
the number of optimization iterations is indicated.
Specifically, the error state of the electric energy metering device when the time t is executed is
Figure 763290DEST_PATH_IMAGE033
Downward operation and maintenance action
Figure 313351DEST_PATH_IMAGE034
Rear end
Figure 695791DEST_PATH_IMAGE051
If the error state score is smaller, the initial score threshold parameter of the set error state score of the electric energy metering device is indicated
Figure 673106DEST_PATH_IMAGE016
Figure 711469DEST_PATH_IMAGE017
And
Figure 986592DEST_PATH_IMAGE018
if the setting is too small, the scoring threshold parameter needs to be increased.
And guiding the operation and the maintenance of the electric energy metering device according to the optimized error state grading threshold of the electric energy metering device and an operation and maintenance model based on a DDFG algorithm.
According to the operation and maintenance strategy optimization method of the electric energy metering device based on the DDPG algorithm, provided by the embodiment of the invention, the accuracy of error state evaluation of the electric energy metering device is ensured by adopting a weighted comprehensive scoring method and setting of a dynamic threshold; an operation and maintenance model based on a DDPG algorithm is established, the operation and maintenance strategy of the electric energy metering device is optimized, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved; the idea of reinforcement learning is used for solving the optimization problem of the operation and maintenance strategy of the electric energy metering device, so that the constraint of human experience in the traditional operation and maintenance mode is eliminated, the operation and maintenance cost is reduced, and the operation and maintenance efficiency is improved.
It should be noted that, in the foregoing embodiments, the description of each embodiment has an emphasis, and reference may be made to the related description of other embodiments for a part that is not described in detail in a certain embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (10)

1. An operation and maintenance strategy optimization method of an electric energy metering device based on a DDPG algorithm is characterized by comprising the following steps:
step 1, selecting each error state parameter of an electric energy metering device in historical data to establish a decision characteristic vector representing the error state of the electric energy metering device, calculating the error state score of the electric energy metering device according to the decision characteristic vector, and evaluating whether the error state of the electric energy metering device is stable, good or early-warning based on the error state score and a set threshold range;
step 2, establishing an operation and maintenance model based on a DDFG algorithm, wherein an Actor network of the operation and maintenance model generates operation and maintenance actions for the electric energy metering device; the Critic network of the operation and maintenance model evaluates the performance of the operation and maintenance action and guides a strategy function to generate the operation and maintenance action of the next stage; performing iterative update training on the operation and maintenance model according to the error state and the corresponding operation and maintenance action obtained in the step 1;
and 3, performing iterative optimization on the threshold range corresponding to the electric energy metering device to be evaluated based on the operation and maintenance model, and determining the operation and maintenance strategy of the electric energy metering device to be evaluated according to the optimized threshold range and the operation and maintenance model.
2. The operation and maintenance strategy optimization method according to claim 1, wherein in the step 1, the decision feature vector is
Figure 491502DEST_PATH_IMAGE001
Wherein, in the step (A),
Figure 742486DEST_PATH_IMAGE002
for the purpose of real-time error estimation,
Figure 225420DEST_PATH_IMAGE003
for a short-term prediction error estimate sequence,
Figure 245459DEST_PATH_IMAGE004
for the long-term prediction error estimate sequence, W is the stability state.
3. The operation and maintenance strategy optimization method according to claim 2, wherein in the step 1, the method for obtaining the error state parameter of the electric energy metering device comprises:
calculating to obtain the real-time error estimated value by adopting an algorithm based on data driving
Figure 421226DEST_PATH_IMAGE002
Preprocessing error data of the electric energy metering device, stripping the error data into self error and additional error, constructing a trend prediction model by utilizing an ARIMA algorithm, inputting the self error into the trend prediction model to obtain a self error short-term prediction value of the electric energy metering device, calculating the additional error short-term prediction value of the electric energy metering device to be measured according to temperature information and frequency information, and adding the self error prediction value and the additional error prediction value to obtain a short-term prediction error estimation value sequence
Figure 159506DEST_PATH_IMAGE003
Processing the plurality of self errors into a time sequence, and inputting the time sequence into a trained LSTM model to obtain a self error long-term predicted value of the electric energy metering device; calculating an additional error long-term predicted value of the electric energy metering device according to the temperature information and the frequency information; fusing the self error long-term predicted value and the additional error long-term predicted value to obtain the long-term prediction error estimated value sequence
Figure 242868DEST_PATH_IMAGE004
The method comprises the following steps of constructing and obtaining a stability state evaluation index of the electric energy metering device, and establishing a stability state index data model of the electric energy metering device, wherein the stability state evaluation index comprises the following steps: a sudden change error stable frequency function model, a sudden change error unstable frequency function model, a gradual change error monotonous significance function model and a gradual change error standard deviation function model; comparing the importance of each state evaluation index by adopting a hierarchical analysis theory, determining the weight of the state evaluation index, calculating the stability state score of the electric energy metering device according to the result of each state evaluation index of the stability state index data model of the electric energy metering device and the corresponding weight, and evaluating whether the stability state W of the electric energy metering device is stable, slightly stable, moderately stable or severely stable according to the stability state score of the electric energy metering device;
in the step 1, after each error state parameter of the decision feature vector is calculated by a weighted comprehensive scoring method, the error state score is obtained.
4. The operation and maintenance strategy optimization method according to claim 2, wherein the step 1 of calculating the error state score of the electric energy metering device according to the decision feature vector comprises:
101, estimating the error value based on the electric energy metering device
Figure 385924DEST_PATH_IMAGE005
Standard deviation of error estimate
Figure 935854DEST_PATH_IMAGE006
Establishing an error state scoring model of the electric energy metering device according to the precision k;
102, respectively calculating real-time error estimation values based on the error state scoring model
Figure 895850DEST_PATH_IMAGE002
Short-term prediction error estimation sequence
Figure 782904DEST_PATH_IMAGE003
And long-term prediction error estimate sequence
Figure 777536DEST_PATH_IMAGE004
Respectively is
Figure 295105DEST_PATH_IMAGE007
Figure 7977DEST_PATH_IMAGE008
And
Figure 636405DEST_PATH_IMAGE009
103, setting the corresponding weights of the stability W of the electric energy metering device to be stable, slightly stable, moderately stable and severely stable according to the stability degree respectively
Figure 217034DEST_PATH_IMAGE010
Figure 171084DEST_PATH_IMAGE011
Figure 105673DEST_PATH_IMAGE012
And
Figure 537791DEST_PATH_IMAGE013
to obtain
Figure 507015DEST_PATH_IMAGE014
104, calculating the error state scores of the error state parameters by adopting a weighted comprehensive scoring method to obtain the error state scores of the electric energy metering device
Figure 366387DEST_PATH_IMAGE015
105, setting each threshold value range of corresponding error state scores representing that the error state of the electric energy metering device is stable, good or early-warning, and scoring according to the error states
Figure 788272DEST_PATH_IMAGE015
The threshold range is used for evaluating the error state of the electric energy metering device.
5. The operation and maintenance strategy optimization method according to claim 4, wherein a scoring threshold parameter is used in the step 1 and the step 3
Figure 31604DEST_PATH_IMAGE016
Figure 104602DEST_PATH_IMAGE017
And
Figure 885607DEST_PATH_IMAGE018
represents the threshold range:
error state of the electric energy metering device
Figure 44056DEST_PATH_IMAGE019
Wherein the content of the first and second substances,
Figure 568710DEST_PATH_IMAGE020
Figure 761794DEST_PATH_IMAGE021
and
Figure 713700DEST_PATH_IMAGE022
respectively showing stability, good and early warning;
Figure 93866DEST_PATH_IMAGE023
scoring an error state of an electric energy metering device, a scoring threshold parameter
Figure 153701DEST_PATH_IMAGE016
Figure 201292DEST_PATH_IMAGE017
And
Figure 324100DEST_PATH_IMAGE018
satisfy the requirement of
Figure 722720DEST_PATH_IMAGE024
6. The operation and maintenance strategy optimization method according to claim 4, wherein the error state score model established in the step 101 is:
Figure 589176DEST_PATH_IMAGE025
;
wherein the content of the first and second substances,
Figure 756852DEST_PATH_IMAGE026
based on error estimation
Figure 784982DEST_PATH_IMAGE005
Error truth value of electric energy metering device obtained by calculation
Figure 955719DEST_PATH_IMAGE027
Out of range [ -k, k]Probability of (c):
Figure 157024DEST_PATH_IMAGE028
7. the operation and maintenance strategy optimization method according to claim 4, wherein the error state score in the step 102 is obtained
Figure 179207DEST_PATH_IMAGE007
Figure 909397DEST_PATH_IMAGE008
And
Figure 751451DEST_PATH_IMAGE009
the calculation formulas of (A) and (B) are respectively as follows:
Figure 959709DEST_PATH_IMAGE029
Figure 836399DEST_PATH_IMAGE030
Figure 924440DEST_PATH_IMAGE031
where i and j are constants.
8. The operation and maintenance strategy optimization method according to claim 4, wherein the error state score of the electric energy metering device is calculated in the step 104
Figure 798331DEST_PATH_IMAGE015
Comprises the following steps:
Figure 810280DEST_PATH_IMAGE032
9. the operation and maintenance strategy optimization method according to claim 5, wherein the calculation process of the target Q value of the operation and maintenance model in the step 2 comprises:
step 201, setting the error state of the electric energy metering device at time t as
Figure 275897DEST_PATH_IMAGE033
Operation and maintenance actions as
Figure 347889DEST_PATH_IMAGE034
Figure 695694DEST_PATH_IMAGE035
Figure 511334DEST_PATH_IMAGE036
Wherein the content of the first and second substances,
Figure 455893DEST_PATH_IMAGE037
Figure 308573DEST_PATH_IMAGE038
and
Figure 612516DEST_PATH_IMAGE039
respectively representing forward extending of a verification period, and performing and arranging field verification according to a specified period;
step 202, define the function
Figure 966268DEST_PATH_IMAGE040
Representing a deterministic operation and maintenance action strategy, and the operation and maintenance actions of the electric energy metering device at any time t
Figure 406476DEST_PATH_IMAGE034
The calculation formula of (2) is as follows:
Figure 551762DEST_PATH_IMAGE041
step 203, define parameters
Figure 343001DEST_PATH_IMAGE042
And a function J, the parameter
Figure 828340DEST_PATH_IMAGE042
For the function
Figure 139367DEST_PATH_IMAGE043
The parameters of the strategy network for simulation are measured by the function J
Figure 973330DEST_PATH_IMAGE043
A function of the performance of the represented operation and maintenance action policy;
Figure 737018DEST_PATH_IMAGE044
wherein the content of the first and second substances,
Figure 681840DEST_PATH_IMAGE045
is a distribution function;
Figure 584724DEST_PATH_IMAGE046
indicating different error states as a function
Figure 855168DEST_PATH_IMAGE043
Q value which can be generated when the represented operation and maintenance action strategy is adopted;
Figure 106152DEST_PATH_IMAGE047
is an error state
Figure 854666DEST_PATH_IMAGE033
According to
Figure 77968DEST_PATH_IMAGE045
When distributed
Figure 50472DEST_PATH_IMAGE046
The expected value of (d);
step 204, in the error state of the electric energy metering device, the error state is
Figure 992014DEST_PATH_IMAGE033
Take operation and maintenance actions
Figure 809797DEST_PATH_IMAGE034
Thereafter, and continuously executing the function
Figure 212572DEST_PATH_IMAGE043
Calculating the target under the represented operation and maintenance action strategy
Figure 293661DEST_PATH_IMAGE046
The values are:
Figure 519237DEST_PATH_IMAGE048
wherein the content of the first and second substances,
Figure 609553DEST_PATH_IMAGE049
r is the return value for the discount factor.
10. The operation and maintenance strategy optimization method according to claim 9, wherein the step 3 comprises:
step 301, defining an error state of an electric energy metering device
Figure 604185DEST_PATH_IMAGE033
The operation and maintenance actions
Figure 121754DEST_PATH_IMAGE034
A function of relationship of
Figure 569047DEST_PATH_IMAGE050
Satisfies the following conditions: when the error states of the electric energy metering device are respectively
Figure 197474DEST_PATH_IMAGE020
Figure 303403DEST_PATH_IMAGE021
And
Figure 991873DEST_PATH_IMAGE022
the operation and maintenance actions executed are respectively corresponding to
Figure 129724DEST_PATH_IMAGE037
Figure 358580DEST_PATH_IMAGE038
And
Figure 327805DEST_PATH_IMAGE039
step 302, based on the target in the operation and maintenance model in the step 2
Figure 187176DEST_PATH_IMAGE051
Value and function of
Figure 609061DEST_PATH_IMAGE050
Corresponding deterministic action strategy to set the initial scoring threshold parameter
Figure 579291DEST_PATH_IMAGE016
Figure 603355DEST_PATH_IMAGE017
And
Figure 633628DEST_PATH_IMAGE018
performing iterative optimization to obtain the optimized grading threshold parameter
Figure 542809DEST_PATH_IMAGE052
Figure 316730DEST_PATH_IMAGE053
And
Figure 260546DEST_PATH_IMAGE054
Figure 461720DEST_PATH_IMAGE055
indicating the number of optimization iterations.
CN202210515703.2A 2022-05-12 2022-05-12 Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm Active CN114648178B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210515703.2A CN114648178B (en) 2022-05-12 2022-05-12 Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210515703.2A CN114648178B (en) 2022-05-12 2022-05-12 Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm

Publications (2)

Publication Number Publication Date
CN114648178A true CN114648178A (en) 2022-06-21
CN114648178B CN114648178B (en) 2022-08-26

Family

ID=81997106

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210515703.2A Active CN114648178B (en) 2022-05-12 2022-05-12 Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm

Country Status (1)

Country Link
CN (1) CN114648178B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018209A (en) * 2022-08-08 2022-09-06 国网湖北省电力有限公司营销服务中心(计量中心) Long-term prediction method and equipment for operation error of digital electric energy metering system

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030023466A1 (en) * 2001-07-27 2003-01-30 Harper Charles N. Decision support system and method
JP2005168251A (en) * 2003-12-05 2005-06-23 Chugoku Electric Power Co Inc:The Device, system, program, and method for predicting power demand, and recording medium
JP2010063320A (en) * 2008-09-05 2010-03-18 Toshiba Corp Method and apparatus for controlling supply and demand for power system
CN104218570A (en) * 2014-08-21 2014-12-17 国家电网公司 Method and system for online evaluating overall measuring errors of electric energy measuring device
CN107784440A (en) * 2017-10-23 2018-03-09 国网辽宁省电力有限公司 A kind of power information system resource allocation system and method
KR20180130945A (en) * 2017-05-31 2018-12-10 세림티에스지(주) Energy consumption and electic fee prediction information system of Domestic using prediction analysis with perodic characteristics of energy consumption big data
CN110365056A (en) * 2019-08-14 2019-10-22 南方电网科学研究院有限责任公司 A kind of distributed energy participation power distribution network pressure regulation optimization method based on DDPG
CN110365057A (en) * 2019-08-14 2019-10-22 南方电网科学研究院有限责任公司 Distributed energy based on intensified learning participates in power distribution network peak regulation method for optimizing scheduling
CN110535146A (en) * 2019-08-27 2019-12-03 哈尔滨工业大学 The Method for Reactive Power Optimization in Power of Policy-Gradient Reinforcement Learning is determined based on depth
CN110766236A (en) * 2019-10-31 2020-02-07 云南电网有限责任公司昆明供电局 Power equipment state trend prediction method based on statistical analysis and deep learning
CN111324358A (en) * 2020-02-14 2020-06-23 南栖仙策(南京)科技有限公司 Training method for automatic operation and maintenance strategy of information system
CN111600851A (en) * 2020-04-27 2020-08-28 浙江工业大学 Feature filtering defense method for deep reinforcement learning model
US20210044444A1 (en) * 2019-08-06 2021-02-11 Cisco Technology, Inc. Controller-based management of noncompliant power over ethernet devices
CN112686445A (en) * 2020-12-30 2021-04-20 南京工程学院 Photovoltaic power generation prediction method based on ARIMA-LSTM-DBN
CN113852082A (en) * 2021-09-03 2021-12-28 清华大学 Method and device for preventing and controlling transient stability of power system
CN114188997A (en) * 2021-12-07 2022-03-15 国网甘肃省电力公司电力科学研究院 Dynamic reactive power optimization method for high-ratio new energy power access area power grid
CN114243718A (en) * 2021-12-23 2022-03-25 华北电力大学(保定) Power grid reactive voltage coordination control method based on DDPG algorithm

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030023466A1 (en) * 2001-07-27 2003-01-30 Harper Charles N. Decision support system and method
JP2005168251A (en) * 2003-12-05 2005-06-23 Chugoku Electric Power Co Inc:The Device, system, program, and method for predicting power demand, and recording medium
JP2010063320A (en) * 2008-09-05 2010-03-18 Toshiba Corp Method and apparatus for controlling supply and demand for power system
CN104218570A (en) * 2014-08-21 2014-12-17 国家电网公司 Method and system for online evaluating overall measuring errors of electric energy measuring device
KR20180130945A (en) * 2017-05-31 2018-12-10 세림티에스지(주) Energy consumption and electic fee prediction information system of Domestic using prediction analysis with perodic characteristics of energy consumption big data
CN107784440A (en) * 2017-10-23 2018-03-09 国网辽宁省电力有限公司 A kind of power information system resource allocation system and method
US20210044444A1 (en) * 2019-08-06 2021-02-11 Cisco Technology, Inc. Controller-based management of noncompliant power over ethernet devices
CN110365057A (en) * 2019-08-14 2019-10-22 南方电网科学研究院有限责任公司 Distributed energy based on intensified learning participates in power distribution network peak regulation method for optimizing scheduling
CN110365056A (en) * 2019-08-14 2019-10-22 南方电网科学研究院有限责任公司 A kind of distributed energy participation power distribution network pressure regulation optimization method based on DDPG
CN110535146A (en) * 2019-08-27 2019-12-03 哈尔滨工业大学 The Method for Reactive Power Optimization in Power of Policy-Gradient Reinforcement Learning is determined based on depth
CN110766236A (en) * 2019-10-31 2020-02-07 云南电网有限责任公司昆明供电局 Power equipment state trend prediction method based on statistical analysis and deep learning
CN111324358A (en) * 2020-02-14 2020-06-23 南栖仙策(南京)科技有限公司 Training method for automatic operation and maintenance strategy of information system
CN111600851A (en) * 2020-04-27 2020-08-28 浙江工业大学 Feature filtering defense method for deep reinforcement learning model
CN112686445A (en) * 2020-12-30 2021-04-20 南京工程学院 Photovoltaic power generation prediction method based on ARIMA-LSTM-DBN
CN113852082A (en) * 2021-09-03 2021-12-28 清华大学 Method and device for preventing and controlling transient stability of power system
CN114188997A (en) * 2021-12-07 2022-03-15 国网甘肃省电力公司电力科学研究院 Dynamic reactive power optimization method for high-ratio new energy power access area power grid
CN114243718A (en) * 2021-12-23 2022-03-25 华北电力大学(保定) Power grid reactive voltage coordination control method based on DDPG algorithm

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
林祥勋: "浅析电能计量装置运行误差分析及状态评价方法", 《技术与市场》 *
程瑛颖: "《电能计量装置运行误差分析及状态评价方法研究》", 《电工电能新技术》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018209A (en) * 2022-08-08 2022-09-06 国网湖北省电力有限公司营销服务中心(计量中心) Long-term prediction method and equipment for operation error of digital electric energy metering system
CN115018209B (en) * 2022-08-08 2022-11-08 国网湖北省电力有限公司营销服务中心(计量中心) Long-term prediction method and equipment for operation error of digital electric energy metering system

Also Published As

Publication number Publication date
CN114648178B (en) 2022-08-26

Similar Documents

Publication Publication Date Title
CN111047085B (en) Hybrid vehicle working condition prediction method based on meta-learning
CN108710905A (en) One kind being based on the united spare part quantitative forecasting technique of multi-model and system
CN110781969A (en) Air conditioner air volume control method and device based on deep reinforcement learning and medium
WO2020147450A1 (en) Ai model automatic generation method based on computational graph evolution
CN114648178B (en) Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm
CN115940294B (en) Multi-stage power grid real-time scheduling strategy adjustment method, system, equipment and storage medium
CN109242558A (en) A kind of management data prediction technique, readable storage medium storing program for executing and forecasting system neural network based
CN111695754A (en) Electric power Internet of things information security risk assessment method and device
CN110794308B (en) Method and device for predicting train battery capacity
CN115115389A (en) Express customer loss prediction method based on value subdivision and integrated prediction
CN111008790A (en) Hydropower station group power generation electric scheduling rule extraction method
Cui et al. Stochastic planning with lifted symbolic trajectory optimization
Neumann et al. Scaling laws for a multi-agent reinforcement learning model
CN111400964B (en) Fault occurrence time prediction method and device
CN112241808A (en) Road surface technical condition prediction method, device, electronic equipment and storage medium
CN116777281A (en) ARIMA model-based power equipment quality trend prediction method and device
CN116384251A (en) New energy generated power combination prediction method and system considering risk avoidance
CN113300884B (en) GWO-SVR-based step-by-step network flow prediction method
CN113435927B (en) User willingness prediction method, device, equipment and storage medium
CN110555742A (en) Generation method and system for generator agent quotation
CN117129895A (en) Battery state of health calculation method, device, storage medium and vehicle
CN115219910A (en) Analysis method and device for battery residue prediction error
CN108053266A (en) A kind of patent value predictor method and device
CN113987261A (en) Video recommendation method and system based on dynamic trust perception
CN113762591A (en) Short-term electric quantity prediction method and system based on GRU and multi-core SVM counterstudy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP01 Change in the name or title of a patent holder
CP01 Change in the name or title of a patent holder

Address after: Room 1803-1805, building 2-07, guanggu.core center, 303 Guanggu Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430000

Patentee after: Wuhan Gelanruo Intelligent Technology Co.,Ltd.

Address before: Room 1803-1805, building 2-07, guanggu.core center, 303 Guanggu Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430000

Patentee before: WUHAN GLORY ROAD INTELLIGENT TECHNOLOGY Co.,Ltd.

PE01 Entry into force of the registration of the contract for pledge of patent right
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: An optimization method for operation and maintenance strategy of electric energy metering device based on DDPG algorithm

Effective date of registration: 20221230

Granted publication date: 20220826

Pledgee: Bank of China Limited Wuhan Donghu New Technology Development Zone Branch

Pledgor: Wuhan Gelanruo Intelligent Technology Co.,Ltd.

Registration number: Y2022420000404

PC01 Cancellation of the registration of the contract for pledge of patent right
PC01 Cancellation of the registration of the contract for pledge of patent right

Granted publication date: 20220826

Pledgee: Bank of China Limited Wuhan Donghu New Technology Development Zone Branch

Pledgor: Wuhan Gelanruo Intelligent Technology Co.,Ltd.

Registration number: Y2022420000404