CN111582903A - Generator intelligent agent considering electric power futures change influence and quotation method - Google Patents
Generator intelligent agent considering electric power futures change influence and quotation method Download PDFInfo
- Publication number
- CN111582903A CN111582903A CN202010207845.3A CN202010207845A CN111582903A CN 111582903 A CN111582903 A CN 111582903A CN 202010207845 A CN202010207845 A CN 202010207845A CN 111582903 A CN111582903 A CN 111582903A
- Authority
- CN
- China
- Prior art keywords
- generator
- action
- quotation
- price
- value table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0206—Price or cost determination based on market factors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0611—Request for offers or quotes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S50/00—Market activities related to the operation of systems integrating technologies related to power network operation or related to communication or information technologies
- Y04S50/10—Energy trading, including energy flowing from end-user application to grid
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S50/00—Market activities related to the operation of systems integrating technologies related to power network operation or related to communication or information technologies
- Y04S50/14—Marketing, i.e. market research and analysis, surveying, promotions, advertising, buyer profiling, customer management or rewards
Abstract
The invention discloses a power generation merchant intelligent agent and a quotation method considering the influence of electric power futures change, wherein the intelligent agent comprises: the Q value table building module is used for building a Q value table consisting of a state set consisting of market clearing prices and an action set consisting of quotation actions and initializing the Q value table; the action selection module is used for establishing a market bidding model of the power generator on the electric energy and selecting quoted actions based on a Q value table according to the established market bidding model; the action correction module is used for judging the influence degree of the future goods considered by the generator and correcting the quoted price according to the variation range of the price of the future goods; and the Q value table updating module is used for submitting the corrected quote to the ISO for clearing and updating the Q value table according to the income and clearing price. The invention considers the influence of the price change of the futures market on the spot market, and is beneficial to the generator to quote in the electric power market more accurately.
Description
Technical Field
The invention relates to the power technology, in particular to a generator intelligent agent and a quotation method considering the influence of power futures change.
Background
With the emergence of the electric power spot market in the domestic market, the power generators will gradually participate in bidding of the electric power market to obtain own benefits, and in the market environment, participants always continuously optimize own bidding strategies for obtaining higher profits. At present, the electric power market in China is still in the stage of just starting, and a generator is not familiar with the market environment and needs a perfect quotation strategy theory as a guide. An efficient quotation decision tool can help decision makers and quotation staff to make a successful quotation and thereby obtain a high amount of revenue. In addition, the method is helpful for a supervisory organization of the power market to investigate the behavior of the power generator, so as to identify the existing loopholes in the market rule and continuously improve the policy and regulation of the power market in China, and therefore, the method is necessary for researching the behavior of the power generator in the power market in China.
Electric power futures have been widely used in the electric power markets in many countries as one of the main means for electric power market participants to circumvent market risks. Research shows that the futures trading of market participants has certain influence on the bidding strategy of power generation manufacturers, the change of the futures price can positively influence the trading of the spot market, and although the risk-avoiding market participants can effectively avoid the fluctuation risk of the spot electricity price by using the electric futures, the risk-neutral market participants can also enter the futures market for increasing the income or expanding the market share by using the strategic influence of the electric futures trading on the spot market. Therefore, a scientific and reasonable decision model is urgently needed by market participants to evaluate the influence of the electric futures on the electric power market equilibrium and the strategy behaviors of the market participants so as to provide a decision basis for avoiding market risks and realizing profit maximization, and meanwhile, a market supervision mechanism also needs to provide a basis for scientifically guiding and supervising the market behaviors according to a proper decision model.
The traditional generator quotation strategy research method mainly considers a spot market model, but the trading of the spot market has great influence on the spot market, and the generator should consider the change of the electric power future price when making quotation, so the research on the generator quotation method considering the influence of the electric power future price change has important promoting significance on the construction of the electric power future and the spot market.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a generator agent and a quotation method considering the influence of electric power futures change, aiming at the problem that the price change of the electric power futures is not considered in the prior art.
The technical scheme is as follows: the generator agent that considers the effect of the change of the electric futures comprises:
the Q value table building module is used for building a Q value table consisting of a state set consisting of market clearing prices and an action set consisting of quotation actions and initializing the Q value table;
the action selection module is used for establishing a market bidding model of the power generator on the electric energy and selecting quoted actions based on a Q value table according to the established market bidding model;
the action correction module is used for judging the influence degree of the future goods considered by the generator and correcting the quoted price according to the variation range of the price of the future goods;
and the Q value table updating module is used for submitting the corrected quote to the ISO for clearing and updating the Q value table according to the income and clearing price.
Further, the Q-value table constructing module specifically includes:
the state set establishing unit is used for establishing a state set state formed by market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator;
the action set establishing unit is used for establishing an action set action formed by quotation actions according to the action intervals and the interval sizes of the intelligent agents of the power generators;
the Q value table establishing unit is used for establishing a corresponding Q value table by adopting a generator agent intelligent agent state set and an action set action, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action;
and the initialization unit is used for initializing all the values in the Q value table to 0.
Further, the action selection module specifically includes:
the generator set electric energy quotation model establishing unit is used for establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciRespectively a first order coefficient and a second order coefficient of the fuel costCoefficient and constant term coefficient, G represents a power generation quotient set; ci m(PGi) A marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiThe electric energy bidding coefficient submitted for the generator i;
the bidding model establishing unit is used for establishing a market bidding model of the power generator in the electric energy according to the electric energy bidding model of the power generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOThe sum of the costs quoted for all generators;
the action selection unit is used for selecting the quotation action based on the Q value table, and the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state of the nth iteration, wherein the initial state is random selection, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(sn,a) At state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
Further, the action modification module specifically includes:
an futures price variation amplitude calculation unit, configured to calculate an futures price variation amplitude according to a current electric power futures price and a previous electric power futures price:
in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price;
the correction probability setting unit is used for respectively setting different correction probabilities p to be 0.1, 0.5 and 0.9 according to the set quotation habits of the power generator, wherein the three types of the price insensitivity to the electric power future price, the sensitivity to the future price and the sensitivity to the future price are included, and the larger the correction probability is, the larger the probability of correcting the quotation influenced by the future price of the power generator is;
an action correcting unit for generating a random number range between 0 and 1, if the corresponding probability value is greater than the range, selecting to correct the quotation of the generator,the corrected quotation coefficient is kipOtherwise, the coefficient k is still adoptedi:
Further, the Q-value table updating module specifically includes:
a quotation submitting unit for submitting quotation according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxRespectively the upper and lower technical output limits of the generator i;
and the profit calculation unit is used for calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning bid amount:
in the formula, λeNode electricity price for node i;
and the Q value table updating unit is used for updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Is a Q valueBefore and after table update, in state s, action a is adoptednThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
a state updating unit for updating the state to s according to the clearing resultn+1。
The method for reporting the price of the power generation trader considering the influence of the change of the electric power futures comprises the following steps:
(1) establishing a Q value table consisting of a state set consisting of market clearing prices and an action set of quotation actions, and initializing;
(2) establishing a market bidding model of a generator on electric energy, and selecting a quotation action based on a Q value table according to the established market bidding model;
(3) judging the influence degree of the future goods considered by the generator, and correcting the quoted price according to the variation range of the price of the future goods;
(4) and submitting the corrected quote to ISO for clearing, and updating a Q value table according to the income and clearing price.
Further, the step (1) specifically comprises:
(1-1) establishing a state set state consisting of market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator;
(1-2) establishing an action set action consisting of quotation actions according to the action intervals and intervals of the intelligent agent of the power generator;
(1-3) adopting a generator agent intelligent agent state set state and an action set action to construct a corresponding Q value table, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action;
(1-4) all the values in the Q value table are initialized to 0.
Further, the step (2) specifically comprises:
(2-1) establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciA first-order coefficient, a second-order coefficient and a constant-term coefficient of the fuel cost respectively, wherein G represents a power generation quotient set; ci m(PGi) A marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiThe electric energy bidding coefficient submitted for the generator i;
(2-2) establishing a market bidding model of the generator in the electric energy according to the electric energy quotation model of the generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOThe sum of the costs quoted for all generators;
(2-3) selecting a quotation action based on the Q value table, wherein the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state of the nth iteration, wherein the initial state is random selection, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(snA) is the state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
Further, the step (3) specifically comprises:
(3-1) calculating the future price variation amplitude according to the current power future price and the previous power future price:
in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price;
(3-2) according to the set quotation habits of the generator, wherein the quotation habits comprise three types of insensitivity to the electric power future price, sensitivity to the future price and sensitivity to the future price, different correction probabilities p are respectively set to be 0.1, 0.5 and 0.9, and the larger the correction probability is, the larger the probability of correcting the quotation of the generator under the influence of the future price is;
(3-3) generating a random number range between 0 and 1, and if the corresponding probability value is greater than the random, selecting to correct the quotation of the power generator, wherein the quotation coefficient after correction is kipOtherwise, the coefficient k is still adoptedi:
Further, the step (4) specifically comprises:
(4-1) according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxRespectively the upper and lower technical output limits of the generator i;
(4-2) calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning electricity quantity:
in the formula, λeNode electricity price for node i;
(4-3) updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Adopt action a in state s before and after updating Q-value tablenThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
(4-4) updating the status to s according to the clearing resultn+1(ii) a Judging whether the iteration times reach a preset maximum time or not; if yes, ending updating the Q value table, and if not, continuing to iteratively update the Q value table.
Has the advantages that: compared with the prior art, the invention has the following remarkable advantages: the method considers the influence of the change of the electric power future price in the future market on the electric power price and the yield of the electric power spot market in the actual situation, corrects the quotation on the basis of selecting the quotation by using the prior reinforcement learning method, and compared with the prior quotation decision method, the method considers the influence of the price change of the future market on the spot market, and is beneficial to a generator to more accurately quote in the electric power market.
Drawings
FIG. 1 is a block diagram of one embodiment of a generator agent that accounts for the effects of power futures changes provided by the present invention;
FIG. 2 is a five-machine three-node network topology;
FIG. 3 is a comparison of magnitude of corrections by different types of generators based on futures prices;
FIG. 4 is a schematic diagram of a generator quote strategy learning process.
Detailed Description
As shown in fig. 1, the generator agent that considers the influence of the change of the electric power futures provided in this embodiment includes a Q-value table building module, an action selecting module, an action correcting module, and a Q-value table updating module, where the Q-value table building module is configured to build a Q-value table that is composed of a state set composed of market clearing prices and an action set of quotation actions, and to perform initialization; the action selection module is used for establishing a market bidding model of the power generator on the electric energy and selecting quotation actions based on a Q value table according to the established market bidding model; the action correction module is used for judging the influence degree of the future goods considered by the generator and correcting the quoted price according to the variation range of the price of the future goods; and the Q value table updating module is used for submitting the corrected quote to ISO for clearing and updating the Q value table according to the income and clearing price. Each module is described in detail below.
The Q value table building module specifically comprises: the state set establishing unit is used for establishing a state set state formed by market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator; the action set establishing unit is used for establishing an action set action formed by quotation actions according to the action intervals and the interval sizes of the intelligent agents of the power generators; the Q value table establishing unit is used for establishing a corresponding Q value table by adopting a generator agent intelligent agent state set and an action set action, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action; and the initialization unit is used for initializing all the values in the Q value table to 0. Specifically, the state interval of the generator agent can be set to 200 to 360, the interval size is 20, a state set state (electric energy market clearing price) is established, the action interval of the generator agent is set to 1 to 1.5, the interval size is 0.1, an action set action (generator agent price reporting policy) is established, and the income of each generator agent is recorded by using a Q value table. Since the number of rows is 8 for state and the number of columns is 6 for action in the Q-value table, the Q-value table is an 8 × 6 matrix, and all matrix values are set to 0 in the initial case.
The action selection module specifically comprises:
the generator set electric energy quotation model establishing unit is used for establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciA first-order coefficient, a second-order coefficient and a constant-term coefficient of the fuel cost respectively, wherein G represents a power generation quotient set;a marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiAnd submitting the electric energy bidding coefficient for the generator i. The generator establishes an electric energy quotation curve according to the fuel cost model, the cost of the generator can be regarded as a quadratic function of the unit output, and therefore the marginal cost is a linear function of the output. And multiplying the marginal cost by a quotation coefficient to obtain an electric energy quotation curve based on the linear supply function model.
The bidding model establishing unit is used for establishing a market bidding model of the power generator in the electric energy according to the electric energy bidding model of the power generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOIs the sum of the costs quoted for all generators;
the action selection unit is used for selecting the quotation action based on the Q value table, and the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state of the nth iteration, wherein the initial state is random selection, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(snA) is the state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm; initially set to 100000;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
The action correction module specifically comprises: an futures price variation amplitude calculation unit, configured to calculate an futures price variation amplitude according to a current electric power futures price and a previous electric power futures price:in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price; the correction probability setting unit is used for respectively setting different correction probabilities p to be 0.1, 0.5 and 0.9 according to the set quotation habits of the power generator, wherein the three types of the price insensitivity to the electric power future price, the sensitivity to the future price and the sensitivity to the future price are included, and the larger the correction probability is, the larger the probability of correcting the quotation influenced by the future price of the power generator is; an action correcting unit for generating a random number range between 0 and 1, if the corresponding probability value is greater than the range, selecting to correct the quotation of the generator, and the quotation coefficient after correction is kipOtherwise, the coefficient k is still adoptedi:
The Q value table updating module specifically includes:
a quotation submitting unit for submitting quotation according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxRespectively the upper and lower technical output limits of the generator i;
and the profit calculation unit is used for calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning bid amount:
in the formula, λeNode electricity price for node i;
and the Q value table updating unit is used for updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Adopt action a in state s before and after updating Q-value tablenThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
a state updating unit for updating the state to s according to the clearing resultn+1。
The embodiment further provides a power generation trader pricing method considering the influence of power futures change, which comprises the following steps:
(1) establishing a Q value table consisting of a state set consisting of market clearing prices and an action set of quotation actions, and initializing;
(2) establishing a market bidding model of a generator on electric energy, and selecting a quotation action based on a Q value table according to the established market bidding model;
(3) judging the influence degree of the future goods considered by the generator, and correcting the quoted price according to the variation range of the price of the future goods;
(4) and submitting the corrected quote to ISO for clearing, and updating a Q value table according to the income and clearing price.
The step (1) specifically comprises the following steps:
(1-1) establishing a state set state consisting of market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator;
(1-2) establishing an action set action consisting of quotation actions according to the action intervals and intervals of the intelligent agent of the power generator;
(1-3) adopting a generator agent intelligent agent state set state and an action set action to construct a corresponding Q value table, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action;
(1-4) all the values in the Q value table are initialized to 0.
The step (2) specifically comprises the following steps:
(2-1) establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciA first-order coefficient, a second-order coefficient and a constant-term coefficient of the fuel cost respectively, wherein G represents a power generation quotient set;a marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiThe electric energy bidding coefficient submitted for the generator i;
(2-2) establishing a market bidding model of the generator in the electric energy according to the electric energy quotation model of the generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOIs the sum of the costs quoted for all generators;
(2-3) selecting a quotation action based on the Q value table, wherein the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state of the nth iteration, wherein the initial state is random selection, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(snA) is the state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
The step (3) specifically comprises the following steps:
(3-1) calculating the future price variation amplitude according to the current power future price and the previous power future price:
in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price;
(3-2) according to the set quotation habits of the generator, wherein the quotation habits comprise three types of insensitivity to the electric power future price, sensitivity to the future price and sensitivity to the future price, different correction probabilities p are respectively set to be 0.1, 0.5 and 0.9, and the larger the correction probability is, the larger the probability of correcting the quotation of the generator under the influence of the future price is;
(3-3) generating a random number range between 0 and 1, and if the corresponding probability value is greater than the random, selecting to correct the quotation of the power generator, wherein the quotation coefficient after correction is kipOtherwise, the coefficient k is still adoptedi:
The step (4) specifically comprises the following steps:
(4-1) according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxRespectively the upper and lower technical output limits of the generator i;
(4-2) calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning electricity quantity:
in the formula, λeNode electricity price for node i;
(4-3) updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Adopt action a in state s before and after updating Q-value tablenThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
(4-4) updating the status to s according to the clearing resultn+1(ii) a Judging whether the iteration times reach a preset maximum time or not; if yes, ending updating the Q value table, if not, continuing to iteratively update the Q value table, namely returning to execute (2-4).
Next, a 5-machine 3-node test system is adopted, and as shown in fig. 2, simulation analysis is performed on generator behaviors in the power market. The 3-node test system contains 5 generators. G1 access node 1, rest nodes access node 2, load access node 3. The basic information of the power generator is shown in table 1.
TABLE 1
Case setting simulation parameters are as follows: the cycle number was 3000 and the load demand was 100000 MW.
Solving is carried out according to a reinforcement learning algorithm, and the amplitude of correction of different types of power generators obtained through simulation according to the future price is shown in figure 3. Fig. 3 adds the absolute values of the corrections of the power generators for each 30 iterations, and it can be seen that the magnitude of the correction of power generator No. 1 is affected most by the futures price, and the magnitude of the correction of power generator No. 2 is second and insensitive to the futures price is the smallest. Fig. 4 illustrates the quotation strategy learning process of the generator No. 1 when considering the influence of the price change of the power future, and it can be seen that the final generator No. 1 quotation strategy converges to a stable range.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A generator agent that accounts for the effects of power futures changes, comprising:
the Q value table building module is used for building a Q value table consisting of a state set consisting of market clearing prices and an action set consisting of quotation actions and initializing the Q value table;
the action selection module is used for establishing a market bidding model of the power generator on the electric energy and selecting quoted actions based on a Q value table according to the established market bidding model;
the action correction module is used for judging the influence degree of the future goods considered by the generator and correcting the quoted price according to the variation range of the price of the future goods;
and the Q value table updating module is used for submitting the corrected quote to the ISO for clearing and updating the Q value table according to the income and clearing price.
2. A generator agent accounting for power futures change effects as claimed in claim 1 wherein: the Q value table building module specifically includes:
the state set establishing unit is used for establishing a state set state formed by market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator;
the action set establishing unit is used for establishing an action set action formed by quotation actions according to the action intervals and the interval sizes of the intelligent agents of the power generators;
the Q value table establishing unit is used for establishing a corresponding Q value table by adopting a generator agent intelligent agent state set and an action set action, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action;
and the initialization unit is used for initializing all the values in the Q value table to 0.
3. A generator agent accounting for power futures change effects as claimed in claim 1 wherein: the action selection module specifically comprises:
the generator set electric energy quotation model establishing unit is used for establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciA first-order coefficient, a second-order coefficient and a constant-term coefficient of the fuel cost respectively, wherein G represents a power generation quotient set;a marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiThe electric energy bidding coefficient submitted for the generator i;
the bidding model establishing unit is used for establishing a market bidding model of the power generator in the electric energy according to the electric energy bidding model of the power generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOThe sum of the costs quoted for all generators;
the action selection unit is used for selecting the quotation action based on the Q value table, and the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state at the nth iteration, initiallyThe state is selected randomly, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(snA) is the state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
4. A generator agent accounting for power futures change effects as claimed in claim 1 wherein: the action correction module specifically comprises:
an futures price variation amplitude calculation unit, configured to calculate an futures price variation amplitude according to a current electric power futures price and a previous electric power futures price:
in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price;
the correction probability setting unit is used for respectively setting different correction probabilities p to be 0.1, 0.5 and 0.9 according to the set quotation habits of the power generator, wherein the three types of the price insensitivity to the electric power future price, the sensitivity to the future price and the sensitivity to the future price are included, and the larger the correction probability is, the larger the probability of correcting the quotation influenced by the future price of the power generator is;
an action correcting unit for generating a random number range between 0 and 1, if the corresponding probability value is greater than the range, selecting to correct the quotation of the generator, and the quotation coefficient after correction is kipOtherwise, the coefficient k is still adoptedi:
5. A generator agent accounting for power futures change effects as claimed in claim 1 wherein: the Q-value table updating module specifically includes:
a quotation submitting unit for submitting quotation according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxEach being a generator iUpper and lower technical output limits;
and the profit calculation unit is used for calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning bid amount:
in the formula, λeNode electricity price for node i;
and the Q value table updating unit is used for updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Adopt action a in state s before and after updating Q-value tablenThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
a state updating unit for updating the state to s according to the clearing resultn+1。
6. A method for reporting prices of power generators considering the influence of changes in electric futures, comprising:
(1) establishing a Q value table consisting of a state set consisting of market clearing prices and an action set of quotation actions, and initializing;
(2) establishing a market bidding model of a generator on electric energy, and selecting a quotation action based on a Q value table according to the established market bidding model;
(3) judging the influence degree of the future goods considered by the generator, and correcting the quoted price according to the variation range of the price of the future goods;
(4) and submitting the corrected quote to ISO for clearing, and updating a Q value table according to the income and clearing price.
7. The generator pricing method considering the effect of power futures changes according to claim 6, characterized by: the step (1) specifically comprises the following steps:
(1-1) establishing a state set state consisting of market clearing prices according to the state interval and the interval size of the intelligent agent of the power generator;
(1-2) establishing an action set action consisting of quotation actions according to the action intervals and intervals of the intelligent agent of the power generator;
(1-3) adopting a generator agent intelligent agent state set state and an action set action to construct a corresponding Q value table, wherein in the Q value table, the number of lines is the dimension of the state, and the number of columns is the dimension of the action;
(1-4) all the values in the Q value table are initialized to 0.
8. The generator pricing method considering the effect of power futures changes according to claim 6, characterized by: the step (2) specifically comprises the following steps:
(2-1) establishing a generator set electric energy quotation model:
in the formula: ci(PGi) A fuel cost function for generator i; pGiThe output of the generator i; a isi、bi、ciA first-order coefficient, a second-order coefficient and a constant-term coefficient of the fuel cost respectively, wherein G represents a power generation quotient set;a marginal cost function for generator i; p (P)Gi) An electric energy bidding curve for the generator i; p is a radical ofiThe electric energy bidding coefficient submitted for the generator i;
(2-2) establishing a market bidding model of the generator in the electric energy according to the electric energy quotation model of the generator set:
in the formula: f. ofGFor the profit of the generator i, λeFor clearing price of electric energy, kiThe quotation coefficient of the generator i in the electric energy market; k is a radical ofimin、kimaxThe minimum value and the maximum value of the electric energy quotation coefficient of the generator i are respectively; pDhThe load requirement of the h-th user; l is a network node set; pGimin、PGimaxRespectively the upper and lower technical output limits, f, of the generator iISOThe sum of the costs quoted for all generators;
(2-3) selecting a quotation action based on the Q value table, wherein the specific selection method comprises the following steps:
randomly selecting an offer action a from the Q value table based on a uniform random algorithmrWherein, in the step (A),is a state snLower selection action arProbability of(s)nRepresenting the state of the nth iteration, wherein the initial state is random selection, and N is the number of actions;
selecting an offer action a based on greedy policypThat is, the quotation action with the maximum Q value in the current state is selected from the Q value table as the optimal quotation action, wherein ap=argmaxQn-1(sn,a),Qn-1(snA) is the state s when the Q value table is not updatednAdopting the Q value corresponding to the action a;
the Metropolis criterion of the simulated annealing method is adopted to dynamically select the parameter values:
wherein, the Temperature is the Temperature in the simulated annealing algorithm;
generating a random number rand between 0 and 1, comparing the rand with the value of the random number rand, and determining the final selected quotation action a according to the following formulanFinally according to the final selected quotation action anFinding out the corresponding quotation coefficient k in the Q value table action seti
9. The generator pricing method considering the effect of power futures changes according to claim 6, characterized by: the step (3) specifically comprises the following steps:
(3-1) calculating the future price variation amplitude according to the current power future price and the previous power future price:
in the formula: correction is the variation range of the future price, futures is the current power future price, and futures _ old is the previous power future price;
(3-2) according to the set quotation habits of the generator, wherein the quotation habits comprise three types of insensitivity to the electric power future price, sensitivity to the future price and sensitivity to the future price, different correction probabilities p are respectively set to be 0.1, 0.5 and 0.9, and the larger the correction probability is, the larger the probability of correcting the quotation of the generator under the influence of the future price is;
(3-3) generating a random number range between 0 and 1, and if the corresponding probability value is greater than the random, selecting to correct the quotation of the power generator, wherein the quotation coefficient after correction iskipOtherwise, the coefficient k is still adoptedi:
10. The generator pricing method considering the effect of power futures changes according to claim 6, characterized by: the step (4) specifically comprises the following steps:
(4-1) according to the corrected quotation coefficient kipAnd correcting the quoted price, and submitting the quoted price to ISO for clearing, wherein the ISO market clearing model is as follows:
in the formula: pGiThe output of the generator i; a isi、biRespectively a first-order coefficient and a second-order coefficient of the fuel cost, wherein L is a network node set; branch is a branch set; lambda [ alpha ]elClearing the price for the market of the node l; pDhThe load requirement of the h-th user; xijThe reactance value for branch ij; thetai、θjPhase angles corresponding to the nodes i and j respectively; pijmaxFor the current limit of line ij, PGimin、PGimaxRespectively the upper and lower technical output limits of the generator i;
(4-2) calculating the profit r according to the electricity price of the generator node fed back by the ISO and the winning electricity quantity:
in the formula, λeNode electricity price for node i;
(4-3) updating the corresponding value in the Q value table according to the profit r:
in the formula, Qn(s,an)、Qn+1(s,an) Adopt action a in state s before and after updating Q-value tablenThe corresponding Q value, α, is the learning rate of reinforcement learning, rnFor immediate benefit of the current action, gamma is the reinforcement learning discount rate,is a benefit in memory, is the state sn+1Maximum utility value that can be given;
(4-4) updating the status to s according to the clearing resultn+1(ii) a Judging whether the iteration times reach a preset maximum time or not; if yes, ending updating the Q value table, and if not, continuing to iteratively update the Q value table.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010207845.3A CN111582903A (en) | 2020-03-23 | 2020-03-23 | Generator intelligent agent considering electric power futures change influence and quotation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010207845.3A CN111582903A (en) | 2020-03-23 | 2020-03-23 | Generator intelligent agent considering electric power futures change influence and quotation method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111582903A true CN111582903A (en) | 2020-08-25 |
Family
ID=72112513
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010207845.3A Withdrawn CN111582903A (en) | 2020-03-23 | 2020-03-23 | Generator intelligent agent considering electric power futures change influence and quotation method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111582903A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112488389A (en) * | 2020-11-30 | 2021-03-12 | 国网浙江省电力有限公司电力科学研究院 | Automatic checking and correcting method and system for spot market clearing declaration parameters |
CN113159825A (en) * | 2021-03-04 | 2021-07-23 | 中国电力科学研究院有限公司 | Power generator collusion simulation method and device and storage medium |
CN113240459A (en) * | 2021-04-27 | 2021-08-10 | 东南大学 | Market member quotation method based on deep reinforcement learning algorithm and module thereof |
-
2020
- 2020-03-23 CN CN202010207845.3A patent/CN111582903A/en not_active Withdrawn
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112488389A (en) * | 2020-11-30 | 2021-03-12 | 国网浙江省电力有限公司电力科学研究院 | Automatic checking and correcting method and system for spot market clearing declaration parameters |
CN113159825A (en) * | 2021-03-04 | 2021-07-23 | 中国电力科学研究院有限公司 | Power generator collusion simulation method and device and storage medium |
CN113240459A (en) * | 2021-04-27 | 2021-08-10 | 东南大学 | Market member quotation method based on deep reinforcement learning algorithm and module thereof |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111582903A (en) | Generator intelligent agent considering electric power futures change influence and quotation method | |
Carriere et al. | An integrated approach for value-oriented energy forecasting and data-driven decision-making application to renewable energy trading | |
Liu et al. | Multi-objective optimization scheduling of wind–photovoltaic–hydropower systems considering riverine ecosystem | |
Yang et al. | A multi-objective stochastic optimization model for electricity retailers with energy storage system considering uncertainty and demand response | |
CN111210144A (en) | Power generation risk management method and system for electric power spot market | |
CN108182507A (en) | A kind of meter and probabilistic sale of electricity quotient the best electric price decision-making technique | |
Jiang et al. | Reserve cost allocation mechanism in renewable portfolio standard-constrained spot market | |
Zhang et al. | Cost-oriented load forecasting | |
CN109191017B (en) | Simulation method, device, equipment and storage medium of comprehensive energy system | |
CN112036611A (en) | Power grid optimization planning method considering risks | |
CN107644370A (en) | Price competing method and system are brought in a kind of self-reinforcing study together | |
CN111082451A (en) | Incremental distribution network multi-objective optimization scheduling model based on scene method | |
CN113887973A (en) | Multi-target planning method and device for comprehensive energy system and terminal equipment | |
CN111476399A (en) | Electricity purchasing method and system for electricity selling company under spot transaction mechanism | |
CN108985897A (en) | A kind of smart grid Generation Side Differential evolution game price competing method | |
CN111311012A (en) | Multi-agent-based micro-grid power market double-layer bidding optimization method | |
Wang et al. | Peer-to-peer electricity transaction decisions of the user-side smart energy system based on the SARSA reinforcement learning | |
CN112598473A (en) | Generator intelligent agent and quotation method based on depth certainty strategy gradient algorithm | |
CN111598719A (en) | New energy seller transaction method and system for spot power market | |
Liu et al. | Comparison of centralized and peer-to-peer decentralized market designs for community markets | |
CN115907575A (en) | ACP and carbon-green certificate transaction-based industrial comprehensive energy system cooperative operation method | |
Vincent et al. | Sustainable microgrid design with peer-to-peer energy trading involving government subsidies and uncertainties | |
Tang et al. | Power generation mix evolution based on rolling horizon optimal approach: A system dynamics analysis | |
CN110648191A (en) | Joint quotation method and system for power generator electric energy and standby market | |
Bublitz et al. | An agent-based model of the German electricity market with short-time uncertainty factors |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200825 |
|
WW01 | Invention patent application withdrawn after publication |