CN113706029A - Self-adaptive management method, device and system for household energy - Google Patents
Self-adaptive management method, device and system for household energy Download PDFInfo
- Publication number
- CN113706029A CN113706029A CN202111013941.5A CN202111013941A CN113706029A CN 113706029 A CN113706029 A CN 113706029A CN 202111013941 A CN202111013941 A CN 202111013941A CN 113706029 A CN113706029 A CN 113706029A
- Authority
- CN
- China
- Prior art keywords
- decision
- model
- meta
- environment
- management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 148
- 230000003044 adaptive effect Effects 0.000 claims abstract description 63
- 230000002787 reinforcement Effects 0.000 claims abstract description 41
- 230000006872 improvement Effects 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000000034 method Methods 0.000 claims description 36
- 230000008859 change Effects 0.000 claims description 29
- 230000006399 behavior Effects 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 13
- 230000007613 environmental effect Effects 0.000 claims description 10
- 238000009826 distribution Methods 0.000 claims description 9
- 238000005265 energy consumption Methods 0.000 claims description 7
- 230000006978 adaptation Effects 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 230000003542 behavioural effect Effects 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims 1
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Evolutionary Computation (AREA)
- Entrepreneurship & Innovation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Software Systems (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Life Sciences & Earth Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a self-adaptive management method, a device and a system for household energy. The adaptive management device comprises a data acquisition unit, an adaptive improvement unit and a decision management unit. The self-adaptive management system comprises a self-adaptive management module, an environment monitoring module and a cloud server. The self-adaptive management method, the device and the system improve the self-adaptability of the family energy management to different environments by training a first decision meta-model in advance, carrying out self-adaptive improvement on the first decision meta-model according to a reinforcement learning algorithm and a current environment data set to obtain a second decision meta-model and carrying out energy decision management on the current family environment according to the second decision meta-model.
Description
Technical Field
The invention relates to the field of self-adaptive management of household energy, in particular to a self-adaptive management method, a device and a system of household energy.
Background
The household energy management system is an intelligent decision-making system which performs optimization management on various energy consumption devices in a household, thereby realizing energy conservation and cost reduction and improving the comfort level of a user. The household energy management system works in a household environment, wherein the environment comprises household equipment managed by the household energy management system, the indoor and outdoor space environment of the household and user behavior influence, and different household environments can be generated by different equipment parameters and different probability distributions of random quantity. The existing household energy management optimization decision-making method generally has the problem of environmental dependence. That is, both modeling the environment and predicting the random quantities are environment-specific, even a data-driven reinforcement learning decision model can only make good decisions under a certain probability distribution of the environmental parameters when it is trained. Under the background of continuous development of smart homes, the development space of a home energy management system is larger and larger, the feasibility and the effectiveness of floor implementation are improved continuously, and the home energy management system is expected to become a core component of a new-generation smart home system.
In the prior art, the most common method is an integer programming method based on mathematical modeling, namely, a mathematical model of the working characteristics of the household electrical appliance is established firstly, the switching state of the mathematical model is expressed by integer variables, future environmental parameters are predicted firstly, and then the integer programming method such as Branch-and-Cut is adopted to solve by taking the minimum deviation of the integral household energy cost and the comfort degree of a user as an optimization target. On the basis, some researches consider the environment randomness, and the randomness is processed by adopting a random planning method such as a scene method and opportunity constraint. In addition, there are some studies on a reinforcement learning method based on data driving.
However, the prior art still has the following disadvantages: the self-adaptability of the household energy management system is poor, and the built-in optimization decision strategy of a manufacturer can hardly make good decisions in different environments in specific users, so that the operation effect of the household energy management system is seriously influenced; in addition, when the living environment or the behavior habit of the user changes, the home energy management system is difficult to perform decision self-adaptation, so that the decision capability is poor.
Therefore, there is a need for a method, apparatus and system for adaptive management of home energy, which can solve the above-mentioned problems in the prior art.
Disclosure of Invention
In view of the above technical problems, an object of the present invention is to provide a method, an apparatus and a system for adaptive management of home energy, so as to improve the adaptivity of home energy management.
The invention provides a self-adaptive management method of household energy, which comprises the following steps: acquiring a current environment data set;
according to the current environment data set and a preset reinforcement learning algorithm, performing self-adaptive improvement on a preset first decision-making meta-model so as to obtain a second decision-making meta-model; the first decision meta-model is a model trained in advance in a cloud server; and performing energy decision management on the current home environment according to the second decision meta-model.
In one embodiment, before the acquiring the current environment data set, the method further includes: and performing energy decision management on the current home environment through the first decision meta-model.
In one embodiment, after performing energy decision management on the current home environment according to the second decision meta-model, the method further includes: judging whether the current home environment changes; if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model; if so, performing self-adaptive improvement on a preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model; and performing energy decision management on the current family environment according to the third decision meta-model.
In one embodiment, the first decision meta-model is a model trained in advance in a cloud server, and specifically includes: establishing an environment model training set according to a plurality of preset parameter probability distributions; and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
In an embodiment, the adaptively improving the first decision meta-model according to the current environment data set and a preset reinforcement learning algorithm to obtain a second decision meta-model specifically includes: training the first decision-making meta-model according to the current environment data set by a random gradient descent method, and acquiring a convergence error of the trained first decision-making meta-model; judging whether the convergence error is smaller than a preset error threshold value or not; if the first decision meta-model is smaller than the second decision meta-model, finishing the training, and outputting the trained first decision meta-model as a second decision meta-model; and if not, continuing to train the first decision meta-model according to the current environment data set by a random gradient descent method.
The invention also provides a self-adaptive management device of the family energy, which comprises a data acquisition unit, an adaptive improvement unit and a decision management unit, wherein the data acquisition unit is used for acquiring the current environment data set; the adaptive improvement unit is used for carrying out adaptive improvement on a preset first decision-making meta-model according to the current environment data set and a preset reinforcement learning algorithm so as to obtain a second decision-making meta-model; the first decision meta-model is a model trained in advance in a cloud server; and the decision management unit is used for performing energy decision management on the current family environment according to the second decision meta-model.
In one embodiment, the adaptation-improving unit is further configured to: judging whether the current family environment change degree is higher than a preset environment change threshold value or not; if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model; and if so, carrying out self-adaptive improvement on the preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model.
The invention also provides a self-adaptive management system for the household energy, which comprises a self-adaptive management module, an environment monitoring module and a cloud server, wherein the environment monitoring module and the cloud server are respectively connected to the self-adaptive management module in a communication manner; wherein the adaptive management module is used for executing the adaptive management method of the household energy; the environment monitoring module is used for acquiring and storing a current environment data set and judging whether the current family environment change degree is higher than a preset environment change threshold value or not according to the current environment data set and a preset environment monitoring method; the cloud server is used for establishing an environment model training set according to a plurality of preset parameter probability distributions; and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
In one embodiment, the adaptive management system further comprises an analysis optimization module for: recording the management behavior of the self-adaptive management module; analyzing the energy consumption behavior habit of the current family and sending an analysis report to the current family according to the management behavior, the current environment data set and a preset energy consumption optimization strategy library; the analysis report includes a behavioral optimization suggestion.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the invention provides a self-adaptive management method, a device and a system for household energy, which are characterized in that a first decision-making meta-model is trained in advance, the first decision-making meta-model is subjected to self-adaptive improvement according to a reinforcement learning algorithm and a current environment data set to obtain a second decision-making meta-model, and energy decision management is performed on the current household environment according to the second decision-making meta-model.
Further, according to the adaptive management method, device and system for the household energy, provided by the invention, when the change degree of the current household environment is judged to be higher than the preset environment change threshold value, the changed environment data set is collected again, the preset first decision-making meta-model is subjected to adaptive improvement again according to the changed environment data set and the reinforcement learning method, so that a third decision-making meta-model is obtained, and then the energy decision-making management is performed on the current household environment according to the third decision-making meta-model, so that the accuracy of the adaptive management of the household energy is improved.
Drawings
The invention will be further described with reference to the accompanying drawings, in which:
fig. 1 shows a flow chart of an embodiment of a method for adaptive management of home energy according to the invention;
fig. 2 shows a flow chart of another embodiment of a method for adaptive management of home energy according to the present invention;
fig. 3 shows a block diagram of an embodiment of an apparatus for adaptive management of home energy according to the present invention;
fig. 4 shows a block diagram of an embodiment of an adaptive management system for home energy according to the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Detailed description of the preferred embodiment
The embodiment of the invention firstly describes a self-adaptive management method of household energy. Fig. 1 shows a flow chart of an embodiment of a method for adaptive management of home energy according to the present invention. As shown in fig. 1, the method comprises the steps of:
and S1, acquiring the current environment data set.
In order to adapt to the currently deployed home environment, the current environment data set needs to be acquired so as to perform adaptive improvement on the preset first decision meta-model subsequently.
And S2, carrying out self-adaptive improvement on a preset first decision-making meta-model according to the current environment data set and a preset reinforcement learning algorithm so as to obtain a second decision-making meta-model.
In one embodiment, a preset first decision meta-model is adaptively improved according to the current environment data set and a preset reinforcement learning algorithm, so as to obtain a second decision meta-model, specifically:
a: using a first decision meta-model
Performing K Episodes in a real environment and calculating a loss function
B: using gradient descent pair parameters
Performing one-time updating to obtain
C: judgment of
And if the value is less than epsilon, continuing the next step, otherwise, turning to the step A.
D: the trained first decision meta-model
The output is a second decision meta-model.
Each Episode is defined as an operation period or characteristic operation duration of specific controlled equipment in the household energy system; ε is the convergence tolerance error constant; α is a learning rate constant.
And S3, performing energy decision management on the current family environment according to the second decision meta-model.
Through step S2, the first decision meta-model originally set in factory is already adaptively trained to sufficiently adapt to the current home environment, so that energy decision management can be performed on the current home environment through the second decision meta-model (the adaptively trained first decision meta-model).
The invention provides a self-adaptive management method of household energy, which improves the self-adaptability of household energy management to different environments by training a first decision-making meta-model in advance, carrying out self-adaptive improvement on the first decision-making meta-model according to a reinforcement learning algorithm and a current environment data group to obtain a second decision-making meta-model and carrying out energy decision-making management on the current household environment according to the second decision-making meta-model.
Detailed description of the invention
Furthermore, the embodiment of the invention also describes a method for adaptively managing the household energy. Fig. 2 shows a flow chart of another embodiment of a method for adaptive management of home energy according to the present invention.
As shown in fig. 2, the method comprises the steps of:
a1: and performing energy decision management on the current home environment through the first decision meta-model.
The first decision meta-model is a model trained in advance at a cloud server. Specifically, in one embodiment, the first decision meta-model training process is as follows: establishing an environment model training set according to a plurality of preset parameter probability distributions; and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
Further, in an embodiment, the first decision meta-model training process specifically includes the following steps:
and a101, establishing an environment model training set according to a plurality of preset parameter probability distributions.
a102, establishing a preset reinforcement learning decision model fθAnd initializes the reinforcement learning parameter θ.
a103, sampling N environments from the environment model training set, and recording as e1,e2,…,eNThe loss function corresponding to each environment is L1,L2,…,LN。
and a104, making i equal to 0.
a105, judging whether i is smaller than N, if so, changing i to i +1, and continuing the next step; if not, the step a109 is executed.
a106 adopting a strategy fθIn the environment eiPerforms K Episodes and calculates the loss function Li(fθ)。
a107, updating theta once by adopting gradient descent to obtain
a108, adopting the updated strategy fθiIn the environment eiPerforms K Episodes and calculates the loss function Li(fθi)。
a109, updating the element parameter theta,
a110 judging
And if not, skipping to execute the step a 104.
Wherein, the environment model training set comprises a plurality of environments, and each environment comprises a controlled electrical appliance model and a working environmentA model and a user behavior model; α is a learning rate constant, θiThe parameters of the decision model obtained by training in the environment i are obtained; ε is the convergence tolerance error constant, where β is the learning rate constant of the meta-learning; each Episode is defined as the operating duration of one operating cycle or characteristic of a particular controlled device.
A2: and acquiring a current environment data set.
In order to adapt to the currently deployed home environment, the current environment data set needs to be acquired so as to perform adaptive improvement on the preset first decision meta-model subsequently.
A3: and performing self-adaptive improvement on a preset first decision-making meta-model according to the current environment data set and a preset reinforcement learning algorithm so as to obtain a second decision-making meta-model.
In one embodiment, a preset first decision meta-model is adaptively improved according to the current environment data set and a preset reinforcement learning algorithm, so as to obtain a second decision meta-model, specifically:
a: using a first decision meta-model
Performing K Episodes in a real environment and calculating a loss function
B: using gradient descent pair parameters
Performing one-time updating to obtain
C: judgment of
And if the value is less than epsilon, continuing the next step, otherwise, turning to the step A.
D: the trained first decision meta-model
The output is a second decision meta-model.
Each Episode is defined as an operation period or characteristic operation duration of specific controlled equipment in the household energy system; ε is the convergence tolerance error constant; α is a learning rate constant.
A4: and performing energy decision management on the current home environment according to the second decision meta-model.
Through step S2, the first decision meta-model originally set in factory is already adaptively trained to sufficiently adapt to the current home environment, so that energy decision management can be performed on the current home environment through the second decision meta-model (the adaptively trained first decision meta-model).
A5: and judging whether the current family environment change degree is higher than a preset environment change threshold value.
When the environment changes greatly, the second decision meta-model may no longer be suitable for a new environment, so that, while performing energy decision management on the current home environment according to the second decision meta-model, it should be monitored in real time whether the current home environment changes, and corresponding measures should be taken according to the monitoring result.
A61: and if not, continuing to perform energy decision management on the current family environment according to the second decision meta-model.
A62: if so, performing self-adaptive improvement on a preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model; and performing energy decision management on the current family environment according to the third decision meta-model.
When the change degree of the current home environment is higher than a preset environment change threshold value, theoretically, the second decision-making meta-model is not suitable for the changed home environment any more, so that a changed environment data set needs to be collected again, the preset first decision-making meta-model is subjected to adaptive improvement again according to the changed environment data set and a preset reinforcement learning algorithm, a third decision-making meta-model is obtained, and then energy decision management is performed on the current home environment according to the third decision-making meta-model.
By adding the judgment and correction steps (A5-A62), the embodiment of the invention can directly re-collect the changed environment data set and re-perform model adaptive training when the current family environment has great change, thereby improving the speed and efficiency of the re-adaptation on the basis of rapid deployment and adaptation.
The embodiment of the invention provides a self-adaptive management method of household energy, which is characterized in that a first decision-making meta-model is trained in advance through a reinforcement learning algorithm, the first decision-making meta-model is subjected to self-adaptive improvement according to the reinforcement learning algorithm and a current environment data group to obtain a second decision-making meta-model, and energy decision-making management is carried out on a current household environment according to the second decision-making meta-model, so that the self-adaptive management method improves the self-adaptability of household energy management to different environments; further, the adaptive management method for the family energy provided by the invention also acquires the changed environment data set again when the change degree of the current family environment is judged to be higher than the preset environment change threshold value, performs adaptive improvement on the preset first decision-making meta-model again according to the changed environment data set and the reinforcement learning method, so as to obtain a third decision-making meta-model, and performs energy decision-making management on the current family environment according to the third decision-making meta-model, so as to improve the accuracy of the adaptive management of the family energy.
Detailed description of the preferred embodiment
Besides the method, the embodiment of the invention also describes a device for adaptively managing the household energy. Fig. 3 shows a block diagram of an embodiment of an apparatus for adaptive management of home energy according to the present invention.
As shown in fig. 3, the adaptive management apparatus includes a data acquisition unit 11, an adaptation improvement unit 12, and a decision management unit 13.
The data acquisition unit 11 is used for acquiring the current environment data set.
The adaptive improvement unit 12 is configured to perform adaptive improvement on a preset first decision meta-model according to the current environment data set and a preset reinforcement learning algorithm, so as to obtain a second decision meta-model. The first decision meta-model is a model trained in advance at a cloud server.
The decision management unit 13 is configured to perform energy decision management on the current home environment according to the second decision meta-model. In one embodiment, the decision management unit 13 is further configured to perform energy decision management on the current home environment through the first decision meta-model before acquiring the current environment data set. In an embodiment, the decision management unit 13 is further configured to perform energy decision management on the current home environment according to the third decision meta-model.
In one embodiment, the adaptation-improving unit 12 is further configured to: judging whether the current family environment change degree is higher than a preset environment change threshold value or not; if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model; and if so, carrying out self-adaptive improvement on the preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model.
When self-adaptive management of family energy is needed, firstly, a decision management unit 13 performs energy decision management on the current family environment through a first decision meta-model trained in advance in a cloud server, so as to generate a current environment data set, and a data acquisition unit 11 acquires the current environment data set; then, the adaptive improvement unit 12 performs adaptive improvement on a preset first decision meta-model according to the current environment data set and a preset reinforcement learning algorithm, so as to obtain a second decision meta-model; finally, the decision management unit 13 performs energy decision management on the current home environment according to the second decision meta-model; when the decision management unit performs energy decision management on the current home environment according to the second decision meta-model, the adaptive improvement unit 12 judges whether the change degree of the current home environment is higher than a preset environment change threshold value in real time; if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model; if so, performing adaptive improvement on a preset first decision meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision meta-model, and enabling a decision management unit 13 to perform energy decision management on the current home environment according to the third decision meta-model.
The invention provides a self-adaptive management device for household energy, which is characterized in that a first decision-making meta-model is trained in advance, the first decision-making meta-model is subjected to self-adaptive improvement according to a reinforcement learning algorithm and a current environment data group to obtain a second decision-making meta-model, and energy decision-making management is performed on a current household environment according to the second decision-making meta-model, so that the self-adaptive management device improves the self-adaptability of household energy management to different environments; furthermore, the adaptive management device for the family energy provided by the invention also acquires the changed environment data set again when the change degree of the current family environment is judged to be higher than the preset environment change threshold value, performs adaptive improvement on the preset first decision-making meta-model again according to the changed environment data set and the reinforcement learning method, so as to obtain a third decision-making meta-model, and performs energy decision-making management on the current family environment according to the third decision-making meta-model, so as to improve the accuracy of the adaptive management of the family energy.
Detailed description of the invention
In addition to the above method and apparatus, the present invention also describes an adaptive management system for home energy. Fig. 4 shows a block diagram of an embodiment of an adaptive management system for home energy according to the invention.
As shown in the figure, the adaptive management system comprises an adaptive management module 1, an environment monitoring module 2 and a cloud server 3, wherein the environment monitoring module 2 and the cloud server 3 are respectively connected to the adaptive management module 1 in a communication manner.
The adaptive management module 1 is configured to execute the adaptive management method for home energy as described above, so that the adaptive management system can be quickly adaptively deployed in different home environments and is adaptively deployed and managed again when the home environment is changed halfway, thereby implementing adaptive management of home energy.
The environment monitoring module 2 is used for collecting and storing a current environment data set, and judging whether the current family environment change degree is higher than a preset environment change threshold value according to the current environment data set and a preset environment monitoring method.
The cloud server 3 is used for establishing an environment model training set according to a plurality of preset parameter probability distributions; and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
In one embodiment, the adaptive management system further comprises an analysis optimization module for: recording the management behavior of the self-adaptive management module; and analyzing the energy consumption behavior habit of the current family and sending an analysis report to the current family according to the management behavior, the current environment data set and a preset energy consumption optimization strategy library. In one embodiment, the analysis report includes a behavior optimization suggestion.
The invention provides a self-adaptive management system for household energy, which is characterized in that a first decision-making meta-model is trained in advance, the first decision-making meta-model is subjected to self-adaptive improvement according to a reinforcement learning algorithm and a current environment data group to obtain a second decision-making meta-model, and energy decision-making management is carried out on the current household environment according to the second decision-making meta-model, so that the self-adaptive management system improves the self-adaptability of household energy management to different environments; furthermore, the adaptive management system for the family energy provided by the invention also acquires the changed environment data set again when the change degree of the current family environment is judged to be higher than the preset environment change threshold value, performs adaptive improvement on the preset first decision-making meta-model again according to the changed environment data set and the reinforcement learning method, so as to obtain a third decision-making meta-model, and performs energy decision-making management on the current family environment according to the third decision-making meta-model, so as to improve the accuracy of the adaptive management of the family energy.
The above-mentioned embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, and it should be understood that the above-mentioned embodiments are only examples of the present invention and are not intended to limit the scope of the present invention. It should be understood that any modifications, equivalents, improvements and the like, which come within the spirit and principle of the invention, may occur to those skilled in the art and are intended to be included within the scope of the invention.
Claims (9)
1. A method for adaptive management of home energy, comprising:
acquiring a current environment data set;
according to the current environment data set and a preset reinforcement learning algorithm, performing self-adaptive improvement on a preset first decision-making meta-model so as to obtain a second decision-making meta-model; the first decision meta-model is a model trained in advance in a cloud server;
and performing energy decision management on the current home environment according to the second decision meta-model.
2. The adaptive management method of home energy according to claim 1, further comprising, before said obtaining the current set of environmental data:
and performing energy decision management on the current home environment through the first decision meta-model.
3. The adaptive home energy management method of claim 2, further comprising, after performing energy decision management on the current home environment according to the second decision meta-model:
judging whether the current family environment change degree is higher than a preset environment change threshold value or not;
if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model;
if so, performing self-adaptive improvement on a preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model; and performing energy decision management on the current family environment according to the third decision meta-model.
4. The adaptive management method for home energy according to claim 3, wherein the first decision meta-model is a model trained in advance at a cloud server, and specifically comprises:
establishing an environment model training set according to a plurality of preset parameter probability distributions;
and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
5. The adaptive management method for home energy according to any one of claims 1 to 4, wherein the adaptive improvement is performed on the first decision meta-model according to the current environment data set and a preset reinforcement learning algorithm, so as to obtain a second decision meta-model, specifically:
training the first decision-making meta-model according to the current environment data set by a random gradient descent method, and acquiring a convergence error of the trained first decision-making meta-model;
judging whether the convergence error is smaller than a preset error threshold value or not;
if the first decision meta-model is smaller than the second decision meta-model, finishing the training, and outputting the trained first decision meta-model as a second decision meta-model;
and if not, continuing to train the first decision meta-model according to the current environment data set by a random gradient descent method.
6. An adaptive management device for home energy, comprising a data acquisition unit, an adaptation improvement unit, and a decision management unit, wherein,
the data acquisition unit is used for acquiring a current environment data set;
the adaptive improvement unit is used for carrying out adaptive improvement on a preset first decision-making meta-model according to the current environment data set and a preset reinforcement learning algorithm so as to obtain a second decision-making meta-model; the first decision meta-model is a model trained in advance in a cloud server;
and the decision management unit is used for performing energy decision management on the current family environment according to the second decision meta-model.
7. The adaptive management device of home energy according to claim 6, characterized in that the adaptation improvement unit is further configured to:
judging whether the current family environment change degree is higher than a preset environment change threshold value or not;
if not, continuing to perform energy decision management on the current home environment according to the second decision meta-model;
and if so, carrying out self-adaptive improvement on the preset first decision-making meta-model according to the changed environment data set and a preset reinforcement learning algorithm so as to obtain a third decision-making meta-model.
8. The self-adaptive management system for the household energy is characterized by comprising a self-adaptive management module, an environment monitoring module and a cloud server, wherein the environment monitoring module and the cloud server are respectively connected to the self-adaptive management module in a communication mode; wherein the content of the first and second substances,
the adaptive management module is configured to perform an adaptive management method of home energy according to any one of claims 1-5;
the environment monitoring module is used for acquiring and storing a current environment data set and judging whether the current family environment change degree is higher than a preset environment change threshold value or not according to the current environment data set and a preset environment monitoring method;
the cloud server is used for establishing an environment model training set according to a plurality of preset parameter probability distributions; and training a preset reinforcement learning decision model according to the environmental model training group by a meta-learning method, thereby obtaining a first decision meta-model.
9. The adaptive management system of home energy as claimed in claim 8, further comprising an analysis optimization module for:
recording the management behavior of the self-adaptive management module;
analyzing the energy consumption behavior habit of the current family and sending an analysis report to the current family according to the management behavior, the current environment data set and a preset energy consumption optimization strategy library; the analysis report includes a behavioral optimization suggestion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111013941.5A CN113706029A (en) | 2021-08-31 | 2021-08-31 | Self-adaptive management method, device and system for household energy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111013941.5A CN113706029A (en) | 2021-08-31 | 2021-08-31 | Self-adaptive management method, device and system for household energy |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113706029A true CN113706029A (en) | 2021-11-26 |
Family
ID=78658127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111013941.5A Pending CN113706029A (en) | 2021-08-31 | 2021-08-31 | Self-adaptive management method, device and system for household energy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706029A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106322656A (en) * | 2016-08-23 | 2017-01-11 | 海信（山东）空调有限公司 | Air conditioner control method, server and air conditioner system |
| CN112161322A (en) * | 2020-09-24 | 2021-01-01 | 深圳市合信达控制系统有限公司 | Heating equipment and control method thereof |
| CN112198853A (en) * | 2020-10-10 | 2021-01-08 | 珠海格力电器股份有限公司 | Control method and device of intelligent household equipment |
| CN112611080A (en) * | 2020-12-10 | 2021-04-06 | 浙江大学 | Intelligent air conditioner control system and method based on federal learning |
| CN113044064A (en) * | 2021-04-01 | 2021-06-29 | 南京大学 | Vehicle self-adaptive automatic driving decision method and system based on meta reinforcement learning |
-
2021
- 2021-08-31 CN CN202111013941.5A patent/CN113706029A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106322656A (en) * | 2016-08-23 | 2017-01-11 | 海信（山东）空调有限公司 | Air conditioner control method, server and air conditioner system |
| CN112161322A (en) * | 2020-09-24 | 2021-01-01 | 深圳市合信达控制系统有限公司 | Heating equipment and control method thereof |
| CN112198853A (en) * | 2020-10-10 | 2021-01-08 | 珠海格力电器股份有限公司 | Control method and device of intelligent household equipment |
| CN112611080A (en) * | 2020-12-10 | 2021-04-06 | 浙江大学 | Intelligent air conditioner control system and method based on federal learning |
| CN113044064A (en) * | 2021-04-01 | 2021-06-29 | 南京大学 | Vehicle self-adaptive automatic driving decision method and system based on meta reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110647900B (en) | Intelligent safety situation prediction method, device and system based on deep neural network | |
| CN107690176B (en) | Network selection method based on Q learning algorithm | |
| CN112272074B (en) | Information transmission rate control method and system based on neural network | |
| CN113485144A (en) | Intelligent household control method and system based on Internet of things | |
| CN112329997A (en) | Power demand load prediction method and system, electronic device, and storage medium | |
| US20230216737A1 (en) | Network performance assessment | |
| CN113706029A (en) | Self-adaptive management method, device and system for household energy | |
| CN112312299A (en) | Service unloading method, device and system | |
| CN104778495A (en) | Bayesian network optimization method based on particle swarm algorithm | |
| CN110705756A (en) | Electric power energy consumption optimization control method based on input convex neural network | |
| CN110262275A (en) | A kind of smart home system and its control method | |
| Singh et al. | Grey Wolf Optimization Based CNN-LSTM Network for the Prediction of Energy Consumption in Smart Home Environment | |
| Verma et al. | Making smart homes smarter: optimizing energy consumption with human in the loop | |
| CN107942693A (en) | Control method, apparatus, storage medium and the computer equipment of home appliance | |
| CN110987062A (en) | Intelligent metering box early warning method and system based on data fusion | |
| CN112926777A (en) | Day-ahead load prediction method and system considering commercial building energy consumption behaviors | |
| CN111047465A (en) | Power grid friendly load response method based on electric power big data | |
| KR102542488B1 (en) | Method and Apparatus for Predicting Appliance Power Usage State Using Non Intrusive Load Monitoring Based on Long-Short Term Memory | |
| Kang et al. | Multi-objective evolutionary optimization of 3D differentiated sensor network deployment | |
| CN114460855B (en) | AIOT module based on intelligence house | |
| KR101330526B1 (en) | Method, Apparatus and Recording Medium for Migration of Prediction Systems | |
| CN117371681A (en) | Power resource scheduling method, device, equipment and storage medium | |
| CN117348431A (en) | Device control method, device and computer readable medium | |
| CN117277346A (en) | Energy storage frequency modulation method, device and equipment based on multi-agent system | |
| bin Dahari et al. | Energy Efficiency of IoT-Edge Computing in Smart Grid Using Deep Reinforcement Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |