CN112231306B - Big data based energy data analysis system and method - Google Patents
Big data based energy data analysis system and method Download PDFInfo
- Publication number
- CN112231306B CN112231306B CN202010853260.9A CN202010853260A CN112231306B CN 112231306 B CN112231306 B CN 112231306B CN 202010853260 A CN202010853260 A CN 202010853260A CN 112231306 B CN112231306 B CN 112231306B
- Authority
- CN
- China
- Prior art keywords
- data
- energy
- stack
- energy data
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007405 data analysis Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000011218 segmentation Effects 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000013500 data storage Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000013144 data compression Methods 0.000 claims abstract description 7
- 238000011550 data transformation method Methods 0.000 claims abstract description 7
- 238000013499 data model Methods 0.000 claims description 30
- 238000012549 training Methods 0.000 claims description 25
- 238000003745 diagnosis Methods 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 4
- 230000008676 import Effects 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract description 8
- 238000013136 deep learning model Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 12
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000005183 dynamical system Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 239000003345 natural gas Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000003208 petroleum Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B31/00—Predictive alarm systems characterised by extrapolation or other computation using updated historic data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/80—Management or planning
- Y02P90/82—Energy audits or management systems therefor
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Databases & Information Systems (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Probability & Statistics with Applications (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Water Supply & Treatment (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Emergency Management (AREA)
- Primary Health Care (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Complex Calculations (AREA)
Abstract
The invention belongs to the technical field of data analysis, and particularly relates to an energy data analysis system and method based on big data. The system comprises: a data acquisition unit comprising: the system comprises various acquisition units deployed on the site, wherein the acquisition units are configured to acquire original energy data; a data storage unit: carrying out data segmentation on the acquired original energy data, and then carrying out classified storage to remove data redundancy; a data preprocessing unit: checking the integrity of the data, removing noise and singular values, reducing the number of effective variables or data samples to be considered by using a data compression and transformation method according to the task target, or finding an invariant identifier of the data, and finally carrying out data standardization. The redundancy of the energy data is effectively reduced and the efficiency of subsequent data processing is improved by carrying out data segmentation and classification on the energy data; meanwhile, the deep learning model is used for carrying out multi-scale analysis on the energy data, and the accuracy of data analysis results is improved.
Description
Technical Field
The invention belongs to the technical field of data analysis, and particularly relates to an energy data analysis system and method based on big data.
Background
Big data refers to a collection of data whose content cannot be captured, managed, and processed within a certain time using conventional software tools. Big data technology, refers to the ability to quickly obtain valuable information from a wide variety of types of data. Technologies applicable to big data include Massively Parallel Processing (MPP) databases, data mining grids, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.
At present, the acquisition and integration of mass energy data are the basis for building energy big data, but the information island problem which generally exists in the energy field becomes an important restriction factor for promoting the energy data resource integration. On one hand, in the informatization process of energy enterprises such as electric power, coal, petroleum, natural gas, cooling/heating and the like, a plurality of independent energy management systems exist in the energy enterprises due to the lack of an effective unified management mechanism, and the data of the independent systems can be collected through respective sensors. However, due to the fact that the system architectures, protocols and the like are inconsistent, the data acquired by the systems cannot be shared, and further analysis and mining of the energy big data are restricted. On the other hand, the traditional power and other energy systems keep the situation of respective planning, independent operation and strip and block division for a long time, cross-system industrial barriers are serious, data standards and contents are different, information intercommunication among different energy systems is closed, coordination and cooperation are lacked among the different energy systems, so that the total energy utilization efficiency is not high, economic and efficient operation of an energy supply system is not facilitated, the shortage problem of traditional fossil energy is more and more serious, and the problem of environmental pollution is endless. .
Disclosure of Invention
In view of this, the main objective of the present invention is to provide a big data-based energy data analysis system and method, which effectively reduce redundancy of energy data and improve efficiency of subsequent data processing by performing data segmentation and classification on the energy data; meanwhile, the deep learning model is used for carrying out multi-scale analysis on the energy data, and the accuracy of data analysis results is improved.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a big-data based energy data analysis system, the system comprising: a data acquisition unit comprising: the system comprises various acquisition units deployed on the site, wherein the acquisition units are configured to acquire original energy data; the data storage unit is configured for carrying out data segmentation on the acquired original energy data, then carrying out classified storage and removing data redundancy; a data preprocessing unit: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; the data analysis model establishing unit is configured for establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; and the early warning unit is configured to monitor and early warn a data analysis result according to a preset index.
Further, the data analysis model establishing unit includes: the system comprises a frame establishing unit, a data model establishing unit and a data model establishing unit, wherein the frame establishing unit is configured to determine a deep learning basic frame, import energy training data and establish a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; the model establishing unit is configured to establish a data model of each node by adopting a mathematical equation, and preset relevant parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; and the initialization unit is configured to initialize the parameter values, compare the output values of the nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correct the parameter values of the nodes, and sequentially circulate to finally obtain the parameter values in the nodes, which enable the output values of the nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum.
Further, the data storage unit: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
Further, the model building unit, each node uses mathematical equation to build the data model of the node, and the method comprises the following steps: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F j1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; and (3) establishing a data model of the node by using the following formula according to the calculated probability:
where y is a defined category parameter, which may be any value, but y is different for each data category.
Further, p (F) is obtained according to calculationj) Classifying, specifically executing the following steps: setting a threshold value, and calculating all the obtained p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
A big-data based energy data analysis method based on the system of one of claims 1 to 5, the method performing the steps of: step 1: collecting original energy data; step 2: carrying out data segmentation on the acquired original energy data, and then carrying out classified storage to remove data redundancy; and step 3: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; and 4, step 4: establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; and 5: and monitoring and early warning the data analysis result according to a preset index.
Further, the step 4: the method for establishing the data analysis model comprises the following steps: determining a deep learning basic framework, importing energy training data, establishing a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, wherein the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; establishing a data model of each node by adopting a mathematical equation, presetting related parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; initializing the parameter values, comparing the output values of the nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correcting the parameter values of the nodes, sequentially circulating, and finally obtaining the parameter values of the nodes which enable the output values of the nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum.
Further, step 2: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
Further, the method for establishing the data model of each node by using the mathematical equation comprises the following steps: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F j1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; and (3) establishing a data model of the node by using the following formula according to the calculated probability:
where y is a defined category parameter, which may be any value, but y is different for each data category.
Further, p (F) is obtained according to calculationj) Classifying, specifically executing the following steps: setting a threshold value, and calculating all the obtained p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
The energy data analysis system and method based on big data have the following beneficial effects: the redundancy of the energy data is effectively reduced and the efficiency of subsequent data processing is improved by carrying out data segmentation and classification on the energy data; meanwhile, the deep learning model is used for carrying out multi-scale analysis on the energy data, and the accuracy of data analysis results is improved. The method is mainly realized by the following steps: 1. the method comprises the steps of carrying out data segmentation on the acquired original energy data, and then carrying out classified storage to remove data redundancy; in the process of removing data redundancy, received energy data is uniformly converted into binary data, the binary data is subjected to slicing processing with a fixed size, a unique stack value is generated for each energy source data block, the energy source data blocks are connected in a stack structure, and a root stack is generated to serve as a stack identifier of the energy data, so that the divided energy data are stored in different stacks, on one hand, the confidentiality of the data is improved because the same data is stored in different stack structures, and even if the data stored in one stack structure is obtained, the complete data is still difficult to obtain; on the other hand, because the data is divided and then the divided data is sequentially stored in a stack, the redundancy between the original data storage is greatly reduced, the space utilization rate of the data storage is improved, and meanwhile, the efficiency is also improved when the stack identifier is used for calling; 2Modeling of data analysis: when the model is built, importing energy training data, and building a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics for the energy training data, wherein the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; the establishment of the neural network for data analysis is completed, and the established neural network model can perform multi-node data analysis in parallel on one hand and has autonomous learning capability on the other hand; meanwhile, in the process of establishing a node mathematical equation aiming at each data node, randomly extracting data from energy training data, and inputting the extracted data, wherein the class set of the data is expressed as: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F j1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
the data type probability distribution model for the nodes is established in the way, the prior probability of the data type of each node, namely each independently analyzed data node, can be obtained through the established mathematical equation, so that the data subjected to the stack splitting processing can be analyzed in advance, the analyzed results are compared, and the accuracy of the analysis is further adjusted continuously.
Drawings
Fig. 1 is a schematic system structure diagram of a big data-based energy data analysis system according to an embodiment of the present invention;
fig. 2 is a schematic method flow diagram of a big data-based energy data analysis method according to an embodiment of the present invention.
Fig. 3 is a schematic flowchart of a method for establishing a data analysis model of a big data-based energy data analysis method according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a principle of data segmentation of the big data-based energy data analysis system and method according to an embodiment of the present invention.
Fig. 5 is a graph illustrating a curve of the data analysis accuracy rate varying with the number of experiments according to the system and method for analyzing energy data based on big data according to the embodiment of the present invention, and a comparison experiment effect diagram in the prior art.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
Example 1
As shown in fig. 1, a big data based energy data analysis system, the system comprising: a data acquisition unit comprising: the system comprises various acquisition units deployed on the site, wherein the acquisition units are configured to acquire original energy data; the data storage unit is configured for carrying out data segmentation on the acquired original energy data, then carrying out classified storage and removing data redundancy; a data preprocessing unit: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; the data analysis model establishing unit is configured for establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; and the early warning unit is configured to monitor and early warn a data analysis result according to a preset index.
By adopting the technical scheme, the redundancy of the energy data is effectively reduced and the efficiency of subsequent data processing is improved by carrying out data segmentation and classification on the energy data; meanwhile, the deep learning model is used for carrying out multi-scale analysis on the energy data, and the accuracy of data analysis results is improved.
Example 2
On the basis of the above embodiment, the data analysis model establishing unit includes: the system comprises a frame establishing unit, a data model establishing unit and a data model establishing unit, wherein the frame establishing unit is configured to determine a deep learning basic frame, import energy training data and establish a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; the model establishing unit is configured to establish a data model of each node by adopting a mathematical equation, and preset relevant parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; and the initialization unit is configured to initialize the parameter values, compare the output values of the nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correct the parameter values of the nodes, and sequentially circulate to finally obtain the parameter values in the nodes, which enable the output values of the nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum.
Specifically, when the model is built, energy training data are imported, a data model comprising an input layer, at least one hidden layer and an output layer is built according to data characteristics of the energy training data, the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; the establishment of the neural network for data analysis is completed, and the established neural network model can perform multi-node data analysis in parallel on one hand and has autonomous learning capability on the other hand; meanwhile, in the process of establishing a node mathematical equation aiming at each data node, randomly extracting data from energy training data, and inputting the extracted data, wherein the class set of the data is expressed as: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as Fj1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
the data type probability distribution model for the nodes is established in the way, the prior probability of the data type of each node, namely each independently analyzed data node, can be obtained through the established mathematical equation, so that the data subjected to the stack splitting processing can be analyzed in advance, the analyzed results are compared, and the accuracy of the analysis is further adjusted continuously.
Example 3
On the basis of the above embodiment, the data storage unit: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
Specifically, the data redundancy is removed by carrying out data segmentation on the acquired original energy data and then carrying out classified storage; in the process of removing data redundancy, received energy data is uniformly converted into binary data, the binary data is subjected to slicing processing with a fixed size, a unique stack value is generated for each energy source data block, the energy source data blocks are connected in a stack structure, and a root stack is generated to serve as a stack identifier of the energy data, so that the divided energy data are stored in different stacks, on one hand, the confidentiality of the data is improved because the same data is stored in different stack structures, and even if the data stored in one stack structure is obtained, the complete data is still difficult to obtain; on the other hand, because the data are divided, and then the divided data are sequentially stored in a stack, the redundancy between the original data during storage is greatly reduced, the space utilization rate of data storage is improved, and meanwhile, the efficiency is also improved when the stack identifier is used for calling.
Example 4
On the basis of the above embodiment, the model building unit, the data model method for building the node by each node using the mathematical equation, performs the following steps: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F j1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; and (3) establishing a data model of the node by using the following formula according to the calculated probability:
where y is a defined category parameter, which may be any value, but y is different for each data category.
Specifically, in the process of establishing a node mathematical equation for each data node, data are randomly extracted from energy training data, the extracted data are input, and a class set of the data is represented as: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F ，1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
the data type probability distribution model for the nodes is established in the way, the prior probability of the data type of each node, namely each independently analyzed data node, can be obtained through the established mathematical equation, so that the data subjected to the stack splitting processing can be analyzed in advance, the analyzed results are compared, and the accuracy of the analysis is further adjusted continuously.
Example 5
Based on the above embodiment, the p (F) is obtained by calculationj) Classifying, specifically executing the following steps: setting a threshold value, all the metersCalculated p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
Example 6
A big-data based energy data analysis method based on the system of one of claims 1 to 5, the method performing the steps of: step 1: collecting original energy data; step 2: carrying out data segmentation on the acquired original energy data, and then carrying out classified storage to remove data redundancy; and step 3: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; and 4, step 4: establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; and 5: and monitoring and early warning the data analysis result according to a preset index.
In particular, data redundancy can hinder the integrity (integrity) of data in a database and also can result in wasted storage space. Reducing data redundancy as much as possible is one of the main goals of database design. One of the main ideas in the normalization theory of relational schema (hereinafter NF theory) is the principle of minimum redundancy, i.e. the normalized relational schema should in some sense have minimum redundancy. However, the NF theory has no standard concept available yet, and according to the principle of equivalence, under different premises with or without universal relationship assumption (universal relationship assumption), the definition of redundancy may be better.
Example 7
On the basis of the above embodiment, the step 4: the method for establishing the data analysis model comprises the following steps: determining a deep learning basic framework, importing energy training data, establishing a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, wherein the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; establishing a data model of each node by adopting a mathematical equation, presetting related parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; initializing the parameter values, comparing the output values of the nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correcting the parameter values of the nodes, sequentially circulating, and finally obtaining the parameter values of the nodes which enable the output values of the nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum.
Specifically, the artificial neural network can be roughly classified into a feedforward network (also called a multi-layer perceptron network) and a feedback network (also called a Hopfield network) according to model structures, wherein the feedforward network can be regarded as a large-scale nonlinear mapping system mathematically, and the feedback network is a large-scale nonlinear dynamical system. According to the learning mode, the artificial neural network can be divided into three types of supervised learning, unsupervised learning and semi-supervised learning; the method can be divided into two categories of determinacy and randomness according to the working mode; the temporal characteristics can be further classified into a continuous type or a discrete type.
Example 8
On the basis of the above embodiment, step 2: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
Example 9
On the basis of the previous embodiment, the method for establishing the data model of each node by adopting the mathematical equation executes the following steps: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as F j1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; and (3) establishing a data model of the node by using the following formula according to the calculated probability:
where y is a defined category parameter, which may be any value, but y is different for each data category.
Example 10
Based on the above embodiment, the p (F) is obtained by calculationj) Classifying, specifically executing the following steps: setting a threshold value, and calculating all the obtained p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the system provided in the foregoing embodiment is only illustrated by dividing the functional units, and in practical applications, the functions may be distributed by different functional units according to needs, that is, the units or steps in the embodiments of the present invention are further decomposed or combined, for example, the units in the foregoing embodiment may be combined into one unit, or may be further decomposed into multiple sub-units, so as to complete all or the functions of the units described above. The names of the units and steps involved in the embodiments of the present invention are only for distinguishing the units or steps, and are not to be construed as unduly limiting the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative elements, method steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the elements, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or unit/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or unit/apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (6)
1. Big data based energy data analysis system, characterized in that the system comprises: a data acquisition unit comprising: the system comprises various acquisition units deployed on the site, wherein the acquisition units are configured to acquire original energy data; the data storage unit is configured for carrying out data segmentation on the acquired original energy data, then carrying out classified storage and removing data redundancy; a data preprocessing unit: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; the data analysis model establishing unit is configured for establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; the early warning unit is configured for monitoring and early warning a data analysis result according to a preset index;
the data analysis model establishing unit includes: the system comprises a frame establishing unit, a data model establishing unit and a data model establishing unit, wherein the frame establishing unit is configured to determine a deep learning basic frame, import energy training data and establish a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; the model establishing unit is configured to establish a data model of each node by adopting a mathematical equation, and preset relevant parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; the initialization unit is configured to initialize the parameter values, compare the output values of the nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correct the parameter values of the nodes, and sequentially circulate to finally obtain the parameter values in the nodes, which enable the output values of the nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum;
the data storage unit: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
2. The system of claim 1, wherein the model building unit, the data modeling method for each node using a mathematical equation to build the node, performs the steps of: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; make itCalculating and saving all data categories as F by the following stepsj1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; and (3) establishing a data model of the node by using the following formula according to the calculated probability:
wherein y is a defined category parameter, which is any value, but y corresponding to each data category is different from each other.
3. The system of claim 2, wherein p (F) is calculated based onj) Classifying, specifically executing the following steps: setting a threshold value, and calculating all the obtained p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
4. A big data based energy data analysis method, characterized in that the method performs the following steps: step 1: collecting original energy data; step 2: carrying out data segmentation on the acquired original energy data, and then carrying out classified storage to remove data redundancy; and step 3: checking the integrity of data, removing noise and singular values, reducing the number of effective variables to be considered or the number of data samples by using a data compression and transformation method according to the task target, or finding out an invariant identifier of the data, and finally carrying out data standardization; and 4, step 4: establishing a data analysis model, and performing data analysis on the energy data processed by the data preprocessing unit by using the established data analysis model to obtain a data analysis result; and 5: monitoring and early warning the data analysis result according to a preset index;
the step 4: the method for establishing the data analysis model comprises the following steps: determining a deep learning basic framework, importing energy training data, establishing a data model comprising an input layer, at least one hidden layer and an output layer according to data characteristics of the energy training data, wherein the input layer comprises a plurality of nodes with data characteristics, the output layer comprises a plurality of nodes with energy diagnosis data characteristics, and each hidden layer comprises a plurality of nodes with mapping corresponding relations with the output value of the previous layer; establishing a data model of each node by adopting a mathematical equation, presetting related parameter values in the mathematical equation by adopting an artificial or random method, wherein the input value of each node in an input layer is the data characteristic, the input value of each node in each hidden layer and an output layer is the output value of an upper layer, and the output value of each node in each layer is the value obtained after the node is operated by the mathematical equation; initializing the parameter values, comparing the output values of all nodes in the output layer with the energy diagnosis data characteristics of the corresponding nodes, repeatedly correcting the parameter values of all nodes, circulating in sequence, and finally obtaining the parameter values of all nodes which enable the output values of all nodes in the output layer to generate the output values corresponding to the output values when the energy diagnosis data characteristic similarity is locally maximum;
the step 2: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: the method for carrying out data segmentation on the acquired original energy data comprises the following steps: uniformly converting the received energy data into binary data, carrying out fixed-size slicing processing on the binary data, generating a unique stack value for each energy source data block, simultaneously linking the energy source data blocks in a stack structure, and generating a root stack as a stack identifier of the energy data; generating an algorithm for generating an energy source data block stack and a root stack according to the actual content of energy data, wherein different types of energy data can generate different stack values; after the writing of the energy data is completed, prompting that the energy data is successfully written; when new energy data are written, a plurality of energy data dividers are arranged and are synchronously written into tasks, the energy data dividers executing the writing tasks can all slice the energy data according to the same algorithm and then store the sliced energy data, and when a plurality of energy data copies exist in a verification network, the writing energy data task is terminated; each energy data divider creates a distributed stack table, and the distributed stack table contains energy data divider information and all energy data and energy data structure relations stored under the energy data divider; when new energy data are written in, the stack table is updated, and information is synchronized with other energy data dividers; and after the energy data segmentation is completed, establishing an index association family by using the address of the energy data in the stack table based on the stack table corresponding to each sub-energy data.
5. The method of claim 4, wherein the step of using the mathematical equation to model the data for each node is performed by: randomly extracting data from energy training data, inputting the extracted data, and expressing the category set of the data as follows: f ═ F1，F2，F3，...，FnAnd the attribute feature set of the data is expressed as: o ═ M1，M2，M3，...，Mn}; using the following steps, all data classes are calculated and saved as Fj1, 2, 3,.., n; the category F to which the data with the feature O belongs is calculated using the following formulaiThe probability distribution of (c) is:
wherein, p (F)j|Mj) Indicating a certain data category as FjThe probability with the attribute characteristic O, the lambda bit adjustment coefficient, the value range is: 0.3 to 0.9; is obtained by calculationUsing the following formula, the data model of the node is established as follows:
wherein y is a defined category parameter, which is any value, but y corresponding to each data category is different from each other.
6. The method of claim 5, wherein p (F) is calculated based onj) Classifying, specifically executing the following steps: setting a threshold value, and calculating all the obtained p (F)j) And performing difference value operation between every two data, classifying the two data of which the calculated difference value is within a set threshold range into the same category, corresponding to the same y value, and representing by using the same data structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010853260.9A CN112231306B (en) | 2020-08-23 | 2020-08-23 | Big data based energy data analysis system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010853260.9A CN112231306B (en) | 2020-08-23 | 2020-08-23 | Big data based energy data analysis system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112231306A CN112231306A (en) | 2021-01-15 |
| CN112231306B true CN112231306B (en) | 2021-05-28 |
Family
ID=74117029
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010853260.9A Active CN112231306B (en) | 2020-08-23 | 2020-08-23 | Big data based energy data analysis system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112231306B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113777985A (en) * | 2021-09-09 | 2021-12-10 | 国网河北省电力有限公司辛集市供电分公司 | Energy digital management and control platform for enterprise management |
| CN115719151A (en) * | 2022-11-28 | 2023-02-28 | 内蒙古电力(集团)有限责任公司内蒙古电力科学研究院分公司 | Energy management system, method, electronic device and storage medium |
| CN116628728B (en) * | 2023-07-24 | 2023-11-14 | 江苏华存电子科技有限公司 | Data storage analysis method and system based on feature perception |
| CN116866391B (en) * | 2023-09-01 | 2023-11-14 | 睿至科技集团有限公司 | Big data-based energy data acquisition method and system |
| CN117271677A (en) * | 2023-09-28 | 2023-12-22 | 大作(江苏)云科技有限公司 | Data processing method based on cloud computing |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275185A (en) * | 2020-01-16 | 2020-06-12 | 珠海格力电器股份有限公司 | Energy use state early warning method, device, equipment and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8380440B2 (en) * | 2008-06-02 | 2013-02-19 | Westerngeco L.L.C. | 3D residual binning and flatness error correction |
| CN113421652A (en) * | 2015-06-02 | 2021-09-21 | 推想医疗科技股份有限公司 | Method for analyzing medical data, method for training model and analyzer |
| EP3427228A1 (en) * | 2016-05-12 | 2019-01-16 | Hewlett-Packard Development Company, L.P. | Adapting a 3d printing file |
| CN106844636A (en) * | 2017-01-21 | 2017-06-13 | 亚信蓝涛（江苏）数据科技有限公司 | A kind of unstructured data processing method based on deep learning |
| CN108345842B (en) * | 2018-01-24 | 2022-03-04 | 中电长城圣非凡信息系统有限公司 | Big data based processing method |
| CN109375590A (en) * | 2018-09-03 | 2019-02-22 | 安徽伟迈信息技术有限公司 | Highly energy-consuming unit source monitoring system |
| CN110298488A (en) * | 2019-05-31 | 2019-10-01 | 武汉烽火富华电气有限责任公司 | A kind of multi-energy data analysis method and system based on data mining |
| CN110288127A (en) * | 2019-05-31 | 2019-09-27 | 武汉烽火富华电气有限责任公司 | A kind of energy big data processing method |
-
2020
- 2020-08-23 CN CN202010853260.9A patent/CN112231306B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275185A (en) * | 2020-01-16 | 2020-06-12 | 珠海格力电器股份有限公司 | Energy use state early warning method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 深度学习在泛在电力物联网中的应用与挑战;谢小瑜 等;《电力自动化设备》;20200402;第40卷(第4期);第77-78页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112231306A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112231306B (en) | Big data based energy data analysis system and method | |
| Fan et al. | A review on data preprocessing techniques toward efficient and reliable knowledge discovery from building operational data | |
| Gaur | Neural networks in data mining | |
| Jain et al. | Data mining techniques: a survey paper | |
| Bagnall et al. | A run length transformation for discriminating between auto regressive time series | |
| CN111597247A (en) | Data anomaly analysis method and device and storage medium | |
| Cismondi et al. | Computational intelligence methods for processing misaligned, unevenly sampled time series containing missing data | |
| Su-Hui et al. | Hadoop-based college student behavior warning decision system | |
| CN117522607A (en) | Enterprise financial management system | |
| Zhang | Research on adaptive recommendation algorithm for big data mining based on Hadoop platform | |
| Guo et al. | Learning hypersphere for few-shot anomaly detection on attributed networks | |
| Dhoot et al. | Efficient Dimensionality Reduction for Big Data Using Clustering Technique | |
| Aparajita et al. | Comparative analysis of clustering techniques in cloud for effective load balancing | |
| CN114610234B (en) | Storage system parameter recommendation method and related device | |
| Li et al. | University Students' behavior characteristics analysis and prediction method based on combined data mining model | |
| Chen et al. | Online cleaning method of power grid energy anomaly data based on improved random forest | |
| CN112579667B (en) | Data-driven engine multidisciplinary knowledge machine learning method and device | |
| Reena | Preprocessing Big Data using Partitioning Method for Efficient Analysis | |
| Barruetabena et al. | Optimizing Communication Data Streams in Edge Computing Systems Using Bayesian Algorithms | |
| Shivaraju et al. | A Map-Reduce Model of Decision Tree Classifier using Attribute Partitioning | |
| Banga et al. | Proposed Intelligent Software System for Early Fault Detection | |
| El Hassak et al. | Safeguarding Industry 4.0: A Machine Learning Approach for Cyber-Physical Systems Security and Sustainability | |
| Salmerón et al. | Parallel filter-based feature selection based on balanced incomplete block designs | |
| Chen et al. | Employment Prediction Models Based on Weighted Feature Selection and Semi-Supervised Machine Learning | |
| Wang et al. | Command and Control Network Fault Detection Based on XGBoost-RF Algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211213 Address after: Room 2203, 22 / F, building 1, Aosheng building, 1166 Xinluo street, Jinan area, China (Shandong) pilot Free Trade Zone, Jinan City, Shandong Province Patentee after: Shandong Hanlin Technology Co.,Ltd. Patentee after: BEIJING SGITG-ACCENTURE INFORMATION TECHNOLOGY Co.,Ltd. Patentee after: State Grid Siji digital technology (Beijing) Co.,Ltd. Address before: Room 2203, 22 / F, building 1, Aosheng building, 1166 Xinluo street, Jinan area, China (Shandong) pilot Free Trade Zone, Jinan City, Shandong Province Patentee before: Shandong Hanlin Technology Co.,Ltd. |