CN109871375A - The information platform and its control method of distributed new scale access - Google Patents
The information platform and its control method of distributed new scale access Download PDFInfo
- Publication number
- CN109871375A CN109871375A CN201811063755.0A CN201811063755A CN109871375A CN 109871375 A CN109871375 A CN 109871375A CN 201811063755 A CN201811063755 A CN 201811063755A CN 109871375 A CN109871375 A CN 109871375A
- Authority
- CN
- China
- Prior art keywords
- data
- distributed
- source
- distributed new
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of information platforms of distributed new scale access, solve the deficiencies in the prior art, technical solution are as follows: including distributed new, distributed data base, middleware and outer net, the data source of the distributed new is connect with the distributed data base, distributed data base is stored with the fused big data by multi-source heterogeneous big data, data interaction request between the middleware and outer net and distributed data base is carried out by XML form, data result transmitting between the middleware and outer net and distributed data base is carried out also by XML form.
Description
Technical field
The present invention relates to a kind of efficiency data platforms, and in particular to a kind of information of distributed new scale access
Platform and its control method.
Background technique
China needs to develop a kind of efficiency control platform and its control method, realizes acquisition to city electricity consumption data, prison
It surveys, analysis, excavate, so that government is understood electric energy supply, dispatching and electricity consumption situation in time, grasp power consumption situation and electricity consumption
Trend etc., being able to carry out reasonable manage to entire urban energy reduces with allotment, realization rational utilization of electricity and uses energy cost, simultaneously
It is formulated for energy policy, energy-saving and emission-reduction INDEX MANAGEMENT and macroeconomic operation analysis provide decision support；By extending to user
The wide covering of inside, fine-grained data acquisition network grasp user's energy situation in real time, realize and consume energy between power grid and user
The online interaction of data makes Utilities Electric Co. regulate and control urban area load, and Optimizing City load curve realizes peak load shifting, reduces
Plant maintenance frequency ensures stabilization of power grids economical operation, promotes urban economy development, realizes energy-saving, response national energy
Strategic requirement.But current equipment is due to problem left over by history, there are the data collector of many different form structures, because
The data structure that this is obtained is multifarious, and data type diversity factor is very big, therefore, makes to a unified efficiency platform is established
At great puzzlement.
Summary of the invention
Lack a kind of information of distributed new scale access it is an object of the invention to solve the above-mentioned prior art
Platform and its control method provide the information platform and its control method of a kind of distributed new scale access.
The technical solution adopted by the present invention to solve the technical problems is: a kind of distributed new scale access
Information platform, including distributed new, distributed data base, middleware and outer net, the data source of the distributed new
It being connect with the distributed data base, distributed data base is stored with the fused big data by multi-source heterogeneous big data,
Data interaction request between the middleware and outer net and distributed data base is carried out by XML form, the centre
Data result transmitting between part and outer net and distributed data base is carried out also by XML form.
It is solved in the present invention using the method for the fusion of multi-source heterogeneous big data, there are many different form structures
Data collector, therefore the data structure obtained is multifarious, the very big problem of data type diversity factor, middleware is intermediate
Layer, intermediate equipment transmit data by XML form and play the role of isolation, while XML form is also on hardware configuration
A kind of transparent transmission form, has met open and clear requirement, and will not influence data safety, can meet distributed new energy
The requirement of the information platform of source scale access.
A kind of information platform control method of distributed new scale access, is suitable for as described above distributed
The information platform of new energy scale access, comprising the following steps:
Step 1 carries out big data pretreatment to the data source of the distributed new；
Step 2 carries out multi-source heterogeneous big number to based on the pretreated data of big data according to distributed database architecture
According to fusion；
Step 3, all types of users safe access control distributed new scale after authentication access
Information platform.
Preferably, in the step 1, including following sub-step:
Sub-step one is pre-processed, attributive analysis is carried out to the data source of distributed new, property index is established and divides
Class；
Sub-step two is pre-processed, attributive analysis is carried out according to the data source of target distribution formula new energy as a result, choosing data
Assessment step by step, data recombination step by step, data cleansing step by step, data pick-up step by step, data filtering step by step and data
Reduction step by step at least two conversions for carrying out data matrix to the data source of target distribution formula new energy step by step.We
Method corresponds to step 1, realizes the preliminary treatment of data.
Preferably, pretreatment sub-step two in, be first carried out data assessment step by step, data recombination step by step；So
Attributive analysis is carried out according to the data source of target distribution formula new energy afterwards as a result, data cleansing is executed step by step, further according to data
The size of collection, which determines, carries out data pick-up step by step, then concentrates to data and do not meet the data progress data filtering for excavating format
Step by step, if the redundant attributes in data set are more, the format that data regularization saves as data matrix afterwards step by step is carried out,
Otherwise the format of data matrix is directly saved as.
Preferably, being executed after carrying out big data pretreatment to the data source of the distributed new in step 1
Data quality control sub-step, the data quality control sub-step include:
Data quality control sub-step one establishes the structuring low-rank representation model of multi-source data, characterizes multi-modal data
Between structural relation,
Data quality control sub-step two is constrained by the low-rank and structural sparse of matrix and is carried out to the quality of data
Detection, recovers relational matrix from sparse error.
Preferably, in step 2, including following sub-step:
The distributed new related data of N number of mode is denoted as by the fusion steps one of multi-source heterogeneous big data
{X1,X2,...,XN, the data set of each mode includes p observation sampleUsing Multiple Kernel Learning algorithm
One kernel function K is separately designed to the data of N number of modem(xi,xj), each kernel function implicitly determines a Nonlinear Mapping letter
Number φm(xi)；
The fusion steps two of multi-source heterogeneous big data, pass through nonlinear mapping function φm(xi) obtain the sight of each mode
Survey data Xm, by the observational data X of each modemCorresponding kernel function K is inputted respectivelym(xi,xj) mapped, generate M together
The K of dimensionm∈Rp×pNuclear matrix, thus the multicore member space of one same sex of insertion；
The fusion steps three of multi-source heterogeneous big data are more using insertion projection algorithm progress to the data of polynary nuclear space
Source fusion.
Preferably, the data to polynary nuclear space are carried out in multi-source fusion using insertion projection algorithm using based on super
The distributed new related data blending algorithm of map insertion, the distributed new related data fusion of hypergraph spectrum insertion
The super side in hypergraph structure in algorithm is one group of data subset at least one identical characteristic.
Substantial effect of the invention is: the method solution of the fusion of multi-source heterogeneous big data is used in the present invention, is deposited
In the data collector of many different form structures, therefore the data structure obtained is multifarious, and data type diversity factor is very big
The problem of, middleware, that is, middle layer, intermediate equipment transmit data by XML form and play the work of isolation on hardware configuration
With, while XML form is also a kind of transparent transmission form, has met open and clear requirement, and will not influence data safety,
The requirement of the information platform of distributed new scale access can be met.
Detailed description of the invention:
A kind of Fig. 1: structural schematic diagram of the information platform of distributed new scale access in the present invention；
Fig. 2: the flow chart of the information platform control method of distributed new scale access in the present invention.
Specific embodiment
Below by specific embodiment, in conjunction with attached drawing, technical scheme of the present invention will be further explained in detail.
Embodiment 1:
A kind of information platform (referring to attached drawing 1) of distributed new scale access, including distributed new, point
Cloth database, middleware and outer net, the data source of the distributed new are connect with the distributed data base, distribution
Formula database purchase has the fused big data by multi-source heterogeneous big data, the middleware and outer net and distribution
Data interaction request between database is carried out by XML form, the middleware and outer net and distributed data base it
Between data result transmitting carried out also by XML form.
A kind of information platform control method (referring to attached drawing 2) of distributed new scale access, is suitable for institute as above
The information platform for the distributed new scale access stated, comprising the following steps:
Step 1 carries out big data pretreatment to the data source of the distributed new；
Step 2 carries out multi-source heterogeneous big number to based on the pretreated data of big data according to distributed database architecture
According to fusion；
Step 3, all types of users safe access control distributed new scale after authentication access
Information platform.
In the step 1, including following sub-step:
Sub-step one is pre-processed, attributive analysis is carried out to the data source of distributed new, property index is established and divides
Class；
Sub-step two is pre-processed, attributive analysis is carried out according to the data source of target distribution formula new energy as a result, choosing data
Assessment step by step, data recombination step by step, data cleansing step by step, data pick-up step by step, data filtering step by step and data
Reduction step by step at least two conversions for carrying out data matrix to the data source of target distribution formula new energy step by step.
Pretreatment sub-step two in, be first carried out data assessment step by step, data recombination step by step；Then according to mesh
The data source for marking distributed new carries out attributive analysis as a result, data cleansing is executed step by step, further according to the size of data set
It determines and carries out data pick-up step by step, then data are concentrated and do not meet the data progress data filtering for excavating format step by step, if
Redundant attributes in data set are more, then carry out the format that data regularization saves as data matrix afterwards step by step, otherwise directly protect
Save as the format of data matrix.In step 1, held after carrying out big data pretreatment to the data source of the distributed new
Row data quality control sub-step, the data quality control sub-step include:
Data quality control sub-step one establishes the structuring low-rank representation model of multi-source data, characterizes multi-modal data
Between structural relation,
Data quality control sub-step two is constrained by the low-rank and structural sparse of matrix and is carried out to the quality of data
Detection, recovers relational matrix from sparse error.
In step 2, including following sub-step:
The distributed new related data of N number of mode is denoted as by the fusion steps one of multi-source heterogeneous big data
{X1,X2,...,XN, the data set of each mode includes p observation sampleUsing Multiple Kernel Learning algorithm
One kernel function K is separately designed to the data of N number of modem(xi,xj), each kernel function implicitly determines a Nonlinear Mapping letter
Number φm(xi)；
The fusion steps two of multi-source heterogeneous big data, pass through nonlinear mapping function φm(xi) obtain the sight of each mode
Survey data Xm, by the observational data X of each modemCorresponding kernel function K is inputted respectivelym(xi,xj) mapped, generate M together
The K of dimensionm∈Rp×pNuclear matrix, thus the multicore member space of one same sex of insertion；
The fusion steps three of multi-source heterogeneous big data are more using insertion projection algorithm progress to the data of polynary nuclear space
Source fusion.
The data of polynary nuclear space are carried out in multi-source fusion using insertion projection algorithm using based on hypergraph spectrum insertion
Distributed new related data blending algorithm, hypergraph are composed super in the distributed new related data blending algorithm of insertion
Super side in graph structure is one group of data subset at least one identical characteristic.
It is solved in the present embodiment using the method for the fusion of multi-source heterogeneous big data, there are many different form structures
Data collector, therefore obtain data structure it is multifarious, the very big problem of data type diversity factor, middleware be in
Interbed, intermediate equipment transmit data by XML form and play the role of isolation, while XML form on hardware configuration
It is a kind of transparent transmission form, has met open and clear requirement, and will not influence data safety, can meets distributed new
The requirement of the information platform of energy scale access.
In the present embodiment, there is each attribute for the relevant multi-source heterogeneous data of distributed new, are being pre-processed
With the analysis for needing to carry out data attribute before data fusion, suitable property index and classification are established, is the pretreatment of data
It lays the foundation.Data prediction can improve the quality of data, eliminate imperfection, redundancy and ambiguity of data etc., from
And help to improve the precision and performance of mining process thereafter.It is taken out by data assessment, data recombination, data cleansing, data
It takes, the technologies such as data filtering, data regularization realize the pretreatments of the relevant multi-source heterogeneous data of distributed new.Data control
The accuracy of fusion and correlativity analysis of the relationship between quality of system to big data.This project is intended in matrix Renew theory frame
The low-rank representation model of lower building structuring is established structural sparse constraint and is constrained with low-rank, efficiently separates error information, repair
Data incomplete is mended, realizes data integrity, consistency and accuracy.
In the present embodiment, the relevant multi-source heterogeneous data of distributed new representation, in terms of deposit
In many difference, Combined Treatment can not be directly carried out.Therefore the present embodiment is first against the relevant multi-source of distributed new
The characteristics of various data of isomery, analyzes its otherness, and data are classified, and then studies and how to map to each data together
One same sex member space finally carries out the fusion of data based on same sex member space.The present embodiment design and each modal data itself
The kernel function that feature matches is mapped to same " multicore member space ", is then composed embedding grammar by hypergraph and is realized and divides
The fusion of the relevant multi-source heterogeneous data of cloth new energy.
Above-mentioned embodiment is only a preferred solution of the present invention, not the present invention is made in any form
Limitation, on the premise of not exceeding the technical scheme recorded in the claims there are also other variations and modifications.
Claims (7)
1. a kind of information platform of distributed new scale access, which is characterized in that including distributed new, distribution
Database, middleware and outer net, the data source of the distributed new are connect with the distributed data base, distributed data
Inventory contains the fused big data by multi-source heterogeneous big data, the middleware and outer net and distributed data base it
Between data interaction request by XML form progress, data between the middleware and outer net and distributed data base
As a result it transmits and is carried out also by XML form.
2. a kind of information platform control method of distributed new scale access, is suitable for as described in claim 1 point
The information platform of cloth new energy scale access, which comprises the following steps:
Step 1 carries out big data pretreatment to the data source of the distributed new；
Step 2 carries out multi-source heterogeneous big data to based on the pretreated data of big data according to distributed database architecture
Fusion；
Step 3, all types of users information that the scale of safe access control distributed new accesses after authentication are flat
Platform.
3. the information platform control method of distributed new scale access according to claim 2, it is characterised in that:
In the step 1, including following sub-step:
Sub-step one is pre-processed, attributive analysis is carried out to the data source of distributed new, establishes property index and classification；
Sub-step two is pre-processed, attributive analysis is carried out according to the data source of target distribution formula new energy as a result, choosing data assessment
Step by step, data recombination step by step, data cleansing step by step, data pick-up step by step, data filtering step by step and data regularization
At least two carry out the conversion of data matrix to the data source of target distribution formula new energy step by step in step by step.
4. the information platform control method of distributed new scale access according to claim 3, it is characterised in that:
Pretreatment sub-step two in, be first carried out data assessment step by step, data recombination step by step；Then new according to target distribution formula
The data source of the energy carries out attributive analysis as a result, executing data cleansing step by step, and the size determination further according to data set is counted
Step by step according to extraction, then to data it concentrates and does not meet the data progress data filtering for excavating format step by step, if in data set
Redundant attributes are more, then carry out the format that data regularization saves as data matrix afterwards step by step, otherwise directly save as data square
The format of battle array.
5. the information platform control method of distributed new scale access according to claim 2, it is characterised in that:
In step 1, data quality control sub-step is executed after carrying out big data pretreatment to the data source of the distributed new
Suddenly, the data quality control sub-step includes:
Data quality control sub-step one, establishes the structuring low-rank representation model of multi-source data, characterizes between multi-modal data
Structural relation,
Data quality control sub-step two detects the quality of data by the low-rank and structural sparse constraint of matrix,
Relational matrix is recovered from sparse error.
6. the information platform control method of distributed new scale access according to claim 2, it is characterised in that:
In step 2, including following sub-step:
The distributed new related data of N number of mode is denoted as { X by the fusion steps one of multi-source heterogeneous big data1,
X2,...,XN, the data set of each mode includes p observation sampleUsing Multiple Kernel Learning algorithm to N
The data of a mode separately design a kernel function Km(xi,xj), each kernel function implicitly determines a nonlinear mapping function
φm(xi)；
The fusion steps two of multi-source heterogeneous big data, pass through nonlinear mapping function φm(xi) obtain the observational data of each mode
Xm, by the observational data X of each modemCorresponding kernel function K is inputted respectivelym(xi,xj) mapped, generate the M K with dimensionm
∈Rp×pNuclear matrix, thus the multicore member space of one same sex of insertion；
The fusion steps three of multi-source heterogeneous big data carry out multi-source using insertion projection algorithm to the data of polynary nuclear space and melt
It closes.
7. the information platform control method of distributed new scale access according to claim 6, it is characterised in that:
The data of polynary nuclear space are carried out using the distribution based on hypergraph spectrum insertion new in multi-source fusion using insertion projection algorithm
Energy-relevant data blending algorithm, hypergraph are composed in the hypergraph structure in the distributed new related data blending algorithm of insertion
Super side is one group of data subset at least one identical characteristic.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811063755.0A CN109871375A (en) | 2018-09-12 | 2018-09-12 | The information platform and its control method of distributed new scale access |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811063755.0A CN109871375A (en) | 2018-09-12 | 2018-09-12 | The information platform and its control method of distributed new scale access |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109871375A true CN109871375A (en) | 2019-06-11 |
Family
ID=66915456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811063755.0A Pending CN109871375A (en) | 2018-09-12 | 2018-09-12 | The information platform and its control method of distributed new scale access |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109871375A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112765653A (en) * | 2021-01-06 | 2021-05-07 | 山财高新科技(山西)有限公司 | Multi-source data fusion privacy protection method based on multi-privacy policy combination optimization |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902286A (en) * | 2014-03-12 | 2014-07-02 | 郑州轻工业学院 | Hierarchy type multi-source data fusion method based on SOA |
| CN104008135A (en) * | 2014-05-07 | 2014-08-27 | 南京邮电大学 | Multi-source heterogeneous database fusion system and data query method thereof |
| CN106777141A (en) * | 2016-12-19 | 2017-05-31 | 国网山东省电力公司电力科学研究院 | A kind of acquisition for merging multi-source heterogeneous electric network data and distributed storage method |
| CN107133885A (en) * | 2017-05-16 | 2017-09-05 | 国网江苏省电力公司电力科学研究院 | Distributed new service platform and its construction method based on mobile Internet |
| CN108446364A (en) * | 2018-03-14 | 2018-08-24 | 湖南商学院 | A kind of visual analysis method towards campus big data |
-
2018
- 2018-09-12 CN CN201811063755.0A patent/CN109871375A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902286A (en) * | 2014-03-12 | 2014-07-02 | 郑州轻工业学院 | Hierarchy type multi-source data fusion method based on SOA |
| CN104008135A (en) * | 2014-05-07 | 2014-08-27 | 南京邮电大学 | Multi-source heterogeneous database fusion system and data query method thereof |
| CN106777141A (en) * | 2016-12-19 | 2017-05-31 | 国网山东省电力公司电力科学研究院 | A kind of acquisition for merging multi-source heterogeneous electric network data and distributed storage method |
| CN107133885A (en) * | 2017-05-16 | 2017-09-05 | 国网江苏省电力公司电力科学研究院 | Distributed new service platform and its construction method based on mobile Internet |
| CN108446364A (en) * | 2018-03-14 | 2018-08-24 | 湖南商学院 | A kind of visual analysis method towards campus big data |
Non-Patent Citations (2)
| Title |
|---|
| 刘兵: "《基于流形正则化的核方法研究与应用》", 30 June 2016, 徐州：中国矿业大学出版社 * |
| 周杨珺: "基于多源异构数据的配电网运行分析平台:系统架构与技术实现", 《南方电网技术》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112765653A (en) * | 2021-01-06 | 2021-05-07 | 山财高新科技(山西)有限公司 | Multi-source data fusion privacy protection method based on multi-privacy policy combination optimization |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106505593B (en) | A kind of analysis of distribution transforming three-phase imbalance and the method for load adjustment based on big data | |
| CN107330056A (en) | Wind power plant SCADA system and its operation method based on big data cloud computing platform | |
| CN102684223B (en) | Optimized evaluating method for wind power output under multi-constraint condition for reducing transmission loss | |
| CN103093007B (en) | Power transmission iron tower three-dimensional virtual assembly method | |
| CN112150025A (en) | Economic evaluation method and system for comprehensive energy service project | |
| CN114861539A (en) | Modeling and implementation method for 5G urban comprehensive energy Internet digital twin system | |
| Jing et al. | An optimized method of HDFS for massive small files storage | |
| Liu et al. | Study on coupling optimization model of node enterprises for energy storage-involved photovoltaic value chain in China | |
| CN111260206A (en) | Photovoltaic power generation influence factor evaluation model, construction method and application | |
| Liu et al. | Multi‐scenario comprehensive benefit evaluation model of a multi‐energy micro‐grid based on the matter‐element extension model | |
| CN109871375A (en) | The information platform and its control method of distributed new scale access | |
| CN105305502B (en) | Distribution network distributed electrical operated control method and system based on coordinative coherence | |
| CN111881131A (en) | Three-dimensional GIS platform applied to power grid planning | |
| CN104933491A (en) | Industrial enterprise energy consumption diagnosis early warning system | |
| CN104933537A (en) | Industrial enterprise energy utilization information management system | |
| Feng et al. | Internet of things system of spatial structure sports events health monitoring based on cloud computing | |
| CN106470242A (en) | A kind of large scale scale heterogeneous clustered node fast quantification stage division of cloud data center | |
| Wu et al. | The design of distributed power big data analysis framework and its application in residential electricity analysis | |
| Xincong et al. | Location evaluation of multi-station integration based on ahp-fuzzy comprehensive evaluation method | |
| Tang et al. | Application technology of big data in smart grid and its development prospect | |
| Yongqi et al. | Comprehensive benefit analysis for integrated energy systems projects based on ahp-fuzzy | |
| Hu et al. | Research on Key technologies of Internet of Things Data Terminal and Integrated Management System for Power mobile devices | |
| Shen et al. | Research on the design of power enterprise central data platform framework based on big data | |
| CN107317362A (en) | A kind of distributed power source monitoring system | |
| Tian | Research on big data platform of power grid enterprise accounting business based on cloud computing analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190611 |
|
| RJ01 | Rejection of invention patent application after publication |