CN102722756A - Portable power information management system based on distributed data warehouse - Google Patents
Portable power information management system based on distributed data warehouse Download PDFInfo
- Publication number
- CN102722756A CN102722756A CN2012101295114A CN201210129511A CN102722756A CN 102722756 A CN102722756 A CN 102722756A CN 2012101295114 A CN2012101295114 A CN 2012101295114A CN 201210129511 A CN201210129511 A CN 201210129511A CN 102722756 A CN102722756 A CN 102722756A
- Authority
- CN
- China
- Prior art keywords
- data
- server
- management system
- reference data
- distributed data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007792 addition Methods 0.000 claims description 6
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 claims description 3
- 230000005012 migration Effects 0.000 claims description 3
- 238000013508 migration Methods 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000000034 method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a portable management system based on a distributed data warehouse, in particular to a portable power information management system based on the distributed data warehouse. The portable power information management system based on the distributed data warehouse provided by the invention comprises a network server, a configuration server, a reference data server and a dynamic data server. The computation storage capacity and data security of the whole system are improved, the expansion investment of the system is reduced, and an idea and powerful technical support are provided for implementation of a power system smart grid and ultra-large scale computation.
Description
Technical field
The present invention relates to the portable management system in distributed data warehouse, the spy relates to distributed data warehouse portable power information management system.
Background technology
Along with the quickening of the flourish of China's Technology of Electrical Power Generation and electric system internal information process, the independently electric system internal information network system and various electric power management system have been set up and have formed at present.
The core of infotech is information processing and storage.Along with the sharp increase of data volume, data storage technology faces huge challenge.The improving constantly of CPU frequency, when operating system version is constantly upgraded; Should not ignore such fact; I.e. information processing technology advanced person how no matter must be with information stores on certain medium, and information and infotech itself all need rely on certain storage medium and exist.
To electric system processor resource and storage resources require increasingly high; Power System Intelligent distributed data warehouse has been proposed; Intelligent distributed data warehouse can perhaps utilize the hardware of cheap PC level, to be significantly less than the cost that uses private server and memory device under the constant situation of existing processor, storage facilities; Utilize system's Intranet to integrate the electric system existing resources; Improve the calculating storage capacity and the data security of total system, reduce system's expansion investments, for realizing that China Power system intelligence electrical network and ultra-large calculating provide thinking and strong technical support.
Summary of the invention:
To in electric system internal network infosystem; The problem that does not also have special-purpose portable, distributed data warehouse resource management apparatus; The invention provides a kind of based on distributed data warehouse portable power information management system; Improve the calculating storage capacity and the data security of total system, reduce system's expansion investments, for realizing that Power System Intelligent electrical network and ultra-large calculating provide thinking and strong technical support.
Technical scheme of the present invention is: based on the portable distributed data warehouse facilities of power information management system; Comprise the webserver; Configuration server; Reference data server and dynamic data server is characterized in that: be used to provide the webserver of data interaction platform, support configuration server, reference data server and dynamic data server data transmission each other; The reference data server backs up reference data on many different physical equipment; The dynamic data server is preserved additions and deletions and is changed the attitude data in internal memory; Configuration server is used to carry out mechanical disorder detection, load balance calculating, load migration scheduling.Its beneficial effect is: adopt form portable, the distributed data warehouse, in power information system, realize the major function of large server and large scale business database software, replaceable large server and the business database that falls former costliness.
Aforesaid portable distributed data warehouse facilities is characterized in that: said reference data has been preserved 3 parts and be stored on the different reference data servers.Its beneficial effect is: the service disruption that can avoid the hardware and software fault to cause.
Aforesaid portable distributed data warehouse facilities is characterized in that: the dynamic data server record is submitted daily record to and is used two-node cluster hot backup.Its beneficial effect is: the service disruption that can avoid the hardware and software fault to cause.
Aforesaid portable distributed data warehouse facilities is characterized in that: said configuration server and the shared physical machine of dynamic data server.Its beneficial effect is: can reduce number of computers, reduce cost.
Description of drawings:
Fig. 1 apparatus structure synoptic diagram;
Fig. 2 device workflow synoptic diagram.
Embodiment:
Explanation of nouns:
Reference data and dynamic data: system changes with the additions and deletions of the list data of incremental mode in record a period of time; Thereby keeping the form body data relatively stable in a period of time; Wherein the data that change of additions and deletions are called dynamic data (usually at internal memory; Be also referred to as memory table), and metastable body data is called reference data in a period of time, the dynamic data that (is saved in SSD solid state hard disc or disk) after reference data and the dump is with classification character format.
Below in conjunction with accompanying drawing the present invention is done further explanation:
As shown in Figure 1, the portable distributed data warehouse facilities based on the power information management system comprises the webserver, configuration server, and reference data server and dynamic data server, the concrete function of various servers is following:
Be used to provide the webserver of data interaction platform: support configuration server, reference data server and dynamic data server data transmission each other.
Reference data server (ChunkServer) is called the server of preserving reference data again, is about to data backup on many different physical equipment.For fear of the service disruption that the hardware and software fault causes, this device has been preserved 3 parts and be stored on the different reference data servers usually with a reference data.
Dynamic data server (UpdateServer) is called the server of preserving dynamic data again, generally is the separate unit server.For fear of the service disruption that the hardware and software fault causes, the dynamic data server record is submitted daily record to and is used two-node cluster hot backup usually.
Configuration server (RootServer) generally is the separate unit server.For fear of the service disruption that the hardware and software fault causes, the configuration server record is submitted daily record to and is adopted two-node cluster hot backup usually.Because the configuration server load is generally all very light, thus it usually with the shared physical machine of dynamic data server.
The course of work based on distributed portable data warehouse power information management system:
Portable distributed data warehouse facilities system work process based on the power information management system is following: temporarily abandoned many unwanted database information Functions of Management System in the time of system design and realization,
Temporary table for example; View (view); Sql like language support etc.; System retouching operation such as changes to the additions and deletions in a period of time and notes (be referred to as dynamic data, be generally held in the internal memory) with incremental form, has so also made body data in a period of time, keep relatively stable (being referred to as reference data).
Because dynamic data is less relatively, generally, is kept at it in the internal memory of dynamic data server.In internal memory, preserve additions and deletions and change relative recording, greatly improved the access performance of system data.In addition, if every revise mean consumption 100 Bytes, the 10GB internal memory can write down 100M (promptly 100,000,000) bar is revised, and expansion dynamic data server memory space, has also just increased the index word that can hold in the internal memory so.Moreover and since freeze after memory table no longer revise, it also can convert to the classification character format and be saved in solid state hard disc or disk on.Shared internal memory can discharge after being dumped to solid state hard disc, and the service of reading of superior performance still can be provided, and this has also alleviated the memory requirements of dynamic data server under the extreme case.In order to tackle mechanical disorder, the dynamic data server write is submitted daily record to and is taked two-shipper (even multimachine) heat to be equipped with.Because the active and standby machine of dynamic data server is synchronous, so standby host also can provide the service of reading simultaneously.
Because reference data is relatively stable; System it according to major key (primary key; Be also referred to as row key) preserve a plurality of copies (generally being 3) after the segmentation to the reference data server, the service disruption of having avoided the single machine fault to cause, a plurality of copies have also promoted the system service ability.The size of individual data table can be configured according to the application data characteristics, and disposing too small tables of data relatively can merge, and excessive tables of data then can divide.Because tables of data deposits by the major key piecemeal continuously, thus system by the range query of major key corresponding continuous disk read, very efficient.
For freezing/dynamic data of dump, the reference data server of system can start reference data and freeze when oneself not being too busy/merging of dump memory table, and generate new reference data.This merging process is a kind of range query in fact, is that a string continuous disk is read to write with continuous disk, also is very efficiently.
The traditional data base management system provides powerful transactional, good consistance and very short query modification response time, but data scale receives serious restriction, lacks extendability; Modern cloud computing provides great data scale, good extendability, strides that table affairs, data consistency are also weak, the query modification response time is also longer usually but lack inter-bank, and the advantage of the two has been merged in the design of system and realization:
1, dynamic data server: be similar to the database role in the data base management system (DBMS), response time that inter-bank strides table affairs and very short query modification and good consistance are provided
2, reference data server: be similar to the working machine in the cloud computing; Have characteristics such as the many copies of data (normally 3), medium-scale data granularity (the about 256MB of tables of data size), ALB, the machine of delaying recovery, machine plug and play, power system capacity and performance can be expanded at any time
3, merging/submission data server (MergeServer): combine reference data server and dynamic data server, obtain latest data, realize data consistency
4, configuration server: be similar to the main control computer in the cloud computing, carry out mechanical disorder detection, load balance calculating, load migration scheduling etc.
The inter-bank that the mutual supplement with each other's advantages of above-mentioned data base management system (DBMS) and cloud computing technology makes system both have the traditional data base management system is striden table affairs, the strong consistency of data and the very short query modification response time, also has Mass Data Management ability, automatic fault recovery, ALB and the good extendability of cloud computing.
Claims (4)
1. based on the portable distributed data warehouse facilities of power information management system; Comprise the webserver; Configuration server; Reference data server and dynamic data server is characterized in that: be used to provide the webserver of data interaction platform, support configuration server, reference data server and dynamic data server data transmission each other; The reference data server backs up reference data on many different physical equipment; The dynamic data server is preserved additions and deletions and is changed the attitude data in internal memory; Configuration server is used to carry out mechanical disorder detection, load balance calculating, load migration scheduling.
2. portable distributed data warehouse facilities as claimed in claim 1 is characterized in that: said reference data has been preserved 3 parts and be stored on the different reference data servers.
3. according to claim 1 or claim 2 portable distributed data warehouse facilities, it is characterized in that: the dynamic data server record submits to daily record also to use two-node cluster hot backup.
4. according to claim 1 or claim 2 portable distributed data warehouse facilities is characterized in that: said configuration server and the shared physical machine of dynamic data server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101295114A CN102722756A (en) | 2012-04-28 | 2012-04-28 | Portable power information management system based on distributed data warehouse |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101295114A CN102722756A (en) | 2012-04-28 | 2012-04-28 | Portable power information management system based on distributed data warehouse |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102722756A true CN102722756A (en) | 2012-10-10 |
Family
ID=46948505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012101295114A Pending CN102722756A (en) | 2012-04-28 | 2012-04-28 | Portable power information management system based on distributed data warehouse |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102722756A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103077189A (en) * | 2012-12-20 | 2013-05-01 | 中国电力科学研究院 | Large-grid-oriented parallel real-time database node positioning system |
| CN104408097A (en) * | 2014-11-17 | 2015-03-11 | 深圳市比一比网络科技有限公司 | Hybrid indexing method and system based on character field hot update |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937435A (en) * | 2009-07-01 | 2011-01-05 | 北京科东电力控制系统有限责任公司 | Online synchronized method of distributed real-time database confronting power system |
| CN102063439A (en) * | 2009-11-18 | 2011-05-18 | 南京南瑞继保电气有限公司 | Online memory database loading method |
-
2012
- 2012-04-28 CN CN2012101295114A patent/CN102722756A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937435A (en) * | 2009-07-01 | 2011-01-05 | 北京科东电力控制系统有限责任公司 | Online synchronized method of distributed real-time database confronting power system |
| CN102063439A (en) * | 2009-11-18 | 2011-05-18 | 南京南瑞继保电气有限公司 | Online memory database loading method |
Non-Patent Citations (2)
| Title |
|---|
| 全中伟: "分布式内存数据库存储研究", 《中国优秀硕士论文全文数据库》 * |
| 武君胜 等: "面向电力系统的分布式实时数据库关键技术及应用", 《中国科技论文在线》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103077189A (en) * | 2012-12-20 | 2013-05-01 | 中国电力科学研究院 | Large-grid-oriented parallel real-time database node positioning system |
| CN103077189B (en) * | 2012-12-20 | 2016-01-20 | 中国电力科学研究院 | A kind of parallel real-time database node positioning system towards bulk power grid |
| CN104408097A (en) * | 2014-11-17 | 2015-03-11 | 深圳市比一比网络科技有限公司 | Hybrid indexing method and system based on character field hot update |
| CN104408097B (en) * | 2014-11-17 | 2018-07-20 | 深圳市比一比网络科技有限公司 | One kind is based on the newer hybrid index method and system of character field heat |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102222085B (en) | Data de-duplication method based on combination of similarity and locality | |
| CN107169083B (en) | Mass vehicle data storage and retrieval method and device for public security card port and electronic equipment | |
| CN102521269B (en) | Index-based computer continuous data protection method | |
| CN107291889A (en) | A kind of date storage method and system | |
| CN101582076A (en) | Data de-duplication method based on data base | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| Yin et al. | Scalable real time data management for smart grid | |
| CN104572505A (en) | System and method for ensuring eventual consistency of mass data caches | |
| CN110196818A (en) | Data cached method, buffer memory device and storage system | |
| JP2018169941A (en) | Information processing device, method, and program | |
| CN103593420A (en) | Method for constructing heterogeneous database clusters on same platform by sharing online logs | |
| CN105516313A (en) | Distributed storage system used for big data | |
| CN102820998B (en) | Realize the dual computer fault-tolerant service system towards office application and date storage method thereof | |
| CN103198119A (en) | Method for fast searching all chained files having same repeating data deleting identification | |
| CN103365601A (en) | Cloud storage system based on Loongson 3A processor | |
| CN104035522A (en) | Large database appliance | |
| Mukhopadhyay et al. | Addressing name node scalability issue in Hadoop distributed file system using cache approach | |
| CN102722756A (en) | Portable power information management system based on distributed data warehouse | |
| Takatsu et al. | PPFS: A scale-out distributed file system for post-petascale systems | |
| CN102385624A (en) | DFS (distributed file system)-oriented log data organization method | |
| CN205263797U (en) | Adopt memory of solid state hard drives SSD as L2 cache | |
| CN202720676U (en) | Portable electric power information management system based on distributed data warehouse | |
| CN209784990U (en) | High-performance server mainboard | |
| CN204102026U (en) | Large database concept all-in-one | |
| CN103714022A (en) | Mixed storage system based on data block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: STATE ELECTRIC NET CROP. Free format text: FORMER OWNER: SUBSTATION MAINTENANCE CENTRAL WUHAN POWER SUPPLY CO., LTD. Effective date: 20130419 Owner name: HUBEI ELECTRIC POWER COMPANY WUHAN POWER SUPPLY CO Effective date: 20130419 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 430051 WUHAN, HUBEI PROVINCE TO: 100031 XICHENG, BEIJING |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20130419 Address after: 100031 Xicheng District West Chang'an Avenue, No. 86, Beijing Applicant after: State Grid Corporation of China Applicant after: Wuhan Power Supply Company of Hubei Electric Power Company Address before: 430051 No. 484, Hanyang Avenue, Wuhan, Hubei Applicant before: Substation Maintenance Central, Wuhan Power Supply Co.,Ltd. |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20121010 |