CN103955400A - Online checking method of parallel computing in electrical power system - Google Patents
Online checking method of parallel computing in electrical power system Download PDFInfo
- Publication number
- CN103955400A CN103955400A CN201410152048.4A CN201410152048A CN103955400A CN 103955400 A CN103955400 A CN 103955400A CN 201410152048 A CN201410152048 A CN 201410152048A CN 103955400 A CN103955400 A CN 103955400A
- Authority
- CN
- China
- Prior art keywords
- check
- online
- task
- hadoop
- electric system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The invention discloses an online checking method of parallel computing in an electrical power system. The method comprises the following steps of (1) building a Hadoop environment; (2) initializing Hadoop-based online check tasks; (3) deposing the online check tasks; (4) executing the online check tasks in parallel; (5) combining and computing each online check task. Due to the fact that Hadoop, i.e. a software frame which can process a large amount of data in a distributing way, is applied to online check of parallel computing in the electrical power system, the problem that too much time is used for large data numerical computing is solved, and the speed of online check computing is greatly increased.
Description
Technical field
The invention belongs to the technical field of Automation of Electric Systems, relate to particularly the online check method of parallel computation in a kind of electric system.
Background technology
Along with the development of power industry, the scale expanding day of electric system, the structure of electrical network is day by day complicated, and calculated amount and computation complexity sharply increase.Traditional computing method can not meet the requirement of electric system on arithmetic speed and space.And in traditional parallel computation, HPC(High Performance Computing, i.e. high-performance calculation) cluster passes to computing node from back end by data bus or network by data, easily causes again I/O bottleneck.
This name of Hadoop(Hadoop is not an abbreviation, and it is an imaginary name.The founder of this project, Doug Cutting explains gaining the name of Hadoop: " this name is that my child names to the elephant toy of a brown color.My naming standard is exactly brief, easily pronounces and spells, and there is no too many meaning, and can not be used to other places.Child is the master-hand of this respect exactly.") be a software frame that can carry out to mass data distributed treatment.Concurrent application exploitation on Hadoop is based on MapReduce(mapping stipulations) programming framework.Exactly large data sets is decomposed into hundreds and thousands of small data set in brief, intermediate result is processed and generated to each (or several) data set by a node in cluster (being exactly generally the computing machine that a Daepori is logical) respectively, then these intermediate results are merged by a large amount of nodes again, form net result.
We are applied to this software frame that can carry out distributed treatment to mass data of Hadoop in the online check of parallel computation in electric system, will improve widely check speed.
Summary of the invention
Technology of the present invention is dealt with problems and is: overcome the deficiencies in the prior art, the online check method of parallel computation in a kind of electric system is provided, it has solved big data quantity numerical operation too much problem consuming time, has greatly improved the speed that online check is calculated.
Technical solution of the present invention is: the online check method of parallel computation in this electric system, and the method comprises the following steps:
(1) build Hadoop environment;
(2) initialization of the online checking task based on Hadoop;
(3) decomposition of online checking task;
(4) executed in parallel of online checking task;
(5) every online checking task is combined to calculating.
Because this method is applied to this software frame that can carry out distributed treatment to mass data of Hadoop in the online check of parallel computation in electric system, so solved big data quantity numerical operation too much problem consuming time, greatly improved the speed that online check is calculated.
Brief description of the drawings
Fig. 1 is the fundamental diagram of MapReduce;
Fig. 2 is according to the process flow diagram of the online check method of parallel computation in electric system of the present invention.
Embodiment
As shown in Figure 1, done as given a definition:
JobTracker: check general assignment dispatching center
TaskTracker1: check subtask 1 executor
TaskTracker2: check subtask 2 executors
:
:
:
TaskTrackerN: check subtask N executor
Check while beginning, check unique check mark of definition for each, and be different subtasks according to certain rule by checking task division.These subtasks are relatively independent in the process of implementation, and have identical check mark.In Hadoop the JobTracker of master control by these subtasks be distributed to idle TaskTracker(be divided into check subtask executor TaskTracker1, check subtask executor TaskTracker2 ... check subtask executor TaskTrackerN), allow these tasks in parallel move.After parallel computation completes, the result that has identical check mark is combined, as final result in result.The process of Data Integration and MapReduce programming are similar.
The principle of MapReduce programming model is: the data entirety receiving is divided into multiple isometric data slots by Hadoop, each data slot passes to a Map function, Map function can produce the key/value pair set of a centre, MapReduce gathers together all value with identical key value, pass to Reduce, these values are merged into a Value set that little Key value is identical by Reduce function, by its output.
As shown in Figure 2, the online check method of parallel computation in this electric system, the method comprises the following steps:
(1) build Hadoop environment;
(2) initialization of the online checking task based on Hadoop;
(3) decomposition of online checking task;
(4) executed in parallel of online checking task;
(5) every online checking task is combined to calculating.
Because this method is applied to this software frame that can carry out distributed treatment to mass data of Hadoop in the online check of parallel computation in electric system, so solved big data quantity numerical operation too much problem consuming time, greatly improved the speed that online check is calculated.
Preferably, in step (1) by rivest, shamir, adelman realize Hadoop without cipher key communication.When guarantee information is perfectly safe, also having realized that many intercomputers need not log in is addressable target.
Preferably, in step (2), be that this is checked and produces a unique check mark, region and plant stand quantity in statistics electric system, represent with M, and the quantity of cluster Computer represents with N, and so every maximum executable number of tasks of computing machine are: M/N; If the plant stand quantity that certain region is contained is less than M/N, so corresponding computing machine is only carried out the task of all plant stands in this region, and after all tasks carryings in this region, this computing machine, in idle condition, can be carried out the task in other regions.These subtasks have the identical and independent operating each other of the mark of check, the feature being independent of each other.
Preferably, in step (4), this is checked to the JobTracker of dispatching center's master control in Hadoop of general assignment, this JobTracker operates on any default computing machine in cluster, JobTracker high-ranking officers' nucleon task is distributed to idle TaskTracker, TaskTracker is divided into check subtask executor TaskTracker1, checks subtask executor TaskTracker2 ... check subtask executor TaskTrackerN), these tasks in parallel operations, and the ruuning situation of responsible monitor task; If the executor TaskTracker of some checks subtask is out of order, JobTracker can hand to its responsible task another idle TaskTracker and rerun.Can reduce like this transmission of data on network, reduce the demand to the network bandwidth, avoid I/O obstruction.
Preferably, in step (5), after the subtask under same check mark is all finished, this time parallel computation completes, and carries out data merging to checking intermediate result, will in result, have the complete check result that is merged into of identical check mark.
The above; it is only preferred embodiment of the present invention; not the present invention is done to any pro forma restriction, any simple modification, equivalent variations and modification that every foundation technical spirit of the present invention is done above embodiment, all still belong to the protection domain of technical solution of the present invention.
Claims (5)
1. an online check method for parallel computation in electric system, is characterized in that: the method comprises the following steps:
(1) build Hadoop environment;
(2) initialization of the online checking task based on Hadoop;
(3) decomposition of online checking task;
(4) executed in parallel of online checking task;
(5) every online checking task is combined to calculating.
2. the online check method of parallel computation in electric system according to claim 1, is characterized in that: in step (1) by rivest, shamir, adelman realize Hadoop without cipher key communication.
3. the online check method of parallel computation in electric system according to claim 2, it is characterized in that: in step (2), check and produce a unique check mark for this, region and plant stand quantity in statistics electric system, represent with M, the quantity of cluster Computer represents with N, and so every maximum executable number of tasks of computing machine are: M/N; If the plant stand quantity that certain region is contained is less than M/N, so corresponding computing machine is only carried out the task of all plant stands in this region, and after all tasks carryings in this region, this computing machine, in idle condition, can be carried out the task in other regions.
4. the online check method of parallel computation in electric system according to claim 3, it is characterized in that: the JobTracker that in step (4), this is checked to dispatching center's master control in Hadoop of general assignment, this JobTracker operates on any default computing machine in cluster, JobTracker high-ranking officers' nucleon task is distributed to idle TaskTracker, TaskTracker is divided into check subtask executor TaskTracker1, check subtask executor TaskTracker2 ... check subtask executor TaskTrackerN), these tasks in parallel operations, and the ruuning situation of responsible monitor task, if the executor TaskTracker of some checks subtask is out of order, JobTracker can hand to its responsible task another idle TaskTracker and rerun.
5. the online check method of parallel computation in electric system according to claim 4, it is characterized in that: after the subtask in step (5) under same check identifies is all finished, this time parallel computation completes, carry out data merging to checking intermediate result, will in result, have the complete check result that is merged into of identical check mark.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410152048.4A CN103955400A (en) | 2014-04-17 | 2014-04-17 | Online checking method of parallel computing in electrical power system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410152048.4A CN103955400A (en) | 2014-04-17 | 2014-04-17 | Online checking method of parallel computing in electrical power system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103955400A true CN103955400A (en) | 2014-07-30 |
Family
ID=51332676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410152048.4A Pending CN103955400A (en) | 2014-04-17 | 2014-04-17 | Online checking method of parallel computing in electrical power system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103955400A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530126A (en) * | 2016-10-28 | 2017-03-22 | 云南电网有限责任公司 | Calculation method and calculation system for online checking of relay protection setting value |
| CN110874271A (en) * | 2019-11-20 | 2020-03-10 | 山东省国土测绘院 | Method and system for rapidly calculating mass building pattern spot characteristics |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101685481A (en) * | 2008-09-27 | 2010-03-31 | 国家电力调度通信中心 | Method and system for calculating on-line power transmission margin based on parallel algorithm |
| CN101685479A (en) * | 2008-09-27 | 2010-03-31 | 国家电力调度通信中心 | Power grid online comprehensive pre-warning method and system based on massively parallel processing |
| CN102662639A (en) * | 2012-04-10 | 2012-09-12 | 南京航空航天大学 | Mapreduce-based multi-GPU (Graphic Processing Unit) cooperative computing method |
-
2014
- 2014-04-17 CN CN201410152048.4A patent/CN103955400A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101685481A (en) * | 2008-09-27 | 2010-03-31 | 国家电力调度通信中心 | Method and system for calculating on-line power transmission margin based on parallel algorithm |
| CN101685479A (en) * | 2008-09-27 | 2010-03-31 | 国家电力调度通信中心 | Power grid online comprehensive pre-warning method and system based on massively parallel processing |
| CN102662639A (en) * | 2012-04-10 | 2012-09-12 | 南京航空航天大学 | Mapreduce-based multi-GPU (Graphic Processing Unit) cooperative computing method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530126A (en) * | 2016-10-28 | 2017-03-22 | 云南电网有限责任公司 | Calculation method and calculation system for online checking of relay protection setting value |
| CN110874271A (en) * | 2019-11-20 | 2020-03-10 | 山东省国土测绘院 | Method and system for rapidly calculating mass building pattern spot characteristics |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102611622B (en) | Dispatching method for working load of elastic cloud computing platform | |
| US9846589B2 (en) | Virtual machine placement optimization with generalized organizational scenarios | |
| CN104536937B (en) | Big data all-in-one machine realization method based on CPU GPU isomeric groups | |
| CN110278249A (en) | A kind of distribution group intelligence system | |
| CN104866374A (en) | Multi-task-based discrete event parallel simulation and time synchronization method | |
| CN107729138B (en) | Method and device for analyzing high-performance distributed vector space data | |
| Farhat et al. | Stochastic modeling and optimization of stragglers | |
| Brandão et al. | A biased random‐key genetic algorithm for single‐round divisible load scheduling | |
| CN103729257A (en) | Distributed parallel computing method and system | |
| Kchaou et al. | Towards an offloading framework based on big data analytics in mobile cloud computing environments | |
| CN103149839A (en) | Operational control method for electrical equipment based on Kuhn-Munkres algorithm | |
| CN104219226A (en) | Method for determining number of optimal communication agent nodes in cloud platform | |
| CN103617494A (en) | Wide-area multi-stage distributed parallel power grid analysis system | |
| CN109960579A (en) | A kind of method and device of adjustment business container | |
| CN103955400A (en) | Online checking method of parallel computing in electrical power system | |
| CN103793281A (en) | Load balancing method of compute-intensive simulation task | |
| CN104166581A (en) | Virtualization method for increment manufacturing device | |
| Ravie et al. | Enhancing the Simulation of Membrane System on the GPU for the N‐Queens Problem | |
| CN105190599A (en) | Cloud application bandwidth modeling | |
| CN109064049A (en) | A kind of dynamic divides the method, apparatus and terminal device of risk zones | |
| Kholod et al. | Creation of data mining cloud service on the actor model | |
| Tejaswi et al. | A GA based approach for task scheduling in multi-cloud environment | |
| Ma et al. | Cloud-based multidimensional parallel dynamic programming algorithm for a cascade hydropower system | |
| CN106357676A (en) | Method for optimizing overhead of cloud service resource | |
| CN106301960A (en) | A kind of cloud resource coordinating management method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140730 |
|
| RJ01 | Rejection of invention patent application after publication |