CN107078541B - System, method and apparatus for the data processing in electric system - Google Patents
System, method and apparatus for the data processing in electric system Download PDFInfo
- Publication number
- CN107078541B CN107078541B CN201480082936.1A CN201480082936A CN107078541B CN 107078541 B CN107078541 B CN 107078541B CN 201480082936 A CN201480082936 A CN 201480082936A CN 107078541 B CN107078541 B CN 107078541B
- Authority
- CN
- China
- Prior art keywords
- data
- system resource
- data flow
- power system
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
- H02J13/00006—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by information or instructions transport means between the monitoring, controlling or managing units and monitored, controlled or operated power network element or electrical equipment
- H02J13/00028—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by information or instructions transport means between the monitoring, controlling or managing units and monitored, controlled or operated power network element or electrical equipment involving the use of Internet protocols
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
- H02J13/00032—Systems characterised by the controlled or operated power network elements or equipment, the power network elements or equipment not otherwise provided for
- H02J13/00034—Systems characterised by the controlled or operated power network elements or equipment, the power network elements or equipment not otherwise provided for the elements or equipment being or involving an electric power substation
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
- H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E60/00—Enabling technologies; Technologies with a potential or indirect contribution to GHG emissions mitigation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S40/00—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them
- Y04S40/20—Information technology specific aspects, e.g. CAD, simulation, modelling, system security
Landscapes
- Engineering & Computer Science (AREA)
- Power Engineering (AREA)
- Supply And Distribution Of Alternating Current (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Disclose the system for the data processing in electric system, method and apparatus and visible computer readable medium.In one embodiment, the system comprises at least one processors；And at least one processor, store computer executable instructions.At least one processor and computer executable instructions are configured to promote the system using at least one described processor: the data in data flow are based on power system resource group associated there at one or more data reception nodes and are grouped, the power system resource group be based in part on it is defining the hierarchical structure of electric power net operation, for the electric system data model and determination；The data in the data flow are based on the power system resource group associated there and are sent to one or more data processing nodes；And the data in the data flow are based on the power system resource group associated there at one or more of data processing nodes and are handled.It provides for the extensive low cost solution handled in real time in electric system, which is efficient, expansible, and makes it possible to realize the online change of data handling procedure.
Description
Technical field
Embodiment of the disclosure is related to field of power system, and particularly data processing in electric system is
System, method and apparatus and visible computer readable medium.
Background technique
Various industries all have network associated therewith.One such industry is to manage the public utilities production of power network
Industry.Power network may include one or all in the following terms: power generation, transmission of electricity and distribution.Can be used such as coal-fired plant,
The power stations such as nuclear power plant generate electricity.For efficiency purposes, the electric power of generation gradually boosted to very high voltage (such as
345K volt) and transmitted by transmission line.Transmission line can with long distance delivery electric power, such as across state circle or across international boundary, until
Until it reaches its wholesale customers, which can be the company for possessing local power distribution network.Transmission line can become in transmission
It terminates, very high voltage can be gradually reduced to middle pressure (such as 138K volt) at power station.Lesser transmission line is (such as sub
Transmission line) by it is middle pressure from transmission substation be transmitted to power distribution station.At power distribution station, which can be gradually reduced again
To " middle pressure " (being such as that 4K is lied prostrate to 23K volt).One or more feed circuits can be issued from power distribution station.For example, four extremely
Dozens of feed circuit can be issued from power distribution station.Feed circuit be include that (three conducting wires are for every phase in 3 phases for 4 conducting wires
And conducting wire is used for zero curve) 3 circuitry phases.Feed circuit can on the ground (on utility poles) or underground laying.It can be with
Periodically the voltage tap on feed circuit is come out using distribution transformer, this gradually reduces voltage to visitor from " middle pressure "
Family voltage (such as 120V).Then client's voltage can be utilized by the customer.
One or more Utilities Electric Co.s' (its major responsibility is that reliable and economic electric power is supplied to its client) manage electric power
Net, including planning related with power network, operation and maintenance.In order to improve the efficiency of management and reduce management cost, electric power is public
Department has attempted by upgrading power network at " smart grid " using prior art IT and electric power project engineering.Smart grid
The sensing device or system of demand for development huge amount, such as advanced measurement basis facility (AMI), phase measurement cells (PMU)
And other in-service monitoring devices.These devices will generate a large amount of data, and data scale can be TB grades.For example, in
State, provincial electric power company usually plan to install the instrument more than 10,000,000, generate nearly 4 TB data possible every month.In addition,
Fig. 1 is illustrated from the data growth trend studied recently, from where it can be seen that data volume with advanced power distribution automation and
The development of AMI and sharply increase, and almost 800TB, this is the quite huge data of number.Meanwhile it is required that quickly
Data processing to providing useful information in time.Therefore, utility company is faced with huge challenge, is especially depositing
Reservoir calculates and database purchase aspect.
Therefore, it needs in the art for data processing and storage, the improvement of more specifically to a large amount of real time datas
Solution.
Summary of the invention
For this purpose, according to the disclosure in a first aspect, providing a kind of system for the data processing in electric system.Institute
The system of stating includes at least one processor；And at least one processor, store computer executable instructions.Described at least one
A memory and computer executable instructions are configured to promote the system using at least one described processor: at one or
The data in data flow are based on power system resource group associated there at multiple data reception nodes to be grouped, institute
Stating power system resource group is to be based in part on number that define the hierarchical structure of electric power net operation, for the electric system
The determination according to model；By the data in the data flow be based on the power system resource group associated there and
It is sent to one or more data processing nodes；And it will be in the data flow at one or more of data processing nodes
The data be based on the power system resource group associated there handled.
In embodiment of the disclosure, the data grouping by data flow may include based on included in the data
In power system resource and measurement point defined in the measurement point identification information and the data model in the data in stream
Correlation between identification information, to determine the electric system money associated with the data in the data flow respectively
Source；And according to identified power system resource and in the data model defined in described in the electric power net operation point
Level structure, to determine the power system resource associated with the data in the stream.
In another embodiment of the present disclosure, the data sent in the data flow can depend on current slow
Rush message size and current waiting time.
According to another embodiment of the present disclosure, current network load condition is considered when the Current buffer message size reaches
Size threshold when, the data in the data flow can be sent.
In another embodiment of the present disclosure, when the current waiting time, which is longer than, considers current network load condition
Between threshold value when, the data in the data flow can be sent.
In another embodiment of the present disclosure, the data high-speed in the data flow can be buffered in one
Or to be used to be handled in multiple data processing nodes, and the data in the data flow are further stored in number
According in library, wherein in the case where the fault recovery of data processing node or restarting, corresponding data are from the database
It is loaded into the Cache of the data processing node.
In another embodiment of the present disclosure, the data in the data flow can be stored in table per year, and
And the title for table includes information about the year, and wherein for the keyword of each data in the data be by
Power system resource associated there identifies the time of measuring with the data time and is formed, and the time of measuring is institute
In the form of stating time migration of the data time within this year as unit of millisecond.
It according to another embodiment of the present disclosure, can be by the data in the data flow according to electricity associated there
Force system resource and be divided into one or more segmentations, and it is one or more of segmentation by based on the institute for the data
It states keyword and concurrently and is in a distributed manner written to the multiple regions of the database sequentially in time.Particularly, described
The data in data flow are stored in institute to be respectively adapted to its own data type and be suitable for the format of data processing
It states in database.
In another embodiment of the present disclosure, the data in the processing data flow can be based on distributed stream
Computing architecture, and data storage is based on distributed data base.
According to the second aspect of the disclosure, a kind of method for the data processing in electric system is provided.The side
Method may include: that the data in data flow are based on electric system associated there at one or more data reception nodes
Group of resources is grouped, the power system resource group be based in part on define electric power net operation hierarchical structure,
For the electric system data model and determination；The data in the data flow are based on institute associated there
It states power system resource group and is sent to one or more data processing nodes；And in one or more of data processings
The data in the data flow power system resource group associated there is based at node to handle.
According to the third aspect of the disclosure, a kind of equipment for the data processing in electric system is provided.It is described to set
Standby includes: data grouping module, is configured to be based on the data in data flow at one or more data reception nodes
Power system resource group associated there is grouped, and the power system resource group is based in part on described in definition
The hierarchical structure of electric power net operation, for electric system data model and determination；Data transmission blocks are configured to
The data in the data flow are based on the power system resource group associated there and are sent to one or more
A data processing node；And data processing module, it is configured to institute at one or more of data processing nodes
The data stated in data flow are handled based on the power system resource group associated there.
According to the fourth aspect of the disclosure, another equipment for the data processing in electric system is provided.It is described to set
Standby includes: for the data in data flow to be based on electric system associated there at one or more data reception nodes
The device that group of resources is grouped, the power system resource group are to be based in part on point for defining the electric power net operation
Level structure, for electric system data model and determination；For by the data in the data flow be based on and its
The associated power system resource group and the device for being sent to one or more data processing nodes；And in institute
It states and the data in the data flow is based on the power train associated there at one or more data processing nodes
The device that system group of resources is handled.
According to the 5th of the disclosure the aspect, a kind of visible computer readable medium with multiple instruction is provided, it is described
Instruction can be executed by processor to execute the data processing in electric system.The visible computer readable medium may include by
It is configured to the instruction for the step of executing the method according to the second aspect of the disclosure.
Using embodiment of the disclosure, provided at one or more receiving nodes based on electric system associated with data
Source group and by data stream packet, be distributed to according to power system resource group at processing node, and at processing node into
Row processing.In the disclosure, a kind of extensive low cost solution handled in real time in electric system is provided,
It is efficient, expansible, and makes it possible to realize the online change of data handling procedure.
Detailed description of the invention
By the detailed description about the embodiment such as illustrated in embodiment with reference to attached drawing, addressed in the disclosure
Other feature will become apparent, wherein spread disclosure attached drawing, identical appended drawing reference indicates same or like portion
Part, and in the drawing:
Fig. 1 schematically illustrates the song increased according to the electron electric power data of nearest result of study in the prior art
Line；
Fig. 2, which is schematically illustrated, according to an embodiment of the present disclosure wherein may be implemented being used for for embodiment of the disclosure
The diagram of the exemplary architecture of power network；
Fig. 3 schematically illustrates the diagram of data processing according to an embodiment of the present disclosure and data storage；
Fig. 4 schematically illustrates the method according to an embodiment of the present disclosure for the data processing in electric system
Flow chart；
Fig. 5 A to 5C schematically illustrates electric power net operation hierarchical structure according to an embodiment of the present disclosure, exemplary original
The diagram of beginning data and exemplary map data；
Fig. 6 schematically illustrates the data processing operation in the case where fault recovery according to an embodiment of the present disclosure
Diagram；
Fig. 7 A and 7B schematically illustrate the diagram of exemplary data tables design according to an embodiment of the present disclosure；
Fig. 8 A and 8B schematically illustrate the diagram of example data storage mode according to an embodiment of the present disclosure；
Fig. 9 A and 9B schematically illustrate simulation case result according to an embodiment of the present disclosure；
Figure 10 schematically illustrates the system according to an embodiment of the present disclosure for the data processing in electric system
Block diagram；
Figure 11 schematically illustrates the equipment according to an embodiment of the present disclosure for the data processing in electric system
Block diagram；
Figure 12 schematically illustrates the data processing being used in electric system according to another embodiment of the present disclosure
The block diagram of another equipment；And
Figure 13 schematically illustrates general-purpose computing system, which is programmed for special purpose computer system
System, can represent referenced herein any calculating equipment.
Specific embodiment
Hereinafter, embodiment of the disclosure will be described in reference to the drawings.In the following description, many certain details are elaborated
Save the thorough understanding in order to provide embodiment.However, it will be apparent to those skilled in the art that the implementation of the disclosure
Mode can not have these details, and the present disclosure is not limited to the specific embodiments being such as described herein.On the contrary, can be with
Any any combination of following characteristics and element is considered to realize and implement the disclosure, whether is it being related to different realities
Apply example.Therefore, following aspect, feature and embodiment being merely illustrative property purpose, and should not be understood as to appended right
It is required that element or limitation, unless in addition clearly indicating in the claims.In addition, in some cases, not in detail
Well-known method and structure is described in order to avoid unnecessarily making embodiment of the disclosure ambiguous hard to understand.
In current electric system, a large amount of data will be generated daily, and require quick data processing so as to timely
Ground provides useful information.Therefore, in the disclosure, propose a kind of solution for the data processing in electric system so that
Can in real time or near real-time handle data, to meet the requirement of current electric system.
Fig. 2, which is schematically illustrated, according to an embodiment of the present disclosure wherein may be implemented being used for for embodiment of the disclosure
The diagram of the exemplary architecture of electric system.As shown in Figure 2, data source 110-1 to 110-n (such as various monitoring systems, prison
View apparatus, sensor, instrument etc.) various types of data are generated, these data pass through real time data bus 120-1 or real-time thing
Part bus 120-2 is sent to processing & storage system 130 in real time.In processing & storage system 130 in real time, mould is handled in real time
Block 140 receives these data and based on the information obtained from data model 160 and includes that information in data is handled, and
And processed result is stored in reservoir 150.Via unified information access modules 170, various analyses and application program 180
Accessible data model 160, and handle result or other data in reservoir 150.As shown in the disclosure this
Framework will not be merely provided for the real time data processing and storage of mass data, and provides fairly good reliability, can contract
Putting property and easy data access will hereinafter be described in detail real-time processing & storage with reference to Fig. 3-Figure 12.
Fig. 3 schematically illustrates the diagram of data processing according to an embodiment of the present disclosure and data storage.As schemed
Show, the data source 110 for setting device AMI etc. including such as supervisory control and data acquisition (SCADA) (SCADA) system, device monitoring will be generated by it
Or the data obtained are supplied to real time processing system 140.Herein, real time processing system 140 can be based at distributed stream
Reason technology, such as Apache Storm system are the distributed real time computation systems for handling quick large data stream.
Real-time processing module 140 may include one or more receiving node 141-1,141-2 ..., 141-m；One or more data
Handle node 142-1 142-2,142-3,142-4 ..., 142-k-1,142-k；And optionally one or more data are defeated
Egress 143-1 ..., 143-j.One or more of receiving nodes are configured to receive data flow from data source 110,
Data spout (spout) is properly termed as in Apache Storm system.One or more of data processing node 142-1,
142-2,142-3,142-4 ..., 142-k-1,142-k each wherein include multiple processes, and be configured to handle to
Its data sent, this is properly termed as processing pipeline (bolts) in Apache Storm system.One or more of data
Output node 143-1,143-j may be configured to collect from processing node processed as a result, and sending it to data storage
For device 150 to store, this is referred to as processing pipeline in Apache Storm system.It can use distributed computing (example
Such as based on cloud computing) in a distributed way come execute it is described receive, handle and export.It is, therefore, possible to provide cloud computing platform 190
To execute Data Stream Processing.Finally, the number into data storage device 150 can will be stored from the result that Data Stream Processing obtains
There can be at least one database wherein according to reservoir 150, and optionally, which can be distributed data
Library, such as HbaseTM, it is Hadoop database, according to the open source, distribution, versioned of the Bigtable of Google modeling
Non-relevant data library.
The process of the method according to an embodiment of the present disclosure for the data processing in electric system is shown in FIG. 4
Figure.As shown in Figure 4, it is used as step S401 first, the data flow at one or more data reception nodes is based on and its phase
Associated power system resource group and be grouped.Can based on measurement point identification and for electric system data model come
Determine power system resource group adjacent thereto.
In power network, it is typically provided with the data model for defining the hierarchical structure of electric power net operation, can be covered each
Various elements and object in provenance, such as power transmission network, power distribution network and client domain, and the various numbers in power network can be provided
According to unification and centralization Data View.Fig. 5 A schematically illustrates example data mould according to an embodiment of the present disclosure
Type.As shown, in exemplary data model, in the hierarchical structure of the electric power net operation in transmission of electricity, distribution and client domain
Arrange electric device.For example, there is, 500kV, 220kV etc. horizontal for the different voltages of power equipment for transmission of electricity；
For some voltage level, there are different types of power equipment, substation, routes etc.；For in some level
The power equipment of some type, such as in the substation of 500KV, there are different power equipments, such as ST0082, ST0083
Deng.Each substation ST0082, ST0083 further include different electronic devices, such as bus section, busbar section, capacitor, breaker, transformation
Device etc., each of these includes multiple measurement points, active power, reactive power, bottom oil temperature etc..Therefore, such
The hierarchical structure of electric power net operation is clearly defined in data model.In addition, in data model, comprising about electrical equipment, electricity
The identification information of power equipment and measurement point, reflect these components to its identify between it is related.Based on the information, can incite somebody to action
Data in data flow are grouped.
Fig. 5 B illustrate from data sources to exemplary initial data.As shown in Figure 5 B, initial data includes surveying
It measures ID " MeaslD ", is the mark of measurement point associated with data；Time of measuring " timeStamp ", instruction measurement number
According to when time；Measured value " Value " indicates the value of measurement data；The quality of data " Quality ", the matter of designation date
Amount, " 0 " indicate high quality.According to including information in data, especially measurement ID " MeaslD " and mentioned in data model
The correlation of confession can determine power system resource associated with data first, such as by using measurement ID " MeasID "
The respective table in data model is searched for as search condition.It therefore can be by using including measurement point identification in data
Associated measurement point or power system resource are recorded with measurement to determine.In addition, being based on data model, may further determine that
Power equipment belonging to measurement point, such as TR1122.Based on the data model, it can determine that the power equipment belongs to electricity therein
Force system resource.For example, being directed to transformer TR1122, it will be clear that, belong to electric system from data model
Group of resources ST0082, i.e. substation ST0082.
Can will about power system resource identify and power system resource group determination information include into data with
For then using.Exemplary map data are only shown for exemplary purposes in figure 5 c.As shown, except being included in
Except information in initial data, mapping data also include " PSRID " and instruction electric power for the mark for indicating power system resource
" PSRGrpID " of the mark of system resource group.
Referring back to Fig. 4, at step S402, the data in data flow are based on power system resource associated there
Group and be sent to processing node.After data in a stream are grouped, electric system associated there will be based on
Group of resources and be sent to subsequent processing node with for being further processed.That is, belonging to same unified power system
The data of group of resources will be sent to identical one or more processing nodes.
In embodiment of the disclosure, message transmission can be advanced optimized to improve efficiency of transmission.Inventor notices
For some messaging bus, performance is the variable that changes according to message size, and usually exist may be implemented it is best
The optimal message size of performance.For example, with regard to use zeroMQ as the Apache Storm system of internal message bus for, with
Optimum performance may be implemented in the message size of 100k.The following table 1 gives performance and message size in Apache Storm system
Between relationship.
Relationship between 1 performance of table and message size
According to table 1, it is evident that realize optimum performance when message size is 100k, and greater or lesser size will not
Better performance can be provided.Given this it finds, in the disclosure, design optimization algorithm considers bus performance and for real-time
It both deadline dates of data transmission, can for example indicate as follows:
BooleanSend=(t >=T*a*w) or (s > S*b*w)
Wherein, BolleanSend is to determine whether that the Boolean of data to be sent, t indicate the current waiting time；T is needle
To the real time data permitted maximum traffic delay time, S is the amount of Current buffer real time data；S is the data of each message
Maximum number, determined by optimal message size and data size, such as in optimal message size be that 100k and each data have
It is 4.17k (that is, 100k/24) in the case where having the size of 24 bytes；W indicates the network load in the value of range from 0 to 1
Than reaction network loading condiction, i.e. current network load or bandwidth；A is the optimization ratio for delay time；And b is to use
In the optimization ratio of message size.A, the value of b is the optimization ratio that data will be made to be sent with small size, can be according to network
Loading condiction is set.For example, its default value can be set to such as 1, however, the situation relatively light in network load
Under, the value of a and b can be set as lower value.
Therefore, in embodiment of the disclosure, the transmission of the data in data flow can depend on Current buffer message ruler
The very little and current waiting time.For example, when the Current buffer message size reaches the size threshold of consideration current network load condition
When value, the data that will send in the data flow.Additionally, or alternatively, work as when the current waiting time is longer than consideration
When the time threshold of preceding network loading conditions, the data that will send in the data flow.In this way, can permit when message reaches
Message is sent when to specified size to send to obtain optimal performance, or when network idle with smaller size, is allowed to really
Protect the real-time performance of data transmission.
Referring back to Fig. 4, at step S403, the data in data flow at one or more data processing nodes are by base
It is handled in power system resource group associated there.At each processing node, it is currently running one or more mistakes
Journey is to handle the data being sent to it.These processes can be configured according to the type of power system resource group and according to need
To update.It, can be anti-by embedded script executing engine (python etc.) and/or java at each processing node
Mechanism is penetrated, easily online updating is carried out or changes Processing Algorithm or process.The processing of data is well known in the art, and
Therefore the detailed description that the operation of nodes is handled about these will not elaborated herein for purposes of clarity.
In embodiment of the disclosure, the data belonged in the data flow of same power system resource group will be sent to
Identical one or more processing nodes, and at these processing nodes, data can be simultaneously handled, electricity may be implemented in this
Real time data processing in Force system, for extensive real time data.In addition, in the disclosure, treatment process
Node can be based on different distributed stream computing techniques, and therefore its useful good scalability, it means that can be with
Stream calculation cluster is dynamically increased to meet new requirement, such as when data volume increases or needs to reduce time cost.Separately
Outside, according to the disclosure, various types of data processings can be executed according to related PSR group, reduce repeated data transmission
And improve the efficiency of data processing.Further, since the data belonged in the data flow of same PSR will be mapped, be packaged and send
To the same processing node, it is possible to promote to realize the data processing towards electric power net operation.
In addition, in embodiment of the disclosure, being sent to each in order to ensure the high efficiency and high reliability of data processing
The data of processing node are cached in processing node, and at the same time ground, initial data be also stored on such as HBase it
In the database of class.Cached data will be used for data processing, and store data in the database can at one or
It is used in the case where the fault recovery or initialization of multiple processing nodes.For example, when processing node is restarted due to such as failure
When can first from database load data.
Fig. 6 illustrates the flow chart of the data processing operation in the case where fault recovery according to an embodiment of the present disclosure.
As shown, data will be loaded into Cache from database first at step 601, and at step 602, handle
Node starts to receive packet data, and then caches received data high-speed in the cache.At step 603,
Processing node further uses the storage of received data into database in the case where such as fault recovery.Then exist
Data are obtained and handled using each process in step 604, and store processed result into database at step 605.
After data have been successfully processed, at step 606, ACK can be sent to receiving node as answer.
According to flow chart, it can be seen that even if can still obtain initial data from database, and therefore may be used there are failure
To provide higher reliability.In addition, the number when data dramatically increases and needs more to handle node to handle these numbers
According to when can execute similar operations.
Apart from the above, in the disclosure, a kind of Data Storage Solution of application program-oriented method is additionally provided, thus
The high-performance for realizing extensive real-time data memory and access is described in detail next with reference to Fig. 7 A to Figure 12.
Will be referring initially to Fig. 7 A and 7B, it illustrates example data storage table according to an embodiment of the present disclosure designs.
In the disclosure, it proposes and stores measurement data per year, i.e., annual measurement data will be stored in the same table, and be used
It will be stored in different tables in the data of different year.Annual measurement data table can have such as " Historydata
2014 " etc table name with include time of measuring about the data being stored therein year information.The table may include
The row keyword " Rowkey " of a block number evidence for identification.More particularly, row keyword " Rowkey " may include two parts,
I.e. about the information of power system resource and about the information of time of measuring.Information about power system resource can be electric power
System resource identifies " PSRID "." PSRID " is the unified id of power system resource, as described above, with no symbol int
Data type (4 byte).Information about time of measuring can only include about the information of the moon and day without year, because closing
It can be reflected by the title of table in the information in year.Particularly, the information used herein about time of measuring is not general
Logical time format, but indicated with the when in-migration in the year of data time as unit of millisecond.This means that about time of measuring
Information do not include information about the year, and its using and first day 0:00am in such as this year time shift.
It therefore, can will be no symbol int format, 4 bytes about the information design of time of measuring.Therefore, in embodiment of the disclosure
In, row keyword only needs 8 bytes, and it is empty that this will greatly save storage compared with the design of row keyword in the prior art
Between.
Furthermore it is possible to include column family " CF ", column " qualifier " and value Value." CF " indicates race belonging to data,
In, ' r ' refers to initial data；" s " refers to statistical data；" e " refers to event.Column qualifier is the measurement class for indicating value
The code of type and format.Can by using the code come as shown in fig.7b measurement type and format define in table and execute
Search for the measurement type and format to determine value.The format memory data of " Value " to be indicated with column qualifier.Furthermore it is possible to
Including " Region ", indicate that measurement data is stored in position therein.It can be pre- according to data scale and computing resource
" Region " first is distributed, to avoid any influence of the performance to adjust automatically.
In conventional methods where, usually using row keyword, but this will require about 24 bytes, and in the disclosure, lead to
It crosses using PSRID and the time shift as unit of millisecond, it is only necessary to 8 bytes.Therefore, traditional row keyword will occupy the disclosure
Row keyword almost 3 times of memory space, it means that very big memory space is saved.In addition, in the disclosure, data flow
In data by be respectively adapted to the data type of its own and be suitable for the format of data processing rather than just with character
String format stores in the database.For example, numerical value (voltage, electric current, power etc.) is stored with the format of such as double precision,
Rather than general used character string.The data types such as double precision, floating-point need memory space more less than character string,
And it can be directly processed in the case where no conversion.Therefore, this can not only save memory space, but also at improvement
Manage efficiency.In addition, required sole act is to be inserted into one into the table of Fig. 7 B if to add the measurement data of new type
Record is to define measurement type and value format.
Fig. 8 A illustrates the diagram of the parallel memorizing of data according to an embodiment of the present disclosure.It, can be with according to the disclosure
Sequence of event concurrently and in a distributed manner stores data into database.In the disclosure, row keyword includes
PSRID and the time shift as unit of millisecond.Data will be divided into multiple segmentations based on PSRID, and sequentially in time
Concurrently by fragmented storage into different zones, as shown in figure 8B.Due to the special designing of row keyword, in such as HBase
Etc database in, would automatically carry out storing process, because HBase itself can support such mechanism.
Such storage mode can provide the data access of application program-oriented method.As it is known, application program usually will be
Access belongs to the data of same power system resource group in special time period, and the mode of storing data will significantly in the database
Ground promotes the access of data, because required all data are usually consecutively stored in region, as shown in figure 8B.This number
It will ensure that when by PSR and period to access data according to storage mode, data can be accessed with quite high efficiency.Therefore,
In embodiment of the disclosure, help is realized high data access performance by Data Storage Solution.
Above-mentioned storage solution not only supports extensive real-time data memory, but also provides good scalability.This
Outside, be quite appropriate for this large data analysis, such as its can easily with other analytic units of the Hadoop ecosystem
In conjunction with.
Fig. 9 A and Fig. 9 B schematically illustrate simulation case result according to an embodiment of the present disclosure.In simulations, make
With 3 to 15 host nodes, each there is 16G memory, 8 core CPU, 1T reservoirs, 1Gbit network；Use 3,000,000 instrument
As data source, each there are 33 measurement data.Fig. 9 A illustrates data processing and write performance, wherein X-axis is shown
The number of host node, Y-axis show the number of the instrument of processing and write-in per second.According to Fig. 9 A, it is evident that processing and write-in property
Almost can linearly it increase with the increase of cluster, and it realizes 50000 instrument/seconds for 15 host nodes
The write performance of (45,000,000 instrument/15 minute).Fig. 9 B illustrates the reading data performance obtained from simulation case.According to this
Table, it is evident that data can be read within the very short time, and be directed to the instrument of identical number, bigger thread amount can
To provide higher reading performance.
Therefore, using embodiment of the disclosure, make it possible to low cost and high-performance treatments and store a large amount of numbers in real time
According to this facilitate extract information from real time data and carry out better business decision in time.In addition, additionally providing for being directed to
Utility company large-scale and the broadly key of distributed real time data processing and storage and basic solution.
In the disclosure, a kind of system for the data processing in electric system is additionally provided, and hereinafter, it will
It is described with reference to Figure 10 this system, Figure 10 schematically illustrates the number according to an embodiment of the present disclosure in electric system
According to the block diagram of the system of processing.
As shown in Figure 10, system 1000 includes at least one processor 1010；And storage computer executable instructions
1030 at least one processor 1020.At least one processor 1020 and computer executable instructions 1030 are configured to
Promote the system 1000 using at least one described processor 1010: by data flow at one or more data reception nodes
In data be based on power system resource group associated there be grouped, the power system resource group is partly
Data model based on the hierarchical structure for defining electric power net operation, for the electric system and determination；By the data
The data in stream are based on the power system resource group associated there and are sent to one or more data processings
Node；And at one or more of data processing nodes by the data in the data flow be based on it is associated with it
The power system resource group handled.
It particularly, in embodiment of the disclosure, may include: based on packet by the data grouping in data flow to be processed
It is contained in defined in the measurement point identification information and the data model in the data in the data flow in electric system
Correlation between resource and measurement point identification information, to determine institute associated with the data in the data flow respectively
State power system resource；And according to identified power system resource and in the data model defined in the power network
The hierarchical structure of operation, to determine the power system resource associated with the data in the stream.
In embodiment of the disclosure, the transmission of the data in data flow can be optimized to realize superior performance.For example,
The transmission of data can depend on Current buffer message size and current waiting time.When the Current buffer message size reaches
When considering the size threshold of current network load condition, the data in the data flow can be sent.Or alternatively, when
When the current waiting time is longer than the time threshold for considering current network load condition, the institute in the data flow can be sent
State data.
In embodiment of the disclosure, the data high-speed in the data flow can be buffered in one or more
To be used to be handled in a data processing node, and the data in the data flow are further stored in database
In.In the case where handling the fault recovery or restarting of node, corresponding data can be loaded into data from database
Handle the Cache of node.The data in the data flow can be stored in table per year, and for table
Title includes the information about the year.Keyword for each data in the data can be by electric power associated there
System resource identifies the time of measuring with the data time and is formed, and the time of measuring is the data time in the year
In the form of the interior time migration as unit of millisecond.
It in embodiment of the disclosure, can be by the data in the data flow according to power train associated there
System resource and be divided into one or more segmentations, and it is one or more of segmentation by based on the pass for the data
Key word and the multiple regions for concurrently and being in a distributed manner written to the database sequentially in time.In addition, the data flow
In the data can by be respectively adapted to its own data type and be suitable for the format of data processing be stored in it is described
In database.
In embodiment of the disclosure, the processing of the data in data flow can be based on distributed stream computing architecture, such as
Apache Storm system, and data storage is based on distributed data base, such as HBase.
In Figure 11, setting for the data processing in electric system according to another embodiment of the present disclosure is illustrated
It is standby.Equipment 1100 includes: data grouping module 1110, data transmission blocks 1120 and data processing module 1130.Data point
Group module 1110 may be configured to be based on the data in data flow at one or more data reception nodes associated therewith
The power system resource group of connection is grouped, and the power system resource group, which is based in part on, defines electric power net operation
Hierarchical structure, for the electric system data model and determination.Data transmission blocks 1120 may be configured to by
The data in the data flow are based on the power system resource group associated there and are sent to one or more
Data processing node.Data processing module 1130 may be configured to will be described at one or more of data processing nodes
The data in data flow are based on the power system resource group associated there and are handled.
In embodiment of the disclosure, it includes institute in the data flow that data grouping module 1110, which can be primarily based on,
It states defined in the measurement point identification information and the data model in data in power system resource and measurement point identification information
Between correlation, to determine the power system resource associated with the data in the data flow respectively.Then
The hierarchical structure of electric power net operation defined according to identified power system resource and in data model, data grouping module
1110 can determine power system resource group associated with the data in stream in turn.
In another embodiment of the present disclosure, data transmission blocks 1120 can be further configured to according to Current buffer
Message size and current waiting time send the data in data flow.For example, being examined when the Current buffer message size reaches
When considering the size threshold of current network load condition, the data in the data flow can be sent.Additionally, or alternatively,
When the current waiting time being longer than the time threshold for considering current network load condition, can send in the data flow
The data.
In embodiment of the disclosure, the data high-speed in data flow can be buffered in one or more data processing sections
For being handled in point.Meanwhile optional data memory module 1140 can store the data in data flow to database
In.In the case where handling the fault recovery or initialization of node, corresponding data can be loaded into from data from database
Manage the Cache of node.Data memory module 1140 may be configured to the data in data flow being stored in table per year
In.Title for table may include the information about the year, and the keyword for each data in the data can
To be formed, the time of measuring by the mark of power system resource associated there and the time of measuring of the data time
In the form of being time migration of the data time within this year as unit of millisecond.It can be by the number in the data flow
One or more segmentations are divided into according to according to the PSRID for the data, and one or more of segmentations are by based on use
In the data its keyword and concurrently and be in a distributed manner written to the multiple regions of the database sequentially in time.
Particularly, the data in the data flow are by be respectively adapted to its own data type and be suitable for the lattice of data processing
Formula stores in the database.
In addition, in embodiment of the disclosure, the data handled in the data flow are based on distributed stream calculating support
Structure, and data storage is based on distributed data base.
In Figure 12, the data processing being used in electric system according to another embodiment of the present disclosure is further illustrated
Another equipment.Equipment 1200 include at one or more data reception nodes by the data in data flow be based on and its
The device 1210 that associated power system resource group is grouped, the power system resource group are to be based in part on to determine
The hierarchical structure of the justice electric power net operation, for electric system data model and determination；For by the data flow
In the data be based on the power system resource group associated there and be sent to one or more data processing sections
The device 1220 of point；And at one or more of data processing nodes by the data base in the data flow
In the device 1230 that the power system resource group associated there is handled.
As shown in Figure 12, in addition equipment 1200 may include for storing the data in data flow into database
Device 1240.In the case where the fault recovery of data processing node or restarting, corresponding data are loaded from database
To the Cache of data processing node.Device 1240 for storing data can be further configured to will be in data flow
Data be stored in table per year, and the title for table includes information about the year, and is wherein used for the number
The keyword of each data can be by the mark of power system resource associated there and the measurement of the data time in
Time and formed, in the form of the time of measuring is time migration of the data time within this year as unit of millisecond.This
Outside, the data in the data flow can be divided into one or more segmentations according to the PSRID for being used for the data, and
And one or more of segmentations are by sequentially in time concurrently and distributed based on its keyword for the data
Ground is written to the multiple regions of the database.Particularly, the data in the data flow are by be respectively adapted to it certainly
Body data type and be suitable for data processing format storage in the database.
Further it is provided that a kind of visible computer readable medium with multiple instruction, the multiple instruction can be processed
Device is executed to execute the data processing in electric system, which may include being configured to execute basis
The instruction of the method and step of any embodiment of disclosed method.
It should be noted that including the operation of each module or device in system 1000, equipment 1100 and equipment 1200
Correspond essentially to foregoing various method steps.Accordingly, with respect to each in system 1000, equipment 1100 and equipment 1200
The detailed operation of module or device please refers to the previous description referring to figs. 2 to the 9B disclosed method carried out.
Although it is provided merely for the sake of the purpose of illustration in addition, hereinbefore proposing specific embodiment,
So that those skilled in the art can fully and thoroughly understand the thought of the disclosure, makes it possible to and implement this public affairs
The solution opened.According to the teaching content such as provided herein, those skilled in the art is contemplated that various modifications；Institute
There are these modifications that should be within the purview of the appended claims.Although for example, including data reception node, data processing node
Or stream processing system is described in the topology of data out node, at the stream it is also possible to use no independent data out node
Reason topology.In embodiment of the disclosure, some electronic device such as ST0082 in some voltage level will be belonged to)
Data are divided into same power system resource group；However, it is also possible to by belong in some electronic device ST0082 some or
The data of the power equipment (for example, transformer, TRl122) of some type are divided into same power system resource group.In addition,
In order to improve efficiency of transmission, specific optimization method is provided；However, it is also possible to be imagined according to teaching content provided herein
Other different optimization methods.
General-purpose computing system 1300 is further illustrated in Figure 13, can be expressed herein any meter referred to
Calculate device.For example, general purpose computer 1300 can be -partially or completely-to indicate control centre, head end, integrated network fortune
Battalion and management system (NOMS), failure, performance and configuration management (FPCM) module or referenced herein any other meter
Calculate device, such as terminal installation, instrument, telemetry interface module (TIU), collector and/or any networking components, such as at this
Router, interchanger or the server discussed in text.Computer system 1300 may include that can be performed to promote computer
System 1300 executes method disclosed herein or one group of instruction 1302 based on any one or more of computer function
Ordered list.Computer system 1300 can serve as self-contained unit, or can be for example connected to using network 115,125 other
Computer system or peripheral unit.
In networked deployment, computer system 1300 can be in the capacity of server or as server-client
Client user computer in user network environment or as the opposite end computer in equity (or distributed) network environment
System operatio.Computer system 1300 can also be embodied as to various devices or be incorporated therein, such as personal computer or shifting
Dynamic computing device, is able to carry out one group of instruction 1302 of the specified movement that will be taken by the machine, and the movement includes but not
Any type of browser is limited by access network 115,125.In addition, each system may include individually or
Execute one or more instruction set jointly to execute any set of the subsystem of one or more computer functions.
Computer system 1300 may include processor 1307, such as central processing module (CPU) and/or graphics process
Module (GPU).Processor 1307 may include one or more general processors, digital signal processor, specific integrated circuit,
Field programmable gate array, digital circuit, optical circuit, analog circuit, a combination thereof or for parsing and handling the other of data
Device that is currently known or developing later.Instruction set 1302 or other software programs may be implemented in processor 1307, such as
The manual programming or computer for realizing logic function generate code.The logic function or any system element remove other functions
Except can with and handle such as audio data sources of simulation electricity, audio or video signal or combinations thereof etc and/or by its turn
Change the number for audiovisual purpose or other digital processing purposes (such as with the compatibility of computer disposal or connected network communication) into
Data source.
Computer system 1300 may include for transmitting the memory 1305 in the bus 1320 of information.It can grasp
The code for promoting computer system to execute any movement or operation as described herein is acted on to be stored in memory 1305.Storage
Device 1305 can be random access memory, read-only memory, programmable storage, hard disk drive or any other type
Volatibility or nonvolatile memory or storage device.
Computer system 300 can also include disk, solid-state driving CD-ROM module 1315.Disk drive module 1315 can be with
Non-provisional or visible computer readable medium including can wherein be embedded in one or more instruction set 1302 (such as software)
1340.In addition, executable one or more of the operation as described herein of instruction 1302.Instruction 1302 is at it by computer
System 1300 can be resided in completely or at least partially during executing in memory 1305 and/or in processor 1307.Above-mentioned number
It can be stored in memory 1305 and/or disk cell 1315 according to library or any other database.
Memory 1305 and processor 1307 can also include computer-readable medium as discussed above." computer
Readable medium ", " computer readable storage medium ", " machine readable media ", " transmitting signal medium " and/or " signaling bearer Jie
Matter " may include include, store, transmitting, propagating or conveying software for instruction executable system, equipment or equipment using or with
Combined use any equipment.The machine readable media property of can choose it is but not limited to electronics, magnetism, optics, electromagnetism, red
Outer or semiconductor system, device, equipment or propagation medium.
In addition, computer system 1300 may include input unit 1325, such as keyboard or mouse, it is configured for
User and any component of system 1300 interact, user's selection or menu entries including showing menu.It can also include
Display 1330, such as liquid crystal display (LCD), cathode-ray tube (CRT) are suitable for transmitting any other display of information
Device.Display 1330, which can be served as, sees interface or specifically serve as and be stored in that processor 1307 is run for user
The interface of software or drive module 1315 in reservoir 1305.
Computer system 1300 may include making it possible to realize the communication interface of the communication via communication network 125
1336.Network 125 may include cable network, wireless network or combinations thereof.1336 network of communication interface can enable to through
It is communicated by any number of communication standard, such as Ethernet AVB, 802.11,802.13,802.20, WiMax or other lead to
Beacon is quasi-.
Therefore, system can be realized with the combination of hardware, software or hardware and software.It can be with centralized fashion extremely
It is realized in a few computer system or in wherein different elements across the distributed way that multiple interconnected computer systems are spread
The system.Any kind of computer system or other equipment for being adapted for carrying out method described herein are all appropriate.Firmly
The typical combination of part and software can be the general-purpose computing system with computer program, the computer program being loaded and
Computer system is controlled when execution, so that it executes method described herein.Such programmed computer can be considered as dedicated meter
Calculation machine.
As described herein, any module or process block are defined as including software, the hardware that can be executed by processor 1307
Or its certain combination.Software module may include be stored in memory 1305 or its memory device can be by processor
1307 or other processors execute instruction.Hardware module may include, commander executable by processor 1307 and/or control with
Just various devices, component, circuit, door, circuit board for executing etc..
Can also will be in system embedment computer program product comprising make it possible to realize operation described herein and its
All features of these operations are able to carry out when being loaded in computer systems.Computer program herein means to be intended to promote
The system with information processing capability is set to execute specific function directly or after any one of the following terms or both
Any expression with any language, code or annotation of one group of instruction: the conversion of another language, code or annotation a) is arrived；B) it uses
The reproduction of different materials form.
Up to the present, the disclosure has been described with reference to the drawings by certain preferred embodiment.It is to be understood, however, that this
It is open that specific embodiment that is shown and providing is provided, but can carry out various modifications within the scope of this disclosure.
Furthermore, it is possible to realize embodiment of the disclosure with software, hardware or combinations thereof.It can be realized with special logic
Hardware component；Software section can be stored in memory and by the appropriate of such as microprocessor or special designs hardware etc
Instruction execution system executes.It will also be recognized by those skilled in the art that can be can be performed with comprising computer in the processor
Instruction and/or control routine are situated between to realize the above method and system, such as in the carrying of such as disk, CD or DVD-ROM etc
The programmable storage of matter or such as read-only memory (firmware) etc or the data of such as optics or electrical signal carrier etc carry
This category code provided on body.The hardware circuit of such as ultra large scale integrated circuit or gate array etc, such as logic can be used
The semiconductor of chip or transistor etc or such as field programmable gate array or programmable logic device etc it is programmable hard
Part device realizes the equipment and its component in the present embodiment, or with the software realization executed by various processors, Huo Zheyong
Combination (for example, with firmware) Lai Shixian of above-mentioned hardware circuit and software.
Although the various embodiments of the disclosure have been described, will become apparent to for a person skilled in the art
Being within the scope of this disclosure can be there are many embodiment and embodiment.Therefore, the disclosure remove according to appended claims and
It is unrestricted except its equivalence.
Claims (23)
1. a kind of system for the data processing in electric system, comprising:
At least one processor；And
At least one processor stores computer executable instructions,
Wherein at least one processor and the computer executable instructions are configured to utilize at least one described processing
Device promotes the system:
By the data in data flow according to power system resource group associated there at one or more data reception nodes
Group is grouped, and the power system resource group is based in part on the hierarchical structure of definition electric power net operation, is used for institute
State the data model of electric system and determination；
The data in the data flow are based on the power system resource group associated there and are sent to one
Or multiple data processing nodes；And
The data in the data flow are based on institute associated there at one or more of data processing nodes
Power system resource group is stated to be handled.
2. system according to claim 1, wherein the data by the data flow are grouped and include:
Based on including defined in measurement point identification information and the data model in data in the data flow
Correlation between power system resource and measurement point identification information, come determine respectively with the data in the data flow
The associated power system resource；And
The classification knot of the electric power net operation defined according to the determining power system resource and in the data model
Structure, to determine the power system resource associated with the data in the stream.
3. system according to claim 1 or 2, wherein the data sent in the data flow depend on currently
The message size of buffering and current waiting time.
4. system according to claim 3, wherein considering that current network is negative when the message size of the Current buffer reaches
When the size threshold of lotus condition, the data in the data flow are sent.
5. system according to claim 3, wherein considering current network load condition when the current waiting time is longer than
Time threshold when, send the data in the data flow.
6. system according to claim 1 or 2, wherein the data in the data flow are cached in described one
To be used to be handled in a or multiple data processing nodes, and the data in the data flow are further stored in
In database, wherein in the case where the fault recovery of data processing node or initialization, corresponding data are from the database
It is loaded into the Cache of the data processing node.
7. system according to claim 1 or 2, wherein the data in the data flow are stored in table per year,
And the title for table includes the information about the year, and wherein the keyword for each data in the data is
It is formed by the mark of power system resource associated there and the time of measuring of the data time, the time of measuring is
In the form of time migration of the data time within this year as unit of millisecond.
8. system according to claim 7, wherein the data in the data flow are according to electric power associated there
System resource and be divided into one or more segmentations, and it is one or more of segmentation based on for described in the data
Keyword and the multiple regions for concurrently and being in a distributed manner written to the database sequentially in time.
9. system according to claim 7, wherein the data in the data flow are by be respectively adapted to its own
Data type and be suitable for data processing format storage in the database.
10. system according to claim 1 or 2, wherein the data in the processing data flow are to be based on dividing
Cloth stream calculation framework, and the data storage is based on distributed data base.
11. a kind of method for the data processing in electric system, comprising:
The data in data flow are based on power system resource group associated there at one or more data reception nodes
Group is grouped, and the power system resource group is based in part on the hierarchical structure of definition electric power net operation, is used for institute
State the data model of electric system and determination；
The data in the data flow are based on the power system resource group associated there and are sent to one
Or multiple data processing nodes；And
The data in the data flow are based on institute associated there at one or more of data processing nodes
Power system resource group is stated to be handled.
12. according to the method for claim 11, wherein the data grouping by the data flow includes:
Based on including defined in measurement point identification information and the data model in data in the data flow
Correlation between power system resource and measurement point identification information, come determine respectively with the data in the data flow
The associated power system resource；And
The classification of the electric power net operation defined according to identified power system resource and in the data model
Structure, to determine the power system resource associated with the data in the stream.
13. method according to claim 11 or 12, wherein the data sent in the data flow depend on
The message size of Current buffer and current waiting time.
14. according to the method for claim 13, wherein when the message size of the Current buffer reaches consideration current network
When the size threshold of loading condiction, the data in the data flow are sent.
15. according to the method for claim 13, wherein considering current network load item when the current waiting time is longer than
When the time threshold of part, the data in the data flow are sent.
16. method according to claim 11 or 12, wherein the data in the data flow are cached in institute
It states to be used to be handled in one or more data processing nodes, and the data in the data flow are further deposited
In the database, wherein in the case where the fault recovery of data processing node or initialization, corresponding data is from the data for storage
Library is loaded into the Cache of the data processing node.
17. method according to claim 11 or 12, wherein the data in the data flow are stored in table per year
In, and the title for table includes the information about the year, and the wherein key for each data in the data
Word is formed by the mark of power system resource associated there and the time of measuring of the data time, when the measurement
Between be time migration of the data time within this year as unit of millisecond in the form of.
18. according to the method for claim 17, wherein the data in the data flow are according to electricity associated there
Force system resource and be divided into one or more segmentations, and it is one or more of segmentation based on the institute for the data
It states keyword and concurrently and is in a distributed manner written to the multiple regions of the database sequentially in time.
19. according to the method for claim 17, wherein the data in the data flow are by be respectively adapted to it certainly
Body data type and be suitable for data processing format storage in the database.
20. method according to claim 11 or 12, wherein the data in the processing data flow are to be based on
Distributed stream computing architecture, and the data storage is based on distributed data base.
21. a kind of equipment for the data processing in electric system, the equipment include:
Data grouping module, be configured at one or more data reception nodes for the data in data flow being based on and its
Associated power system resource group is grouped, and the power system resource group is to be based in part on to define the electric power
The hierarchical structure of net operation, for electric system data model and determination；
Data transmission blocks are configured to the data in the data flow being based on the power train associated there
System group of resources and be sent to one or more data processing nodes；And
Data processing module, being configured at one or more of data processing nodes will be described in the data flow
Data are based on the power system resource group associated there and are handled.
22. a kind of equipment for the data processing in electric system, the equipment include:
For the data in data flow to be provided based on electric system associated there at one or more data reception nodes
The device that source group is grouped, the power system resource group are to be based in part on the classification for defining the electric power net operation
Structure, for electric system data model and determination；
It is sent to for the data in the data flow to be based on the power system resource group associated there
The device of one or more data processing nodes；And
It is associated with it for being based on the data in the data flow at one or more of data processing nodes
The device that is handled of the power system resource group.
23. a kind of visible computer readable medium with multiple instruction, described instruction can be executed by processor to execute electric power
Data processing in system, the visible computer readable medium include being configured to execute according to claim 1 in 1 to 20
The instruction of the step of described in any item methods.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2014/085227 WO2016029367A1 (en) | 2014-08-26 | 2014-08-26 | System, method and apparatuses for data processing in power system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107078541A CN107078541A (en) | 2017-08-18 |
| CN107078541B true CN107078541B (en) | 2019-11-22 |
Family
ID=55398582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480082936.1A Active CN107078541B (en) | 2014-08-26 | 2014-08-26 | System, method and apparatus for the data processing in electric system |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN107078541B (en) |
| AU (1) | AU2014405046B2 (en) |
| WO (1) | WO2016029367A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115081158B (en) * | 2022-08-23 | 2022-11-11 | 成都中科合迅科技有限公司 | Graphic element display method based on digital expression |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1761124A (en) * | 2005-09-09 | 2006-04-19 | 东南大学 | Full networked digital protection platform |
| CN201518429U (en) * | 2009-10-26 | 2010-06-30 | 江西省电力科学研究院 | Electric energy qualitative data concentrator for digitalization transforming plant |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10845399B2 (en) * | 2007-04-03 | 2020-11-24 | Electro Industries/Gaugetech | System and method for performing data transfers in an intelligent electronic device |
| EP2681822B1 (en) * | 2011-03-03 | 2019-07-03 | ABB Schweiz AG | A control device for controlling a circuit breaker, and methods |
| US20120310559A1 (en) * | 2011-05-31 | 2012-12-06 | Cisco Technology, Inc. | Distributed data collection for utility grids |
| US20130219279A1 (en) * | 2012-02-21 | 2013-08-22 | Ambient Corporation | Aggregating nodes for efficient network management system visualization and operations |
| US8964360B2 (en) * | 2013-02-06 | 2015-02-24 | Jonathan D. Trout | System to connect and multiplex sensor signals |
-
2014
- 2014-08-26 WO PCT/CN2014/085227 patent/WO2016029367A1/en active Application Filing
- 2014-08-26 AU AU2014405046A patent/AU2014405046B2/en active Active
- 2014-08-26 CN CN201480082936.1A patent/CN107078541B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1761124A (en) * | 2005-09-09 | 2006-04-19 | 东南大学 | Full networked digital protection platform |
| CN201518429U (en) * | 2009-10-26 | 2010-06-30 | 江西省电力科学研究院 | Electric energy qualitative data concentrator for digitalization transforming plant |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2016029367A1 (en) | 2016-03-03 |
| AU2014405046B2 (en) | 2018-06-28 |
| AU2014405046A1 (en) | 2017-04-13 |
| CN107078541A (en) | 2017-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Munshi et al. | Big data framework for analytics in smart grids | |
| CN106980669B (en) | A kind of storage of data, acquisition methods and device | |
| US9501555B2 (en) | Real-time data management for a power grid | |
| CN109635127B (en) | Power equipment portrait knowledge map construction method based on big data technology | |
| CN106651188A (en) | Electric transmission and transformation device multi-source state assessment data processing method and application thereof | |
| CN105589874A (en) | ETL task dependence relationship detecting method and device and ETL tool | |
| CN109284346A (en) | A kind of distribution network planning method and device based on cloud computing | |
| BR112013003660B1 (en) | CENTRAL POWER INSTALLATION SYSTEM | |
| CN103489139A (en) | Comprehensive analysis and management system for urban distribution network planning | |
| CN107078541B (en) | System, method and apparatus for the data processing in electric system | |
| CN104299170B (en) | Intermittent energy source mass data processing method | |
| CN107357919A (en) | User behaviors log inquiry system and method | |
| CN110969359A (en) | Power distribution network parallel topology analysis method and device, computer equipment and storage medium | |
| CN107545007A (en) | Electric power big data quick-searching engine | |
| Wang et al. | Block storage optimization and parallel data processing and analysis of product big data based on the hadoop platform | |
| CN103955492A (en) | Method for storing data in electrical power system | |
| Singh et al. | Storage and analysis of synchrophasor data for event detection in Indian power system using hadoop ecosystem | |
| Heng et al. | Integrated analytics system for electric industry asset management | |
| Zhang et al. | Open architecture design of universal terminal for multiple measurement and control scenarios of distribution network | |
| CN111553040A (en) | Power grid topology analysis high-performance computing method and device based on GPU acceleration | |
| CN112257913A (en) | Medium-voltage line load value prediction method, device, equipment and storage medium | |
| Stefan | Automation of Smart Grid operations through spatio-temporal data-driven systems | |
| US20140074869A1 (en) | Autoclassifying compound documents for enhanced metadata search | |
| Kovačević et al. | Combining real-time processing streams to enable demand response in smart grids | |
| CN108763665A (en) | Grid simulation analyzes date storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |