CN115982232A - Hadoop-based power grid data processing method and system - Google Patents

Hadoop-based power grid data processing method and system Download PDF

Info

Publication number
CN115982232A
CN115982232A CN202211595468.0A CN202211595468A CN115982232A CN 115982232 A CN115982232 A CN 115982232A CN 202211595468 A CN202211595468 A CN 202211595468A CN 115982232 A CN115982232 A CN 115982232A
Authority
CN
China
Prior art keywords
data
power grid
hadoop
information
processing method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211595468.0A
Other languages
Chinese (zh)
Inventor
王捷
李晶
黄杰
崔一铂
王晋
朱国威
刘畅
喻潇
周亮
唐泽洋
田里
徐江珮
龙凤
董重重
苏昊扬
徐成伟
赵环
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Hubei Electric Power Co Ltd
Electric Power Research Institute of State Grid Hubei Electric Power Co Ltd
Original Assignee
State Grid Hubei Electric Power Co Ltd
Electric Power Research Institute of State Grid Hubei Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Hubei Electric Power Co Ltd, Electric Power Research Institute of State Grid Hubei Electric Power Co Ltd filed Critical State Grid Hubei Electric Power Co Ltd
Priority to CN202211595468.0A priority Critical patent/CN115982232A/en
Publication of CN115982232A publication Critical patent/CN115982232A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04SSYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a power grid data processing method and system based on Hadoop, wherein the method comprises the following steps: step one: power grid big data are collected through a Hadoop-based power grid data mining and analyzing technology, wherein the power grid big data comprise real-time data information, equipment parameter data and power generation and load data of a power grid; step two: storing and managing the collected power grid big data on a power grid big data storage platform based on a MapReduce technology to store data security situations; step three: and establishing a zero trust framework to realize the safety protection of the power terminal. The invention can improve the data quality, improve the data storage safety, improve the terminal safety protection, reduce the probability of data tampering, damage and leakage, promote data circulation, give full play to the data value of the power grid, meet the national requirements on data sharing and exchange, and provide support for utilizing data innovation, mining data dividend and promoting data economy.

Description

Hadoop-based power grid data processing method and system
Technical Field
The invention relates to the technical field of intelligent power grids, in particular to a power grid data processing method and system based on Hadoop.
Background
Along with the increasing demand of social electric power energy, the intellectualization of the power grid is also under continuous deep development; the energy utilization control system at the client side in the intelligent power grid is used as a link for connecting the client with the intelligent energy service platform, is an important means for supporting the ubiquitous power internet of things at the client side, and is an execution unit for implementing various comprehensive energy services such as demand response, energy efficiency improvement and the like.
With the continuous expansion of the scale of the power grid, the complexity of the operation of the power grid is continuously increased, the problem of data security risk is increasingly highlighted, and for the defects that the common requirements of the power grid for data acquisition monitoring, demand response and the like are not known sufficiently, the development cost is high, the transportability and the reusability are poor and the like, the storage of the security situation of the power grid data, the mining and analysis of the power grid data and the security protection of the power terminal are necessarily realized by using a large data analysis-based related technology.
Disclosure of Invention
The invention aims to provide a power grid data processing method and system based on Hadoop, which are used for realizing power grid data security situation storage, power grid data mining and analysis and power terminal security protection and simultaneously solving the problems of high development cost, poor portability and reusability and the like in the prior art.
A power grid data processing method based on Hadoop comprises the following steps:
the method comprises the following steps: the method comprises the steps that power grid big data are collected through a Hadoop-based power grid data mining and analyzing technology, wherein the power grid big data comprise real-time data information, equipment parameter data and power generation and load data of a power grid;
step two: storing and managing the collected power grid big data on a power grid big data storage platform based on a MapReduce technology to store data security situations;
step three: and establishing a zero trust framework to realize the safety protection of the power terminal.
Furthermore, the Hadoop-based power grid data mining and analyzing technology is realized by adopting a data acquisition layer, a data storage layer, a service application layer and a user layer;
the data acquisition layer adopts a distributed directional acquisition system architecture, takes terminal stations in different networks as a basic task unit for network data acquisition to acquire original network data and gathers and transmits the original network data to the data storage layer, wherein each basic task unit adopts an independent acquisition rule and strategy;
the data storage layer is used for finishing the aggregation, storage and original processing of original data of the data and providing different types of function calling services, and the data storage layer is realized by adopting a Hadoop framework;
the service application layer is used for calling and analyzing the network data processed by the data storage layer to realize the stripping of the public component and the individual service application component and transmitting the result of the network data analysis to the user layer for real-time display;
and the user layer is used for transmitting and displaying the data information of the service application layer.
Further, the basic task unit includes a data acquisition unit, which is configured to acquire data by a dynamic web page acquisition method and a web page information extraction method, extract information by a method based on a row-block distribution function, and further acquire data.
Furthermore, the data acquisition unit acquires the Feed addresses through the breadth traversal site, acquires information corresponding to each Feed address in real time, tracks updated information, and acquires the information in an incremental updating mode.
Furthermore, the acquisition rules and strategies comprise a vertical search template semi-automatic generation technology, a dynamic page optimization access technology and an intelligent capture process scheduling strategy.
Further, the processing of the original data in the data storage layer includes partitioning the data to be processed by using a window technique, describing a change of stream data by using a sliding window model, and saving a mode in the original data by using the sliding window model.
Further, a mode in the original data is saved by using a sliding window model, which specifically comprises the following steps:
storing the mode of the unchanged partial data into a sliding window according to the changed block data of the data; respectively calculating the modes of adding and deleting partial data; updating the mode stored in the sliding window according to the mode of the changed partial data;
using a multi-window method to support the online mining request of a user; the multi-window method divides the data stream into a plurality of segments with fixed length, each segment forms a window, when the number of windows in the memory reaches a certain number, the windows are combined to form a window with higher outline level along with the inflow of the data stream, a plurality of windows with different outline levels form a hierarchical structure, and at the moment, each window is equivalent to a snapshot of data between two predefined time stamps on the data stream.
Further, the data security situation storage based on the MapReduce technology comprises the following steps:
step 2.1: calling data information of a user layer and inputting the data information into a user program;
step 2.2: dividing an input file of a user program into M parts by a MapReduce library, wherein M is defined by a user;
step 2.3: reading input data of the corresponding fragments by a worker to which Map operation is allocated, extracting key value pairs from the input data by the Map operation, transmitting each key value pair to a Map function as a parameter, and caching middle key value pairs generated by the Map function in a memory;
step 2.3: the cached intermediate key values are periodically written into a local disk and are divided into R areas, the size of R is defined by a user, and each area corresponds to one Reduce operation in the future; the position of the middle key-value pair is notified to a master, and the master is responsible for forwarding the information to a Reduce worker;
step 2.5: the master informs the specific position of a partition responsible for a worker distributing Reduce operation, and after the Reduce worker reads all responsible intermediate key values, the intermediate key values are sorted so that the key value pairs of the same key are gathered together;
step 2.5: traversing the sorted intermediate key value pairs by the reduce worker, transmitting the key and the associated value to a reduce function for each unique key, and adding the output generated by the reduce function into the output file of the partition;
step 2.7: when all Map and Reduce jobs are completed, the master wakes up the user program, and the MapReduce function call returns the code of the user program.
Further, the establishing of the zero trust framework to realize the safety protection of the power terminal includes the following steps:
step 3.1, constructing a zero trust module, collecting equipment information of the electric power terminal equipment, carrying out trust grading according to the collected equipment information, giving a trust value, evaluating the electric power terminal equipment according to the trust value, and dividing the electric power terminal equipment into trusted equipment and abnormal equipment;
step 3.2, data acquisition is carried out on the trusted equipment in the step 3.1, and acquired data are obtained;
3.3, constructing a security situation perception module, carrying out situation perception on the data collected in the step 3.2, and converting the collected data into perception data after the perception is qualified;
step 3.4, a real-time management and control module is constructed, the perception data in the step 3.3 are managed and controlled, and a safety instruction is generated;
and 3.5, issuing the safety command in the step 3.4 to the electric power terminal equipment, and performing safety protection and safety reinforcement on the electric power terminal equipment.
A Hadoop-based power grid data processing system comprises: a computer-readable storage medium and a processor;
the computer-readable storage medium is used for storing executable instructions;
the processor is used for reading the executable instructions stored in the computer-readable storage medium and executing the Hadoop-based power grid data processing method.
The invention improves the safety protection capability, improves the identification accuracy of data safety events, and the traceability timeliness for power grid data safety services, reduces the probability of data tampering, damage and leakage, promotes data circulation, fully exerts the power grid data value, meets the national requirements on data sharing and exchange, and provides support for utilizing data innovation, mining data dividends and promoting data economy.
Drawings
FIG. 1 is a flow chart of a Hadoop-based power grid data processing method according to an embodiment of the invention;
fig. 2 is a flowchart of data security situation storage based on MapReduce technology in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Referring to fig. 1, in a first aspect of the present invention, a method for processing power grid data based on Hadoop is provided, including the following steps:
the method comprises the following steps: the method comprises the steps that power grid big data are collected through a Hadoop-based power grid data mining and analyzing technology, wherein the power grid big data comprise real-time data information, equipment parameter data and power generation and load data of a power grid;
step two: storing and managing the collected power grid big data on a power grid big data storage platform to store data security situations based on a MapReduce technology;
step three: and establishing a zero trust framework to realize the safety protection of the power terminal.
The Hadoop-based power grid data mining and analyzing technology is realized by adopting a data acquisition layer, a data storage layer, a service application layer and a user layer.
The data acquisition layer adopts a distributed directional acquisition system architecture and takes terminal stations in different networks as a basic task unit for power grid data acquisition to acquire real-time data information, equipment parameter data, power generation and load data and gather and transmit the real-time data information, the equipment parameter data and the power generation and load data to a data storage layer; the basic task unit comprises a data acquisition unit, wherein the data acquisition unit is used for acquiring data through a dynamic webpage acquisition method and a webpage information extraction method, extracting information through a method based on a line-block distribution function and further acquiring data, and specifically, the data acquisition unit acquires Feed addresses through a breadth traversal site, acquires information corresponding to each Feed address in real time, tracks and updates the information and acquires the information in an increment updating mode. Each basic task unit adopts an independent acquisition rule and strategy; the acquisition rules and strategies comprise a vertical search template semi-automatic generation technology, a dynamic page optimization access technology and an intelligent capture process scheduling strategy.
The data storage layer is used for finishing the aggregation, storage and original processing of original data of the data and providing different types of function calling services; the data storage layer is realized by adopting a Hadoop frame;
and the service application layer is used for calling and analyzing the data processed by the data storage layer to realize the separation of the public component and the individual service application component and transmitting the result of the network data analysis to the user layer for real-time display.
And the user layer is used for transmitting and displaying the data information of the service application layer.
The data security situation storage based on the MapReduce technology is realized by adopting a distributed file system (HDFS) and a MapReduce, wherein the HDFS is a file system of Hadoop and is used for storing oversized files; mapReduce is a Hadoop parallel programming model and is used for performing deep analysis on data stored on a distributed file system (HDFS).
As shown in fig. 2, the data security posture storage based on the MapReduce technology includes the following steps:
step 2.1: calling data information of a user layer and inputting the data information into a user program;
step 2.2: dividing an input file of a user program into M parts by a MapReduce library, wherein M is defined by a user;
step 2.3: starting to read the input data of the corresponding fragments by the worker allocated with the Map operation, wherein the number of the Map operation is determined by M and is in one-to-one correspondence with split; the Map operation extracts key value pairs from input data, each key value pair is used as a parameter and is transmitted to a Map function, and an intermediate key value pair generated by the Map function is cached in a memory;
step 2.3: the cached intermediate key value is regularly written into a local disk and is divided into R areas, the size of R is defined by a user, and each area corresponds to a Reduce operation in the future; the positions of the intermediate key value pairs are notified to a master, and the master is responsible for forwarding information to a Reduce worker;
step 2.5: the master informs the specific position of the partition responsible for the worker assigned with the Reduce operation, and when the Reduce worker reads all the intermediate key-value pairs responsible for the Reduce worker, the intermediate key-value pairs are sorted firstly, so that the key-value pairs of the same key are gathered together;
step 2.5: traversing the sorted intermediate key value pairs by the reduce worker, transmitting the key and the associated value to a reduce function for each unique key, and adding the output generated by the reduce function into the output file of the partition;
step 2.7: when all Map and Reduce jobs are completed, the master wakes up the user program, and the MapReduce function call returns the code of the user program.
The method for establishing the zero trust framework to realize the safety protection of the power terminal comprises the following steps:
step 3.1: constructing a zero trust module, and collecting equipment information of the power terminal equipment; carrying out trust scoring according to the acquired equipment information and giving a trust value; evaluating the electric terminal equipment according to the trust value, and dividing the electric terminal equipment into trusted equipment and abnormal equipment;
the flow of the zero trust module for acquiring the equipment information is as follows: reading equipment data, reading a rule file, analyzing a rule base and collecting equipment information; meanwhile, the zero trust module carries out continuous dynamic equipment identity authentication on the power terminal equipment so as to block the virtual standby information; the trust value is an index of identity authentication, and comprehensive scoring is obtained according to basic attributes and access time delay of the equipment; the maintenance of trust values includes the following:
(1) The maximum trust value is M, and the minimum trust value is N; m > N
(2) The threshold value of the trust value is H, if the threshold value is higher than or equal to H, the user is a legal user, and if the threshold value is lower than H, the user is an illegal user;
(3) Adding T to the trust value after each successful verification;
(4) Subtracting T from the trust value when each verification fails;
the trust value comprises a direct trust value, a time delay evaluation trust value and an abnormal behavior evaluation trust value, and the calculation formula is as follows:
T=T d +T t +T a
t is a trust value, T d As a direct trust value, T t Evaluating trust value, T, for time delay a Evaluating a trust value for the abnormal behavior;
the direct trust value is an S-shaped function, and the calculation formula is as follows:
Figure BDA0003997101630000091
wherein T is d F is a direct trust value, and is a direct trust value constraint coefficient of different equipment; the time delay evaluation trust value and the abnormal behavior evaluation trust value form an indirect trust value;
the time delay evaluation trust value is evaluated according to the response time of the equipment, and the calculation formula is as follows:
Figure BDA0003997101630000092
wherein T is t Evaluating a trust value for time delay, wherein tau is the maximum allowable delay of equipment response, and D is information transmission delay;
the evaluation trust value of the abnormal behavior is evaluated according to the proportion of the abnormal behavior and the normal behavior of the equipment, and the calculation formula is as follows:
Figure BDA0003997101630000101
wherein T is a A trust value is evaluated for the abnormal behavior,
A u for abnormal behavior amount, A n Is the normal behavior amount;
step 3.2: data acquisition is carried out on the trusted equipment in the step 3.1, and acquired data are obtained;
step 3.3: a safety situation perception module is constructed, and situation perception is carried out on the data collected in the step 3.2; when the perception is qualified, converting the acquired data into perception data;
the situation awareness comprises intrusion detection, vulnerability awareness, file integrity detection and log monitoring operation;
step 3.4: a real-time control module is constructed, the sensing data in the step 3.3 are controlled, and a safety instruction is generated;
step 3.5: the safety instruction in the step 3.4 is issued to the electric power terminal equipment, and safety protection and safety reinforcement are carried out on the electric power terminal equipment;
the invention can improve the data quality, improve the data storage safety, improve the terminal safety protection, reduce the probability of data tampering, damage and leakage, promote data circulation, give full play to the data value of the power grid, meet the national requirements on data sharing and exchange, and provide support for utilizing data innovation, mining data dividend and promoting data economy.
In another aspect, the present invention provides a power grid data processing system based on Hadoop, including: a computer-readable storage medium and a processor;
the computer readable storage medium is used for storing executable instructions;
the processor is configured to read executable instructions stored in the computer-readable storage medium, and execute the Hadoop-based power grid data processing method according to the first aspect.
In another aspect, the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method for Hadoop-based grid data processing according to the first aspect.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (10)

1. A power grid data processing method based on Hadoop is characterized by comprising the following steps:
the method comprises the following steps: the method comprises the steps that power grid big data are collected through a Hadoop-based power grid data mining and analyzing technology, wherein the power grid big data comprise real-time data information, equipment parameter data and power generation and load data of a power grid;
step two: storing and managing the collected power grid big data on a power grid big data storage platform based on a MapReduce technology to store data security situations;
step three: and establishing a zero trust framework to realize the safety protection of the power terminal.
2. The Hadoop-based power grid data processing method according to claim 1, wherein the Hadoop-based power grid data mining and analyzing technology is implemented by a data acquisition layer, a data storage layer, a service application layer and a user layer;
the data acquisition layer adopts a distributed directional acquisition system architecture, and terminal stations in different networks are used as a basic task unit for network data acquisition to acquire original network data and gather and transmit the original network data to the data storage layer, wherein each basic task unit adopts an independent acquisition rule and strategy;
the data storage layer is used for finishing the aggregation, storage and original processing of original data of the data and providing different types of function calling services, and the data storage layer is realized by adopting a Hadoop framework;
the service application layer is used for calling and analyzing the network data processed by the data storage layer to realize the stripping of the public component and the individual service application component and transmitting the result of the network data analysis to the user layer for real-time display;
and the user layer is used for transmitting and displaying the data information of the service application layer.
3. The Hadoop-based power grid data processing method as claimed in claim 2, wherein the basic task unit comprises a data acquisition unit for acquiring data by a dynamic web page acquisition method and a web page information extraction method, and extracting information by a method based on a row-block distribution function to further acquire data.
4. The Hadoop-based power grid data processing method as claimed in claim 3, wherein the data acquisition unit acquires Feed addresses through a breadth traversal site, acquires information corresponding to each Feed address in real time, tracks updated information, and acquires information in an incremental updating manner.
5. The Hadoop-based power grid data processing method as claimed in claim 2, wherein the collection rules and policies include vertical search template semi-automatic generation technology, dynamic page optimization access technology, and intelligent capture process scheduling policy.
6. The Hadoop-based power grid data processing method according to claim 2, wherein the processing of the raw data in the data storage layer comprises blocking the data to be processed by using a window technique, describing changes in the stream data by using a sliding window model, and saving a pattern in the original data by using the sliding window model.
7. The Hadoop-based power grid data processing method as claimed in claim 6, wherein a sliding window model is used to store patterns in the original data, specifically:
storing the mode of the unchanged partial data into a sliding window according to the changed block data of the data; respectively calculating modes of adding and deleting partial data; updating the mode stored in the sliding window according to the mode of the changed part of data;
using a multi-window method to support the online mining request of a user; the multi-window method divides the data stream into a plurality of segments with fixed length, each segment forms a window, when the number of windows in the memory reaches a certain number, the windows are combined to form a window with higher summary level along with the inflow of the data stream, the windows with different summary levels form a hierarchical structure, and at the moment, each window is equivalent to a snapshot of data between two predefined time stamps on the data stream.
8. The Hadoop-based power grid data processing method according to claim 1, wherein the data security situation storage based on the MapReduce technology comprises the following steps:
step 2.1: calling data information of a user layer and inputting the data information into a user program;
step 2.2: dividing an input file of a user program into M parts by a MapReduce library, wherein M is defined by a user;
step 2.3: reading input data of the corresponding fragments by a worker to which Map operation is allocated, extracting key value pairs from the input data by the Map operation, transmitting each key value pair to a Map function as a parameter, and caching middle key value pairs generated by the Map function in a memory;
step 2.3: the cached intermediate key value is periodically written into a local disk and is divided into R areas, the size of R is defined by a user, and each area corresponds to a Reduce operation in the future; the position of the middle key-value pair is notified to a master, and the master is responsible for forwarding the information to a Reduce worker;
step 2.5: the master informs the specific position of a partition responsible for a worker distributing Reduce operation, and after the Reduce worker reads all responsible intermediate key values, the intermediate key values are sorted so that the key value pairs of the same key are gathered together;
step 2.5: traversing the sorted intermediate key value pairs by the reduce worker, transmitting the key and the associated value to a reduce function for each unique key, and adding the output generated by the reduce function into the output file of the partition;
step 2.7: when all Map and Reduce jobs are completed, the master wakes up the user program, and the MapReduce function call returns the code of the user program.
9. The Hadoop-based power grid data processing method as claimed in claim 1, wherein the establishing of the zero trust framework to realize the power terminal security protection comprises the following steps:
step 3.1, a zero trust module is constructed, equipment information of the electric power terminal equipment is collected, trust scoring is carried out according to the collected equipment information, a trust value is given, the electric power terminal equipment is evaluated according to the trust value, and the electric power terminal equipment is divided into trusted equipment and abnormal equipment;
step 3.2, data acquisition is carried out on the trusted equipment in the step 3.1, and acquired data are obtained;
3.3, constructing a security situation awareness module, carrying out situation awareness on the acquired data in the step 3.2, and converting the acquired data into awareness data when the awareness is qualified;
step 3.4, a real-time management and control module is constructed, the perception data in the step 3.3 are managed and controlled, and a safety instruction is generated;
and 3.5, issuing the safety command in the step 3.4 to the electric power terminal equipment, and carrying out safety protection and safety reinforcement on the electric power terminal equipment.
10. A Hadoop-based power grid data processing system comprises: a computer-readable storage medium and a processor;
the computer-readable storage medium is used for storing executable instructions;
the processor is used for reading executable instructions stored in the computer-readable storage medium and executing the Hadoop-based power grid data processing method of any one of claims 1 to 9.
CN202211595468.0A 2022-12-13 2022-12-13 Hadoop-based power grid data processing method and system Pending CN115982232A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211595468.0A CN115982232A (en) 2022-12-13 2022-12-13 Hadoop-based power grid data processing method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211595468.0A CN115982232A (en) 2022-12-13 2022-12-13 Hadoop-based power grid data processing method and system

Publications (1)

Publication Number Publication Date
CN115982232A true CN115982232A (en) 2023-04-18

Family

ID=85965753

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211595468.0A Pending CN115982232A (en) 2022-12-13 2022-12-13 Hadoop-based power grid data processing method and system

Country Status (1)

Country Link
CN (1) CN115982232A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117235054A (en) * 2023-08-20 2023-12-15 国网湖北省电力有限公司武汉供电公司 Power grid data security management method, system and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117235054A (en) * 2023-08-20 2023-12-15 国网湖北省电力有限公司武汉供电公司 Power grid data security management method, system and storage medium

Similar Documents

Publication Publication Date Title
CN112102111B (en) Intelligent processing system for power plant data
CN112637127B (en) Cross-block chain communication method and device
CN107645403A (en) Terminal rule engine apparatus, terminal rule operation method
CN110225095B (en) Data processing method, device and system
CN112650762A (en) Data quality monitoring method and device, electronic equipment and storage medium
CN111382956A (en) Enterprise group relationship mining method and device
CN107330580A (en) Power marketing Base data platform construction method
Ma et al. How China’s linked urban–rural construction land policy impacts rural landscape patterns: A simulation study in Tianjin, China
CN110913032A (en) Method and platform for realizing DNS (Domain name System) domain name request analysis by using power grid threat information
CN114398669A (en) Joint credit scoring method and device based on privacy protection calculation and cross-organization
CN106502875A (en) A kind of daily record generation method and system based on cloud computing
CN104750826A (en) Structural data resource metadata automatically-identifying and dynamically-registering method
CN104022913A (en) Test method and device for data cluster
CN115529315B (en) Cloud edge cooperative system
CN115982232A (en) Hadoop-based power grid data processing method and system
CN107347064A (en) Cloud computing platform Tendency Prediction method based on neural network algorithm
CN114356712B (en) Data processing method, apparatus, device, readable storage medium, and program product
Manohar Design of distributed database system based on improved DES algorithm
CN111708844B (en) Data processing method, device and equipment based on block chain
CN109194700B (en) Flow control method and related device
Garg et al. Smart Infrastructure: Role of Information Technology in Smart Cities
Guo Design of anti-intrusion storage system for agricultural products sales data based on multi layer cross entropy
CN113301031A (en) Internet of things security management method and system
CN103916450A (en) Foundation architecture of high-performance data warehouse
US20240037004A1 (en) Preserving Enterprise Artifacts Using Digital Twin Technology And Intelligent Smart Contracts

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination