CN112434924B - Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data - Google Patents
Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data Download PDFInfo
- Publication number
- CN112434924B CN112434924B CN202011291361.8A CN202011291361A CN112434924B CN 112434924 B CN112434924 B CN 112434924B CN 202011291361 A CN202011291361 A CN 202011291361A CN 112434924 B CN112434924 B CN 112434924B
- Authority
- CN
- China
- Prior art keywords
- data
- service
- layer
- suite
- monitoring platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 74
- 238000012544 monitoring process Methods 0.000 title claims abstract description 38
- 238000004458 analytical method Methods 0.000 claims abstract description 19
- 230000000007 visual effect Effects 0.000 claims abstract description 18
- 238000013500 data storage Methods 0.000 claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 11
- 238000007418 data mining Methods 0.000 claims abstract description 11
- 230000010354 integration Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 10
- 230000000903 blocking effect Effects 0.000 claims description 9
- 238000012800 visualization Methods 0.000 claims description 9
- 230000002688 persistence Effects 0.000 claims description 8
- 230000003993 interaction Effects 0.000 claims description 3
- 238000013523 data management Methods 0.000 abstract description 3
- 238000007619 statistical method Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 15
- 238000007726 management method Methods 0.000 description 14
- 230000002159 abnormal effect Effects 0.000 description 9
- 238000011161 development Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000013439 planning Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Marketing (AREA)
- Tourism & Hospitality (AREA)
- Entrepreneurship & Innovation (AREA)
- Health & Medical Sciences (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a risk inspection monitoring platform based on a cloud platform under full-electric-network multi-source data, which comprises an infrastructure layer, a basic software layer, a service supporting layer and an application layer; the infrastructure layer comprises an intelligent electric meter acquisition end and a local edge calculation terminal; the basic software layer comprises database software and a Web middle layer, and provides big data storage service for the risk inspection monitoring platform through the database software and big data basic service for the risk inspection monitoring platform through the Web middle layer; the service support layer provides a big data service suite, and the big data service suite comprises a data integration suite, a data governance suite, a query statistics suite, a numerical analysis suite and a data mining suite; and the risk inspection monitoring platform realizes visual intelligent inspection service through the application layer. The invention completely covers the life cycle of the whole data risk inspection from data integration, data management, data storage, data query, data statistical analysis and data mining.
Description
Technical Field
The invention belongs to the technical field of intelligent inspection of a power grid, and particularly relates to a risk inspection monitoring platform based on a cloud platform under multi-source data of a full power grid.
Background
The method has the advantages that the construction of 'two-coverage' is completed, the deepened application of data is realized, a large amount of electric energy data is generated, a new round of marketing and transformation is more targeted, the outstanding problems of low voltage, neck blockage, power failure and the like in poor areas are effectively solved, and the requirements of the masses from 'power on' to 'power on' are met. Meanwhile, the intelligent service level of the power grid is further improved, and a user can enjoy better power supply service. Under the background of the transformation from the current southern power grid to the digitization, the data value after the 'double coverage' is urgently dug, so that the data value can serve for the development of marketing business work, and a solid data base is provided for the intellectualization of inspection business.
In contrast, the chinese patent application with application number CN201911196973.6 proposes a power marketing inspection management method, which includes: acquiring historical marketing service data of the power marketing system, and carrying out standardized processing on the historical marketing service data; analyzing the historical marketing service data after the standardization processing, and extracting abnormal characteristic data; processing abnormal characteristic data based on machine learning and deep learning, establishing an abnormal rechecking positioning algorithm model, completing self-learning of the abnormal rechecking positioning algorithm model, and obtaining an inspection knowledge map; processing the marketing inspection rule, the problem type corresponding to the marketing inspection rule and the data source corresponding to the problem type based on the inspection knowledge graph, positioning abnormal nodes of current marketing service data, obtaining a marketing inspection result, and establishing the inspection knowledge graph for inspection, so that the traceability is realized, and the accurate inspection of the marketing full service, the full data, the full specialty and the full risk is realized.
The Chinese patent application CN110378808A discloses a power marketing inspection method based on gene recombination and feature clustering, which comprises the following steps: step 100, data cleaning, wherein after provincial inspection is carried out on imported work order data, two results are generated after the first data cleaning, namely effective exception and invalid exception of the work order data; step 200, the valid and abnormal data are issued to a city level unit for data processing, and the invalid and abnormal data are directly processed; and step 300, intelligently processing the data processing of the city level units by an intelligent inspection method. The invention can effectively improve the accuracy of on-line inspection and greatly improve the efficiency of on-line inspection operation.
However, the processing mode of the existing marketing system for inspecting the business is strong in subjectivity and insufficient in intellectualization, which not only increases the workload of business personnel, but also cannot meet the requirement of lean management of the inspection.
The current situation of the inspection module of the current marketing management system is specifically analyzed as follows:
1. the rules are not refined enough, and each problem point cannot be positioned in detail;
2. the method is realized based on a relational database, has large performance loss and cannot perform effective parallel processing;
3. the sampling time is long, and the efficiency is influenced;
4. the problems screened out by the rules are more, and the monthly checking traffic is limited and cannot be effectively tracked;
5. the inspection problem and experience are not managed in a knowledge mode, and the problem of the same kind cannot be processed quickly by effectively utilizing knowledge when being processed by different people;
6. the risk monitoring and controlling capability needs to be improved.
Disclosure of Invention
In order to solve the technical problems, the invention provides a risk inspection monitoring platform based on a cloud platform under full-electric-network multi-source data, which comprises an infrastructure layer, a basic software layer, a service supporting layer and an application layer; the infrastructure layer comprises an intelligent electric meter acquisition end and a local edge calculation terminal; the basic software layer comprises database software and a Web middle layer, and provides big data storage service for the risk inspection monitoring platform through the database software and big data basic service for the risk inspection monitoring platform through the Web middle layer; the service support layer provides a big data service suite, and the big data service suite comprises a data set suite, a data governance suite, a query statistics suite, a numerical analysis suite and a data mining suite; and the risk inspection monitoring platform realizes visual intelligent inspection service through the application layer.
The technical scheme of the invention is divided into a data acquisition layer, a data storage layer, a data calculation layer, a data analysis layer and a platform service layer from the flow of data acquisition, storage, communication and use. The platform is integrally deployed on the cloud platform, and the computing and storage services provided by the cloud platform are used, so that the platform naturally has elastic expansion and contraction capability. The platform is a data convergence center, a unique source of unified data and data service, is a development support platform for large data application of each company, and is a carrier for metadata management, data quality management and data standard management.
Various basic technical components required by the project for processing mass data are integrated according to the big data basic platform, the whole life cycle of data processing from data integration, data management, data storage, data query, data statistical analysis and data mining is completely covered, and the necessary analysis technologies of the project such as memory calculation and anomaly analysis are covered.
Specifically, the invention provides a risk inspection monitoring platform based on a cloud platform under full-electric-network multi-source data, and the risk inspection monitoring platform comprises an infrastructure layer, a basic software layer, a service support layer and an application layer.
The infrastructure layer comprises M intelligent electric meter acquisition ends distributed at different positions in a power grid target range and N local edge computing terminals, wherein M > N >1, and M and N are positive integers;
the basic facility layer is communicated with the basic software layer through a distributed file system, the basic software layer comprises database software and a Web middle layer, big data storage service is provided for the risk inspection monitoring platform through the database software, big data basic service is provided for the risk inspection monitoring platform through the Web middle layer, and the big data basic service comprises SQL query, distributed memory calculation, streaming calculation, numerical analysis and data mining;
the service supporting layer provides a big data service suite, and the big data service suite comprises a data integration suite, a data governance suite, a query statistics suite, a numerical analysis suite and a data mining suite;
and the risk inspection monitoring platform realizes visual intelligent inspection service through the application layer.
The application layer also provides a visual configuration service component, and the visual configuration service component sets grouping visual parameters;
based on the grouping visualization parameters, grouping the M intelligent electric meter acquisition ends to obtain K groups, wherein each group comprises a plurality of intelligent electric meter acquisition ends; k is less than or equal to N;
and sending the power grid state data collected by the plurality of intelligent electric meter collecting ends included in each group to at least one local edge computing terminal corresponding to the group.
Each local edge computing terminal is provided with a data cache queue;
and adjusting the packet or data transmission mode by judging whether the data buffer queue is blocked and whether the duration time of the blocking exceeds a preset time value.
The risk inspection monitoring platform also comprises a display layer, wherein the display layer provides an operation interface for a user and realizes the functions expected by the user through the interaction of the user.
In the invention, in order to ensure the safe and stable operation of the marketing system and the high-performance data processing of the inspection abnormity analysis work, the project is realized by combining a big data technology and a J2EE technology. The structure and function modules of each layer are as follows:
infrastructure layer: is the hardware infrastructure layer of the project. For the requirement of intelligent inspection on application, an application server is considered to publish Web application; database servers are contemplated to enable storage of application and management data. And a server cluster is considered to be constructed on the big data platform by adopting the server so as to support big data distributed high-performance mass data processing and analysis.
The basic software layer: for the aspect of intelligent inspection application consistent with the technical architecture of the southern power grid marketing system, the method considers database software and Web middleware and is also consistent with the marketing system in the aspect of selection; mainstream big data platform components are adopted in the aspect of intelligent inspection support, and Hadoop cloud ecological components are mainly adopted. The intelligent rule analysis service provided by each component and the services provided by each component support the requirements of checking archive data processing, business data, table data processing and abnormal rule checking.
A service support layer: designing various business services by adopting a service design concept based on an SOA architecture; and packaging the basic big data development technology in the aspect of intelligent inspection support, and designing a big data service suite.
An application layer: and realizing specific service functions based on the service support layer. For the application planning of the project, the J2EE technology, the micro service technology and the big data technology are respectively adopted for development according to the functional characteristics.
In addition, the invention adopts service-oriented architecture (SOA) to carry out application design, development and system integration, and tries to get rid of the constraint of a technical-oriented solution, thereby improving the reusability of software and accelerating the implementation of application software.
Further advantages of the invention will be apparent in the detailed description section in conjunction with the drawings attached hereto.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a main body architecture diagram of a risk inspection monitoring platform based on a cloud platform under full-electric-network multi-source data according to an embodiment of the present invention
FIG. 2 is a schematic block diagram of the platform of FIG. 1
FIG. 3 is a schematic diagram of the actual connections of the infrastructure layer in the platform of FIG. 1
FIG. 4 is a schematic diagram of different functional modules provided by the platform of FIG. 1
Detailed Description
The invention is further described with reference to the following drawings and detailed description.
Referring to fig. 1, a main framework diagram of a risk inspection monitoring platform based on a cloud platform under full-power grid multi-source data according to an embodiment of the present invention is shown.
In fig. 1, the risk inspection monitoring platform includes an infrastructure layer, a basic software layer, a service support layer, an application layer, a presentation layer, and a visual configuration service component.
For further explanation, the risk inspection monitoring platform provided by the invention adopts a provincial level centralized deployment mode, the platform relates to inspection rule statistics, the platform is built on a big data analysis platform, an application server and a database server use virtual resources provided by a cloud platform, and service data come from a cloud database server cluster.
In order to ensure safe and stable operation of the marketing system and high-performance data processing of the inspection anomaly analysis work, the embodiment adopts a mode of combining a big data technology and a J2EE technology.
The platform is structurally divided into: the system comprises four layers of an infrastructure layer, a basic software layer, a service supporting layer and an application layer.
More specifically, on the basis of fig. 1, reference is made to fig. 2.
Infrastructure layer: is the hardware infrastructure layer of the project. For the requirement of intelligent inspection on application, an application server is considered to publish Web application; database servers are contemplated to enable storage of application and management data. And a server cluster is considered to be constructed on the big data platform by adopting the server so as to support big data distributed high-performance mass data processing and analysis.
The basic facility layer is communicated with the basic software layer through a distributed file system, the basic software layer comprises database software and a Web middle layer, big data storage service is provided for the risk inspection monitoring platform through the database software, big data basic service is provided for the risk inspection monitoring platform through the Web middle layer, and the big data basic service comprises SQL query, distributed memory calculation, streaming calculation, numerical analysis and data mining;
the basic software layer: for the aspect of intelligent inspection application consistent with the technical architecture of the southern power grid marketing system, the method considers database software and Web middleware and is also consistent with the marketing system in the aspect of selection; mainstream big data platform components are adopted in the aspect of intelligent inspection support, and Hadoop cloud ecological components are mainly adopted. The intelligent rule analysis service provided by each component and the services provided by each component support the requirements of checking archive data processing, business data, table data processing and abnormal rule checking.
A service support layer: designing various business services by adopting a service design concept based on an SOA architecture; and packaging the basic big data development technology in the aspect of intelligent inspection support, and designing a big data service suite.
The service supporting layer provides a big data service suite, and the big data service suite comprises a data integration suite, a data governance suite, a query statistics suite, a numerical analysis suite and a data mining suite;
an application layer: and realizing specific service functions based on the service support layer. For the application planning of the project, the J2EE technology, the micro service technology and the big data technology are respectively adopted for development according to the functional characteristics.
And the risk inspection monitoring platform realizes visual intelligent inspection service through the application layer.
The application layer also provides a visualization configuration service component that sets grouping visualization parameters.
A display layer: and providing an operation interface for the user, and realizing the function expected by the user through the interaction of the user.
In FIG. 2, the distributed file system employs YARN as the big data cluster resource manager.
Referring next to fig. 3, fig. 3 illustrates in particular one particular arrangement of the infrastructure layer according to the present invention.
Generally, the infrastructure layer comprises M smart meter acquisition ends distributed at different positions in a power grid target range and N local edge computing terminals, wherein M > N >1, and M and N are positive integers.
In fig. 3, 4 smart meter collection terminals and 2 local edge computing terminals are shown.
In fig. 3, based on the grouping visualization parameters, 2 groups are obtained after the 4 smart meter collecting terminals are grouped, and each group includes 2 smart meter collecting terminals;
in FIG. 3, four intelligent electric meter collecting ends are marked as No. 1-No. 4 collecting ends from top to bottom, and No. 1 and No. 3 are divided into a group; no. 2 and No. 4 are divided into one group; if 2 local edge computing terminals are marked as a and B respectively, in the state of fig. 3, the local edge computing terminal a receives the power grid state data collected by the No. 1 and No. 3 intelligent electric meter collecting terminals; and the local edge computing terminal B receives the power grid state data collected by the No. 2 and No. 4 intelligent electric meter collecting ends.
Furthermore, each local edge computing terminal is provided with a data buffer queue.
In each embodiment of the invention, data is processed as locally as possible through the combination of the edge computing terminal and the cloud platform, and if the data is blocked, the grouping is preferentially adjusted; and if the adjustment group cannot meet the capacity expansion requirement, uploading the adjustment group to the platform. The data can be processed in real time by the aid of the arrangement, and meanwhile, rapid increase of cloud transmission cost is avoided.
Specifically, if a certain local edge computing terminal is receiving power grid state data sent by a smart meter collection end, a data cache queue is blocked and the duration time of the blocking exceeds a preset time value, part of the power grid state data collected by the smart meter collection ends in a group corresponding to the local edge computing terminal is sent to the cloud platform.
And if the data cache queue is blocked but the duration time of the blocking does not exceed a preset time value when a certain local edge computing terminal receives the power grid state data sent by the intelligent electric meter collecting end, reducing the number of the intelligent electric meter collecting ends contained in the group corresponding to the local edge computing terminal.
In fig. 3, if a data cache queue of an edge computing terminal a is blocked and a duration time of the blocking exceeds a predetermined time value, then the power grid state data parts acquired by the acquisition ends of the intelligent electric meters 1 and 3 need to be subsequently sent to a cloud platform;
if the data cache queue of the edge computing terminal B is blocked but the duration of the blocking does not exceed the predetermined time value, it may be considered that the state data of the power grid collected by the collecting terminal of the smart meter No. 2 or No. 4 is sent to other edge computing terminals, for example, to the edge computing terminal a; or other local edge computing terminal, or initiate more edge computing terminals, etc.
In specific implementation, if a local edge computing terminal receives power grid state data sent by a smart meter acquisition end, and the data cache queue is blocked but the duration of the blocking does not exceed a preset time value, a feedback signal is sent to the visual configuration service assembly, so that the visual configuration service assembly adjusts the grouped visual parameters.
In the above embodiment, the system further provides a data persistence support service;
after the user sets the grouping visualization parameters through the visualization configuration service component, the service support layer receives a processing request submitted by the user, triggers a service logic, and calls the data persistence support service to complete data storage.
More specifically, see fig. 4.
Packet data is acquired by the infrastructure layer,
calling a JPA tool class through the Web middle layer in the basic software layer to perform data paging query;
the service supporting layer intercepts query data through a data access plug-in and stores the data into a big data cluster server.
The risk inspection monitoring platform provides an Apache CXF WebService running environment, and issues Service services to Web Service to remote control terminals at different positions in the target range of the power grid.
With reference to fig. 4, in this embodiment, various basic technical components required for processing mass data of this project are integrated according to the big data base platform, which completely covers the whole data processing life cycle from data integration, data management, data storage, data query, data statistical analysis, and data mining, and covers necessary analysis technologies of this project, such as memory calculation and anomaly analysis:
wherein:
distributed file system: in order to meet the storage of mass data and the data security requirements, a distributed file system is adopted. HDFS is a highly fault-tolerant system that ensures data security and integrity, and is suitable for deployment on inexpensive machines. HDFS provides high throughput data access and is well suited for application on large-scale data sets.
Cluster resource management: the YARN is used as a big data cluster resource manager, is a universal resource management system, carries out uniform resource management and scheduling on resources such as memory, storage, CPU and the like of related equipment distributed in a cluster, and brings great benefits to the cluster in the aspects of utilization rate, uniform resource management, data sharing and the like.
Big data integration: a plurality of data integration components are provided, and integration of data from different data sources such as relational databases (such as ORACLE |, MYSQL databases), text files (such as meter code txt files, CSV files and the like), instant messages and the like to a large data platform can be realized. The big data platform component mainly adopts a Hadoop cloud ecological component. The data integrity and analysis service provided by each component and the service provided by each component support the requirements of intelligent inspection on archive data processing, business data analysis, table code data processing and rule checking.
1. And the Spark On YARN scheme is adopted to realize the intelligent checking of the base file data and the service data of the users in the whole province in the memory of the cluster according to the checking rule.
2. And (3) large data storage: data warehouse storage, columnar storage, memory-based storage and the like based on the HDFS distributed file system are provided. And the storage requirements of the project mass data are met through various storage modes.
Big data base service: the system comprises a plurality of components for carrying out query statistics, data analysis mining and treatment on data.
More specifically, mapping (ORM) of Java objects and relational databases is realized by means of JPA (JAVA persistent API) technology, and differences of the databases are shielded; the transmission parameter based on the entity type is more beneficial to determining the service interface of the service layer; for JPA-not-good operations, such as association query, high-performance database operation, general entity model definition, and the like, the platform reserves a set of custom data access layers. The database access is shielded through the persistent data service layer, and related operations such as SQL statements, JDBC and the like do not exist on the data service layer.
JPA is a default persistence mechanism of a development platform, and a Spring + JPA + Hibernate implementation mode is specifically adopted. The Entity and the naming query of the JPA are declared with Annotation. The platform provides an Eclipse plug-in, realizes the ORM of the database table to the entity, and organizes the database table to a specific Package for unified management.
The user-defined persistence layer is a data access plug-in provided by the development platform and can be selectively installed according to needs. The plug-in can make up the performance defects of JPA in big data correlation query and batch data processing, is compatible with the processing mode of the original platform, and the custom persistence layer is also the basis of the dynamic entity model.
The service layer receives a processing request submitted by a user, triggers business logic, and calls the service of the persistent layer to complete data storage, and the process is not subject to transaction control. Typically, the service layer is the boundary of a transaction.
The platform provides two transaction control modes: JTA global transactions using containers and declarative transaction control provided by the platform.
The platform provides an Apache CXF WebService running environment. The Service is published as a Web Service, and the following approaches can be provided:
1. adding relevant Annotation in a Service interface by using JAX-WS (Java API For XML Web Service) specifications to change the Annotation into SEI, and then issuing the Service as Web Service by using a platform.
2. And configuring the general Service into Web Service in a registration management module by using a Service registration mechanism to realize configurable release.
The platform supports a variety of mainstream browsers including IE (6.0 and above), FireFox and Google's Chrome. The platform supports browser compatibility by:
1. the basic component and the presentation component provided by the platform realize the compatibility of the browser.
2. When designing functions and pages, the display components (such as labels) provided by the platform should be used as much as possible
3. A Jquery library and a public js function of a cross-browser provided by a platform;
4. for functions with serious difference between browsers, such as a universal modal window, the function is provided by a platform to be realized instead, and the difference between browsers is shielded.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. A risk inspection monitoring platform based on a cloud platform under full-electric-network multi-source data comprises an infrastructure layer, a basic software layer, a service supporting layer and an application layer;
the method is characterized in that:
the infrastructure layer comprises M intelligent electric meter acquisition ends distributed at different positions in a power grid target range and N local edge computing terminals, wherein M is greater than N and greater than 1, and M and N are positive integers;
the basic facility layer is communicated with the basic software layer through a distributed file system, the basic software layer comprises database software and a Web middle layer, big data storage service is provided for the risk inspection monitoring platform through the database software, big data basic service is provided for the risk inspection monitoring platform through the Web middle layer, and the big data basic service comprises SQL query, distributed memory calculation, stream calculation, numerical analysis and data mining;
the service supporting layer provides a big data service suite, and the big data service suite comprises a data integration suite, a data governance suite, a query statistics suite, a numerical analysis suite and a data mining suite;
the risk inspection monitoring platform realizes visual intelligent inspection service through the application layer;
the application layer also provides a visual configuration service component, and the visual configuration service component sets grouping visual parameters;
based on the grouping visualization parameters, grouping the M intelligent electric meter acquisition ends to obtain K groups, wherein each group comprises a plurality of intelligent electric meter acquisition ends; k is less than or equal to N;
sending the power grid state data collected by the plurality of intelligent electric meter collecting ends included in each group to at least one local edge computing terminal corresponding to the group;
each local edge computing terminal is provided with a data cache queue;
if a certain local edge computing terminal is receiving power grid state data sent by an intelligent power meter acquisition end, a data cache queue is blocked and the duration time of the blocking exceeds a preset time value, part of the power grid state data acquired by the intelligent power meter acquisition ends in a plurality of intelligent power meter acquisition ends contained in a group corresponding to the local edge computing terminal is sent to the cloud platform;
and if the data cache queue is blocked but the duration time of the blocking does not exceed a preset time value when a certain local edge computing terminal receives the power grid state data sent by the intelligent electric meter collecting end, reducing the number of the intelligent electric meter collecting ends contained in the group corresponding to the local edge computing terminal.
2. The risk inspection monitoring platform based on the cloud platform under full-electric-network multi-source data according to claim 1,
the method is characterized in that:
if a certain local edge computing terminal receives power grid state data sent by a smart electric meter acquisition end, a data cache queue is blocked, but the duration time of the blocking does not exceed a preset time value, a feedback signal is sent to the visual configuration service assembly, and the visual configuration service assembly adjusts the grouping visual parameters.
3. The cloud platform-based risk inspection monitoring platform for multi-source data of the whole power grid according to any one of claims 1-2, wherein:
the risk inspection monitoring platform also comprises a display layer, wherein the display layer provides an operation interface for a user and realizes the functions expected by the user through the interaction of the user.
4. The cloud platform-based risk inspection monitoring platform for multi-source data of the whole power grid according to any one of claims 1-2, wherein:
the system also provides data persistence support services;
after the user sets the grouping visualization parameters through the visualization configuration service component, the service supporting layer receives a processing request submitted by the user, triggers a business logic, and calls the data persistence support service to complete data storage.
5. The risk inspection monitoring platform based on the cloud platform under full-electric-network multi-source data according to claim 4,
the method is characterized in that:
the invoking the data persistence support service to complete data storage specifically includes:
packet data is acquired by the infrastructure layer,
calling a JPA tool class through the Web middle layer in the basic software layer to perform data paging query;
and the service supporting layer intercepts query data through a data access plug-in and stores the data into the big data cluster server.
6. The risk inspection monitoring platform based on the cloud platform under the full-grid multi-source data according to any one of claims 1-2,
the method is characterized in that:
the distributed file system adopts the YARN as a big data cluster resource manager.
7. The risk inspection monitoring platform based on the cloud platform under the full-grid multi-source data according to any one of claims 1-2,
the method is characterized in that:
the risk inspection monitoring platform provides an Apache CXF WebService running environment, and issues Service services to Web Service to remote control terminals at different positions in the target range of the power grid.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011291361.8A CN112434924B (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data |
| CN202110969168.3A CN113656379A (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform in full power grid |
| CN202110970696.0A CN113688118A (en) | 2020-11-18 | 2020-11-18 | Risk inspection platform for power grid multi-source data based on cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011291361.8A CN112434924B (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110970696.0A Division CN113688118A (en) | 2020-11-18 | 2020-11-18 | Risk inspection platform for power grid multi-source data based on cloud platform |
| CN202110969168.3A Division CN113656379A (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform in full power grid |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112434924A CN112434924A (en) | 2021-03-02 |
| CN112434924B true CN112434924B (en) | 2021-09-14 |
Family
ID=74692827
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110970696.0A Withdrawn CN113688118A (en) | 2020-11-18 | 2020-11-18 | Risk inspection platform for power grid multi-source data based on cloud platform |
| CN202011291361.8A Active CN112434924B (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data |
| CN202110969168.3A Withdrawn CN113656379A (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform in full power grid |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110970696.0A Withdrawn CN113688118A (en) | 2020-11-18 | 2020-11-18 | Risk inspection platform for power grid multi-source data based on cloud platform |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110969168.3A Withdrawn CN113656379A (en) | 2020-11-18 | 2020-11-18 | Risk inspection monitoring platform based on cloud platform in full power grid |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN113688118A (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283772A (en) * | 2021-06-04 | 2021-08-20 | 云南电网有限责任公司信息中心 | Electric power marketing inspection business rule analysis and application method |
| CN114696468B (en) * | 2022-05-31 | 2022-09-09 | 南方电网数字电网研究院有限公司 | Digital power distribution network edge calculation control device and method |
| CN115576927A (en) * | 2022-11-11 | 2023-01-06 | 加华地学(武汉)数字技术有限公司 | Multi-professional universal digital system and design method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709623A (en) * | 2016-11-14 | 2017-05-24 | 深圳供电局有限公司 | Power grid marketing inspection risk management and control method based on risk calculation model |
| CN109754179A (en) * | 2018-12-29 | 2019-05-14 | 国网北京市电力公司 | A kind of integrated distribution formula low pressure monitoring deployment system and method |
| CN109783229A (en) * | 2018-12-17 | 2019-05-21 | 平安普惠企业管理有限公司 | The method and device of thread resources distribution |
| CN110109745A (en) * | 2019-05-15 | 2019-08-09 | 华南理工大学 | A kind of task cooperation on-line scheduling method for edge calculations environment |
| CN111427930A (en) * | 2020-03-30 | 2020-07-17 | 广东电科院能源技术有限责任公司 | Low-voltage photovoltaic energy storage microgrid device monitoring management system, method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108093401B (en) * | 2017-12-13 | 2020-06-09 | 电子科技大学 | Mobile intelligent terminal privacy protection system and method based on edge calculation |
| CN109743401A (en) * | 2019-01-30 | 2019-05-10 | 南京我在智能科技有限公司 | A kind of intelligent device management system based on Internet of Things edge calculations server |
| CN110336703A (en) * | 2019-07-12 | 2019-10-15 | 河海大学常州校区 | Industrial big data based on edge calculations monitors system |
| CN110794785A (en) * | 2019-11-14 | 2020-02-14 | 北京国电通网络技术有限公司 | Internet of things-based construction site edge calculation Internet of things analysis and alarm system and method |
| CN111260924B (en) * | 2020-02-10 | 2021-01-26 | 北京中交国通智能交通系统技术有限公司 | Traffic intelligent control and service release strategy method adapting to edge calculation |
-
2020
- 2020-11-18 CN CN202110970696.0A patent/CN113688118A/en not_active Withdrawn
- 2020-11-18 CN CN202011291361.8A patent/CN112434924B/en active Active
- 2020-11-18 CN CN202110969168.3A patent/CN113656379A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709623A (en) * | 2016-11-14 | 2017-05-24 | 深圳供电局有限公司 | Power grid marketing inspection risk management and control method based on risk calculation model |
| CN109783229A (en) * | 2018-12-17 | 2019-05-21 | 平安普惠企业管理有限公司 | The method and device of thread resources distribution |
| CN109754179A (en) * | 2018-12-29 | 2019-05-14 | 国网北京市电力公司 | A kind of integrated distribution formula low pressure monitoring deployment system and method |
| CN110109745A (en) * | 2019-05-15 | 2019-08-09 | 华南理工大学 | A kind of task cooperation on-line scheduling method for edge calculations environment |
| CN111427930A (en) * | 2020-03-30 | 2020-07-17 | 广东电科院能源技术有限责任公司 | Low-voltage photovoltaic energy storage microgrid device monitoring management system, method and device |
Non-Patent Citations (3)
| Title |
|---|
| 6G异构边缘计算;王鹏飞等;《物联网学报》;20200331;第4卷(第1期);第121-130页 * |
| 华为技术有限公司.华为自由流收费大数据稽核解决方案及应用.《中国交通信息化》.2019,第1-4页. * |
| 华为自由流收费大数据稽核解决方案及应用;华为技术有限公司;《中国交通信息化》;20191231;第1-4页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113688118A (en) | 2021-11-23 |
| CN113656379A (en) | 2021-11-16 |
| CN112434924A (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112434924B (en) | Risk inspection monitoring platform based on cloud platform under full-electric-network multi-source data | |
| Zheng et al. | Real-time big data processing framework: challenges and solutions | |
| CN104769582B (en) | For the real time data releasing of power grid | |
| CN109582717A (en) | A kind of database unified platform and its read method towards electric power big data | |
| CN106709012A (en) | Method and device for analyzing big data | |
| CN104881352A (en) | System resource monitoring device based on mobile terminal | |
| CN113179173B (en) | Operation and maintenance monitoring system for expressway system | |
| CN107103064A (en) | Data statistical approach and device | |
| CN115335821B (en) | Offloading statistics collection | |
| Ma et al. | Review of power spatio-temporal big data technologies for mobile computing in smart grid | |
| Liu et al. | On construction of an energy monitoring service using big data technology for smart campus | |
| Malik et al. | A common data architecture for energy data analytics | |
| CN103595792A (en) | Environmental pollution trend monitoring system | |
| CN105653523A (en) | Energy consumption supervise network of things basis platform system building method | |
| CN105512154A (en) | Method and system for environmental-protection data collection based on embedded database | |
| Jin et al. | Research on Wide-area Distributed Power Quality Data Fusion Technology of Power Grid | |
| Shouaib et al. | Survey on iot-based big data analytics | |
| Sun et al. | Mechanism design for unified management of power grid planning data | |
| Kakanakov | Big data analytics in electricity distribution systems | |
| Wang et al. | Power grid data monitoring and analysis system based on edge computing | |
| Zhao et al. | Application research on application performance management system in big data of power grid | |
| Lu et al. | Technology Architecture for Source-Grid-Load-Storage Collaborative Control | |
| Fu et al. | The Whole Process Monitoring System of Meteorological Big Data Cloud Platform Based on “Tianjing” | |
| Chang et al. | Distributed storage strategy and visual analysis for economic big data | |
| CN113919167A (en) | Fine evaluation method for new energy power generation performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Gong Shu Inventor after: Xu Zhongwen Inventor after: Qi Penghui Inventor after: Zhang Xiongbao Inventor after: Cao Wei Inventor after: Liu Feng Inventor before: Liu Feng |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210831 Address after: No. 6 Democracy Road, Xingning District, Nanning City, Guangxi Zhuang Autonomous Region, 530000 Applicant after: GUANGXI POWER GRID Co.,Ltd. Address before: Yunnan University, No.1, Yuhua District, Chenggong District, Kunming City, Yunnan Province, 650500 Applicant before: Liu Feng |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |