CN114978850B - Heterogeneous high concurrency big data collaborative access system of smart power grid - Google Patents
Heterogeneous high concurrency big data collaborative access system of smart power gridInfo
- Publication number
- CN114978850B CN114978850B CN202210565009.1A CN202210565009A CN114978850B CN 114978850 B CN114978850 B CN 114978850B CN 202210565009 A CN202210565009 A CN 202210565009A CN 114978850 B CN114978850 B CN 114978850B
- Authority
- CN
- China
- Prior art keywords
- data
- sgepon
- access
- equipment
- power grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 claims abstract description 73
- 238000012544 monitoring process Methods 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 174
- 238000007726 management method Methods 0.000 claims description 142
- 230000003068 static effect Effects 0.000 claims description 70
- 230000008859 change Effects 0.000 claims description 40
- 230000008569 process Effects 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 34
- 230000004044 response Effects 0.000 claims description 32
- 238000013461 design Methods 0.000 claims description 29
- 108050009189 Translin Proteins 0.000 claims description 18
- 102100032834 Translin-associated protein X Human genes 0.000 claims description 18
- 230000006978 adaptation Effects 0.000 claims description 18
- 230000007246 mechanism Effects 0.000 claims description 17
- 230000004927 fusion Effects 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000013500 data storage Methods 0.000 claims description 10
- 238000013499 data model Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 9
- 238000007405 data analysis Methods 0.000 claims description 6
- 238000011084 recovery Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000002955 isolation Methods 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000005611 electricity Effects 0.000 abstract description 22
- 238000012360 testing method Methods 0.000 abstract description 17
- 238000012423 maintenance Methods 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 11
- 210000001503 joint Anatomy 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000002580 esophageal motility study Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004146 energy storage Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Abstract
According to the application, SGEPON is adopted as a communication channel, so that the problems of multiple manufacturers, scattered management, incapability of sharing information and the like of SGEPON equipment in an electricity consumption information acquisition system are solved, a first-level power grid collaborative access SGEPON data network management system is provided, unified management of a plurality of second-level power grids, multiple manufacturers and tens of thousands of SGEPON equipment is realized, the problems of isomerism and high concurrency existing in different SGEPON data network management access layers are effectively solved, the operation efficiency of the whole SGEPON data network management is improved, and the requirement for comprehensive and unified management of the first-level power grid scale SGEPON network is met; through unit testing and comprehensive testing, the application realizes the specific service function required by the provincial power grid, meets the performance requirements of SGEPON equipment on data accuracy, consistency and timeliness, realizes integrated monitoring, effectively improves the management and operation and maintenance level of the whole electricity consumption information acquisition system, and has great effect and great practical value in the fields of large-scale power grid big data collaborative access and the like.
Description
Technical Field
The application relates to a smart grid big data collaborative access system, in particular to a smart grid heterogeneous high concurrency big data collaborative access system, and belongs to the technical field of smart grid access systems.
Background
The intelligent power grid combines SGEPON formed by Ethernet and passive optical network, uses PON technology at physical layer, adopts Ethernet multi-source protocol at link layer, and realizes Ethernet access by utilizing the topological structure of PON. The OLT and the ONU belong to active devices, and are respectively placed at two ends of the SGEPON network. The OLT is local side equipment, mainly completes the uplink access of service data, and is connected with the ONU equipment through the ODN to control and manage the ONU. The ONU is a terminal device, and mainly receives the service transmitted from the OLT, and provides various broadband services to the connected users. The ODN is composed of a passive optical splitter and an optical fiber, and mainly provides a transmission channel for optical signals.
SGEPON has the excellent characteristics of high bandwidth, low cost, easy use and easy upgrading, the national power grid communication department also distributes SGEPON in the power consumption information acquisition system and the intelligent power distribution system as a communication network for transmitting power data, and a large amount of business data such as data, pictures, voice, video and the like can be efficiently and quickly transmitted from a backbone network to a large number of terminal devices by virtue of a SGEPON access network.
Currently, each manufacturer develops a network element management system EMS for managing its own devices, such as U2000 for hua, ANM2000 for beacon, and N2000 for happy. In the initial stage of network deployment, customers can manage corresponding SGEPON power grid equipment well through EMS of manufacturers. However, with the deployment of SGEPON networks, customers hope to realize the comprehensive management of multi-manufacturer and multi-region SGEPON networks, even the maintenance of SGEPON is in butt joint with other service systems, so that the integrated management of the communication network and the service information is realized, further resources are saved, the operation and maintenance level is improved, and the whole service system is efficiently, safely and well operated for protecting the navigation.
In SGEPON grid management system, SGEPON devices belong to network element layer, EMS provided by each manufacturer is network element layer management system, so to realize the comprehensive management of SGEPON network, it is necessary to establish network layer management system, and data access is performed according to corresponding protocol and communication mechanism through north interface of manufacturer EMS, so as to realize the comprehensive management of multi-manufacturer and multi-region SGEPON grid devices.
The current provincial power grid electricity consumption information acquisition system SGEPON communication channel and acquisition equipment integrated monitoring has two main targets: firstly, developing a first-level power grid SGEPON data network management system to realize unified management of multiple manufacturer devices in a second-level power grid electricity consumption acquisition system; and secondly, the primary power grid SGEPON data network management system and the electricity consumption information acquisition service management system are in butt joint, so that data sharing is realized, and further integrated monitoring of a communication channel and an acquisition terminal is realized. To accomplish these two goals, it is most critical how to efficiently implement SGEPON device data access through the northbound interface of each vendor EMS. In addition, the accuracy of the access data, the timeliness of important information and the consistency of data expression of the power company have clear requirements, and the requirements for the access of SGEPON equipment data are more strict. However, in the process of developing SGEPON data network management, when SGEPON device data access is performed, the following problems are found:
(1) Isomerism: including multi-source protocol layer isomerism and data layer isomerism. Multi-source protocol layer heterogeneous refers to that a plurality of northbound interface multi-source protocols exist, and different multi-source protocols have differences in communication mechanism, data format and the like. The data layer is heterogeneous, and part of the difference is caused by different data formats of the multi-source protocol, and the difference is caused by different SGEPON device types of various manufacturers, so that the device attribute and the description data are different.
(2) High concurrency: the SGEPON deployment in the electric power information acquisition system has the characteristics of wide distribution and large scale, the total number of devices of the secondary power grid SGEPON is about tens of thousands, the provincial electric power SGEPON data network manager is required to realize the access of all SGEPON device data of all the secondary power grids in the whole provincial area, various access tasks are required to be executed through multithreading concurrency, the high concurrency can generate great pressure on the processing capacity of a CPU (Central processing Unit) of a server and system resources, and if task scheduling and thread management cannot be well realized, the efficiency of data access of SGEPON devices can be restricted, so that the performance of the whole SGEPON data network manager is affected.
The TMN structure is based on the experimental study of China electric science and Chongqing electric science, a communication data network management system in an electric power system is proposed, the problem that a plurality of northbound interface multisource protocols exist between network control and data acquisition layers of the system is considered, an interface adaptation method and a unified data model building method are provided for supporting the multisource protocols, the existing isomerism problem is shielded, the concrete performance of isomerism among different multisource protocols is not analyzed, and how to realize the interface adaptation and build the unified data model is not described.
In summary, the big data access of the power grid in the prior art still has problems, and the difficulties and the problems to be solved of the present application are mainly focused on the following aspects:
Firstly, in the prior art electricity consumption information acquisition system, the problems of multiple manufacturers of SGEPON equipment, scattered management, incapability of sharing information and the like are solved, unified management of a plurality of secondary power grids, multiple manufacturers and tens of thousands of SGEPON equipment is not realized, because the northbound interface multisource protocols of the network element management systems of different SGEPON equipment manufacturers are various, the SGEPON network scale is large, the problems of isomerism and high concurrency exist in the implementation process of SGEPON data network management access layers, the problems seriously affect the operation efficiency of the whole SGEPON data network management, so that the operation efficiency of the whole SGEPON data network management cannot meet the requirements of comprehensively and uniformly managing the primary power grid scale SGEPON network, the specific business functions required by a provincial power grid cannot be realized, the performance requirements of SGEPON equipment data accuracy, consistency and timeliness are not met, the robustness is poor, the anti-noise capability is not realized, and the management and the operation and maintenance level of the whole electricity consumption information acquisition system are lower;
Secondly, the current provincial power grid electricity consumption information acquisition system SGEPON communication channel and acquisition equipment integrated monitoring has two obvious defects: firstly, a data network management system of a primary power grid SGEPON cannot realize unified management of multiple manufacturer devices in a power consumption information acquisition system of a secondary power grid; the first-level power grid SGEPON data network management system cannot be in butt joint with the electricity consumption information acquisition service management system, so that data sharing is realized, integrated monitoring of a communication channel and an acquisition terminal cannot be realized, and data access of SGEPON equipment cannot be realized through a northbound interface of EMS of each manufacturer; in addition, the access of data of the SGEPON equipment in the prior art cannot meet the requirements of accuracy of the access data, timeliness of important information and consistency of data expression of an electric company;
thirdly, in the prior art SGEPON, the data network management access layer has the problem of isomerism: the multi-source protocol layer heterogeneous method comprises multi-source protocol layer heterogeneous and data layer heterogeneous, wherein the multi-source protocol layer heterogeneous problem mainly exists in a plurality of northbound interface multi-source protocols, and different multi-source protocols have differences in the aspects of communication mechanisms, data formats and the like; the heterogeneous problem of the data layer is caused by different data formats of the multi-source protocol, and in addition, the device attribute and the description data are different due to different SGEPON device types of each manufacturer, so that the requirements of most application scenes such as heterogeneous big data access of the smart grid can not be met;
Fourth, the high concurrency problem in the data network management access layer in the prior art SGEPON is that: the SGEPON deployment in the electric power electricity information acquisition system has the characteristics of wide distribution and large scale, the total number of the devices of the secondary power grid SGEPON is about tens of thousands, the provincial power SGEPON data network manager is required to realize the access of all SGEPON device data of all the secondary power grids in the whole province, various access tasks are required to be executed through multithreading concurrency, the high concurrency can generate great pressure on the processing capacity of a CPU (central processing unit) of a server and system resources, the prior art cannot well realize task scheduling and thread management, the efficiency of data access of SGEPON devices is restricted, the performance of the whole SGEPON data network manager is further influenced, the prior art does not analyze the concrete manifestation of isomerism among different multi-source protocols, the interface adaptation and the establishment of a unified data model are also not realized, the multithreading concurrency control cannot be realized, the electric power communication, the data processing and the energy storage capacity are weak, and the operation and maintenance level and the management efficiency of the power grid are restricted.
Therefore, the intelligent power grid data system is constructed based on SGEPON network management, and is applied to the process of comprehensively managing the data access in the isomerism SGEPON aiming at the isomerism and high concurrency problems in the data access process, so that the isomerism of different manufacturers and different northbound interface multi-source protocols is shielded, the unified expression of the data is realized, the multithreading concurrency control method is optimized, the efficient communication, the data processing and the storage are realized, and the operation and maintenance level and the management efficiency are improved.
Disclosure of Invention
According to the application, SGEPON is adopted as a communication channel, so that the problems of multiple manufacturers, scattered management, incapability of sharing information and the like of SGEPON equipment in an electricity consumption information acquisition system are solved, a first-level power grid collaborative access SGEPON data network management system is provided, unified management of a plurality of second-level power grids, multiple manufacturers and tens of thousands of SGEPON equipment is realized, the problems of multiple north interface multisource protocols, large SGEPON network scale and high isomerism and high concurrency of SGEPON data network management access layers of different SGEPON equipment manufacturer network element management systems are effectively solved, the operation efficiency of the whole SGEPON data network management system is improved, and the requirements of comprehensive and unified management on a first-level power grid scale SGEPON network are met; through unit testing and comprehensive testing, the application realizes the specific service function required by the provincial power grid, meets the performance requirements of SGEPON equipment on data accuracy, consistency and timeliness, realizes integrated monitoring, effectively improves the management and operation and maintenance level of the whole electricity consumption information acquisition system, and has great effect and great practical value in the fields of large-scale power grid big data collaborative access and the like.
In order to achieve the technical advantages, the technical scheme adopted by the application is as follows:
The intelligent power grid heterogeneous high concurrency big data collaborative access system adopts SGEPON as a communication channel, proposes a primary power grid collaborative access SGEPON data network management system, and uniformly manages a plurality of secondary power grids, a plurality of manufacturers and tens of thousands of SGEPON devices;
Firstly, aiming at the isomerism problem existing in SGEPON data network management access layer, a unified access model of big data of intelligent SGEPON equipment is provided, which comprises the following steps: the method comprises the steps of firstly, dynamically expanding access of a multi-source protocol, including design of a multi-source protocol dynamic adapter, realization of a multi-source protocol adaptation interface, realization of a multi-source protocol dynamic adaptation abstract class and realization of a multi-source protocol dynamic adaptation class, secondly, unifying heterogeneous big data of a power grid, including object analysis of power grid SGEPON equipment, collaborative coding of intelligent SGEPON equipment data, and thirdly, designing a heterogeneous unified access database;
Firstly, designing and realizing a multi-source protocol adapter of an access layer to shield the isomerism of a multi-source protocol based on the relation between an object-oriented interface and a class, and realizing the dynamic and expansion access of the multi-source protocol; then designing SGEPON a metadata fusion model of the equipment through UML based on SGEPON equipment objects, object attributes and relationships among the objects, and providing templates for data processing unification by specifying the coding of key data; finally, an Oracle database is adopted in the primary power grid SGEPON data network management, and the design of a database table structure is carried out according to a SGEPON device metadata fusion model, so that unified storage and unified display of SGEPON device data are realized;
Secondly, aiming at the high concurrency problem existing in SGEPON data network management access layers, the data access concurrency control method of the intelligent SGEPON device is provided, and comprises the following steps: the concurrent power grid control method comprises an intelligent SGEPON concurrent thread pool, a priority task queue control and a power grid C3PO database connection pool control;
and analyzing different subtasks and specific implementation flows contained in each service module in the access layer, merging a Java thread pool, a priority task queue and a C3PO database connection pool, and introducing the merged Java thread pool, the priority task queue and the C3PO database connection pool into the access layer to realize high concurrency control and improve the access efficiency of SGEPON equipment.
Preferably, the multi-source protocol dynamic adapter design: the access layer of the first-level power grid SGEPON data network manager supports a multi-source protocol by adding a multi-source protocol adapter, and the multi-source protocol adapter provides unified multi-source protocol call for SGEPON network element equipment data access by extracting the same characteristics of the multi-source protocol and shielding differences;
the multi-source protocol adapter of the first-level power grid SGEPON data network management access layer utilizes the design of the realization relation between abstract classes and interfaces in Java language and the inheritance relation between the classes and abstract classes, the uppermost layer defines a unified father interface Northbound, the middle layer defines abstract classes of each multi-source protocol, realizes the father interface, and the lowermost layer defines the realization class of multi-source protocol concrete of each manufacturer.
Preferably, implementation of the multi-source protocol adaptation interface: the uppermost layer of the multi-source protocol adapter defines an interface Northbound, according to the abstract implementation of the functions of the multi-source protocol, the method stated in the interface Northbound corresponds to the same functions abstracted by various protocols one by one, when an access layer establishes communication according to a multi-source protocol mechanism, the IP address of a server deployed by a manufacturer EMS and a port bound by a northbound interface are obtained, a method connect (StringlP, intport) corresponds to the communication establishment of the multi-source protocol, and the method defines that the input parameters are the IP of a character string type and the port of an integer type, and is specifically implemented in a lower abstract class according to the transport layer protocols of different protocols; method sendOrder (String order) corresponds to a sending command of a multi-source protocol, and specifies that the input parameter is an order of a character string type; method getEquipments () obtains SGEPON static data of the equipment corresponding to the multi-source protocol, obtains the response static data after the access layer sends the query command, and returns the response static data by the result of the character String set ArrayList < String >; in addition, getOLTState (), getPONState (), getBoardState (), getONUState () methods correspond to real-time states of OLT, PON port, board and ONU returned by the EMS northbound interface of the secondary grid manufacturer, and all return in a String set arranlist < String >; method LISTENCHANGE () is used for monitoring and pushing SGEPON equipment change data corresponding to a multi-source protocol, an access layer monitors equipment change data actively pushed by a manufacturer EMS northbound interface through the method, and after a message is read, the device change data is added into a character String set ArrayList < String > to return; method LISTENALARME () corresponds to the listening push SGEPON device alert data of the multi-source protocol, also returns as a String set ArrayList < String >; the method close () corresponds to the closing communication of the multi-source protocol, is used in pair with the method connect (StringIP, intport), and releases the occupied IO and port resources after finishing data interaction with the northbound interface of the manufacturer EMS, and timely closes the communication through the close ().
Preferably, the implementation of the multi-source protocol dynamic adaptation abstract class: the second layer of the multi-source protocol adapter sequentially defines abstract classes according to different protocols, the naming rule is "protocol name is abbreviated as_ Northbound", the father interface Northbound of the upper layer is realized, and each definition not only rewrites and realizes the method stated in Northbound, but also keeps the method stated but not the method;
The access layer is used as a client, each secondary power grid EMS is used as a server, the access layer realizes the code of the client through a Socket class, and the abstract class TLl _ Northbound is specifically defined as: firstly, declaring a Socket type sk object, obtaining a word BufferedWriter type object bw and a BufferedReader type object br related to IO through the sk, then in a method connect (StringlP, intport), creating and initializing the sk, bw and br objects, and judging whether the access layer successfully establishes communication with a northbound interface of a manufacturer EMS according to a result returned by the method;
The method for realizing connection (StringIP, int port) commonly owned in each vendor subclass in the TL 1-Northbound abstract class is realized, other similar methods are realized one by one, partial methods are realized differently due to the difference of vendor equipment, original explanation is reserved, and other protocol SNMP_ Northbound, CORBA _ Northbound abstract classes are also specifically defined according to the rule.
Preferably, the dynamically adapted class of the multi-source protocol implements: the lowest layer is specific to the realization class of each protocol of each manufacturer, the naming rule is ' manufacturer name is abbreviated as multi-source protocol name is abbreviated as ' Northbound ', the name rules are inherited to the protocol abstract class of the upper layer, and the method realized in the parent class is inherited, and the method with difference of each manufacturer is realized;
Specific implementation of getEquipments () method in fh_ TLl _ Northbound class: after sending a command for inquiring the static data of the OLT equipment, the access layer receives response information through the method, firstly, a getEquipments () method judges whether the response is corresponding SGEPON equipment data or not, then, a getEquipments () method judges the inquiry command corresponding to the response information through analyzing fields in response_id and whether the inquiry is successful or not, if the inquiry is failed, the getEquipments () method obtains the reason of error through the response_block field, if the inquiry is successful, the response information message number and the record number of the static data of the OLT equipment in each message are obtained through the response_block field, finally, the access layer reads the static data of all the OLT equipment according to the specific record number and adds the static data into a character String set ArrayList < String >;
Similarly, the multi-source protocol adapter of the data network management access layer of the primary power grid SGEPON is also specifically realized in this way for various northbound interface protocols of other manufacturers, and part of northbound interfaces of the EMS do not realize other protocols, so that the methods in the classes are realized in a null way for later expansion.
Preferably, unification of grid heterogeneous big data: the access layer performs data conversion in the analysis process, shields the isomerism of data, establishes a unified SGEPON network element equipment data model in advance, and provides a template for data conversion and data unification in the analysis process;
1) Power grid SGEPON device object resolution
SGEPON the data network management object comprises OLT, SHELF, BOARD, PON port and ONU, and the attribute information of the device object comprises: device background information, SGEPON static information, device real-time state information, SGEPON real-time alarm information and device log information, wherein the device background information comprises manufacturer and secondary power grid position information of a device, and the SGEPON static information comprises device basic address positioning and software and hardware version information; the equipment real-time state information comprises real-time management and running states; SGEPON the real-time alarm information comprises generation and recovery of the alarm information of the current equipment; the equipment log information comprises historical alarm information and equipment change information;
The association relation between managed objects is defined in SGEPON data network management, SGEPON is a point-to-multipoint topological structure on a network structure, and then the topological association relation of a multi-branch tree of ' province-city-OLT-SHELF-BOARD-PON-ONU ' is formed by combining province-city-OLT-SHELF-BOARD-PON-ONU ' with province-city-geographical position information, wherein province is a root node, and ONU is a leaf node;
2) Intelligent SGEPON device data collaborative coding
SGEPON the equipment metadata fusion model provides a unified conversion model, when the data network management of the first-level power grid SGEPON performs network element equipment data access, a processing thread of an access layer performs data analysis, filtering and field attribute extraction according to the model, encodes the second-level power grid information, encodes manufacturer information through a unique identity of a unique type CITY_CODE, performs final unification through a unique identity of the MAU_CODE, and stores consistent data into a table of a database.
Preferably, the grid concurrency data model:
1) Access layer overall architecture model
The access layer of the data network management of the primary power grid SGEPON needs to realize the access of the data of the secondary power grid SGEPON at the same time, independent processes are designed aiming at each manufacturer in the access layer, the concurrency of multiple processes is realized, and each process comprises SGEPON equipment data access tasks of a plurality of secondary power grids;
Based on the concrete management realized by SGEPON data network management application layers, the SGEPON equipment data access program of each secondary power grid realizes corresponding service data access, and the method specifically comprises 4 steps: SGEPON network element equipment static data access, change data access, state data access and alarm data access, wherein an access program of each secondary power grid opens a separate thread for each service, and the access program of each service data comprises communication control, data acquisition, data processing and data storage processes;
The access layer of SGEPON data network management is communicated with the northbound interface of each secondary power grid EMS, data access of SGEPON network element equipment is carried out, and the implementation of each secondary power grid of the access layer is divided into 4 modules according to service and function division;
2) Static data access model
Firstly, an access layer establishes communication connection with a service port of a manufacturer EMS northbound interface according to a communication mechanism of a multi-source protocol, if the communication connection is not established successfully, the upper limit of reconnection times is 3 times, if the connection is established continuously for 3 times, the current task is directly ended, the next opportunity is waited, after the communication connection is established successfully, the access layer sends an authentication command according to the regulation and command format of the multi-source protocol, reads and analyzes the returned authentication result, if the authentication fails, the authentication is revised according to the returned error reason, and the authentication is carried out again; after authentication is successful, a query command is sent according to a multi-source protocol to query the static data of SGEPON equipment of different types, then data acquisition, data processing, data unification and data storage are carried out, finally, after the query is completed, the connection is disconnected, and resources such as occupied ports are released;
3) Altering data access models
The method comprises the steps that a push mode is adopted in data changing access, firstly, an access layer establishes communication connection with a northbound interface of an EMS (enhanced management system) of a manufacturer, authentication is successful, then, the access layer sends a registration command for enabling the EMS to actively report the changed data according to a multi-source protocol rule, judges whether registration is successful or not according to reply information, if the registration is failed, the registration command is corrected according to error reasons in the reply information, and then, the registration is repeated, after the registration is successful, SGEPON equipment pushed by the EMS is monitored, the complete data message is read once the pushed changed data is monitored by the access layer, analysis is carried out, the changed network element equipment is judged through the analyzed data, and what change happens, according to the type and the change condition of the changed equipment, the access layer interacts with the data layer, insert, delete, update operation is carried out on a corresponding data table, the consistency of the data is guaranteed, and the equipment change information is stored in an equipment change table, so that the data is shared to an information collecting service management system;
The access layer establishes long connection with the northbound interface of the manufacturer EMS, always keeps a monitoring state, creates subtasks by using a heartbeat packet mechanism of a multi-source protocol, uses the same communication connection, periodically sends heartbeat commands to the northbound interface, and keeps long connection with the northbound interface of the manufacturer EMS.
Preferably, the intelligence SGEPON concurrent thread pool: the method comprises the steps that different service data access modes in an access layer are different, thread pool configurations to be adopted are also different, firstly, an access layer program directly uses a concurrent framework tool class Executors to call a static method newScheduledThreadPool (int) to create a ScheduledExecutorService type thread pool, the object is named execService, the core thread number of the thread pool is set to be 8, the delay and periodic execution of the task are supported, then, an independent task is created for real-Time state access of each secondary power grid SGEPON device, the task of each secondary power grid further comprises a communication thread, a data processing thread and a storage thread, finally, a thread pool object execService calls scheduleAtFixedRate (Runnable command, long initialDelay, longperiod, a Time Unit) to submit the SGEPON device state data access task of each secondary power grid, the first parameter command transmitted represents a specific task, the second parameter INITIALDELAY represents initialization delay, the third parameter period represents the task execution period, the fourth parameter represents a Time Unit of the delay and period, the delay and period are staggered, the task is set in each secondary power grid 372, the task is accessed in a state, the period is set, the period is further, the period is changed, and the period is set by setting up and the period is further, the period is changed, and the period is changed by setting up of the period is changed.
Preferably, the priority task queue control: all tasks in the priority task queue PriorityBlockingQueue realize a complete interface and rewrite compareTo () method, the method transmits the same type of task object, and compares the priorities of the two task objects, thereby realizing sequencing, one task is defined in a SGEPON data network management access layer program to be inherited by a Thread class or realize Runnable interface realization, but none of the tasks realize the complete interface, for realizing code multiplexing, priorityThread and PriorityRunnable classes are defined first, a priority member variable is declared in the Thread class, if not set, the priority class is defaulted to 5, the abstract class is defined by inheriting the Thread class and realizing the complete interface, and the compareTo () method is rewritten, the PriorityRunnable class is required to be defined as an abstract class, meanwhile, the Runnable interface and the complete interface are realized, but only the compareTo () method is rewritten, the run () method is not rewritten, and the priority of the task is represented by defining the member variable priority, and the range of the priority class is also taken as the integer between 1 and 10;
In addition, the access layer performs level assessment and setting for each service data access and subtasks included in the service data access, the priority level of the subtasks for alarm data acquisition, alarm data analysis and storage is set to be the highest 10, and the priority level of the corresponding heartbeat packet is set to be 3, because the user pays more attention to the alarm information of SGEPON devices, if the alarm information exists, the user processes the alarm information preferentially, the heartbeat packet is used for ensuring that when no data is transmitted on a communication connection for a long time, the connection is prevented from being disconnected, and the alarm data is transmitted on the communication connection at present, the heartbeat packet is not transmitted temporarily, so the priority level for transmitting the heartbeat packet is set to be low.
Preferably, the grid C3PO database is connected to pool control: the configuration of the CPO database connection pool in the access layer is based on the C3PO database connection pool, the configuration of the CPO database connection pool adopts xml file configuration, the file is named as C3p0-config.xml, the file is placed under the class path of the item, jdbcUrl. Driverclass. User, password, acquireIncrement, initialPoolSize, minPoolSize and maxPoolSize parameters are set in the configuration file, jdbcUrl represents url addresses of database connection, different database url addresses represent different formats, the connection number in the pool is initialized to 10 through initialPoolSize, the minimum connection number in the pool is set to 10 through minPoolSize, when the database connection in the pool is insufficient, the connection number is increased by acquireIncrement each time, but the maximum connection number in the pool is maxPoolSize is 50, when the connection number in the pool reaches the maximum and is used, new tasks are queued for waiting for connection to be free;
Setting corresponding parameters of a C3PO database connection pool through an xml configuration file, importing a required Jar package, writing a database connection management tool class JDBCUtils through Java language, firstly, defining JDBCUtils through static member variables and initializing a thread pool dataSource object, defining proprietary connection related to a database transaction, binding the proprietary connection on a corresponding thread to prevent data security problems in a multithreaded environment, then, defining JDBCUtils a static method getConnection () for acquiring the database connection, acquiring the database connection pool through defining a static method getDataSource (), releasing the connection through defining releaseConnection (Connectionconnection) method, putting the connection after completion into the database connection pool, and ensuring atomicity, consistency, isolation and durability of the database transaction processing by calling corresponding transaction methods in a program, wherein the real database physical connection is not closed, and the three static methods beginTransaction (), commitTransaction (), and rollbackTransaction () are related to transactions in the database, and are sequentially opened, committed and rolled back.
Compared with the prior art, the technical scheme has the following innovation points and advantages:
Firstly, SGEPON is adopted as a communication channel, so that the problems of multiple manufacturers, scattered management, incapability of sharing information and the like of SGEPON equipment in an electricity consumption information acquisition system are solved, a first-level power grid collaborative access SGEPON data network management system is provided, unified management of a plurality of second-level power grids, multiple manufacturers and tens of thousands of SGEPON equipment is realized, the problems of multiple north interface multisource protocols, large SGEPON network scale and high isomerism and high concurrency of SGEPON data network management access layers of different SGEPON equipment manufacturer network element management systems are effectively solved, the operation efficiency of the whole SGEPON data network management is improved, and the requirement of comprehensive and unified management of a first-level power grid scale SGEPON network is met; through unit testing and comprehensive testing, the application realizes the specific service function required by the provincial power grid, meets the performance requirements of SGEPON equipment on data accuracy, consistency and timeliness, realizes integrated monitoring, effectively improves the management and operation and maintenance level of the whole electricity consumption information acquisition system, and has great effect and great practical value in the fields of large-scale power grid big data collaborative access and the like;
Secondly, the application designs a power grid electricity consumption information acquisition system SGEPON communication channel and acquisition equipment integrated monitoring system: firstly, developing a first-level power grid SGEPON data network management system to realize unified management of multiple manufacturer devices in a second-level power grid electricity consumption acquisition system; the first-level power grid SGEPON data network management system and the power consumption information acquisition service management system are in butt joint to realize data sharing, so that integrated monitoring of a communication channel and an acquisition terminal is realized, and SGEPON equipment data access is realized through a northbound interface of EMS of each manufacturer, and the method has the advantages of accurate access of SGEPON equipment data, resource saving, high efficiency, safety, good operation, timely important information, consistent data expression, good robustness and obvious advantages when the smart power grid is in heterogeneous high concurrent big data collaborative access;
Thirdly, aiming at the problem of isomerism in SGEPON data network management access layers, the application provides the relation between the object-oriented interfaces and classes by analyzing and comparing different points of a plurality of main flow north interface multisource protocols and specific application of each multisource protocol in different manufacturers, designs a multisource protocol adapter to shield isomerism of the multisource protocols and realizes dynamic and expansion access of the multisource protocols; in addition, based on SGEPON equipment objects, object attributes and association relations, a SGEPON equipment metadata fusion model is designed through UML, coding of key data is regulated, templates are provided for data processing and unification, table structure design is conducted in an Oracle database according to a SGEPON equipment metadata fusion model, isomerism of the data is shielded, and unified storage and unified display of SGEPON equipment data are achieved.
Fourth, the application aims at the high concurrency problem existing in SGEPON data network management access layer, and by analyzing different subtasks and specific implementation processes contained in each service module in the access layer, java thread pool, priority task queue and C3PO database connection pool are integrated and then introduced into the access layer program, so as to realize high concurrency control, optimize the utilization rate of system resources, realize good scheduling of tasks, improve the access efficiency of SGEPON equipment data, meet the performance requirements of SGEPON equipment data accuracy, consistency and timeliness, realize integrated monitoring, and effectively improve the management and operation and maintenance levels of the whole electricity consumption information acquisition system.
Drawings
Fig. 1 is an access layer protocol dynamic adapter design class diagram.
Fig. 2 is a flowchart of a multi-source protocol dynamic adaptation abstract class Socket communication.
Fig. 3 is an access database table design summary diagram.
Fig. 4 is a table structural design diagram of the secondary grid information CITY.
Fig. 5 is a table configuration design diagram of the device OLT CHANGE information record table olt_change.
Fig. 6 is a SGEPON static data access flow diagram.
Fig. 7 is a flow chart of an ONU change contrast mechanism.
Fig. 8 is a SGEPON device state data access flow diagram.
FIG. 9 is a SGEPON device alert data access model flow diagram.
FIG. 10 is a diagram of a specific implementation of abstract class PriorityRunnable of the application.
Fig. 11 is an access stratum per-service data subtask class assessment and setup diagram.
Fig. 12 is a static data access statistical diagram of the G-municipality SGEPON equipment in the experiment.
Fig. 13 is a statistical chart of the polling time of the state of the integrated network management equipment in the experiment SGEPON.
Detailed description of the preferred embodiments
The present application will be described in detail with reference to the drawings, wherein the application is described in detail. Those skilled in the art can make similar generalizations without departing from the spirit of the application, and therefore the application is not limited by the specific embodiments disclosed below.
The application adopts SGEPON as a communication channel, solves the problems of more manufacturers, scattered management, incapability of sharing information and the like of SGEPON equipment in an electricity consumption information acquisition system, proposes a first-level power grid collaborative access SGEPON data network management system, and realizes unified management of a plurality of second-level power grids, a plurality of manufacturers and tens of thousands of SGEPON equipment. The north interface multisource protocols of network element management systems of different SGEPON equipment manufacturers are various, and the SGEPON network scale is large, so that the SGEPON data network management access layer has the problems of isomerism and high concurrency in the implementation process, and the problems seriously affect the operation efficiency of the whole SGEPON data network management, so that the requirements of comprehensive and unified management on the first-level power grid scale SGEPON network cannot be met.
Aiming at the problem of isomerism in SGEPON data network management access layers, the application provides the relation between an object-oriented interface and a class by analyzing and comparing different points of a plurality of main-stream northbound interface multisource protocols and specific application of each multisource protocol in different manufacturers, designs a multisource protocol adapter to shield isomerism of the multisource protocol and realizes dynamic and expanded access of the multisource protocol. In addition, based on SGEPON equipment objects, object attributes and association relations, a SGEPON equipment metadata fusion model is designed through UML, coding of key data is regulated, templates are provided for data processing and unification, table structure design is conducted in an Oracle database according to a SGEPON equipment metadata fusion model, isomerism of the data is shielded, and unified storage and unified display of SGEPON equipment data are achieved.
Aiming at the high concurrency problem in SGEPON data network management access layer, the application integrates a Java thread pool, a priority task queue and a C3PO database connection pool and then introduces the integrated Java thread pool, the priority task queue and the C3PO database connection pool into an access layer program by analyzing different subtasks and specific implementation flows contained in each service module in the access layer, thereby realizing high concurrency control, optimizing the utilization rate of system resources, realizing good scheduling of tasks and improving the access efficiency of SGEPON equipment data. Through unit test and comprehensive test, SGEPON data network management access layer realizes specific service function required by provincial power grid, and satisfies performance requirements of SGEPON equipment for data accuracy, consistency and timeliness, realizes integrated monitoring, and effectively improves management and operation and maintenance levels of the whole electricity consumption acquisition system.
1. Intelligent SGEPON equipment big data unified access model
The method solves the problem of isomerism in a data network management access layer of a primary power grid SGEPON; firstly, realizing unified access of a multi-source protocol by designing a multi-source protocol adapter; then, through designing SGEPON equipment metadata fusion models and prescribing data collaborative coding, unified access of heterogeneous data is realized; and finally, designing a table structure in the database based on SGEPON equipment metadata fusion models.
Dynamic extended access of (one) multi-source protocol
When the access layer of the primary power grid SGEPON data network manager accesses SGEPON equipment data through the northbound interfaces of the EMSs of the secondary power grids, the northbound interfaces of EMSs of different manufacturers of the secondary power grids need to realize the same multi-source protocol, so that the primary power grid SGEPON data network manager can be rapidly developed and deployed. However, the problem of different northbound interface protocols exists truly, which sets a barrier for the realization of the data network management access layer of the primary power grid SGEPON. Aiming at the problem of multiple heterogeneous north interface protocols, the application designs the support of the multi-source protocol by dynamically adapting the multi-source protocol based on the relation between the object-oriented heterogeneous interfaces and classes.
1. Multi-source protocol dynamic adapter design
The access layer of the first-level power grid SGEPON data network manager supports the multi-source protocol by adding a multi-source protocol adapter, the multi-source protocol adapter provides unified multi-source protocol call for SGEPON network element equipment data access by extracting the same characteristics of the multi-source protocol and shielding the difference,
The multi-source protocol adapter of the first-level power grid SGEPON data network management access layer utilizes the design of the realization relation between abstract classes and interfaces in Java language and the inheritance relation between the classes and abstract classes, the design class diagram is shown in figure 1, the uppermost layer defines a uniform father interface Northbound, the middle layer defines abstract classes of each multi-source protocol, realizes the father interface, and the lowermost layer defines specific realization classes of multi-source protocols of all manufacturers.
2. Implementation of a multi-source protocol adaptation interface
The uppermost layer of the multi-source protocol adapter defines an interface Northbound that is implemented according to the functional abstraction of the multi-source protocol.
The method stated in the interface Northbound corresponds to the same function abstracted by various protocols one by one, when the access layer establishes communication according to a multi-source protocol mechanism, the IP address of a server deployed by a manufacturer EMS and a port bound by a northbound interface are obtained, the method connect (StringlP, intport) corresponds to the communication establishment of the multi-source protocol, the input parameters are specified to be the IP of a character string type and the port of an integer type, and the method is realized in a lower abstract class according to the transmission layer protocol of different protocols; method sendOrder (String order) corresponds to a sending command of a multi-source protocol, and specifies that the input parameter is an order of a character string type; method getEquipments () obtains SGEPON static data of the equipment corresponding to the multi-source protocol, obtains the response static data after the access layer sends the query command, and returns the response static data by the result of the character String set ArrayList < String >; in addition, getOLTState (), getPONState (), getBoardState (), getONUState () methods correspond to real-time states of OLT, PON port, board and ONU returned by the EMS northbound interface of the secondary grid manufacturer, and all return in a String set arranlist < String >; method LISTENCHANGE () is used for monitoring and pushing SGEPON equipment change data corresponding to a multi-source protocol, an access layer monitors equipment change data actively pushed by a manufacturer EMS northbound interface through the method, and after a message is read, the device change data is added into a character String set ArrayList < String > to return; method LISTENALARME () corresponds to the listening push SGEPON device alert data of the multi-source protocol, also returns as a String set ArrayList < String >; the method close () corresponds to the closing communication of the multi-source protocol, is used in pair with the method connect (StringIP, intport), and releases the occupied IO and port resources after finishing data interaction with the northbound interface of the manufacturer EMS, and timely closes the communication through the close ().
3. Implementation of dynamic adaptation abstract class of multi-source protocol
The second layer of the multi-source protocol adapter sequentially defines abstract classes according to different protocols, the naming rule is "protocol name is abbreviated as_ Northbound", the parent interface Northbound of the upper layer is realized, and each definition not only rewrites and realizes the method declared in Northbound, but also keeps the declaration of the method and does not realize. Different manufacturer devices can cause mutual difference due to the characteristics of the different manufacturer devices, and even if the northbound interfaces of the manufacturer EMS support the same kind of protocol, the protocols can also have difference in packaging data messages, so that the data has isomerism. However, under the same protocol, the differences existing in different equipment manufacturers mainly affect data processing, no difference exists in the aspects of communication mechanism and the like, the same parts are extracted and implemented in abstract classes of the protocol, inheritance classes of all lower-layer manufacturers commonly have the same method, and other methods with differences still remain stated and are implemented in all manufacturer subclasses. The specific flow is shown in fig. 2. The access layer is used as a client, each secondary power grid EMS is used as a server, the access layer realizes the code of the client through a Socket class, and the abstract class TLl _ Northbound is specifically defined as: firstly, a Socket type sk object is declared, words BufferedWriter type objects bw and BufferedReader type objects br related to IO are obtained through the sk, then in a method connect (StringlP, intport), creation and initialization of the sk, bw and br objects are carried out, and whether the access layer and a northbound interface of a manufacturer EMS successfully establish communication is judged according to a result returned by the method.
The method for realizing connection (StringIP, int port) commonly owned in each vendor subclass in the TL 1-Northbound abstract class is realized, other similar methods are realized one by one, partial methods are realized differently due to the difference of vendor equipment, original explanation is reserved, and other protocol SNMP_ Northbound, CORBA _ Northbound abstract classes are also specifically defined according to the rule.
4. Class implementation of dynamic adaptation of multi-source protocols
The lowest layer is specific to the realization class of each protocol of each manufacturer, the naming rule is 'manufacturer name is abbreviated as' multi-source protocol name is abbreviated as 'Northbound', the naming rules are inherited to the protocol abstract class of the upper layer, and the method realized in the parent class is inherited, and the method that the manufacturers have differences is realized.
Specific implementation of getEquipments () method in fh_ TLl _ Northbound class: after sending a command for inquiring the static data of the OLT equipment, the access layer receives response information through the method, firstly, a getEquipments () method judges whether the response is corresponding SGEPON equipment data, then, a getEquipments () method judges the inquiry command corresponding to the response information through analyzing fields in response_id and whether the inquiry is successful, if the inquiry is failed, the getEquipments () method obtains the reason of error through the response_block field, if the inquiry is successful, the number of response information messages and the record number of the static data of the OLT equipment in each message are obtained through the response_block field, finally, the access layer reads the static data of all the OLT equipment according to the specific record number and adds the static data into a character String set ArrayList < String >.
Similarly, the multi-source protocol adapter of the data network management access layer of the primary power grid SGEPON is also specifically realized in this way aiming at various northbound interface protocols of other manufacturers. The partial EMS northbound interface does not implement other protocols, and the methods in these classes are implemented empty for later expansion.
(II) unification of heterogeneous big data of power grid
The problem of data isomerism still exists because different protocols prescribe respective unique message formats and different vendor equipment attributes differ. In order to provide consistent presentation for users at an application layer, an access layer converts data in the analysis process and shields the heterogeneity of the data. Therefore, a unified SGEPON network element equipment data model is established in advance, and a template is provided for data conversion and data unification in the analysis process.
1. Power grid SGEPON device object resolution
SGEPON the data network management object comprises OLT, SHELF, BOARD, PON port and ONU, and the attribute information of the device object comprises: device background information, SGEPON static information, device real-time state information, SGEPON real-time alarm information and device log information, wherein the device background information comprises manufacturer and secondary power grid position information of a device, and the SGEPON static information comprises device basic address positioning and software and hardware version information; the equipment real-time state information comprises real-time management and running states; SGEPON the real-time alarm information comprises generation and recovery of the alarm information of the current equipment; the device log information includes historical alert information and device change information.
And SGEPON, defining the association relationship between managed objects in a data network management, wherein SGEPON is a point-to-multipoint topological structure on a network structure, and combining with province-city geographic position information to form the topological association relationship of a multi-branch tree of 'province-city-OLT-SHELF-BOARD-PON-ONU', wherein province is a root node, and ONU is a leaf node.
2. Intelligent SGEPON device data collaborative coding
SGEPON the equipment metadata fusion model provides a unified conversion model for the data heterogeneous problem caused by multi-source protocol and multi-manufacturer. When the data network manager of the primary power grid SGEPON accesses the network element equipment data, the processing thread of the access layer performs data analysis, filtering and field attribute extraction according to the model. However, in the data unification process, differences may exist due to different device vendor attribute description fields. In order to facilitate data storage and transmission, the secondary power grid information is encoded, manufacturer information is encoded through a unique identity of the integer CITY_CODE, and final unification is performed through a data collaborative encoding method through the unique identity of the MAU_CODE so as to store consistent data into a table of a database.
(III) heterogeneous unified Access database design
In order to provide better data service for clients, the primary power grid SGEPON is used for data management data layer, database table design and creation are carried out according to the relation between the SGEPON equipment metadata fusion model and each equipment, and the implementation adopts a third paradigm to reduce data redundancy. The database of the primary power grid SGEPON data network management comprises a user table, a log table, a device table and an alarm table, the main SGEPON network element device association table comprises static information, change information, real-time state and alarm information of the secondary power grid, manufacturers and various devices, and the total number of the 19 data tables is 19, and the summarized information is shown in figure 3.
FIG. 4 is a table structural design of secondary grid information CITY, in which CITY CODE CITY_ID is used as a primary key to uniquely identify the secondary grid, and MAU_CODE is associated with vendor information MANUFACTURE table by external key vendor CODE to indicate which vendor equipment the secondary grid uses.
The manufacturer information MANUFACTURE is a table structure design in which the manufacturer encodes mau_code as a primary key, which uniquely identifies the manufacturer. And the table structural design of the equipment OLT, wherein the OLT_IP is a main key, the OLT is uniquely identified, and the OLT is indicated to which CITY the OLT belongs by associating the external key CITY_CODE with a secondary power grid CITY table. Fig. 5 is a table structure design of the device OLT CHANGE information record table olt_change, in which olt_chg_id is used as a primary key, and the OLT CHANGE record is uniquely identified and associated with the table OLT by an external key olt_ip, which OLT is changed. The table structure design of the device OLT real-time STATE table OLT_STATE, wherein the OLT_IP is used as a main key to uniquely identify the real-time STATE of the OLT, and meanwhile, the OLT_IP is also an external key and is associated with the table OLT, so that the joint query of the real-time STATE information and the static information of the OLT is realized.
And SGEPON, carrying out table structural design of an ALARM information table ALARM of equipment, wherein an ALARM alarm_id is used as a main key, carrying out unique identification on ALARM information, and associating the ALARM information with the table OLT through an external key alarm_olt_ip to indicate which OLT has a fault, and obtaining the specific POSITION of the fault through the alarm_position.
2. Data access concurrency control method for intelligent SGEPON equipment
Power grid concurrency data model
1. Access layer overall architecture model
The access layer of the data network management of the primary power grid SGEPON needs to realize the access of the data of the secondary power grid SGEPON at the same time, independent processes are designed for each manufacturer in the access layer, the concurrency of multiple processes is realized, and each process comprises the data access tasks of SGEPON equipment of a plurality of secondary power grids.
Based on the concrete management realized by SGEPON data network management application layers, the SGEPON equipment data access program of each secondary power grid realizes corresponding service data access, and the method specifically comprises 4 steps: SGEPON network element equipment static data access, change data access, state data access and alarm data access, wherein an access program of each secondary power grid opens a separate thread for each service, and the access program of each service data comprises communication control, data acquisition, data processing and data storage processes.
The access layer of SGEPON data network management is communicated with the northbound interface of each secondary power grid EMS, data access of SGEPON network element equipment is carried out, and the implementation of each secondary power grid of the access layer is divided into 4 modules according to service and function division.
2. Static data access model
And SGEPON, in the development stage, the data layer does not have any data, and static data of all network element devices OLT, machine frames, single boards and ONUs, including identification information, basic attribute information, association information and the like, are required to be obtained from the secondary power grid network management during system initialization. As shown in FIG. 6, the access process of the static data access adopts a pull mode, firstly, the access layer establishes communication connection with a service port of a northbound interface of an EMS manufacturer according to a communication mechanism of a multi-source protocol, if the communication connection is not successfully established, the reconnection is attempted, the upper limit of reconnection times is 3 times, if the connection is not successfully established continuously for 3 times, the current task is directly ended, the next opportunity is waited, after the communication connection is successfully established, the access layer sends an authentication command according to the regulation and command format of the multi-source protocol, reads and analyzes the returned authentication result, and if the authentication fails, the authentication is revised according to the returned error reason, and the authentication is carried out again. And after authentication is successful, sending a query command according to a multi-source protocol, querying static data of SGEPON equipment of different types, then performing data acquisition, data processing, data unification and data storage, and finally disconnecting after the query is completed, and releasing resources such as occupied ports.
3. Altering data access models
The method comprises the steps that a push mode is adopted in data changing access, firstly, an access layer establishes communication connection with a northbound interface of an EMS (enhanced management system) of a manufacturer, authentication is successful, then, the access layer sends a registration command for enabling the EMS to actively report the changed data according to a multi-source protocol rule, judges whether registration is successful or not according to reply information, if the registration is failed, the registration command is corrected according to error reasons in the reply information, and then, the registration is repeated, after the registration is successful, SGEPON equipment pushed by the EMS is monitored, the complete data message is read once the pushed changed data is monitored by the access layer, analysis is carried out, the changed network element equipment is judged through the analyzed data, and what change happens, according to the type and the change condition of the changed equipment, the access layer interacts with the data layer, insert, delete, update operation is carried out on a corresponding data table, the consistency of the data is guaranteed, and the equipment change information is stored in an equipment change table, so that the data is shared to an information collecting service management system;
The access layer establishes long connection with the northbound interface of the manufacturer EMS, always keeps a monitoring state, creates subtasks by using a heartbeat packet mechanism of a multi-source protocol, uses the same communication connection, periodically sends heartbeat commands to the northbound interface, and keeps long connection with the northbound interface of the manufacturer EMS. However, even if a heartbeat mechanism is added between the access layer and the northbound interface of the manufacturer EMS to keep long connection, the connection is disconnected due to some unavoidable factors, and if equipment change occurs in the period of the unconnected connection, the access layer of SGEPON data network manager cannot monitor the changed data, which can cause inconsistent data of the primary power network manager and the secondary power network manager, and affects management and maintenance. Aiming at the problem, a comparison mechanism is added to perform double-layer protection, so that the consistency of data is ensured, and the flow of the comparison mechanism is shown in figure 7.
4. State data access model
The state data access adopts a pull+polling mode, the access process is shown in fig. 8, firstly, the access layer establishes communication connection with a northbound interface of a manufacturer EMS, authentication is successful, then, the access layer inquires positioning information of corresponding equipment from an equipment table of a database, sequentially inquires OLT, BOARD, PON real-time running states of ONU and obtains response data returned by the northbound interface of the manufacturer EMS in an inquiry command of the corresponding positioning information package, finally, the access layer analyzes and filters the obtained data according to a multi-source protocol format, and respectively stores the data according to different types of equipment, and the access layer adopts a polling strategy to update the real-time state data in the realization.
5. Alarm data access model
The alarm information is accessed by adopting a push mode, firstly, the access layer establishes communication connection with a northbound interface of a manufacturer EMS, authentication is successful, secondly, the access layer sends a registration command for enabling the EMS to actively report equipment alarms according to the specification of a multi-source protocol, monitors alarm data pushed by the EMS after the registration is successful, reads complete data messages once the alarm data are monitored, starts a data processing thread again, analyzes and filters the alarm data messages according to a specified format to obtain the serial number, the position, the time, the reason and the like of the alarm, judges whether the reported new alarm data or the recovery data of the previous alarm according to an identification field, finally, carries out corresponding insert, update operation on an alarm table in a database according to whether the alarm is generated or recovered, and ensures the real-time performance and the accuracy of the alarm data of each second-level power grid in the data network management of the first-level power grid SGEPON, and ensures long connection with the northbound interface of the manufacturer EMS through a heartbeat package.
In order to ensure that the alarm data in SGEPON data network management is completely consistent with the alarm data in EMS of each secondary power grid manufacturer, an access layer is added with a pull mode to acquire alarm data which is not successfully reported, firstly, after the network is normal, the access layer is in communication connection with a northbound interface of the EMS of the secondary power grid again, after authentication is successful, the time of the last alarm information of the secondary power grid is read from an alarm table of a database, then the last alarm information is packaged into a command and is sent to the northbound interface of the EMS of the corresponding secondary power grid manufacturer, all alarm occurrence and alarm recovery data from the time of incoming to the time are inquired, after the access layer acquires a complete alarm data message, a data processing thread is started, corresponding operation is carried out on the alarm table of the database according to alarm field data obtained through processing, meanwhile, when the data processing thread is started, the original communication thread registers an alarm active report command to the northbound interface of the EMS of the manufacturer, and monitoring is started again, and the implementation process is shown in fig. 9.
(II) concurrent power grid control method
1. Intelligent SGEPON concurrent thread pool
The access layer of the first-level power grid SGEPON data network management has the conditions of multi-process and multi-thread concurrency, so that the system resources are better utilized, the running efficiency of the access layer program is improved, efficient and accurate data service is provided for users, and the Java thread pool technology can be applied to the programs of the access layer.
The method comprises the steps that different service data access modes in an access layer are different, thread pool configurations to be adopted are also different, firstly, an access layer program directly uses a concurrent framework tool class Executors to call a static method newScheduledThreadPool (int) to create a ScheduledExecutorService type thread pool, the object is named execService, the core thread number of the thread pool is set to be 8, the delay and periodic execution of the task are supported, then, an independent task is created for real-Time state access of each secondary power grid SGEPON device, the task of each secondary power grid further comprises a communication thread, a data processing thread and a storage thread, finally, a thread pool object execService calls scheduleAtFixedRate (Runnable command, long initialDelay, longperiod, a Time Unit) to submit the SGEPON device state data access task of each secondary power grid, the first parameter command transmitted represents a specific task, the second parameter INITIALDELAY represents initialization delay, the third parameter period represents the task execution period, the fourth parameter represents a Time Unit of the delay and period, the delay and period are staggered, the task is set in each secondary power grid 372, the task is accessed in a state, the period is set, the period is further, the period is changed, and the period is set by setting up and the period is further, the period is changed, and the period is changed by setting up of the period is changed.
2. Priority task queue control
All tasks in the priority task queue PriorityBlockingQueue realize the complete interface and rewrite compareTo () method, the method transmits the same type of task object, and compares the priorities of the two task objects, and further realizes the sorting, one task is defined in the SGEPON data network management access layer program to inherit the Thread class or realize Runnable interface realization, but none of them realize the complete interface, for realizing code multiplexing, firstly define PriorityThread and PriorityRunnable classes, declare priority member variables in the Thread class, if not set, default to 5, through inheriting the Thread class and realizing the complete interface, rewrite compareTo () method, priorityRunnable class needs to be defined as abstract class, simultaneously realize Runnable and complete interfaces, but rewrite compareTo () method, but not rewrite run () method, and express the priority of the task by defining member variable priority, the range of which also takes the whole number between 1 and 10, and the specific realization of PriorityRunnable is shown in fig. 10.
In addition, the access layer performs level assessment and setting for each service data access and subtasks included therein, as shown in fig. 11. The priority of the subtasks for acquiring, analyzing and storing the alarm data is set to be 10 at the highest, and the priority of the corresponding heartbeat packet is set to be 3, because the user attaches importance to the alarm information of SGEPON devices, if the alarm information exists, the alarm information is preferentially processed, the heartbeat packet is used for ensuring that the connection is prevented from being disconnected when no data is transmitted on the communication connection for a long time, and the heartbeat packet is not transmitted temporarily when the alarm data exists on the communication connection, so the priority of transmitting the heartbeat packet is set to be low.
3. Power grid C3PO database connection pool control
The first-level power grid SGEPON data network management access layer is used for realizing the access of the data of the second-level power grid SGEPON equipment by a manufacturer, a second-level power grid and a business, after the data is acquired and processed, the access layer stores or updates the data which are consistent in conversion into a corresponding table corresponding to an Oracle database, wherein part of programs also need to inquire corresponding data from the database, the access layer and the database are connected with an Oracle database service port in the interaction process, so that the time is saved, the resource utilization rate is improved, the multiplexing of the database connection is realized, and the access layer is based on a C3PO database connection pool.
The configuration of the CPO database connection pool adopts xml file configuration, the file is named as c3p0-config.xml, the file is placed under the class path of the project, jdbcUrl. Driverclass. User, password, acquireIncrement, initialPoolSize, minPoolSize and maxPoolSize parameters are set in the configuration file, jdbcUrl represents url addresses of database connection, different database url addresses represent different formats, the connection number in the pool is initialized to 10 through initialPoolSize, the minimum connection number in the pool is set to 10 through minPoolSize, when the database connection in the pool is insufficient, the connection number is increased through acquireIncrement, namely 5 connections are increased each time, but the maximum connection number maxPoolSize in the pool is 50, and when the connection number in the pool reaches the maximum and is used, new tasks are queued for waiting for connection to be free.
Setting corresponding parameters of a C3PO database connection pool through an xml configuration file, importing a required Jar package, writing a database connection management tool class JDBCUtils through Java language, firstly, defining JDBCUtils through static member variables and initializing a thread pool dataSource object, defining proprietary connection related to a database transaction, binding the proprietary connection on a corresponding thread to prevent data security problems in a multithreaded environment, then, defining JDBCUtils a static method getConnection () for acquiring the database connection, acquiring the database connection pool through defining a static method getDataSource (), releasing the connection through defining releaseConnection (Connectionconnection) method, putting the connection after completion into the database connection pool, and ensuring atomicity, consistency, isolation and durability of the database transaction processing by calling corresponding transaction methods in a program, wherein the real database physical connection is not closed, and the three static methods beginTransaction (), commitTransaction (), and rollbackTransaction () are related to transactions in the database, and are sequentially opened, committed and rolled back.
3. Experimental testing and analysis
Cell testing
The access layer of the first-level power grid SGEPON data network management mainly analyzes whether the access layer can successfully establish connection with the northbound interface of the second-level power grid manufacturer EMS through unit test, realizes the access of various service data of SGEPON equipment and meets the power performance requirement, and the unit test result and analysis of G city are described in detail below.
(1) Gcity SGEPON device static data access
As can be seen from the response information obtained by the access layer inquiring the static data of the OLT equipment in the G city, the response data comprises the response of the login command and the response of the inquiring the static data command of the OLT, and the two parts are divided into "; the response of the login command comprises manufacturer identification, G city server IP, response time and login success, and the OLT static equipment data response also comprises the number of data blocks, the number of current data blocks and the record number of the current data blocks, and complete response data can be read according to the three data blocks. Firstly, the query result of the static data of the current OLT equipment only comprises one data packet, wherein the data packet comprises 12 pieces of OLT equipment data records in total, the static layout of the OLT equipment is between two broken lines, the static data of each piece of OLT equipment is added into a set, then the obtained result is transmitted to a data processing thread, the data processing thread carries out data analysis and data filtering, the static layout of the OLT equipment is uniformly converted according to a SGEPON equipment metadata fusion model and a specified uniform coding rule, and finally, the uniformly processed static data of the OLT equipment is stored into an OLT table of a database through a data storage module.
Similarly, the access layer can sequentially query the static data of SHELF, BOARD, ONU devices in the same manner, store the static data in a table corresponding to the database after data processing and unification, and the statistics result of various SGEPON devices accessed by SGEPON data network management is shown in fig. 12. The statistical result is compared with the statistical data in the ANM2000 network management in G city, and the number of the collected static data of the equipment can be determined to be accurate.
(2) G-city SGEPON device change data access
As can be seen from the data obtained by monitoring the change of the G city SGEPON, the response data comprises three parts of the response of the login command, the response of the registration change reporting command and the change data, and the parts are divided into'; "segmentation". By analyzing the response data, the access layer can know that the access layer successfully remotely logs in the resource change service port of the North interface of the ANM2000, and successfully registers the device change information reporting command, and meanwhile, the access layer is exactly in 2015-08-1316:50: time 05 resulted in OLTIP being 13.140.61.50, the ONU delete data with ponid=1-1-2-3,0NUID =5. After the access layer processes the changed data, deleting the corresponding data in the database table ONU, and storing the record into the ONU_CHANGE table so as to perform data sharing and synchronous updating with the electricity consumption information acquisition service management system, thereby realizing integrated monitoring.
(3) G-city SGEPON device real-time status data access
The partial response data inquiring the real-time state of the ONU can see that the state of 10 ONU devices exists in the data message, the IP of the OLT to which the data message belongs is 13.140.61.82, the PON ports are 1-1-4-4, wherein the running state of the ONU devices with the ONUID of 3 is offline, and the other ONU devices run normally. After the access layer processes the ONU STATE data, the STATE information is encoded according to the specification and then stored in a database ONU_STATE table. The same kind of access layer realizes the access and storage of OLT, BOARD, PON equipment real-time state data in turn. The time statistics result of the real-time state of the G-city SGEPON equipment is that the polling time is 124.748 seconds, so that the requirement of the user on timeliness is met.
(4) G-city SGEPON device alert data access
As can be seen from a piece of ALARM data obtained by monitoring the G city SGEPON ALARM, the parameters and values of the ALARM data exist in the form of key value pairs, and the access layer can encode the ALARM type, the ALARM grade and the ALARM identifier according to the regulations after processing the ALARM data and store the ALARM type, the ALARM grade and the ALARM identifier in an ALARM table of a database.
The access layer queries according to OLTIP and PONID when the ONU equipment is accessed, so that two records exist in the same ONU belonging to two OLTs, the situation of inconsistent data can be caused, but the data in the database of the southward communication network management of O city is directly checked, the total records of the ONU are 1209, and the access result is proved to be accurate.
(II) comprehensive test
The access layer of SGEPON data network management realizes complete and unified access of static data, change data, state data and alarm data of 16 secondary power grids and 4 manufacturer SGEPON devices through comprehensive test, and stores the data in a table corresponding to a database, so that detailed and comprehensive data can be provided for each management module of an application layer. Therefore, the functional requirement of the provincial power grid on SGEPON data network management is met.
The access layer of SGEPON data network management passes the comprehensive test, and the data of 16 secondary power grids SGEPON are subjected to data conversion and unification according to SGEPON equipment metadata fusion model and coding rules in the access process, and then stored in a table corresponding to a database, so that a manager can see the consistent data at an application layer. Meanwhile, SGEPON data network management can realize the butt joint with the electricity consumption information acquisition service management system through a webservice interface, and unified data is provided for the electricity consumption information acquisition service management system. Therefore, the requirement of the provincial power grid on SGEPON data network management data consistency is met.
The access layer of SGEPON data network management passes the comprehensive test, the real-time state data of 16 secondary power grids SGEPON devices can be updated within 15 minutes, and the statistical result of the test time is shown in fig. 13. In addition, the alert data may be acquired within 1 minute of occurrence and presented to the administrator at the application layer. Therefore, the requirement of the provincial power grid on the data timeliness of SGEPON data network management is met. Analysis shows that the application meets the functional requirement of the provincial power grid on SGEPON data network management and meets the performance requirement on the accuracy, consistency and timeliness of SGEPON data network management data.
Claims (10)
1. The intelligent power grid heterogeneous high concurrency big data collaborative access system is characterized in that SGEPON is adopted as a communication channel, a primary power grid collaborative access SGEPON data network management system is provided, and a plurality of secondary power grids, a plurality of manufacturers and tens of thousands of SGEPON devices are managed in a unified mode;
Firstly, aiming at the isomerism problem existing in SGEPON data network management access layer, a unified access model of big data of intelligent SGEPON equipment is provided, which comprises the following steps: the method comprises the steps of firstly, dynamically expanding access of a multi-source protocol, including design of a multi-source protocol dynamic adapter, realization of a multi-source protocol adaptation interface, realization of a multi-source protocol dynamic adaptation abstract class and realization of a multi-source protocol dynamic adaptation class, secondly, unifying heterogeneous big data of a power grid, including object analysis of power grid SGEPON equipment, collaborative coding of intelligent SGEPON equipment data, and thirdly, designing a heterogeneous unified access database;
Firstly, designing and realizing a multi-source protocol adapter of an access layer to shield the isomerism of a multi-source protocol based on the relation between an object-oriented interface and a class, and realizing the dynamic and expansion access of the multi-source protocol; then designing SGEPON a metadata fusion model of the equipment through UML based on SGEPON equipment objects, object attributes and relationships among the objects, and providing templates for data processing unification by specifying the coding of key data; finally, an Oracle database is adopted in the primary power grid SGEPON data network management, and the design of a database table structure is carried out according to a SGEPON device metadata fusion model, so that unified storage and unified display of SGEPON device data are realized;
Secondly, aiming at the high concurrency problem existing in SGEPON data network management access layers, the data access concurrency control method of the intelligent SGEPON device is provided, and comprises the following steps: the concurrent power grid control method comprises an intelligent SGEPON concurrent thread pool, a priority task queue control and a power grid C3PO database connection pool control;
and analyzing different subtasks and specific implementation flows contained in each service module in the access layer, merging a Java thread pool, a priority task queue and a C3PO database connection pool, and introducing the merged Java thread pool, the priority task queue and the C3PO database connection pool into the access layer to realize high concurrency control and improve the access efficiency of SGEPON equipment.
2. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein a multi-source protocol dynamic adapter design: the access layer of the first-level power grid SGEPON data network manager supports a multi-source protocol by adding a multi-source protocol adapter, and the multi-source protocol adapter provides unified multi-source protocol call for SGEPON network element equipment data access by extracting the same characteristics of the multi-source protocol and shielding differences;
the multi-source protocol adapter of the first-level power grid SGEPON data network management access layer utilizes the design of the realization relation between abstract classes and interfaces in Java language and the inheritance relation between the classes and abstract classes, the uppermost layer defines a unified father interface Northbound, the middle layer defines abstract classes of each multi-source protocol, realizes the father interface, and the lowermost layer defines the realization class of multi-source protocol concrete of each manufacturer.
3. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein the implementation of a multi-source protocol adaptation interface: the uppermost layer of the multi-source protocol adapter defines an interface Northbound, according to the abstract implementation of the functions of the multi-source protocol, the method stated in the interface Northbound corresponds to the same functions abstracted by various protocols one by one, when an access layer establishes communication according to a multi-source protocol mechanism, the IP address of a server deployed by a manufacturer EMS and a port bound by a northbound interface are obtained, a method connect (StringlP, intport) corresponds to the communication establishment of the multi-source protocol, and the method defines that the input parameters are the IP of a character string type and the port of an integer type, and is specifically implemented in a lower abstract class according to the transport layer protocols of different protocols; method sendOrder (String order) corresponds to a sending command of a multi-source protocol, and specifies that the input parameter is an order of a character string type; method getEquipments () obtains SGEPON static data of the equipment corresponding to the multi-source protocol, obtains the response static data after the access layer sends the query command, and returns the response static data by the result of the character String set ArrayList < String >; in addition, getOLTState (), getPONState (), getBoardState (), getONUState () methods correspond to real-time states of OLT, PON port, board and ONU returned by the EMS northbound interface of the secondary grid manufacturer, and all return in a String set arranlist < String >; method LISTENCHANGE () is used for monitoring and pushing SGEPON equipment change data corresponding to a multi-source protocol, an access layer monitors equipment change data actively pushed by a manufacturer EMS northbound interface through the method, and after a message is read, the device change data is added into a character String set ArrayList < String > to return; method LISTENALARME () corresponds to the listening push SGEPON device alert data of the multi-source protocol, also returns as a String set ArrayList < String >; the method close () corresponds to the closing communication of the multi-source protocol, is used in pair with the method connect (StringIP, intport), and releases the occupied IO and port resources after finishing data interaction with the northbound interface of the manufacturer EMS, and timely closes the communication through the close ().
4. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein the implementation of a multi-source protocol dynamic adaptation abstract class: the second layer of the multi-source protocol adapter sequentially defines abstract classes according to different protocols, the naming rule is "protocol name is abbreviated as_ Northbound", the father interface Northbound of the upper layer is realized, and each definition not only rewrites and realizes the method stated in Northbound, but also keeps the method stated but not the method;
The access layer is used as a client, each secondary power grid EMS is used as a server, the access layer realizes the code of the client through a Socket class, and the abstract class TLl _ Northbound is specifically defined as: firstly, declaring a Socket type sk object, obtaining a word BufferedWriter type object bw and a BufferedReader type object br related to IO through the sk, then in a method connect (StringlP, intport), creating and initializing the sk, bw and br objects, and judging whether the access layer successfully establishes communication with a northbound interface of a manufacturer EMS according to a result returned by the method;
The method for realizing connection (StringIP, int port) commonly owned in each vendor subclass in the TL 1-Northbound abstract class is realized, other similar methods are realized one by one, partial methods are realized differently due to the difference of vendor equipment, original explanation is reserved, and other protocol SNMP_ Northbound, CORBA _ Northbound abstract classes are also specifically defined according to the rule.
5. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein the class of multi-source protocol dynamic adaptation implements: the lowest layer is specific to the realization class of each protocol of each manufacturer, the naming rule is ' manufacturer name is abbreviated as multi-source protocol name is abbreviated as ' Northbound ', the name rules are inherited to the protocol abstract class of the upper layer, and the method realized in the parent class is inherited, and the method with difference of each manufacturer is realized;
Specific implementation of getEquipments () method in fh_ TLl _ Northbound class: after sending a command for inquiring the static data of the OLT equipment, the access layer receives response information through the method, firstly, a getEquipments () method judges whether the response is corresponding SGEPON equipment data or not, then, a getEquipments () method judges the inquiry command corresponding to the response information through analyzing fields in response_id and whether the inquiry is successful or not, if the inquiry is failed, the getEquipments () method obtains the reason of error through the response_block field, if the inquiry is successful, the response information message number and the record number of the static data of the OLT equipment in each message are obtained through the response_block field, finally, the access layer reads the static data of all the OLT equipment according to the specific record number and adds the static data into a character String set ArrayList < String >;
Similarly, the multi-source protocol adapter of the data network management access layer of the primary power grid SGEPON is also specifically realized in this way for various northbound interface protocols of other manufacturers, and part of northbound interfaces of the EMS do not realize other protocols, so that the methods in the classes are realized in a null way for later expansion.
6. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein the heterogeneous big data of the grid are unified: the access layer performs data conversion in the analysis process, shields the isomerism of data, establishes a unified SGEPON network element equipment data model in advance, and provides a template for data conversion and data unification in the analysis process;
1) Power grid SGEPON device object resolution
SGEPON the data network management object comprises OLT, SHELF, BOARD, PON port and ONU, and the attribute information of the device object comprises: device background information, SGEPON static information, device real-time state information, SGEPON real-time alarm information and device log information, wherein the device background information comprises manufacturer and secondary power grid position information of a device, and the SGEPON static information comprises device basic address positioning and software and hardware version information; the equipment real-time state information comprises real-time management and running states; SGEPON the real-time alarm information comprises generation and recovery of the alarm information of the current equipment; the equipment log information comprises historical alarm information and equipment change information;
The association relation between managed objects is defined in SGEPON data network management, SGEPON is a point-to-multipoint topological structure on a network structure, and then the topological association relation of a multi-branch tree of ' province-city-OLT-SHELF-BOARD-PON-ONU ' is formed by combining province-city-OLT-SHELF-BOARD-PON-ONU ' with province-city-geographical position information, wherein province is a root node, and ONU is a leaf node;
2) Intelligent SGEPON device data collaborative coding
SGEPON the equipment metadata fusion model provides a unified conversion model, when the data network management of the first-level power grid SGEPON performs network element equipment data access, a processing thread of an access layer performs data analysis, filtering and field attribute extraction according to the model, encodes the second-level power grid information, encodes manufacturer information through a unique identity of a unique type CITY_CODE, performs final unification through a unique identity of the MAU_CODE, and stores consistent data into a table of a database.
7. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein a grid concurrency data model:
1) Access layer overall architecture model
The access layer of the data network management of the primary power grid SGEPON needs to realize the access of the data of the secondary power grid SGEPON at the same time, independent processes are designed aiming at each manufacturer in the access layer, the concurrency of multiple processes is realized, and each process comprises SGEPON equipment data access tasks of a plurality of secondary power grids;
Based on the concrete management realized by SGEPON data network management application layers, the SGEPON equipment data access program of each secondary power grid realizes corresponding service data access, and the method specifically comprises 4 steps: SGEPON network element equipment static data access, change data access, state data access and alarm data access, wherein an access program of each secondary power grid opens a separate thread for each service, and the access program of each service data comprises communication control, data acquisition, data processing and data storage processes;
The access layer of SGEPON data network management is communicated with the northbound interface of each secondary power grid EMS, data access of SGEPON network element equipment is carried out, and the implementation of each secondary power grid of the access layer is divided into 4 modules according to service and function division;
2) Static data access model
Firstly, an access layer establishes communication connection with a service port of a manufacturer EMS northbound interface according to a communication mechanism of a multi-source protocol, if the communication connection is not established successfully, the upper limit of reconnection times is 3 times, if the connection is established continuously for 3 times, the current task is directly ended, the next opportunity is waited, after the communication connection is established successfully, the access layer sends an authentication command according to the regulation and command format of the multi-source protocol, reads and analyzes the returned authentication result, if the authentication fails, the authentication is revised according to the returned error reason, and the authentication is carried out again; after authentication is successful, a query command is sent according to a multi-source protocol to query the static data of SGEPON equipment of different types, then data acquisition, data processing, data unification and data storage are carried out, finally, after the query is completed, the connection is disconnected, and resources such as occupied ports are released;
3) Altering data access models
The method comprises the steps that a push mode is adopted in data changing access, firstly, an access layer establishes communication connection with a northbound interface of an EMS (enhanced management system) of a manufacturer, authentication is successful, then, the access layer sends a registration command for enabling the EMS to actively report the changed data according to a multi-source protocol rule, judges whether registration is successful or not according to reply information, if the registration is failed, the registration command is corrected according to error reasons in the reply information, and then, the registration is repeated, after the registration is successful, SGEPON equipment pushed by the EMS is monitored, the complete data message is read once the pushed changed data is monitored by the access layer, analysis is carried out, the changed network element equipment is judged through the analyzed data, and what change happens, according to the type and the change condition of the changed equipment, the access layer interacts with the data layer, insert, delete, update operation is carried out on a corresponding data table, the consistency of the data is guaranteed, and the equipment change information is stored in an equipment change table, so that the data is shared to an information collecting service management system;
The access layer establishes long connection with the northbound interface of the manufacturer EMS, always keeps a monitoring state, creates subtasks by using a heartbeat packet mechanism of a multi-source protocol, uses the same communication connection, periodically sends heartbeat commands to the northbound interface, and keeps long connection with the northbound interface of the manufacturer EMS.
8. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein the smart SGEPON concurrency thread pool: the method comprises the steps that different service data access modes in an access layer are different, thread pool configurations to be adopted are also different, firstly, an access layer program directly uses a concurrent framework tool class Executors to call a static method newScheduledThreadPool (int) to create a ScheduledExecutorService type thread pool, the object is named execService, the core thread number of the thread pool is set to be 8, the delay and periodic execution of the task are supported, then, an independent task is created for real-Time state access of each secondary power grid SGEPON device, the task of each secondary power grid further comprises a communication thread, a data processing thread and a storage thread, finally, a thread pool object execService calls scheduleAtFixedRate (Runnable command, long initialDelay, longperiod, a Time Unit) to submit the SGEPON device state data access task of each secondary power grid, the first parameter command transmitted represents a specific task, the second parameter INITIALDELAY represents initialization delay, the third parameter period represents the task execution period, the fourth parameter represents a Time Unit of the delay and period, the delay and period are staggered, the task is set in each secondary power grid 372, the task is accessed in a state, the period is set, the period is further, the period is changed, and the period is set by setting up and the period is further, the period is changed, and the period is changed by setting up of the period is changed.
9. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein priority task queue control: all tasks in the priority task queue PriorityBlockingQueue realize a complete interface and rewrite compareTo () method, the method transmits the same type of task object, and compares the priorities of the two task objects, thereby realizing sequencing, one task is defined in a SGEPON data network management access layer program to be inherited by a Thread class or realize Runnable interface realization, but none of the tasks realize the complete interface, for realizing code multiplexing, priorityThread and PriorityRunnable classes are defined first, a priority member variable is declared in the Thread class, if not set, the priority class is defaulted to 5, the abstract class is defined by inheriting the Thread class and realizing the complete interface, and the compareTo () method is rewritten, the PriorityRunnable class is required to be defined as an abstract class, meanwhile, the Runnable interface and the complete interface are realized, but only the compareTo () method is rewritten, the run () method is not rewritten, and the priority of the task is represented by defining the member variable priority, and the range of the priority class is also taken as the integer between 1 and 10;
In addition, the access layer performs level assessment and setting for each service data access and subtasks included in the service data access, the priority level of the subtasks for alarm data acquisition, alarm data analysis and storage is set to be the highest 10, and the priority level of the corresponding heartbeat packet is set to be 3, because the user pays more attention to the alarm information of SGEPON devices, if the alarm information exists, the user processes the alarm information preferentially, the heartbeat packet is used for ensuring that when no data is transmitted on a communication connection for a long time, the connection is prevented from being disconnected, and the alarm data is transmitted on the communication connection at present, the heartbeat packet is not transmitted temporarily, so the priority level for transmitting the heartbeat packet is set to be low.
10. The smart grid heterogeneous high concurrency big data collaborative access system according to claim 1, wherein a grid C3PO database connection pool controls: the configuration of the CPO database connection pool in the access layer is based on the C3PO database connection pool, the configuration of the CPO database connection pool adopts xml file configuration, the file is named as C3p0-config.xml, the file is placed under the class path of the item, jdbcUrl. Driverclass. User, password, acquireIncrement, initialPoolSize, minPoolSize and maxPoolSize parameters are set in the configuration file, jdbcUrl represents url addresses of database connection, different database url addresses represent different formats, the connection number in the pool is initialized to 10 through initialPoolSize, the minimum connection number in the pool is set to 10 through minPoolSize, when the database connection in the pool is insufficient, the connection number is increased by acquireIncrement each time, but the maximum connection number in the pool is maxPoolSize is 50, when the connection number in the pool reaches the maximum and is used, new tasks are queued for waiting for connection to be free;
Setting corresponding parameters of a C3PO database connection pool through an xml configuration file, importing a required Jar package, writing a database connection management tool class JDBCUtils through Java language, firstly, defining JDBCUtils through static member variables and initializing a thread pool dataSource object, defining proprietary connection related to a database transaction, binding the proprietary connection on a corresponding thread to prevent data security problems in a multithreaded environment, then, defining JDBCUtils a static method getConnection () for acquiring the database connection, acquiring the database connection pool through defining a static method getDataSource (), releasing the connection through defining releaseConnection (Connectionconnection) method, putting the connection after completion into the database connection pool, and ensuring atomicity, consistency, isolation and durability of the database transaction processing by calling corresponding transaction methods in a program, wherein the real database physical connection is not closed, and the three static methods beginTransaction (), commitTransaction (), and rollbackTransaction () are related to transactions in the database, and are sequentially opened, committed and rolled back.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210565009.1A CN114978850B (en) | 2022-05-23 | Heterogeneous high concurrency big data collaborative access system of smart power grid |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210565009.1A CN114978850B (en) | 2022-05-23 | Heterogeneous high concurrency big data collaborative access system of smart power grid |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114978850A CN114978850A (en) | 2022-08-30 |
| CN114978850B true CN114978850B (en) | 2024-07-02 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101179427A (en) * | 2007-11-07 | 2008-05-14 | 西安大唐电信有限公司 | Unified management method of Ethernet passive optical network multi-criterion ONU |
| CN104079436A (en) * | 2014-07-14 | 2014-10-01 | 武汉大学 | Cross-device and cross-protocol EPON element management system |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101179427A (en) * | 2007-11-07 | 2008-05-14 | 西安大唐电信有限公司 | Unified management method of Ethernet passive optical network multi-criterion ONU |
| CN104079436A (en) * | 2014-07-14 | 2014-10-01 | 武汉大学 | Cross-device and cross-protocol EPON element management system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104079436B (en) | A kind of Element management system of striding equipment in the EPON networks of agreement | |
| CN100544282C (en) | A kind of trouble correlation analytic system and method | |
| EP1361761A1 (en) | Telecommunications network management system and method for service monitoring | |
| CN111506412A (en) | Distributed asynchronous task construction and scheduling system and method based on Airflow | |
| CN105721562B (en) | A kind of Heterogeneous service call method based on agency with cooperate with calling system | |
| CN103546343B (en) | The network traffics methods of exhibiting of network traffic analysis system and system | |
| CN102387043B (en) | Alarm analysis method, workstation and system based on simple network management protocol | |
| US10536397B2 (en) | Packet count-based object locking protocol | |
| CN116489214A (en) | Unified service scheduling method, device, medium and equipment based on micro service gateway | |
| CN113923200A (en) | Method and device for realizing massive API gateway service | |
| CN114422542A (en) | Terminal domain management system | |
| CN114978850B (en) | Heterogeneous high concurrency big data collaborative access system of smart power grid | |
| CN102684894B (en) | Method and device for realizing northboundbound interface | |
| CN112149975B (en) | APM monitoring system and method based on artificial intelligence | |
| CN113515267A (en) | PaaS platform based on industrial Internet of things | |
| CN114915533B (en) | Method and architecture for realizing north interface based on platform | |
| CN105429805A (en) | Distributed network management system and information processing method | |
| CN113824801B (en) | Intelligent integration terminal unified access management component system | |
| Cheng et al. | Design of IT infrastructure multicloud management platform based on hybrid cloud | |
| CN106202585B (en) | The more scene Multi-state data systems of electric power and management method | |
| CN114978850A (en) | Heterogeneous high-concurrency big data collaborative access system for smart power grid | |
| Jukić et al. | Fault management API for SNMP agents | |
| CN113888139A (en) | Construction method and system of unified substation data asset management system | |
| CN112422349A (en) | Network management system, method, equipment and medium for NFV | |
| Zheng et al. | Considerations on the Model Selection of SDOTN Controller Northbound Interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240528 Address after: Room 1006, 10th Floor, Building 3, Tianjianhu Intelligent Networking Industrial Park, No. 173 Fengxiang Street, High tech Industrial Development Zone, Zhengzhou City, Henan Province, 450000 Applicant after: HENAN DONGLING ELECTRONIC TECHNOLOGY Co.,Ltd. Country or region after: China Address before: No. 2, South Street, North China Electric Power University, Changping District, Beijing 102200 Applicant before: Chen Pengfei Country or region before: China |
|
| GR01 | Patent grant |