CN107357856B - Method for realizing data integration and data service based on power grid panoramic service model - Google Patents

Method for realizing data integration and data service based on power grid panoramic service model Download PDF

Info

Publication number
CN107357856B
CN107357856B CN201710516052.8A CN201710516052A CN107357856B CN 107357856 B CN107357856 B CN 107357856B CN 201710516052 A CN201710516052 A CN 201710516052A CN 107357856 B CN107357856 B CN 107357856B
Authority
CN
China
Prior art keywords
data
service
power grid
model
micro
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710516052.8A
Other languages
Chinese (zh)
Other versions
CN107357856A (en
Inventor
曹伟
叶桂南
刘媛
韦昌福
阮诗迪
王含乾
农晓宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangxi Power Grid Co Ltd
Original Assignee
Guangxi Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangxi Power Grid Co Ltd filed Critical Guangxi Power Grid Co Ltd
Priority to CN201710516052.8A priority Critical patent/CN107357856B/en
Publication of CN107357856A publication Critical patent/CN107357856A/en
Application granted granted Critical
Publication of CN107357856B publication Critical patent/CN107357856B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06Energy or water supply

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Water Supply & Treatment (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a method for realizing data integration and data service based on a power grid panoramic service model, and relates to the technical field of power grid system informatization. The method for realizing data integration and data service based on the power grid panoramic business model is characterized in that data of all business systems, all level systems and all department systems are dispatched, a modeling tool and a data extraction tool are unified, and data are collected and processed to provide data service in a micro-service architecture mode; the invention relies on a power grid panoramic service model, realizes the integration of scheduling domain data, overcomes barriers among various service application systems, establishes a uniform and standard scheduling data market, provides uniform data support for a report module and other application modules, avoids the problems of inaccurate, non-unique and non-uniform information data of a power grid system, and provides a high-efficiency and convenient data management function.

Description

Method for realizing data integration and data service based on power grid panoramic service model
Technical Field
The invention belongs to the technical field of power grid system informatization, and particularly relates to a method for realizing data integration and data service based on a power grid panoramic business model.
Background
With the continuous and deep application of information technology in the field of power grid dispatching operation, the access of each service system planned and constructed is continuously increased, and the association of each service is more and more intimate. In addition, the development of power grid services and the improvement of management requirements are increased, the requirements for information sharing and cooperation among service applications, between scheduling and other departments of a company and between upper and lower scheduling mechanisms are higher and higher, and the problems of difficult information sharing, incapability of managing and controlling data, island form and the like are more and more prominent. In order to meet the requirements of power grid service development and fine management of scheduling operation, a data support platform capable of integrating and managing various information such as a multi-level scheduling power grid model, data and graphics is urgently needed, the high-efficiency application analysis of a large-scale complex power grid is met, technical support is provided for each application system, the analysis and evaluation of system operation data are realized on the basis, a data basis is provided for other comprehensive data applications, and the main requirement analysis is as follows:
1. various data are dispersed in each business system, and partial systems lack mechanisms and means for integrating the data due to the fact that the systems are not open, the data are difficult to be comprehensively utilized, support cannot be well provided for decision analysis, the return rate of system investment is low, the data of each business system need to be effectively fused, centralized and unified management is carried out, the data are converted into valuable information, and unified data support is provided for related business application.
2. The communication mode is various, and the stipulation is eight, and mostly need independent communication channel, require highly and communication operation, maintenance, management complicacy to communication resource. The network and point-to-point interaction mode is backward, data service is difficult to manage, relationship and continuous operation and maintenance are difficult to clear with the increase of information systems, service encapsulation and standard interface development are carried out according to a standardized interface mode of an operation service bus to establish a data acquisition, verification and processing platform, interconnection, intercommunication and interoperation of systems at all levels are realized, seamless integration of 6+1 and scheduling of the existing service system is realized, and unified resource sharing and coordinated control among the systems and between the upper level and the lower level are improved, so that transverse cooperation and longitudinal communication are realized.
3. The whole scheduling service domain lacks a unified information model, the data and service reuse rate is low, when data or application interaction requirements exist, brand-new development is often needed, the same or similar service data are repeatedly maintained in a plurality of systems, the workload is high, the consistency is poor, the data are uniformly collected through building a standard data model, the standard and a control mode are unified, and different data regions are controlled and provided for application in different scheduling grades.
4. Data are externally provided with applications through different service modes, accuracy of data sources cannot be guaranteed, data quality is uneven, integrated control over data quality and data applications is lacked, reliability of the data cannot be guaranteed, a data quality tracking service needs to be established, data acquisition and application are tracked, a data quality problem tracking report is acquired through association with a data quality detection and data quality audit platform, support is provided for accurate data extraction, and effectiveness and accuracy of the data are guaranteed.
5. At present, data are repeated from multiple sources, effective summarization and analysis services cannot be provided, a unified statistic summarization analysis service needs to be established, unified management of the data is realized, one data, one inlet, one outlet and multi-stage application are really realized, related data are automatically obtained from multiple heterogeneous data sources, validity verification is carried out according to the attribute relationship of the data, the operation modes of automatic calculation, statistics, summarization and automatic definition are realized, and the requirements of different service scene applications and different service domain application are met.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method for realizing data integration and data service based on a power grid panoramic business model, which is characterized in that data of all business systems, all levels of systems and all departments are dispatched, unified modeling tools and data extraction tools are adopted, and data are collected and processed to provide data service in a micro-service architecture mode; the data of all service systems, all levels of systems and all department systems can be effectively and accurately fused together, data service according with data uniqueness is provided, and the problems that information data of a power grid system is inaccurate, not unique and not uniform are solved.
The invention solves the technical problems through the following technical scheme: a method for realizing data integration and data service based on a power grid panoramic service model comprises the following steps:
(1) establishing a unified power grid information model, and performing standard modeling: the unified power grid information model is informationized by using an object-oriented concept, a model structure is established by using a modeling language expression method through a modeling tool, a model physical table is generated, and the unified power grid information model is defined into a group of packages;
(2) extracting multi-system data: through the establishment of the power grid information model and the generation of the physical table structure which are completed in the step (1), the unified extraction work of the multi-system data is realized; determining the content of system integration data, extracting data in each service system, each level system and each department system, completing data integration, enabling the integrated data to meet the data requirements of a power grid information model, and providing data support for subsequent micro-service access;
(3) deployment of the microservice: the method comprises the following steps of developing a data support platform by adopting a micro-service architecture, and decomposing complex business and huge single application into a plurality of relatively small services for development; in the case where the original functionality provided is unchanged, the application is broken down into a number of manageable branches or services; so as to solve the complex or heterogeneous deployment environment and provide favorable conditions for subsequent maintenance; and starting the micro-service;
(4) micro-service access: the micro-service access is used for providing data service for the outside, and power grid information data of multiple systems are obtained by accessing the micro-server; the micro-service access comprises authority authentication and service access, and when the micro-service is subjected to data access, identity verification is required to be performed so as to prevent data information from being leaked and maliciously accessed, and the safety and normal access of the data information are ensured.
Further, the modeling tool in the step (1) adopts a UM L modeling tool.
Further, the power grid information model in the step (1) comprises a power grid resource package, an asset equipment package, a power consumption customer package, a topology connection package, a measurement resource package and a public support package; the Power grid Resource package is used for describing the relationship among substations, lines and transformers in a Power grid and is represented by taking Power System Resource (Power System Resource) objects as cores; the Asset equipment package is used for describing assets and relationships between the assets and functional positions and is represented by taking Asset (Asset) objects as cores; the power utilization customer package is used for describing the relationship between the transformer and the customer in the power grid; the topology connection package is used for describing the connection relation between the conductive devices in the power grid; the measurement resource package is used for describing the relation between the operation data in the power grid and the power grid resources; the common support package is used for providing classes for organization, location, etc. to be used by other packages and global data type definitions.
Further, the step (1) comprises the following substeps:
1.1, establishing an entity class and a virtual class, establishing entity identifiers, Chinese identifiers, English names and the like of the entity class and the virtual class, selecting a parent entity, and enabling the entity class and the virtual class to inherit all attributes of the parent entity without reconstruction, establishing attribute identifiers, Chinese identifiers, English identifiers, descriptions, data types, lengths, precision, whether a main key is unique, whether the main key is not empty or not, whether the main key is displayed in a model or not, establishing entity relationship classification such as aggregation, self-aggregation, inheritance and relationship, and establishing the relationship types such as 1-1, 1-N, 0-1 and the like;
1.2, generating a model physical table, namely releasing a new version of the input and edited power grid information model data, wherein the released version is used for establishing a physical table model and providing a power grid information model basis for data service; selecting a server where the data are located to generate a model physical table, wherein the model physical table is used for storing a table structure of the data; the model reflection provides the requirement of uniform export of the report forms, and the model reflection generates a maintainable interface model by adopting a physical table reflection technology for the existing data structure, wherein the interface model is used for providing a basis for the micro-service;
1.3, establishing a data quality check strategy: establishing a rule template according to service constraint, for example, a network area meets a highest value, a line bearing capacity maximum value and the like; inputting a check rule according to business requirements, wherein the recording time is less than filing time, or selecting from a rule template; the data quality is checked according to the service requirement, for example, the data quality check is executed once every day, every month or every half hour, and the data quality check strategy is provided by adopting a micro-service mode; outputting a verification result of data quality verification to a corresponding log, and recording information such as the number of data which do not accord with a verification rule in detail in the log, so as to provide a basis for alarm information and guarantee the accuracy and uniqueness of the data;
sequentially inputting IEC61970 in the power grid information model through the steps: : core (Core), IEC 61970: : measurement (Meas), IEC 61970: : topology (Topology), IEC 61970: : electric wire (wirees), IEC 61970: : protection (Protection), IEC 61968: : common (Common), IEC 61968: : the asset (Assets) packages model class data, attributes, and their associations.
Further, the multisystem data extraction in the step (2) adopts an ET L tool.
Further, the multiple systems in the step (2) include provincial dispatching OCS systems, each regional OCS system, DMIS systems, energy-saving power generation dispatching systems, water dispatching automation systems, meteorological comprehensive systems and all-level PMS systems.
Further, the step (2) comprises the following substeps:
2.1 configuring data sources: the data source management module realizes the management function of data source connection information, realizes the connection of the data source by configuring information such as IP, user name, password and the like, realizes the functions of adding, modifying and deleting the data source, and supports the connection of data sources of Oracle, Mysql, easy whale, SqlServer, Sybase, Ftp and the like; sequentially inputting source data sources of each service system, each level system and each door system, and then inputting target data source information of the data support platform;
2.2 data table management: the data sheet management realizes the management function of collecting data sheet information, and carries out classification management according to a data source and a target library, and the data sheet management also comprises the functions of adding and deleting a data sheet and creating a target sheet; sequentially selecting a data table of a system to be extracted from a source data source, and generating source data table information after selecting a target source as a data support platform; generating a data support platform target table according to the requirement of a source data table, and adding and selecting database addresses of each system to be acquired in a data source of a data table management function; adding a database address of a selected data support platform in a target database, and executing the operation of creating a target table;
2.3 task scheduling configuration: the task scheduling configuration is used for scheduling the task configuration and executing the type, the executing time, the timing time and other information; filling a scheduling name in a newly-added scheduling page, selecting trigger time and execution time, and selecting and using the scheduling task in a data task after the scheduling task is configured;
2.4, a structured data task, namely realizing a management function of a collection loading task of structured data by the structured data task, supporting a mode of extracting according to an original table of a data table, supporting complex SQ L query of the data table and dynamic table query of a user-defined SQ L extraction function mode, sequentially adding tasks, selecting the data table needing to be added with the tasks, selecting execution nodes, setting scheduling time and a writing mode according to actual requirements, and completing operation after selecting options;
2.5 unstructured data tasks: the unstructured data task module realizes the management function of the collection and loading tasks of unstructured data and supports the transmission of unstructured data between FTP and local or between FTP and FTP;
and 2.6, integrating data, namely analyzing and integrating the data extracted into the data support platform library by using an SQ L statement on the basis of the power grid system service, converting the data into a data form meeting the requirement of a unified power grid information model, and inserting the data into a physical table structure corresponding to the model to finish data integration.
Further, the step (3) comprises the following substeps:
3.1 starting a resource service manager;
3.2 starting and deploying the authority management micro-service;
3.3 starting and deploying the metadata micro-service;
3.4 starting and deploying the data service microservice;
3.5 starting and deploying the micro service gateway;
3.6 starting and deploying data quality check micro service;
through the steps, the start of the micro service is completed, and the micro service automatically provides data service access according to the power grid information model for other systems to call; after the power grid information model is newly added or the release version is changed every time, the micro-service automatically provides data access according to the model modeling information without manual interference, and favorable conditions are provided for subsequent service expansion integration.
Further, the microservice access in the step (4) provides two data forms, namely XM L format data based on SOAP and RESTful format data based on HTTP protocol.
Compared with the prior art, the method for realizing data integration and data service based on the power grid panoramic business model, provided by the invention, has the advantages that data of all business systems, all levels of systems and all departments are dispatched, a modeling tool and a data extraction tool are unified, and data are collected and processed to provide data service in a micro-service architecture mode; the method depends on a power grid panoramic service model, realizes the integration of scheduling domain data, overcomes barriers among various service application systems, establishes a uniform and standard scheduling data market, provides uniform data support for a report module and other application modules, avoids the problems of inaccurate, non-unique and non-uniform information data of the power grid system, and provides a high-efficiency and convenient data management function; according to the invention, a power grid dispatching maintenance worker can quickly generate a unified power grid information model only by simple operation and release data service by depending on the model; under the condition that information modification is needed, only simple modification is needed to be carried out on model information, service is automatically generated without re-developing a service interface, and the workload of power grid dispatching maintenance personnel and information system management personnel is reduced.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only one embodiment of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a schematic diagram of the structure of the core classes and their relationships of the grid resource package of the present invention;
FIG. 2 is a schematic diagram of a device asset relationship structure of an asset device package of the present invention;
FIG. 3 is a schematic diagram of a relationship between a customer and a device of the electricity customer package according to the present invention;
FIG. 4 is a schematic diagram of a relationship between measurement of a measurement resource package and a power system resource according to the present invention;
fig. 5 is a schematic diagram of the topology relationship structure and the devices of the topology connection packet according to the present invention.
Detailed Description
The technical solutions in the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The basic idea of the implementation method of data integration and data service based on the power grid panoramic service model is as follows:
1. and checking and analyzing the modeling content, the model structure and the classification basis of the unified power grid information model and checking the incidence relation among the classes, analyzing the attribute information content by the model structure, and analyzing the difference after the modeling tool is converted into a physical table model of the computer.
2. And analyzing the data scattering positions of the systems where the various attributes are located according to the information content of the model, checking and determining the principle of extracting a data source, and realizing the regular collection and updating of data by using a data extraction tool.
3. Analyzing a data integration principle according to the model information providing content and the bias point of the service among the systems; and (4) sorting and analyzing the difference of data among the systems, for example, the asset information is basically stored in PMS systems at all levels, and the data of data scheduling OCS systems in 500kV and 220kV are the most accurate.
4. Thinking how to provide services according to the model, and generating the data access micro-service of the loosely-coupled architecture which is easy to develop, manage and maintain and conforms to the software development spirit after the corresponding computer physical table model is obtained when the model changes.
The invention provides a method for realizing data integration and data service based on a power grid panoramic service model, which comprises the following steps:
(1) establishing a unified power grid information model, namely performing standard modeling, namely informationizing the unified power grid information model by using an object-oriented concept, expressing the unified power grid information model by using a unified modeling language UM L through a UM L modeling tool, and defining the unified power grid information model into a group of packages;
(2) determining the content of system integration data, extracting data in each service system, each level system and each department system, completing data integration, enabling the integrated data to meet the data requirements of the power grid information model, and providing data support for subsequent micro-service access;
(3) deployment of the microservice: the method comprises the following steps of developing a data support platform by adopting a micro-service architecture, and decomposing complex business and huge single application into a plurality of relatively small services for development; in the case where the original functionality provided is unchanged, the application is broken down into a number of manageable branches or services; and starting the micro-service; so as to solve the complex or heterogeneous deployment environment and provide favorable conditions for subsequent maintenance;
(4) micro-service access: the micro-service access is used for providing data service for the outside, and power grid information data of multiple systems are obtained by accessing the micro-server; the micro-service access comprises authority authentication and service access, and when data access is performed on the micro-service, identity verification is required to be performed so as to prevent data information from being leaked and maliciously accessed and ensure the safety and normal access of the data information;
example 1, micro-service access was performed using the following format:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:mdm="http://mdm.com">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<mdm:SrchRecord>
<mdm:MDMReqHeader>
<mdm:UserInfo>
<mdm:Username>zhangsan</mdm:Username>
<mdm:Token>d8712a0e8c49dc77c688f107d37972f8</mdm:Token>
</mdm:UserInfo>
<mdm:Page>1</mdm:Page>
</mdm:MDMReqHeader>
<mdm:MDMReqBody>
<mdm:SrchClause>
<mdm:Objectuid>basevoltage</mdm:Objectuid>
</mdm:SrchClause>
</mdm:MDMReqBody>
</mdm:SrchRecord>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
the power grid information model in the step (1) comprises a power grid resource package, an asset equipment package, a power utilization customer package, a topology connection package, a measurement resource package and a public support package; the Power grid Resource package is used for describing the relationship among substations, lines and transformers in a Power grid and is represented by taking Power System Resource (Power System Resource) objects as cores; the Asset equipment package is used for describing assets and relationships between the assets and functional positions and is represented by taking Asset (Asset) objects as cores; the power utilization customer package is used for describing the relationship between equipment and customers in the power grid; the topology connection package is used for describing the connection relation between the conductive devices in the power grid; the measurement resource package is used for describing the relation between the operation data in the power grid and the power grid resources; the public support package is used for providing classes for organizations, positions and the like to be used by other packages and global data type definitions;
as shown in fig. 1 to 5, each package includes one or more class diagrams, which graphically illustrate all classes and their relationships in the package; then defining each class in a text form according to the attributes of the classes and the relation with other classes; the model standardization incidence relation among all classes is realized, and the association modes of generalization, aggregation and self-aggregation are supported.
The step (1) comprises the following substeps:
1.1, establishing an entity class and a virtual class, establishing entity identifiers, Chinese identifiers, English names and the like of the entity class and the virtual class, selecting a parent entity, and inheriting all attributes of the parent entity by the entity class and the virtual class without re-establishing, establishing attribute identifiers, Chinese identifiers, English identifiers, descriptions, data types, lengths, precision, whether a main key is unique, whether the main key is not empty or not, whether the main key is displayed in a model or not, establishing entity relationship classification such as aggregation, self-aggregation, inheritance and relationship, wherein the relationship types are 1-1, 1-N, 0-1 and the like;
1.2, generating a model physical table, namely releasing a new version of the input and edited power grid information model data, wherein the released version is used for establishing a physical table model and providing a power grid information model basis for data service; selecting a server where the data are located to generate a model physical table, wherein the model physical table is used for storing a table structure of the data; the model reflection provides the requirement of uniform export of the report forms, and the model reflection generates a maintainable interface model by adopting a physical table reflection technology for the existing data structure, wherein the interface model is used for providing a basis for the micro-service;
1.3, establishing a data quality check strategy: establishing a rule template according to service constraint, for example, a network area meets a highest value, a line bearing capacity maximum value and the like; inputting a check rule according to business requirements, wherein the recording time is less than filing time, or selecting from a rule template; the data is checked according to the business requirements, such as data checking is executed every day, every month or every half hour, and a data quality checking strategy is provided in a micro-service mode; outputting a verification result of data quality verification to a corresponding log, and recording information such as the number of data which do not accord with a verification rule in detail in the log, so as to provide a basis for alarm information and guarantee the accuracy and uniqueness of the data;
sequentially inputting IEC61970 in the power grid information model through the steps: : core (Core), IEC 61970: : measurement (Meas), IEC 61970: : topology (Topology), IEC 61970: : electric wire (wirees), IEC 61970: : protection (Protection), IEC 61968: : common (Common), IEC 61968: : the asset (Assets) packages model class data, attributes, and their associations.
The multiple systems in the step (2) comprise provincial dispatching OCS systems, each regional OCS system, DMIS systems, energy-saving power generation dispatching systems, water dispatching automation systems, meteorological comprehensive systems and all levels of PMS systems.
The step (2) comprises the following substeps:
2.1, configuring a data source, namely, realizing a management function of data source connection information by a data source management module of an ET L tool, realizing connection of the data source by configuring information such as IP, user name, password and the like, realizing the functions of adding, modifying and deleting the data source, supporting the connection of data sources of Oracle, Mysql, easy whale, SqlServer, Sybase, Ftp and the like, sequentially inputting source data sources of each service system, each level system and each door system, and then inputting target data source information of a data support platform;
2.2 data table management: the data sheet management realizes the management function of collecting data sheet information, and carries out classification management according to a data source and a target library, and the data sheet management also comprises the functions of adding and deleting a data sheet and creating a target sheet; sequentially selecting a data table of a system to be extracted from a source data source, and generating source data table information after selecting a target source as a data support platform; generating a data support platform target table according to the requirement of a source data table, and adding and selecting database addresses of each system to be acquired in a data source of a data table management function; adding a database address of a selected data support platform in a target database, and executing the operation of creating a target table;
2.3 task scheduling configuration: the task scheduling configuration is used for scheduling the task configuration and executing the type, the executing time, the timing time and other information; filling a scheduling name in a newly-added scheduling page, selecting triggering time as 12 nights, selecting execution time as daily execution, and selecting the scheduling task for use in a data task after the scheduling task is configured;
2.4, a structured data task, namely realizing a management function of acquiring and loading a task of structured data by the structured data task, supporting a mode of extracting according to an original table of a data table, supporting complex SQ L query of the data table and dynamic table query of a user-defined SQ L extraction function mode, sequentially adding the tasks, selecting the data table needing to be added with the tasks, selecting an execution node, setting scheduling time as 12 o' clock at night according to actual requirements, setting a writing mode as intelligent insertion, and completing operation after selection of options;
the ET L tool extracts the structured data of each system data source automatically at 12 o' clock at night into the data support platform library, and the extracted data are integrated and converted in the subsequent steps and then stored into the model data table;
2.5 unstructured data tasks: the unstructured data task module realizes the management function of the collection and loading tasks of unstructured data and supports the transmission of unstructured data between FTP and local or between FTP and FTP;
because the quasi-real-time data exists in the described system, the storage mode of the system data is mostly stored in an E file format, so that the ET L tool particularly supports the release of unstructured data tasks, and E-format files in each system are transmitted to a data support platform and then are analyzed and converted into structured data;
2.6, integrating data, namely analyzing and integrating the data extracted into the data support platform library by using SQ L statements on the basis of the power grid system service, converting the data into a data form meeting the requirement of a unified power grid information model, and inserting the data into a physical table structure corresponding to the model to finish data integration;
according to the contents of the model class package and the attributes, the scattered positions of the physical table structures of the systems are combed, the service characteristics and the data structures of the systems are arranged, and the integrated data are cleaned; by eliminating error and invalid data in the original data and converting the original data according to a uniform data model, credible, usable and reliable data are provided for subsequent application.
Example 2, taking the principle of data cleaning integration between the OCS system and the PMS system as an example:
determining the specification requirement of a general principle based on a power grid panoramic service model, taking an OCS system as a reference and a PMS system as a supplement, and performing data integration, conversion and processing according to the power grid panoramic service model, wherein the specification requirement specifically comprises the following steps:
2.6.1 acquiring all equipment information of a large user, a power plant and a traction station from an OCS (online charging system) to ensure the integrity of the data;
2.6.2 major types of the ultrahigh-voltage and 220-kV transformer substation and the line have repeatability due to the fact that an OCS system and a PMS system exist at the same time, the OCS system is taken as a main part, plant name and line name information of the ultrahigh-voltage and 220-kV transformer substation is obtained from the OCS system, equipment names and parameter information of the three major types are obtained from the PMS system, and uniqueness and integrity of the information are guaranteed;
2.6.3 station and equipment information which other OCS systems do not have, such as 110kV substation information, is all obtained from the PMS system;
2.6.4, the device information obtained from the PMS system reserves the device naming field, adds the scheduling naming field and carries out unified refreshing according to the scheduling naming rule;
2.6.5 maintaining the station information at two ends of the line, and specifically acquiring the station information at two ends from the PMS to supplement the station information to the line information;
2.6.6 establishing a factory station name table of the OCS system, if new factory stations and circuits are added, taking the table as a unique source of the factory stations, and updating the equipment information of the PMS system according to the table;
the method can integrate the DMIS system, the energy-saving power generation dispatching system, the water regulation automation system, the meteorological integrated system and the power grid metering automation system, and after data is cleaned and integrated, the SQ L statement is inserted into the model table structure to complete data integration of all systems.
The step (3) comprises the following substeps:
3.1 starting deployment resource service manager
The start command is:
nohup java -jar/home/etl/migrate/diaodu/mforcecload/mforce-registry-2.0.0.war --config.location=/home/etl/migrate/diaodu/mforcecload/central-config>registry.out 2>&1
nohup java -jar
3.2 starting deployment Authority management micro-service
The start command is:
/home/etl/migrate/diaodu/mforcecload/mforce-authserver-2.0.0.war--config.host=127.0.0.1>authserver.out 2>&1&
3.3 launching deployment metadata microservices
The start command is:
nohup java -jar /home/etl/migrate/diaodu/mforcecload/mforce-metadata-ms-2.0.0.jar--config.host=127.0.0.1>metadata-ms.out 2>&1&
3.4 initiating deployment of data service microservices
The start command is:
nohup java -jar /home/etl/migrate/diaodu/mforcecload/mforce-dataservice-ms-2.0.0.jar --config.host=127.0.0.1>dataservice-ms.out 2>&1&
3.5 starting deployment of microservice gateway
The start command is:
nohup java -jar /home/etl/migrate/diaodu/mforcecload/mforce-gateway-2.0.0.war --config.host=127.0.0.1 --server.port=8084>gateway.out 2>&1&
3.6 initiating deployment of data quality check microservices
The start command is:
nohup java -jar /home/etl/migrate/diaodu/mforcecload/mforce-qualitycheck-ms-2.0.0.jar --config.host=127.0.0.1 --qc.tmp=/home/etl/migrate/diaodu/mforcecload/tmp/>qualitycheck-ms.out 2>&1&
through the steps, the start of the micro service is completed, and the micro service automatically provides data service access according to the power grid information model for other systems to call; after the power grid information model is newly added or the release version is changed every time, the micro-service automatically provides data access according to the model modeling information without manual interference, and favorable conditions are provided for subsequent service expansion integration.
After the above steps are completed, the micro service is ready for use, but since the micro service is not subjected to security authentication, the service interface has the possibility of being maliciously accessed, and therefore, the access authority authentication is required to ensure normal data access.
The micro-service access in the step 4 provides two data forms, namely XM L format data based on SOAP with higher security level and RESTful format data based on HTTP protocol with higher flexibility.
Example 3, XM L format:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"></SOAP-ENV:Header>
<soap:Body>
<SrchRecordResponse xmlns="http://mdm.com">
<Result>
<MDMRespHeader>
<Messages>
<RespMsg/>
<RespCode>Y</RespCode>
</Messages>
<TotalPage>5</TotalPage>
<Page>1</Page>
<TotalRows>100</TotalRows>
</MDMRespHeader>
<FIELD>
<KEY>ISDC</KEY>
<VALUE/>
</FIELD>
<FIELD>
<KEY>NOMINALVOLTAGE</KEY>
<VALUE>30</VALUE>
</FIELD>
<FIELD>
<KEY>OCSID</KEY>
<VALUE/>
</FIELD>
<FIELD>
<KEY>ALIASNAME</KEY>
<VALUE>±30V</VALUE>
</FIELD>
<FIELD>
<KEY>DESCRIPTION</KEY>
<VALUE/>
</FIELD>
<FIELD>
<KEY>LOCALNAME</KEY>
<VALUE/>
</FIELD>
<FIELD>
<KEY>MRID</KEY>
<VALUE>67</VALUE>
</FIELD>
<FIELD>
<KEY>NAME</KEY>
<VALUE>±30V</VALUE>
</FIELD>
<FIELD>
<KEY>PATHNAME</KEY>
<VALUE/>
</FIELD>
<FIELD>
<KEY>ROWNUM_</KEY>
<VALUE>40</VALUE>
</FIELD>
</Record>
</MDMRespBody>
</Result>
</SrchRecordResponse>
</soap:Body>
</soap:Envelope>
example 4, restdu L format:
{"result":{"mdmrespHeader":{"messages":{"respMsg":"","respCode":"Y"},"page":"1","totalPage":"5","totalRows":"100"},"mdmrespBody":{"record":[{"objectuid":null,"field":[{"key":"ID","value":"26"},{"key":"ISDC","value":""},{"key":"NOMINALVOLTAGE","value":"1.5"},{"key":"OCSID","value":""},{"key":"ALIASNAME","value":"±1.5V"},{"key":"DESCRIPTION","value":""},{"key":"LOCALNAME","value":""},{"key":"MRID","value":"57"},{"key":"NAME","value":"±1.5V"},{"key":"PATHNAME","value":""},{"key":"ROWNUM_","value":"21"}],"subRecord":[]},{"objectuid":null,"field":[{"key":"ID","value":"30"},{"key":"ISDC","value":""},{"key":"NOMINALVOLTAGE","value":"2"},{"key":"OCSID","value":""},{"key":"ALIASNAME","value":"±2V"},{"key":"DESCRIPTION","value":""},{"key":"LOCALNAME","value":""},{"key":"MRID","value":"58"},{"key":"NAME","value":"±2V"},{"key":"PATHNAME","value":""},{"key":"ROWNUM_","value":"22"}],"subRecord":[]},{"objectuid":null,"field":[{"key":"ID","value":"34"},{"key":"ISDC","value":""},{"key":"NOMINALVOLTAGE","value":"2.4"},{"key":"OCSID","value":""},{"key":"ALIASNAME","value":"±2.4V"},{"key":"DESCRIPTION","value":""}"}],"subRecord":[]}]}}}。
the above disclosure is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or modifications within the technical scope of the present invention, and shall be covered by the scope of the present invention.

Claims (7)

1. A method for realizing data integration and data service based on a power grid panoramic service model is characterized by comprising the following steps: the method comprises the following steps:
(1) establishing a unified power grid information model, and performing standard modeling: the unified power grid information model is informationized by using an object-oriented concept, a model structure is established by using a modeling language expression method through a modeling tool, a model physical table is generated, and the unified power grid information model is defined into a group of packages;
the modeling tool adopts a UM L modeling tool;
the method specifically comprises the following substeps:
1.1, establishing an entity class and a virtual class, establishing entity identifiers, Chinese identifiers and English names of the entity class and the virtual class, selecting a parent entity, and inheriting all attributes of the parent entity by the entity class and the virtual class without re-establishing;
1.2 generating a model physical table: the edited power grid information model data is input and released into a new version, and the released version is used for building a physical table structure and providing a power grid information model basis for data service; selecting a server where the data are located to generate a model physical table, wherein the model physical table is used for storing a table structure of the data; the model reflection provides the requirement of uniform export of the report forms, and the model reflection generates a maintainable interface model by adopting a physical table reflection technology for the existing data structure, wherein the interface model is used for providing a basis for the micro-service;
1.3, establishing a data quality check strategy: creating a rule template according to the service constraint; inputting a check rule according to the service requirement, or selecting from a rule template; verifying the data quality according to the service requirement, wherein a data quality verification strategy is provided by adopting a micro-service mode; outputting a verification result of data quality verification to a corresponding log, and recording the number of data which do not accord with the verification rule in detail in the log;
(2) extracting multi-system data: through the establishment of the power grid information model and the generation of the physical table structure which are completed in the step (1), the unified extraction work of the multi-system data is realized; determining the content of system integration data, extracting data in each service system, each level system and each department system, completing data integration, enabling the integrated data to meet the data requirements of a power grid information model, and providing data support for subsequent micro-service access;
(3) deployment of the microservice: the method comprises the following steps of developing a data support platform by adopting a micro-service architecture, and decomposing complex business and huge single application into a plurality of relatively small services for development; in the case where the original functionality provided is unchanged, the application is broken down into a number of manageable branches or services; and starting the micro-service;
(4) micro-service access: the micro-service access is used for providing data service for the outside, and power grid information data of multiple systems are obtained by accessing the micro-server; the micro-service access comprises authority authentication and service access, and identity verification is required when the micro-service is subjected to data access.
2. The method for implementing data integration and data service based on the power grid panoramic service model according to claim 1, characterized in that: the power grid information model in the step (1) comprises a power grid resource package, an asset equipment package, a power utilization customer package, a topology connection package, a measurement resource package and a public support package; the power grid resource package is used for describing the relationship among substations, lines and transformers in a power grid and is represented by taking power system resource objects as cores; the asset equipment package is used for describing assets and the relation between the assets and the functional position and is represented by taking asset objects as cores; the power utilization customer package is used for describing the relationship between the transformer and the customer in the power grid; the topology connection package is used for describing the connection relation between the conductive devices in the power grid; the measurement resource package is used for describing the relation between the operation data in the power grid and the power grid resources; the common support package is used for providing organization, classes for other packages and global data type definitions.
3. The method for realizing data integration and data service based on the power grid panoramic business model as claimed in claim 1, wherein the multisystem data extraction in the step (2) adopts an ET L tool.
4. The method for implementing data integration and data service based on the power grid panoramic service model according to claim 3, characterized in that: the multiple systems in the step (2) comprise provincial dispatching OCS systems, each regional OCS system, DMIS systems, energy-saving power generation dispatching systems, water dispatching automation systems, meteorological comprehensive systems and all levels of PMS systems.
5. The method for implementing data integration and data service based on the power grid panoramic service model as claimed in claim 4, wherein: the step (2) comprises the following substeps:
2.1 configuring data sources: the data source management module realizes the management function of data source connection information, and realizes the connection of the data source by configuring IP, user name and password information; sequentially inputting source data sources of each service system, each level system and each door system, and then inputting target data source information of the data support platform;
2.2 data table management: the data table management realizes the management of the collected data table information, and carries out classification management according to a data source and a target library, and adds and deletes the data table and creates a target table; sequentially selecting a data table of a system to be extracted from a source data source, and generating source data table information after selecting a target source as a data support platform; generating a data support platform target table according to the requirement of a source data table, and adding and selecting database addresses of each system to be acquired in a data source of a data table management function; adding a database address of a selected data support platform in a target database, and executing the operation of creating a target table;
2.3 task scheduling configuration: the task scheduling configuration is used for scheduling task configuration execution type, execution time and timing time; filling a scheduling name in a newly-added scheduling page, selecting trigger time and execution time, and selecting and using the scheduling task in a data task after the scheduling task is configured;
2.4, a structured data task, namely realizing a management function of a collection loading task of structured data by the structured data task, supporting a mode of extracting according to an original table of a data table, supporting complex SQ L query of the data table and dynamic table query of a user-defined SQ L extraction function mode, sequentially adding tasks, selecting the data table needing to be added with the tasks, selecting execution nodes, setting scheduling time and a writing mode according to actual requirements, and completing operation after selecting options;
2.5 unstructured data tasks: the unstructured data task module realizes the management of the collection and loading tasks of unstructured data and supports the transmission of unstructured data between FTP and local or between FTP and FTP;
and 2.6, integrating data, namely analyzing and integrating the data extracted into the data support platform library by using an SQ L statement on the basis of the power grid system service, converting the data into a data form meeting the requirement of a unified power grid information model, and inserting the data into a physical table structure corresponding to the model to finish data integration.
6. The method for implementing data integration and data service based on the power grid panoramic service model according to claim 5, wherein the method comprises the following steps: the step (3) comprises the following substeps:
3.1 starting a resource service manager;
3.2 starting and deploying the authority management micro-service;
3.3 starting and deploying the metadata micro-service;
3.4 starting and deploying the data service microservice;
3.5 starting and deploying the micro service gateway;
3.6 starting and deploying data quality check micro service;
through the steps, the start of the micro service is completed, and the micro service automatically provides data service access according to the power grid information model for other systems to call; after the power grid information model is newly added or the release version is changed every time, the micro-service automatically provides data access according to the model modeling information.
7. The method for implementing data integration and data service based on the panoramic business model of the power grid as claimed in claim 6, wherein the microservice access in the step (4) provides two data forms, namely XM L format data based on SOAP and RESTful format data based on HTTP protocol.
CN201710516052.8A 2017-06-29 2017-06-29 Method for realizing data integration and data service based on power grid panoramic service model Active CN107357856B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710516052.8A CN107357856B (en) 2017-06-29 2017-06-29 Method for realizing data integration and data service based on power grid panoramic service model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710516052.8A CN107357856B (en) 2017-06-29 2017-06-29 Method for realizing data integration and data service based on power grid panoramic service model

Publications (2)

Publication Number Publication Date
CN107357856A CN107357856A (en) 2017-11-17
CN107357856B true CN107357856B (en) 2020-07-14

Family

ID=60273687

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710516052.8A Active CN107357856B (en) 2017-06-29 2017-06-29 Method for realizing data integration and data service based on power grid panoramic service model

Country Status (1)

Country Link
CN (1) CN107357856B (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108052634B (en) * 2017-12-20 2021-11-12 江苏瑞中数据股份有限公司 Integration method of multi-information system of power grid production control large area and asset management large area
CN108040128A (en) * 2018-01-12 2018-05-15 国网福建省电力有限公司 It is a kind of to realize that file data uploads synchronous method automatically based on kettle
CN110097238A (en) * 2018-01-29 2019-08-06 全球能源互联网研究院有限公司 A kind of electrical energy power quality disturbance event methods of exhibiting and system
CN108647265B (en) * 2018-04-28 2021-08-27 新疆熙菱信息技术股份有限公司 Multi-platform data-based interactive system
CN108647320B (en) * 2018-05-10 2021-01-19 国网冀北电力有限公司秦皇岛供电公司 GIS-based power distribution automation master station system
CN108710696A (en) * 2018-05-23 2018-10-26 国网浙江省电力有限公司电力科学研究院 A kind of electric network data mining analysis platform and its implementation
CN109038832A (en) * 2018-09-19 2018-12-18 上海许继电气有限公司 Substation's auxiliary monitoring system based on micro services framework
CN109857570A (en) * 2018-12-29 2019-06-07 航天信息股份有限公司 The standardized data class micromodule of operation system
CN109902105B (en) * 2019-02-28 2021-08-17 携程旅游网络技术(上海)有限公司 Data query system, method, device and storage medium for micro-service architecture
CN110069559A (en) * 2019-03-21 2019-07-30 中国人民解放军陆军工程大学 A kind of analysis of Heterogeneous Information System data and integrated approach with height automatic control
CN110008045B (en) * 2019-04-12 2021-12-10 深圳乐信软件技术有限公司 Method, device and equipment for aggregating microservices and storage medium
CN110781248A (en) * 2019-09-27 2020-02-11 浙江省北大信息技术高等研究院 Multi-source heterogeneous data acquisition method and device
CN110780856B (en) * 2019-10-16 2023-10-20 南方电网科学研究院有限责任公司 Electricity data release platform based on micro-service
CN111860854B (en) * 2019-12-06 2024-05-07 北京嘀嘀无限科技发展有限公司 Model feature management system, model feature management method, and storage medium
CN111210126A (en) * 2019-12-27 2020-05-29 中国电力科学研究院有限公司 Application program system and method for ubiquitous connectable electric energy system
CN111191180A (en) * 2020-01-15 2020-05-22 百望股份有限公司 Method and device for constructing micro-service system in invoice field and storage medium
CN111752937A (en) * 2020-06-30 2020-10-09 海南电网有限责任公司 Data deviation rectifying method, system and readable medium based on micro service
CN111797178A (en) * 2020-07-06 2020-10-20 国网安徽省电力有限公司 Data acquisition and processing method based on report tool
CN112307103A (en) * 2020-10-30 2021-02-02 山东浪潮通软信息科技有限公司 Big data rendering method and device and computer readable medium
CN113642299A (en) * 2021-07-30 2021-11-12 国网江苏省电力有限公司南通供电分公司 One-key generation method based on power grid statistical form
CN115905376A (en) * 2021-08-17 2023-04-04 中兴通讯股份有限公司 Data processing method, data query method, device and storage medium
CN113888139A (en) * 2021-10-29 2022-01-04 广东电网有限责任公司 Construction method and system of unified substation data asset management system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609547A (en) * 2012-03-29 2012-07-25 深圳供电局有限公司 Maintenance method and maintenance system for panorama model of digital power grid
CN102722786A (en) * 2012-05-28 2012-10-10 广东电网公司电力调度控制中心 Intelligent scheduling support platform for power grid professional systems
CN102750606A (en) * 2012-05-16 2012-10-24 中国电力科学研究院 Power grid scheduling cloud system
CN103700032A (en) * 2013-12-19 2014-04-02 国家电网公司 Construction method of power grid regulation and control integration panoramic data platform
CN104200402A (en) * 2014-09-11 2014-12-10 国家电网公司 Publishing method and system of source data of multiple data sources in power grid
CN106056481A (en) * 2016-06-23 2016-10-26 中国南方电网有限责任公司电网技术研究中心 Distribution power multi-source data system and multi-source data processing method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609547A (en) * 2012-03-29 2012-07-25 深圳供电局有限公司 Maintenance method and maintenance system for panorama model of digital power grid
CN102750606A (en) * 2012-05-16 2012-10-24 中国电力科学研究院 Power grid scheduling cloud system
CN102722786A (en) * 2012-05-28 2012-10-10 广东电网公司电力调度控制中心 Intelligent scheduling support platform for power grid professional systems
CN103700032A (en) * 2013-12-19 2014-04-02 国家电网公司 Construction method of power grid regulation and control integration panoramic data platform
CN104200402A (en) * 2014-09-11 2014-12-10 国家电网公司 Publishing method and system of source data of multiple data sources in power grid
CN106056481A (en) * 2016-06-23 2016-10-26 中国南方电网有限责任公司电网技术研究中心 Distribution power multi-source data system and multi-source data processing method

Also Published As

Publication number Publication date
CN107357856A (en) 2017-11-17

Similar Documents

Publication Publication Date Title
CN107357856B (en) Method for realizing data integration and data service based on power grid panoramic service model
CN107402976B (en) Power grid multi-source data fusion method and system based on multi-element heterogeneous model
CN109829125A (en) Show the platform of user management of dispatching of power netwoks operation data
Cavalieri et al. Integration of IEC 61850 SCL and OPC UA to improve interoperability in Smart Grid environment
CN107730394A (en) The polynary isomeric data fusion method of panorama power network based on big data
CN102722764A (en) Integrated power grid optimization auxiliary decision analysis system
Liu et al. Study on data management of fundamental model in control center for smart grid operation
CN103825755A (en) Power secondary system modeling method and system
CN108777637A (en) A kind of data center&#39;s total management system and method for supporting server isomery
CN114153920A (en) Big data edge platform and method
CN111524027B (en) Electric power information statistical method, device, equipment and storage medium
KR101397380B1 (en) System for integrating meta-modeling
CN104361457A (en) Information modeling method for production overhaul management of power distribution network and information interactive system and method
CN102929899A (en) Distributed reporting system based on intermediate table
CN104200331A (en) Power grid model joining method and system of power system
KR101397381B1 (en) Method for integrating meta-modeling
CN103489139A (en) Comprehensive analysis and management system for urban distribution network planning
CN112651872A (en) Community comprehensive treatment system and method based on data middlebox
CN112988919A (en) Power grid data market construction method and system, terminal device and storage medium
CN115934856A (en) Method and system for constructing comprehensive energy data assets
CN112150027A (en) Power supply service command platform system
CN111538720A (en) Method and system for cleaning basic data in power industry
CN114281790A (en) Multi-type load resource aggregator access system and method
CN106657282B (en) Method and device for integrating running state information of converter station equipment
KR101397379B1 (en) System for transformation meta-modeling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant