CN103309977A - Heterogeneous data resource integration method - Google Patents
Heterogeneous data resource integration method Download PDFInfo
- Publication number
- CN103309977A CN103309977A CN2013102366075A CN201310236607A CN103309977A CN 103309977 A CN103309977 A CN 103309977A CN 2013102366075 A CN2013102366075 A CN 2013102366075A CN 201310236607 A CN201310236607 A CN 201310236607A CN 103309977 A CN103309977 A CN 103309977A
- Authority
- CN
- China
- Prior art keywords
- data
- data source
- destination
- data resource
- definition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a data resource integration method. The method comprises the following steps: configuring data source connection, which includes a database IP address, a database name and access permission information; configuring data set maintenance, which includes assigning a specific data list or a specific data file required extracted; configuring a destination data source, and assigning a specific destination database table in which the destination data source is required to be imported; configuring a mapping relationship, and assigning a conversion relationship for importing data in the data source to the destination data source; and integrating the data set to the destination data source according to the mapping relationship. According to the invention, the problem of resource integration is flexibly solved by using a workflow mode to realize loose coupling of data source definition, data set extraction, mapping relationship definition, destination data source definition and data loading on the base of a B/S architecture, the application has proven in the report writing of the power industry, good supports are provided, and the method also has good generalization in other fields.
Description
Technical field
The present invention has mainly proposed a kind of method of isomeric data resource consolidation in the process of automatically writing of electric power report, realized on this basis a cover total solution.
Background technology
Along with informationalized propelling, write in the demand of analyzing with decision data at analysis report as units such as electric power enterprise and government departments, need integrated some long-range isomeric datas as the basis of analyzing.Yet need these Database Systems of integration different, have a large amount of error in data at storage mode; There is very large difference on the data structure of storage information; Lack unified describing standard, a kind of like this tissue has hindered unit main information and digitized process.Therefore how to integrate these data, a unified data access interface is provided, writing for management decision reference and analysis report provides service to become an urgent demand.
In present method and implementation, mainly be based on the Data Integration mode of C/S structure, need to client be installed at subscriber set under this mode, large to very high while of the requirement client software maintenance difficulties of subscriber set; Shortage is to the support of workflow; In the situation that the power industry the special environment needs large system combination not support, cause integral data to be used with data and be separated in the different systems.
Summary of the invention
Based on the problem that exists under the top large background, the invention provides a kind of data resource integrated method, we have realized a whole set of solution on the basis of this method simultaneously.On the basis of B/S framework the definition of data source, the extraction of data set, the flow process that mapping relations definition, the definition of destination data source and data load has solved the problem of resource consolidation flexibly by the mode loose coupling of workflow,
Data resource integrated method comprises step:
Disposition data source connects, and comprises the IP address of database, database-name and access authority information;
The maintenance of configuration data set comprises the appointment of needs being extracted concrete data form or concrete data file;
Which concrete destination data storehouse table configuration purpose data source, appointment need to import in;
The configuration mapping relations, the data importing in the specific data source is to the transforming relationship in destination data source;
According to described mapping relations, described data set is integrated into described destination data source.
Can find out by above method and implementation, Data Integration is put the integration flow process of the upper whole data resource of definition mode series connection of formula, integrate definition for one, be associated with concrete mapping relations, the source and the destination data source that need specific data in the mapping relations like this can be with a kind of mode asynchronous integration data of workflow in concrete the execution.And by data resource integrated realization being integrated in the use system of data, seamless like this solution the problem of digital divide, also avoided the cross-system problem of operation.By using proof that good support is provided in the report of power industry is write, this scheme also has good generalization in other field.
Description of drawings
Fig. 1 is the overall flow schematic diagram of the isomeric data resource consolidation in the invention process;
Fig. 2 is the functional schematic of the data-mapping contextual definition in the invention process;
Fig. 3 is data resource integrated one-piece construction figure.
Embodiment
Below in conjunction with accompanying drawing and specific embodiment, method of the present invention and scheme are realized being further described.
Data resource integrated method comprises step:
Disposition data source connects, and comprises the IP address of database, database-name and access authority information;
The maintenance of configuration data set comprises the appointment of needs being extracted concrete data form or concrete data file;
Which concrete destination data storehouse table configuration purpose data source, appointment need to import in;
The configuration mapping relations, the data importing in the specific data source is to the transforming relationship in destination data source;
According to described mapping relations, described data set is integrated into described destination data source.
Can find out by above method and implementation, Data Integration is put the integration flow process of the upper whole data resource of definition mode series connection of formula, integrate definition for one, be associated with concrete mapping relations, the source and the destination data source that need specific data in the mapping relations like this can be with a kind of mode asynchronous integration data of workflow in concrete the execution.And by data resource integrated realization being integrated in the use system of data, seamless like this solution the problem of digital divide, also avoided the cross-system problem of operation.By using proof that good support is provided in the report of power industry is write, this scheme also has good generalization in other field.
Referring to shown in Figure 1, the method for isomeric data resource consolidation comprises following process:
Process one: data resource integrated definition procedure
The task of a concrete Data Integration of definition, scheduling rule of appointment is integrated in definition title, mapping relations and execution etc. information.The module that data resource management is integrated is got up the information association in the implementation the concrete information of can arranging in pairs or groups flexibly by the binding form of attribute.
Process two: data source connection procedure
Mainly be definition and the source-information of safeguarding the data that need extraction.IP address, type of database, database-name, database user name and password etc. the information that comprises the place of database.By this definition mode, can switch flexibly the mode of connection.The function that in the realization that database connects, adds simultaneously connectivity verification, the validity that can effect connects.
Process three: DSD
Mainly be the data acquisition that defines the former data that need extraction, its definition is to be based upon on the basis of data source connection.Comprise the dataset name definition, be listed as in the concrete database table that extracts and the database table.
Process four: destination data source
Resource imported to after main definition and maintenance were integrated specific purposes database and form.Comprise the definition in destination data storehouse and the appointment of tables of data.
Process five: mapping relations
In this process, defined mapping relations between source data and the destination data, comprised that mainly field connects, field intercepts, field is sued for peace, field rounds, field is averaged, field type conversion and field format conversion etc.Sample is as follows:
Field connects: field Name=" China ", field Province=" and Guangdong ", destination field address=after connecting " and Chinese Guangdong ".
Field intercepting: field Company=" Chinese Microsoft Research, Asia ", destination field Company=after the intercepting " and Microsoft Research, Asia ".
Field summation: field Salary={5000,10000,20000}, destination field TotalSalary=35000 after the summation.Other operation can be as above realization mechanism.
Process six: implementation
By the realization of prior process, extract the data of data source by the workflow form that implementation 1 can be asynchronous, be stored in the destination data storehouse according to mapping relations.
Can find out by above method and implementation, Data Integration is put the integration flow process of the upper whole data resource of definition mode series connection of formula, integrate definition for one, be associated with concrete mapping relations, the source and the destination data source that need specific data in the mapping relations like this can be with a kind of mode asynchronous integration data of workflow in concrete the execution.And by data resource integrated realization being integrated in the use system of data, seamless like this solution the problem of digital divide, also avoided the cross-system problem of operation.On the basis of B/S framework the definition of data source, the extraction of data set, the flow process that mapping relations definition, the definition of destination data source and data load has solved the problem of resource consolidation flexibly by the mode loose coupling of workflow, use proof that good support is provided in the report of power industry is write, this method also has good generalization in other field.
Fig. 3 is data resource integrated one-piece construction figure.
Wherein, this method is based on the framework realization of B/S.This method is that the scheme under the power industry the special environment realizes.Whole integration process is based on the processing mode of workflow in this method.The whole definition procedure of workflow is the definition of going down of the framework at B/S.The final operation of the data that define in the workflow can be supported the mechanism of incremental update.
The above embodiment has only expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to claim of the present invention.Should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.
Claims (6)
1. the method for isomeric data resource consolidation is characterized in that, may further comprise the steps:
Disposition data source connects, and comprises the IP address of database, database-name and access authority information;
The maintenance of configuration data set comprises the appointment of needs being extracted concrete data form or concrete data file;
Which concrete destination data storehouse table configuration purpose data source, appointment need to import in;
The configuration mapping relations, the data importing in the specific data source is to the transforming relationship in destination data source;
According to described mapping relations, described data set is integrated into described destination data source.
2. the method for isomeric data resource consolidation according to claim 1 is characterized in that,
The method of described isomeric data resource consolidation is based on the framework of B/S and realizes.
3. the method for isomeric data resource consolidation according to claim 1 is characterized in that,
The method of described isomeric data resource consolidation realizes under the power industry the special environment.
4. the method for isomeric data resource consolidation according to claim 1 is characterized in that,
The whole integration process of the method for described isomeric data resource consolidation is based on the processing mode of workflow.
5. the method for isomeric data resource consolidation according to claim 4 is characterized in that, the whole definition procedure of workflow is the definition of going down of the framework at B/S.
6. the method for isomeric data resource consolidation according to claim 5 is characterized in that, the final operation of the data that define in the workflow can be supported the mechanism of incremental update.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310236607.5A CN103309977B (en) | 2013-06-14 | 2013-06-14 | Heterogeneous data resource integration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310236607.5A CN103309977B (en) | 2013-06-14 | 2013-06-14 | Heterogeneous data resource integration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103309977A true CN103309977A (en) | 2013-09-18 |
| CN103309977B CN103309977B (en) | 2017-02-08 |
Family
ID=49135195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310236607.5A Active CN103309977B (en) | 2013-06-14 | 2013-06-14 | Heterogeneous data resource integration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103309977B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927344A (en) * | 2014-03-31 | 2014-07-16 | 国网山东省电力公司 | Data integration method |
| CN105653525A (en) * | 2014-11-10 | 2016-06-08 | 金蝶软件（中国）有限公司 | Data importing method and system between data set |
| CN107291884A (en) * | 2017-06-20 | 2017-10-24 | 郑州云海信息技术有限公司 | A kind of data relationship construction method and device |
| CN108255612A (en) * | 2018-01-23 | 2018-07-06 | 山东汇贸电子口岸有限公司 | The data acquisition platform of uniform permission administration and data management platform integrated approach |
| CN104899295B (en) * | 2015-06-09 | 2018-08-03 | 苏州国云数据科技有限公司 | A kind of heterogeneous data source data relation analysis method |
| CN111737235A (en) * | 2020-08-12 | 2020-10-02 | 国网浙江省电力有限公司营销服务中心 | Heterogeneous data migration method for power industry |
| CN112328704A (en) * | 2020-11-03 | 2021-02-05 | 成都中科大旗软件股份有限公司 | Method, system, computer equipment and storage medium for realizing multi-data source combined query |
| CN113628066A (en) * | 2021-06-28 | 2021-11-09 | 南京南瑞继保电气有限公司 | Method for extracting multidimensional feature vector data of wind generating set in loose coupling mode |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697166A (en) * | 2009-10-28 | 2010-04-21 | 浪潮电子信息产业股份有限公司 | Method for accelerating data integration of heterogeneous platform |
| CN102081661A (en) * | 2011-01-19 | 2011-06-01 | 吉林大学 | Data integration method and system of heterogeneous relational database based on XML (Extensive Makeup Language) |
| CN102426587A (en) * | 2011-10-27 | 2012-04-25 | 北京航空航天大学 | Method for customizing and inquiring heterogeneous BOM (Bill of Materiel) based on complex product |
-
2013
- 2013-06-14 CN CN201310236607.5A patent/CN103309977B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697166A (en) * | 2009-10-28 | 2010-04-21 | 浪潮电子信息产业股份有限公司 | Method for accelerating data integration of heterogeneous platform |
| CN102081661A (en) * | 2011-01-19 | 2011-06-01 | 吉林大学 | Data integration method and system of heterogeneous relational database based on XML (Extensive Makeup Language) |
| CN102426587A (en) * | 2011-10-27 | 2012-04-25 | 北京航空航天大学 | Method for customizing and inquiring heterogeneous BOM (Bill of Materiel) based on complex product |
Non-Patent Citations (1)
| Title |
|---|
| 李登元 等: "基于异构数据源整合与集成的信息系统集成技术研究与应用", 《军械工程学院学报》, vol. 16, no. 6, 31 December 2004 (2004-12-31) * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927344A (en) * | 2014-03-31 | 2014-07-16 | 国网山东省电力公司 | Data integration method |
| CN105653525A (en) * | 2014-11-10 | 2016-06-08 | 金蝶软件（中国）有限公司 | Data importing method and system between data set |
| CN105653525B (en) * | 2014-11-10 | 2020-04-10 | 金蝶软件（中国）有限公司 | Method and system for importing data between account sets |
| CN104899295B (en) * | 2015-06-09 | 2018-08-03 | 苏州国云数据科技有限公司 | A kind of heterogeneous data source data relation analysis method |
| CN107291884A (en) * | 2017-06-20 | 2017-10-24 | 郑州云海信息技术有限公司 | A kind of data relationship construction method and device |
| CN108255612A (en) * | 2018-01-23 | 2018-07-06 | 山东汇贸电子口岸有限公司 | The data acquisition platform of uniform permission administration and data management platform integrated approach |
| CN111737235A (en) * | 2020-08-12 | 2020-10-02 | 国网浙江省电力有限公司营销服务中心 | Heterogeneous data migration method for power industry |
| CN112328704A (en) * | 2020-11-03 | 2021-02-05 | 成都中科大旗软件股份有限公司 | Method, system, computer equipment and storage medium for realizing multi-data source combined query |
| CN112328704B (en) * | 2020-11-03 | 2024-02-23 | 成都中科大旗软件股份有限公司 | Method, system, computer device and storage medium for implementing multiple data source joint inquiry |
| CN113628066A (en) * | 2021-06-28 | 2021-11-09 | 南京南瑞继保电气有限公司 | Method for extracting multidimensional feature vector data of wind generating set in loose coupling mode |
| CN113628066B (en) * | 2021-06-28 | 2024-03-19 | 南京南瑞继保电气有限公司 | Multi-dimensional feature vector data loose coupling extraction method for wind generating set |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103309977B (en) | 2017-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103309977A (en) | Heterogeneous data resource integration method | |
| CN104317966B (en) | A kind of dynamic index method inquired about for electric power big data Rapid Combination | |
| CN104933070A (en) | Catalog management system used for government affairs information platform | |
| CN103136249A (en) | System and method of multiplex mode isomerous data integration | |
| CN104573115A (en) | Method and system for achieving integration interface supporting operation of multi-type databases | |
| KR101630752B1 (en) | Data Processing Method for Distributable and Unidentifiable Big Data | |
| CN104216966A (en) | Method supporting index creation in various modes | |
| CN103106573A (en) | Massive email analyzing method and system based on relational graph | |
| CN103399942A (en) | Data engine system supporting SaaS multi-tenant function and working method of data engine system | |
| CN104301149A (en) | Multi-data-center permission management method and system | |
| CN104881506A (en) | Configurable management information system right control method based on database | |
| CN104598587A (en) | Invoice authenticity inquiry method and system | |
| Kim et al. | A study on utilization of spatial information in heterogeneous system based on apache nifi | |
| CN103092866A (en) | Data monitoring method and monitoring device | |
| CN104008107A (en) | Implement method of knowledge base on operation and maintenance management | |
| CN104424530A (en) | Method for realizing layering management of multilevel departments through permission setting | |
| CN105912723A (en) | Storage method of custom field | |
| CN102999329A (en) | Active directory interface developing method based on extensive markup language mapping configuration | |
| CN104917847A (en) | Mobile library based on cloud desktop | |
| CN103713903A (en) | Hard disk dynamic loading method and device | |
| CN102254099A (en) | Method and device for establishing health record based on human social relation | |
| CN107633388A (en) | A kind of office management system suitable for household services enterprise | |
| Ye et al. | The framework and standards of Chinese intangible cultural heritage metadata | |
| CN110489472B (en) | Data exchange method and unit management system using same | |
| Mühlemeier | Resilience and agency in mature sustainability transitions. Theoretical conceptualisation and empirical analysis of actor-& system-level dynamics in sociotechnical energy systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 510080 water Donggang 8, Dongfeng East Road, Yuexiu District, Guangzhou, Guangdong. Patentee after: ELECTRIC POWER RESEARCH INSTITUTE, GUANGDONG POWER GRID CO., LTD. Address before: 510080 water Donggang 8, Dongfeng East Road, Yuexiu District, Guangzhou, Guangdong. Patentee before: Electrical Power Research Institute of Guangdong Power Grid Corporation |