CN108090061A - A kind of difference organization data source system - Google Patents
A kind of difference organization data source system Download PDFInfo
- Publication number
- CN108090061A CN108090061A CN201611023576.5A CN201611023576A CN108090061A CN 108090061 A CN108090061 A CN 108090061A CN 201611023576 A CN201611023576 A CN 201611023576A CN 108090061 A CN108090061 A CN 108090061A
- Authority
- CN
- China
- Prior art keywords
- data
- data source
- heterogeneous
- difference
- management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/83—Querying
Abstract
A kind of difference organization data source system belongs to data source systems technical field more particularly to difference organization data source system.The present invention provides a kind of difference organization data source system that can solve the isomery sex chromosome mosaicism of different data patterns in structure and semantically.The present invention includes predetermined switching part, unified copy data portion, data fusion concentrated part, data source structure part, query planning part, query execution part and result filtration fraction；The predetermined switching part of its structural feature carries out the exchange of data with sharing to customize dedicated diverter tool between different data sources；The unified copy data portion is to establish data warehouse as typical case, by the way that the data in heterogeneous data source are analyzed, converted and loaded, data in each data source are moved into data warehouse, to the carry out centralized management of data, centralised storage in heterogeneous data source, multiple data sources and business rules are integrated.
Description
Technical field
The invention belongs to data source systems technical field more particularly to difference organization data source systems.
Background technology
Data source refers to database used in database application or database server.Data source is to provide certain
The device or original media of data required for kind.At present in enterprise, due to development time or the difference of developing department, often have
Multiple information systems that are heterogeneous, operating on different soft and hard part platform are run simultaneously, and the data source of these systems is independent of one another
And mutually closing so that data are difficult to exchange, share and merge between the systems, therefore form " information island " phenomenon.With
It the continuous of information-based application field to expand and deeply, between enterprises and the demand day for asking information friendship beans of enterprise and external world
It is beneficial strong, there is an urgent need to be integrated to original information system, connect " information island ", shared data.
The content of the invention
The present invention addresses the above problem, and providing a kind of can solve different data patterns in structure and semantically
Isomery sex chromosome mosaicism difference organization data source system.
To achieve the above object, the present invention adopts the following technical scheme that, the present invention includes predetermined switching part, unified copy
Data portion, data fusion concentrated part, data source structure part, query planning part, query execution part and result filtering
Part；The predetermined switching part of its structural feature carries out the friendship of data to customize dedicated diverter tool between different data sources
Stream is with sharing.
The unified copy data portion be to establish data warehouse as typical case, by the data in heterogeneous data source into
It row analysis, conversion and loads, the data in each data source is moved into data warehouse, data in heterogeneous data source are collected
Chinese style management, centralised storage integrate multiple data sources and business rules.
The data fusion concentrated part is to integrate heterogeneous data source using middleware, and middleware is taken as the journey of standard
Sequence interface and agreement, to the data sharing on different soft and hard part platform and application interoperation；To the middleware system of data integration
Between heterogeneous data source and application program, coordinate each Database Systems downwards, be upwards the application system of Access Integration data
System provides global data.
Data source structure part is by being database, Excel forms, semi-structured XML document to each
Data source sets one to be located at services sites different on Web, and data are managed by the way of local.
The query planning part is the son resolved into the standard queries that client is submitted for each heterogeneous database
It inquires about and is submitted to corresponding wrapper.
The query execution part includes queue management and management and running submodule forms, and queue sets son of the management to generation
Inquiry carries out the reception of data, is lined up, and being sent to corresponding wrapper using management and running submodule performs.
The result filtration fraction obtains result set by queried access of the wrapper to each heterogeneous database；As a result
Collection includes the query result of disparate databases, is made of the row data in database；And this this data is judged, it is closed
And it filters.
As a preferred embodiment, the exchange of data of the present invention refers to use various electronic data interchanges soft with shared
Part carries out data exchange.
As another preferred embodiment, wrapper of the present invention includes the coupling part of request, is lined up part, forwarding portion
Point.
In addition, dispatching management module of the present invention uses the queue management mechanism based on FIFO scheduling strategies, line is dispatched
Cheng Shouxian takes the head element of queue, then judges its property parameters, is finally sent to the execution of corresponding package device.
Advantageous effect of the present invention.
Using the present invention, scattered application system still independent work, will not destroy original application architecture originally；It can be with
Integrated multiple data sources and complicated business rules, data integration quality are good.
Middleware Technology of the present invention is due to having many advantages, such as that data source autonomy is good, inquiry is real-time, configuration is flexible.Utilize this
Body carrys out the relation on descriptive semantics level, and the XML descriptions on integrated structure can preferably solve different data patterns and tie
Isomery sex chromosome mosaicism on structure and semantically；Heterogeneous data source is integrated, the purpose is to realize information sharing, it is lonely to solve information
The problems such as typing, the wasting of resources, inconsistent data are repeated caused by island.
Description of the drawings
In order to which technical problem solved by the invention, technical solution and advantageous effect is more clearly understood, below in conjunction with
The drawings and the specific embodiments, the present invention will be described in further detail.It should be appreciated that specific embodiment party described herein
Formula is only used to explain the present invention, is not intended to limit the present invention.
Fig. 1 is present system flow diagram.
Specific embodiment
As shown in the figure, this present invention include predetermined switching part, unified copy data portion, data fusion concentrated part,
Data source structure part, query planning part, query execution part and result filtration fraction；Predetermined switching part is in different numbers
According to dedicated diverter tool is customized between source, the exchange of data is carried out with sharing；
The unified copy data portion is to establish data warehouse as typical case, by dividing the data in heterogeneous data source
Data in each data source are moved into data warehouse by analysis, conversion and loading, to the progress centralization of data in heterogeneous data source
Management, centralised storage integrate multiple data sources and business rules.
The data fusion concentrated part is to integrate heterogeneous data source using middleware, and middleware is taken as the journey of standard
Sequence interface and agreement, to the data sharing on different soft and hard part platform and application interoperation；To the middleware system of data integration
Between heterogeneous data source and application program, coordinate each Database Systems downwards, be upwards the application system of Access Integration data
System provides global data.
Data source structure part is by being database, Excel forms, semi-structured XML document to each
Data source sets one to be located at services sites different on Web, and data are managed by the way of local.
The query planning part is the son resolved into the standard queries that client is submitted for each heterogeneous database
It inquires about and is submitted to corresponding wrapper.
The query execution part includes queue management and management and running submodule forms, and queue sets son of the management to generation
Inquiry carries out the reception of data, is lined up, and being sent to corresponding wrapper using management and running submodule performs.
The result filtration fraction obtains result set by queried access of the wrapper to each heterogeneous database；As a result
Collection includes the query result of disparate databases, is made of the row data in database；And this this data is judged, it is closed
And it filters.
The exchange of the data refers to carry out data exchange using various electronic data interchange softwares with shared.
The wrapper includes the coupling part of request, is lined up part, forwarding part.
The dispatching management module uses the queue management mechanism based on FIFO scheduling strategies, and scheduling thread takes queue first
Head element, then judge its property parameters, be finally sent to corresponding package device execution.
There may be difference semantically between different aforementioned sources, the data after integrating should be according to certain data conversion
Pattern and business rule carry out uniform data structure and field semantics code conversion.
Middleware layer of the present invention shields the distributivity and isomerism of data source from higher level.User thinks all
Data are all local, and in same service-domain, and the specific processing of inquiry request, the return of result are all born by interlayer
Duty.Middleware is mainly made of 3 intermediary device, wrapper and ontology library parts, wherein, intermediary device again include Query Builder,
4 query decomposition engine, query execution engine and results processor functional units.The task of query decomposition engine is to be looked into
Ask disintegration.It, which is mainly responsible for, receives global query's sentence that Query Builder transfers, and analysis local ontology defines information, really
Surely the localized data source to be inquired about, and global body and the correspondence of local ontology in mapping ruler is called to perform and decompose calculation
Global query is decomposed into the subquery of corresponding each localized data source by method.Wrapper is exactly the translater of relation schema.One bag
Dress device is mainly made of inquiry converter and result converter.
The above content is combine specific preferred embodiment to the further description of the invention made, it is impossible to assert this
The specific implementation of invention is confined to these explanations, for those of ordinary skill in the art to which the present invention belongs, not
On the premise of departing from present inventive concept, several simple deduction or replace can also be made, the present invention should be all considered as belonging to and be carried
The protection domain that claims of friendship determine.
Claims (4)
1. a kind of difference organization data source system, in the middle part of predetermined switching part, unified copy data portion, data fusion collection
Divide, data source builds part, query planning part, query execution part and result filtration fraction；It is characterized in that predetermined switching
Part carries out the exchange of data with sharing to customize dedicated diverter tool between different data sources；
The unified copy data portion is to establish data warehouse as typical case, by dividing the data in heterogeneous data source
Data in each data source are moved into data warehouse by analysis, conversion and loading, to the progress centralization of data in heterogeneous data source
Management, centralised storage integrate multiple data sources and business rules；
The data fusion concentrated part is to integrate heterogeneous data source using middleware, and the program that middleware is taken as standard connects
Mouth and agreement, to the data sharing on different soft and hard part platform and application interoperation；The middleware system of data integration is located at
Between heterogeneous data source and application program, coordinate each Database Systems downwards, carried upwards for the application system of Access Integration data
For global data；
Data source structure part is by being database, Excel forms, semi-structured XML document to each data
Source sets one to be located at services sites different on Web, and data are managed by the way of local；
The query planning part is the subquery resolved into the standard queries that client is submitted for each heterogeneous database
And it is submitted to corresponding wrapper；
The query execution part includes queue management and management and running submodule forms, and queue sets subquery of the management to generation
It carries out the reception of data, be lined up, being sent to corresponding wrapper using management and running submodule performs；
The result filtration fraction obtains result set by queried access of the wrapper to each heterogeneous database；Result set bag
Query result containing disparate databases is made of the row data in database；And this this data is judged, it merged
Filter.
2. a kind of difference organization data source system according to claim 1, it is characterised in that the exchange of the data is with sharing
Refer to carry out data exchange using various electronic data interchange softwares.
3. a kind of difference organization data source system according to claim 1, it is characterised in that the wrapper includes request
Coupling part is lined up part, forwarding part.
4. a kind of difference organization data source system according to claim 1, it is characterised in that the dispatching management module uses
Based on the queue management mechanism of FIFO scheduling strategies, scheduling thread takes the head element of queue first, then judges its property parameters,
It is finally sent to the execution of corresponding package device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611023576.5A CN108090061A (en) | 2016-11-21 | 2016-11-21 | A kind of difference organization data source system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611023576.5A CN108090061A (en) | 2016-11-21 | 2016-11-21 | A kind of difference organization data source system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108090061A true CN108090061A (en) | 2018-05-29 |
Family
ID=62168488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611023576.5A Pending CN108090061A (en) | 2016-11-21 | 2016-11-21 | A kind of difference organization data source system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108090061A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109543087A (en) * | 2018-11-20 | 2019-03-29 | 北京爱思沃国际数据科技有限公司 | A kind of construction method and application method of international three pole data interoperation automotive engine system |
-
2016
- 2016-11-21 CN CN201611023576.5A patent/CN108090061A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109543087A (en) * | 2018-11-20 | 2019-03-29 | 北京爱思沃国际数据科技有限公司 | A kind of construction method and application method of international three pole data interoperation automotive engine system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107451214B (en) | Non-primary key query method and distributed NewSQL database system | |
| CN101819596B (en) | Memory-based XML script buffer | |
| CN103631870B (en) | System and method used for large-scale distributed data processing | |
| CN101165683A (en) | Heterogeneous different source different platform data integration method and exchange method | |
| Neumayr et al. | Dual deep modeling: multi-level modeling with dual potencies and its formalization in F-Logic | |
| CN108108466A (en) | A kind of distributed system journal query analysis method and device | |
| CN109815254A (en) | Cross-region method for scheduling task and system based on big data | |
| Khalid et al. | A comparative analysis of big data frameworks: An adoption perspective | |
| CN1689298A (en) | Composition service for autonomic computing | |
| CN108090061A (en) | A kind of difference organization data source system | |
| CN107402941A (en) | Conventional data Fabric Interface and its implementation | |
| Liu et al. | Dynamic Query Processing in DIOM. | |
| EP2343658A1 (en) | Federation as a process | |
| Liu et al. | A dynamic query scheduling framework for distributed and evolving information systems | |
| CN106940724B (en) | Multi-paradigm fusion analysis processing method for big data | |
| Wu et al. | HAMR: A dataflow-based real-time in-memory cluster computing engine | |
| Pan et al. | An investigation into the feasibility of the semantic web | |
| CN108345651A (en) | A kind of data integrated system and method for realizing the data virtualization to interconnect | |
| Tang et al. | Online application of science and technology program oriented distributed heterogeneous data integration | |
| CN103092591A (en) | XML (Extensive Makeup Language) and Web Service based data integration middleware method | |
| CN105931149A (en) | Tourism system platform | |
| CN102903030B (en) | The method that between heterogeneous system, different working flow carries out task process | |
| Panwar | Optimizing Big Data Processing in SQL Server through Advanced Utilization of Stored Procedures | |
| CN117077684A (en) | Multifunctional enterprise service portal platform | |
| Zemni et al. | Privacypreserving business process fragmentation for reusability |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180529 |
|
| WD01 | Invention patent application deemed withdrawn after publication |