CN110750686A - Fusion system and fusion method of global heterogeneous data - Google Patents
Fusion system and fusion method of global heterogeneous data Download PDFInfo
- Publication number
- CN110750686A CN110750686A CN201910967052.9A CN201910967052A CN110750686A CN 110750686 A CN110750686 A CN 110750686A CN 201910967052 A CN201910967052 A CN 201910967052A CN 110750686 A CN110750686 A CN 110750686A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- module
- global
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a fusion system and a fusion method of global heterogeneous data, wherein the fusion system comprises an interface layer: the interface layer interacts with a user and provides a query interface and a result display interface for maintaining data for the user; a logic control layer: the system is used for realizing the main functions of a fusion system, and inquiring and presenting results of a data source of global heterogeneous data; a data access layer: the data access layer is used for realizing the unified query service of a data source in the heterogeneous data fusion system and comprises a global fusion module and a document acquisition module; data source layer: the method combines data security and query synchronization, reduces the processing time of query results and improves the security protection capability of the data on the premise of only increasing a small amount of storage.
Description
Technical Field
The invention relates to the technical field of information fusion, in particular to a fusion system of global heterogeneous data and a fusion method of the global heterogeneous data.
Background
Data fusion faces several problems: the heterogeneous nature of the data source, the integrity of the data constraint, the local semantic conflict, and the global access performance are maintained, which are described below.
(1) And (4) the heterogeneity of data sources.
The data source system itself is diverse in terms of difference in use and universalization of regions. The processing modes of different types of data are basically different, and the processing modes of the same type of data can have various differences. The data includes structured database data, semi-structured data, various text files, multimedia audio and video files and the like. Heterogeneity can be divided into system heterogeneity, data model heterogeneity, and logical heterogeneity. The System heterogeneity refers to the difference of hardware, Operation System, File Management System and the like of the local data source. Data model heterogeneity is different among storage modes such as a relationship mode, an object mode and a document nesting mode. The logical isomerism includes name isomerism, numerical isomerism, semantic isomerism, pattern isomerism and the like.
(2) The integrity of the data constraints is maintained.
One of the objectives of the data fusion work is that the fused data must ensure certain data integrity and data constraint integrity. Therefore, the integrity and the correctness of the data processing work of the fusion system can be fully ensured.
(3) Local semantic conflicts.
There are conflicts in semantic description between data sources, e.g., synonyms of the same name. Such semantic description conflicts interfere with the query processing of the data, and may lead to redundant data and erroneous data if not resolved.
(4) Global access performance.
The user accesses the fused heterogeneous data without knowing the heterogeneous data source to which the data originally belongs, and the access efficiency cannot be reduced.
Disclosure of Invention
The purpose of the invention is as follows: in view of the deficiencies of the prior art, the present invention provides a system for fusing global heterogeneous data, and another object of the present invention is to provide a method for fusing global heterogeneous data, so as to solve the problems that information cannot be timely transmitted to a user and the information transmission is not safe.
The technical scheme is as follows: a system for fusion of global heterogeneous data, comprising:
interface layer: the interface layer interacts with a user and provides a query interface and a result display interface for maintaining data for the user;
a logic control layer: the system is used for realizing the main functions of a fusion system, and inquiring and presenting results of a data source of global heterogeneous data; the system comprises a data query module, a query optimization module, a result fusion module, an authentication and authorization module and a background management module.
A data access layer: the data access layer is used for realizing the unified query service of a data source in the heterogeneous data fusion system and comprises a global fusion module and a document acquisition module; the global fusion module is firstly used for establishing a mapping table, including names and addresses of various data sources, names and addresses of data and security levels of the data sources, and then is used for converting the various data sources into XML files with a uniform format according to the mapping table, integrating the XML files and generating an XML file to be provided for the query optimization module to query; the document acquisition module encapsulates each data source into Web service, and shields the difference between the original database system and the original application system.
Data source layer: is a data source set of global heterogeneous data.
Further, the data query module processes the query for the global mode submitted by the user and transmits the query request to the query optimization module.
Furthermore, the query optimization module establishes an index file with a security tag on the basis of a binary tree traversal XML document coding scheme; and querying data from the index file with the security tag, positioning a data source of the queried data, recording the data source address by the query optimization module, and sending a query result to the result fusion module.
Furthermore, the result fusion module integrates and further screens the query results submitted by the query optimization module, then converts the data form set according to the result presentation view, and displays the query data to the user.
Further, the authentication and authorization module provides an authentication and authorization function, and performs authentication and authorization on a module or a user needing authentication and authorization.
Furthermore, the authority of the common user is different from that of the administrator, the background management module is supplied for the administrator to use, and the user can only perform operation within the authority range of the user, such as data query and access; and the administrator can modify, add or reduce data sources, set data attribute information and result presentation views according to the change of the data sources.
A method of global data fusion according to the global data fusion system, comprising the steps of:
step 1, initialization setting: the administrator sets the setting, only needs to set once, and can modify the setting according to the needs, and the setting steps are as follows:
(1.1) setting a data source;
(1.2) setting local data attribute information which must contain security level information;
(1.3) setting global data attribute information, namely a union set of local data attribute information;
(1.4) setting a data presentation view, namely information options displayed to a user;
(1.5) setting user and application authorities;
step 2, actual operation, after initialization is completed, the system enters an actual operation stage, and the specific steps are as follows:
(2.1) the system establishes a global mapping table;
(2.2) the system acquires an XML document of a data source;
(2.3) the system establishes an index with a security label;
(2.4) the system waits for user query input;
(2.5) the system converting the query condition into a query with an index of security tags;
(2.6) integrating, screening and inquiring results by the system;
and (2.7) the system displays the query result data to the user according to the data presentation view, and after the steps are completed, the system enters the step (2.4) again to wait for the query input of the user.
Has the advantages that: compared with the prior art, the invention has the following remarkable advantages: the invention adds the security label to the data, adds the security label to the data of each security level, and each data has a unique security label. This not only refines the security requirements, but also significantly reduces the cost of modifying the relevant data in the event of a change in the security level of the data. Therefore, the data security and the query synchronization are combined, the processing time of the query result is reduced, the security protection capability of the data is improved on the premise of only increasing a small amount of storage, and in addition, the data conversion is carried out by using the global mapping table. The concept of views in a data source is referred to, and the idea of a global mode is adopted. The method provides conversion from data source data to XML file elements, so that a user can convert the query of the data source data into the query of the XML file elements, and the difference of the original database system and the original application system of the data source is shielded.
Drawings
Fig. 1 is a block diagram of a system for fusing global heterogeneous data according to the present invention.
Detailed Description
The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention and the accompanying drawings.
As shown in fig. 1, a system for fusing global heterogeneous data includes:
interface layer: the interface layer interacts with a user and provides a query interface and a result display interface for maintaining data for the user;
logic control layer 1: the logic control layer 1 comprises five sub-modules which are a data query module 4, a query optimization module 3, a result fusion module 5, an authentication authorization module 6 and a background management module 7 respectively; the data query module 4 processes the query for the global mode submitted by the user and transmits the query request to the query optimization module 3; the query optimization module 3 establishes an index file with a security tag on the basis of a binary tree traversal XML document coding scheme; inquiring data from the index file with the security label, positioning a data source of the inquired data, recording the data source address by the inquiry optimizing module 3, and sending an inquiry result to the result fusion module 5; the result fusion module 5 integrates and further screens the query results submitted by the query optimization module 3, then converts the data form set according to the result presentation view, and displays the query data to the user; the authentication and authorization module 6 provides an authentication and authorization function and performs authentication and authorization on a module or a user needing authentication and authorization; the background management module 7 is provided for an administrator, the user can only carry out operation within the self authority range, the authority of the common user is different from that of the administrator, the administrator can modify the data according to the change of the data source, the common user can only carry out data query access, the administrator adds or reduces the data source, and sets data attribute information and a result presentation view;
data access layer 2: the data access layer 2 is used for realizing a unified query service of a data source in the heterogeneous data fusion system and comprises a global fusion module 8 and a document acquisition module 9; the global fusion module 8 firstly establishes a mapping table, including names and addresses of various data sources, names and addresses of data and security levels thereof, then converts the various data sources into XML files with a uniform format according to the mapping table, integrates the XML files, generates an XML file, and provides the XML file for the query optimization module to query; the document acquisition module 9 encapsulates each data source into a Web service, and shields the difference between the original database system and the original application system;
data source layer: is a data source set of global heterogeneous data.
The invention also provides a fusion method of the global data fusion system, which comprises the following steps:
1) initialization setting: the administrator sets the setting, only needs to set once, and can modify the setting according to the needs, and the setting steps are as follows:
(1) setting a data source;
(2) setting local data attribute information which must contain security level information;
(3) setting global data attribute information, namely a union set of local data attribute information;
(4) setting a data presentation view, namely information options displayed to a user;
(5) setting user and application permissions;
2) actual operation, after initialization is completed, the system enters an actual operation stage, which comprises the following specific steps:
(1) the system establishes a global mapping table;
(2) the system acquires an XML document of a data source;
(3) the system establishes an index with a security label;
(4) the system waits for user query input;
(5) the system converts the query condition into a query with an index of the security label;
(6) the system integrates and screens the query result;
(7) and the system displays the query result data to the user according to the data presentation view, and after the steps are completed, the system enters the step 4 again to wait for the query input of the user.
In fig. 1, the present invention adds a security tag to data, and adds a security tag to data of each security level, and each data has its own security tag. This not only refines the security requirements, but also significantly reduces the cost of modifying the relevant data in the event of a change in the security level of the data. Therefore, the data security and the query synchronization are combined, the processing time of the query result is reduced, the security protection capability of the data is improved on the premise of only increasing a small amount of storage, and in addition, the data conversion is carried out by using the global mapping table. The concept of views in a data source is referred to, and the idea of a global mode is adopted. The method provides conversion from data source data to XML file elements, so that a user can convert the query of the data source data into the query of the XML file elements, and the difference of the original database system and the original application system of the data source is shielded.
Claims (7)
1. A system for fusion of global heterogeneous data, comprising:
the interface layer is used for interacting with a user and providing a query interface and a result display interface for maintaining data for the user;
the system comprises a logic control layer (1) and a background management module (7), wherein the logic control layer is used for inquiring a data source of global heterogeneous data and presenting results, and comprises a data inquiry module (4), an inquiry optimization module (3), a result fusion module (5), an authentication authorization module (6) and the background management module;
the data access layer (2) is used for realizing the unified query service of the data source and comprises a global fusion module (8) and a document acquisition module (9); the global fusion module (8) is firstly used for establishing a mapping table, including names and addresses of various data sources, names and addresses of data and security levels of the data sources, and secondly used for converting the various data sources into XML files with a uniform format according to the mapping table, integrating the XML files, and generating an XML file to be provided for the query optimization module to query; the document acquisition module (9) encapsulates each data source into Web service, and shields the difference between the original database system and the original application system;
and the data source layer is a data source set of the global heterogeneous data.
2. The system for fusing global heterogeneous data according to claim 1, wherein: the data query module (4) is used for processing a query for the global mode submitted by a user and transmitting a query request to the query optimization module (3).
3. The system for fusing global heterogeneous data according to claim 1, wherein: the query optimization module (3) establishes an index file with a security tag on the basis of a binary tree traversal XML document coding scheme; inquiring data from the index file with the security label, positioning a data source of the inquired data, recording the data source address by the inquiry optimizing module (3), and sending the inquiry result to the result fusion module (5).
4. The system for fusing global heterogeneous data according to claim 1, wherein: and the result fusion module (5) integrates and further screens the query results submitted by the query optimization module (3), converts the data form set according to the result presentation view and displays the query data to the user.
5. The system for fusing global heterogeneous data according to claim 1, wherein: the authentication and authorization module (6) is used for providing an authentication and authorization function and performing authentication and authorization on a module or a user needing the authentication and authorization.
6. The system for fusing global heterogeneous data according to claim 1, wherein: the background management module (7) is used by an administrator, and the administrator sets users and application authorities thereof, modifies data sources according to changes of the data sources, sets data attribute information and results presentation views through the background management module.
7. A global data fusion method according to any one of claims 1-6, comprising the steps of:
step 1, an administrator performs initialization setting, and the initialization setting specifically comprises the following contents:
(1.1) setting a data source;
(1.2) setting local data attribute information, wherein the information comprises security level information;
(1.3) setting global data attribute information, namely a union set of local data attribute information;
(1.4) setting a data presentation view, namely information options displayed to a user;
(1.5) setting user and application authorities;
step 2, initialization is completed, and the system enters an actual operation stage, which specifically comprises the following contents:
(2.1) the system establishes a global mapping table;
(2.2) the system acquires an XML document of a data source;
(2.3) the system establishes an index with a security label;
(2.4) the system waits for user query input;
(2.5) the system converting the query condition into a query with an index of security tags;
(2.6) integrating, screening and inquiring results by the system;
and (2.7) the system displays the query result data to the user according to the data presentation view, and after the steps are completed, the system enters the step (2.4) again to wait for the query input of the user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910967052.9A CN110750686A (en) | 2019-10-12 | 2019-10-12 | Fusion system and fusion method of global heterogeneous data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910967052.9A CN110750686A (en) | 2019-10-12 | 2019-10-12 | Fusion system and fusion method of global heterogeneous data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110750686A true CN110750686A (en) | 2020-02-04 |
Family
ID=69278038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910967052.9A Withdrawn CN110750686A (en) | 2019-10-12 | 2019-10-12 | Fusion system and fusion method of global heterogeneous data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110750686A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254518A (en) * | 2021-05-21 | 2021-08-13 | 京软伟业信息技术(北京)有限公司 | Information resource management and analysis method based on particle data |
| CN113722549A (en) * | 2021-09-03 | 2021-11-30 | 优维科技(深圳)有限公司 | Data state fusion storage system and method based on graph |
-
2019
- 2019-10-12 CN CN201910967052.9A patent/CN110750686A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254518A (en) * | 2021-05-21 | 2021-08-13 | 京软伟业信息技术(北京)有限公司 | Information resource management and analysis method based on particle data |
| CN113722549A (en) * | 2021-09-03 | 2021-11-30 | 优维科技(深圳)有限公司 | Data state fusion storage system and method based on graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107819824B (en) | Urban data opening and information service system and service method | |
| US8484230B2 (en) | Dynamic parsing rules | |
| CN104200402A (en) | Publishing method and system of source data of multiple data sources in power grid | |
| CN109213820B (en) | Method for realizing fusion use of multiple types of databases | |
| CN113508403A (en) | System and method for interoperable communication of automation system components with multiple information sources | |
| KR101212778B1 (en) | Cloud computing based smart office system and server for managing the same and method for managing the same | |
| CN111680041B (en) | Safety high-efficiency access method for heterogeneous data | |
| CN110750686A (en) | Fusion system and fusion method of global heterogeneous data | |
| CN102917009A (en) | Method and system for collecting and storing stock data based on cloud computing technology | |
| JP5002729B2 (en) | Data viewer management | |
| US20150205880A1 (en) | Integrating linked data with relational data | |
| CN103390018A (en) | Web service data modeling and searching method based on SDD (service data description) | |
| CN101216824B (en) | Method for publishing tree -type structure database as distributed XML database | |
| CN110888878A (en) | Service-oriented main data management method and system | |
| CN101377737B (en) | Resource management apparatus of application system | |
| US20110302185A1 (en) | Data publication and subscription system | |
| CN104348853A (en) | Electric power system service registration management method and system | |
| CN101883082A (en) | Method, equipment and system for acquiring modeling file information of network configuration protocol server | |
| CN103533094A (en) | Identification code all-in-one machine and identification code system | |
| CN105760532A (en) | Resource sharing system based on Web Service and resource sharing method based on Web Service | |
| CN105719216A (en) | E-government platform information data processing method | |
| CN113849692A (en) | Data exchange method and system, electronic equipment and storage medium | |
| CN102065133B (en) | Method and system for constructing complex service logic by supporting Portlet cooperation | |
| Jeong et al. | A message conversion system, XML-based metadata semantics description language and metadata repository | |
| CN111459907A (en) | Method, system and storage medium for configuring master data through model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200204 |