WO2016015439A1 - 数据库虚拟化微内核数据源的注册与封装方法 - Google Patents

数据库虚拟化微内核数据源的注册与封装方法 Download PDF

Info

Publication number
WO2016015439A1
WO2016015439A1 PCT/CN2014/094368 CN2014094368W WO2016015439A1 WO 2016015439 A1 WO2016015439 A1 WO 2016015439A1 CN 2014094368 W CN2014094368 W CN 2014094368W WO 2016015439 A1 WO2016015439 A1 WO 2016015439A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual
database
data
rest
server
Prior art date
Application number
PCT/CN2014/094368
Other languages
English (en)
French (fr)
Inventor
艾建文
季统凯
Original Assignee
国云科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国云科技股份有限公司 filed Critical 国云科技股份有限公司
Publication of WO2016015439A1 publication Critical patent/WO2016015439A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases

Definitions

  • the invention relates to the field of cloud database technology, in particular to a method for registering and encapsulating a database virtualized microkernel data source.
  • Data "open sharing” is one of the ways that big data applications get data sources.
  • the cloud database treats various relational databases as a series of simple two-dimensional tables and operates on a simplified version of SQL or access objects; thus solving the problem of data concentration and sharing.
  • the existing software application architecture has many physical concepts, and each process in development, deployment, maintenance, and expansion needs to grasp the physical details.
  • the common problems of these applications are: serious information islands, lack of data integration management, lack of Deep understanding of database management and application value, neglect of long-term data quality management, lack of data quality monitoring and update mechanism, lack of knowledge of data analysis and mining, can not obtain effective information and model four problems from the data.
  • the data is granulated from the data management level.
  • different database virtual package instances are generated according to user requirements, and the virtual database package instances are migrated between different operating systems.
  • To achieve data management and personal information asset isolation and security protection to provide data virtual channel DVPN for applications, combined with the data acquisition requirements of big data processing platform, seamlessly migrate existing relational data and popular big data file formats, data One-stop loading, virtualized transfer of tablespaces, transparent export of diverse data files, multi-version, multi-version, homomorphic data preparation Share and recovery, you need to build a concise database virtualization microkernel data source registration and encapsulation method.
  • the technical problem solved by the present invention is to provide a database virtualization microkernel data source registration and encapsulation method, which is mainly used for virtualizing a microkernel distributed MySQL, DB2, Oracle, SQLServer, SyBase database, non-relational file system data source. Registration and encapsulation methods.
  • the method includes virtual library/table registration and encapsulation
  • Virtual library/table registration Virtual data federation maps different virtual metadata directories to different virtual libraries/tables. Each virtual library/table corresponds to a virtual storage address.
  • the host that stores the virtual library/table has separate virtual storage. Pool id; when the host storing the virtual library/table is started, the virtual data information is reported to the corresponding virtual data federation according to the virtual storage pool id, and the automatic registration is performed, and after the virtual data federation receives the report information, the virtual metadata directory is refreshed; When the client accesses this virtual library/table, the host with the id can use the client to mount the virtual library/table;
  • Virtual library/table encapsulation reflects the data abstraction mapping between the virtual database, the virtual table/view and the physical application server, the database server, the database, the data table or the file; the operation of facing the virtual database and the virtual table at the logical layer ultimately Mapping to the operation of the corresponding one or more physical concept entities; the mapping is a many-to-many or cross-constrained; the virtual table can establish a mapping relationship between the physical database and the table or file, and the mapping relationship is non-one-to-one correspondence, A virtual table schema can be mapped to one or more database servers, databases, tables, or files on the network.
  • the registration and encapsulation specifically includes the following steps:
  • the first step is that the client registers from the database/table and non-relational file system of MySQL, DB2, Oracle, SQLServer, SyBase and other data sources through structure mapping and content mapping.
  • the second step is that the virtual data federation starts the data source encapsulation interface according to the user operation operator to implement the parsing and directional code from the bottom layer adjustment core module part;
  • the third step is that the server provides a command according to the virtual data federation, calls the rest server data source encapsulation method, obtains the client database related information through the configuration file, and deploys the rest package interface adapter;
  • the fourth step is that the database virtualization microkernel builds a virtual service to the upper layer.
  • the service consists of a virtual table view.
  • the database virtualization microkernel has a virtual user space, and the user id in the virtual user space is obtained through a virtual table operation operator.
  • the required virtual table view; the virtual table operation operator can perform operations on different virtual libraries/tables, and can generate different virtual table views to realize multi-tenant view extraction of the virtual library/table;
  • the fifth step of the database virtualization microkernel extracts data according to the multi-tenant view of the virtual library/table, and encapsulates according to the rest package code.
  • the client registration and encapsulation specifically includes the following steps:
  • the first step is to add the rest interface registration database interface, including the url resource address, the required port of the rest, and the like, and register the rest mode virtual table;
  • the core query parsing code module remains unchanged, and the underlying jdbc connection database mode is changed to the rest resource access mode to obtain the remote database data;
  • the third step is to modify the library table design.
  • the dbtype adds the rest type in addition to the databases such as MySQL, DB2, Oracle, SQLServer, and SyBase.
  • the fourth step is to increase the underlying implementation class of the DataSourceWrapper interface implementation class RestWrapper.
  • the entire core module part queries the parsing orientation code, and executes the DataSourceWrapper implementation class RestWrapper when the query parses the dbtype to rest;
  • the server registration and encapsulation specifically includes the following steps:
  • the first step is to add the rest server method of the DataSourceWrapper interface.
  • Each method of the interface corresponds to a different url resource.
  • the server obtains the information about the local database through the configuration file, that is, configures the configuration file when the rest interface adapter is deployed;
  • the third step, the rest server method to obtain the customer database data data is equivalent to the local jdbc connection database, the client passes the parameters to the server, the server local jdbc connects and operates the database to encapsulate the result into a resource and return it to the client, the client does You can get the desired data or results by some analytical processing.
  • the method can represent and operate the application servers, database servers, databases and tables in which the various data sources physically exist by means of the two logical concepts and mappings of the virtual database and the virtual table.
  • the method can represent and operate the application servers, database servers, databases and tables in which the various data sources physically exist by means of the two logical concepts and mappings of the virtual database and the virtual table.
  • the user transparently develops, deploys, and manages the network application in the distributed network environment, does not need to pay attention to physical resources, runs the specific location of the application server, and shields complex physical operation details of various data sources; the developed application It is globally accessible, deployable, and portable, ensuring the convenience of using the Internet platform.
  • the invention proposes a database virtualization microkernel data source registration and encapsulation method, and aims at current mainstream
  • the database MySQL, DB2, Oracle, SQLServer, and SyBase can be directly accessed.
  • the data source is encapsulated by a wrapper and registered in the system to form a virtual table with a uniform shape.
  • the virtual library and virtual table creation wizard can be used to build a business virtualization packaged data view, enabling multi-users to rent virtual data online.
  • FIG. 1 is a diagram of a database virtualization microkernel data source registration and encapsulation mechanism of the present invention
  • FIG. 2 is a diagram of a virtualized data source client and server registration and encapsulation method.
  • Virtualized microkernels distribute heterogeneous databases, file data source registration and encapsulation methods.
  • the physical distribution and heterogeneous data are connected to form a single virtual database, which provides a unified data standard and access interface for the application and supports transparent access to the data source.
  • Through the virtualization of diverse data independent access to distributed data sources is achieved, and the distribution and heterogeneous characteristics of diverse data sources are shielded.
  • After the various data sources are encapsulated by wrappers, they are registered in the system to form a virtual table with a unified form. Under the premise of not moving the physical location of the data source, a single virtual database with integrated and unified data standards is formed.
  • the database virtualization microkernel is based on the SOA technology organization system framework, and publishes services in the form of Web services, and is accessible to users through the enterprise portal.
  • the platform builds a business basic data system through a virtual library table administrator providing a system creation wizard, and supports a user to add and delete a system function module online.
  • Virtual library/table registration A virtual table is a logical abstraction of logical data representing multiple data sources, databases, and tables at a logical level. It belongs to a specific virtual data federation.
  • the database virtualization microkernel requires virtual library and virtual table to have the ability to actively register, self-maintain, and self-update.
  • the virtual library/table belongs to a subset of the virtual data federation, and can provide services separately or through encapsulation.
  • the new virtual view provides external services.
  • the virtual data federation maps different virtual metadata directories to different virtual libraries/tables. Each virtual library/table corresponds to a virtual storage address, and the host storing the virtual library/table has a separate virtual storage pool id.
  • the host that stores the virtual library/table When the host that stores the virtual library/table starts, it reports the virtual data information to its corresponding virtual data federation according to the virtual storage pool id, and performs automatic registration. After the virtual data federation receives the report information, it refreshes the virtual metadata directory. When a client accesses this virtual library/table, the host with the id can mount the virtual library/table using the client.
  • Virtual library/table encapsulation Encapsulation relationships reflect virtual data, virtual tables/views, and data abstraction mappings between physical application servers, database servers, databases, and data tables (files). The operation of the virtual database and the virtual table at the logical layer is ultimately mapped to the operation of the corresponding one or more physical concept entities. This mapping is many-to-many and can also be cross-constrained.
  • a virtual table can establish a mapping relationship between a physical database and a table (including files). This mapping relationship is non-one-to-one correspondence.
  • a virtual table schema can be mapped to one or more database servers, databases, and tables on the network ( file).
  • FIG. 1 The database virtualization microkernel data source registration and encapsulation mechanism of the present invention is shown in FIG.
  • the first step is to register the virtual library/table from the data source MySQL, DB2, Oracle, SQLServer, SyBase database/table, non-relational file system, through the rest mode, and virtualize the virtual database. Virtual data tables and virtual files.
  • the second step is that the virtual data federation starts the data source encapsulation interface according to the user operation operator to implement the parsing and directional code from the bottom layer adjustment core module part.
  • the third step is that the server provides a command according to the virtual data federation, calls the rest server data source encapsulation method, obtains the client database related information through the configuration file, and deploys the rest package interface adapter.
  • the fourth step is to build a virtual service for the upper layer of the database virtualization microkernel.
  • the service has a virtual table view.
  • the database virtualization microkernel has a virtual user space, and the user id in the virtual user space is obtained through the virtual table operation operator.
  • the required virtual table view because the virtual table operation operator can operate on different virtual libraries/tables, can generate different virtual table views, and realize multi-tenant view extraction of virtual libraries/tables.
  • the fifth step extracts data according to the multi-tenant view of the virtual library/table according to the database virtualization microkernel, and encapsulates according to the rest package code.
  • the client registration and encapsulation steps are as follows:
  • the first step is to add the rest interface registration database interface, including the url resource address, the rest required port information, etc., to register the rest mode virtual table, as shown in Figure 2.
  • the second step of the core query parsing code module remains unchanged, the underlying jdbc connection database mode is changed to the rest resource access mode to obtain the remote database data.
  • the third step is to modify the library table design, dbtype in addition to MySQL, DB2, Oracle, SQLServer, SyBase and other databases plus rest type
  • the fourth step is to increase the DataSourceWrapper interface implementation class RestWrapper underlying implementation class adjustment core module part query parsing orientation code, such as when the query parsing dbtype is rest, execute DataSourceWrapper implementation class RestWrapper
  • the server package steps are as follows:
  • the first step is to add the rest server method of the DataSourceWrapper interface.
  • Each method of the interface corresponds to a different url resource.
  • the second step is to obtain the information about the local database through the configuration file, that is, when the rest interface adapter is deployed, configure the configuration file.
  • the third step rest server method to obtain the client database data data is equivalent to the local jdbc connection database, the client passes the parameters to the server, the server local jdbc connects and operates the database, the result is encapsulated into a resource and returned to the client, the client does a certain Analytical processing can get the desired data or results.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

数据库虚拟化微内核数据源的注册与封装方法，包括虚拟库/表注册与封装；当存储虚拟库/表的主机启动时，按照虚拟存储池id向其对应的虚拟数据Federation汇报虚拟数据信息，进行自动注册；封装关系反映虚拟数据库、虚拟表/视图与物理的应用服务器、数据库服务器、数据库、数据表或文件之间的数据抽象映射；在逻辑层面对虚拟数据库、虚拟表的操作最终映射到对应的一个或多个物理概念实体的操作；其构建了一个简练的数据库虚拟化微内核数据源注册与封装方法；可以应用于云数据库数据信息的获取上。

Description

数据库虚拟化微内核数据源的注册与封装方法 技术领域
本发明涉及云数据库技术领域，尤其是一种数据库虚拟化微内核数据源的注册与封装方法。
背景技术
数据“开放共享”是大数据应用获取数据源的方法之一。云数据库将各种关系型数据库看成一系列简单的二维表，并基于简化版本的SQL或访问对象进行操作；因而解决了数据集中与共享的问题。
现有的软件应用体系结构物理概念较多，并且在开发、部署、维护、扩展的每个过程都需要把握物理细节，这些应用普遍存在的问题有：信息化孤岛严重，缺乏数据整合管理、缺乏对数据库管理及应用价值的深入认识、忽视长期的数据质量管理，缺乏数据质量监控及更新机制、缺乏数据分析挖掘的认识，无法从数据中获取有效信息和模式四大问题。
针对现有应用的复杂性，从数据管理层级对数据进行颗粒化划分，在分布式环境下，根据用户需求产生不同的数据库虚拟封装实例，并实现虚拟数据库封装实例在不同操作系统之间的迁移，实现数据管理与个人信息资产隔离与安全防护，给应用提供数据虚拟通道DVPN，结合大数据处理平台数据获取的需求，对已有关系型数据和流行大数据文件格式进行无缝迁移，数据的一站式加载，表空间的虚拟化传输，多样数据文件的透明化导出，多样、多版本同态数据备 份与恢复，需要构建一个简练的数据库虚拟化微内核数据源注册与封装方法。
发明内容
本发明解决的技术问题在于提供一种数据库虚拟化微内核数据源的注册与封装方法，主要用于虚拟化微内核分布式MySQL、DB2、Oracle、SQLServer、SyBase数据库、非关系型文件系统数据源注册与封装方法。
本发明解决上述技术问题的技术方案是：
所述方法包括虚拟库/表注册和封装；
虚拟库/表注册：虚拟数据Federation将不同的虚拟元数据目录映射到不同的虚拟库/表，每个虚拟库/表都对应一个虚拟存储地址，存储虚拟库/表的主机有单独的虚拟存储池id；当存储虚拟库/表的主机启动时，按照虚拟存储池id向其对应的虚拟数据Federation汇报虚拟数据信息，进行自动注册，虚拟数据Federation收到汇报信息后，刷新虚拟元数据目录；当客户端访问到此虚拟库/表时，拥有该id的主机可以使用客户端挂载虚拟库/表；
虚拟库/表封装：封装关系反映虚拟数据库、虚拟表/视图与物理的应用服务器、数据库服务器、数据库、数据表或文件之间的数据抽象映射；在逻辑层面对虚拟数据库、虚拟表的操作最终映射到对应的一个或多个物理概念实体的操作；所述映射是多对多或交叉约束；虚拟表可以建立到物理数据库和表或文件的映射关系，这种映射关系是非一一对应的，一个虚拟表schema可以映射到网络上的一个或多个数据库服务器、数据库、表或文件。
所述的注册与封装具体包括以下步骤：
第一步是客户端从MySQL、DB2、Oracle、SQLServer、SyBase等数据源的数据库/表、非关系型文件系统中，通过结构映射和内容映射，通过rest方式注册 虚拟库/表，虚拟出虚拟数据库、虚拟数据表和虚拟文件；
第二步是虚拟数据Federation根据用户操作算子启动数据源封装接口实现从底层调整核心模块部分查询解析定向代码；
第三步是服务端根据虚拟数据Federation提供命令，调用rest服务端数据源封装方法，通过配置文件获取客户端数据库相关信息，部署rest封装接口适配器；
第四步是数据库虚拟化微内核对上层构建虚拟服务，该服务由虚拟表视图构成，数据库虚拟化微内核拥有一个虚拟用户空间，虚拟用户空间中的用户id，通过虚拟表操作算子，获得所需的虚拟表视图；虚拟表操作算子可以对不同的虚拟库/表进行运算，可以产生不同的虚拟表视图，实现虚拟库/表的多租户视图提取；
第五步数据库虚拟化微内核根据虚拟库/表的多租户视图提取数据，按照rest封装代码进行封装。
所述方法的
客户端注册与封装具体包括以下步骤：
第一步，增加rest接口注册数据库界面，包括url资源地址，rest所需端口等信息，注册rest方式虚拟表；
第二步，核心查询解析代码模块保持不变，底层jdbc连接数据库方式改成rest资源访问方式获取远端数据库数据；
第三步，修改库表设计，dbtype除了MySQL、DB2、Oracle、SQLServer、SyBase等数据库之外还加rest类型；
第四步，增加DataSourceWrapper接口实现类RestWrapper底层实现类调 整核心模块部分查询解析定向代码，当查询解析dbtype为rest时执行DataSourceWrapper实现类RestWrapper；
服务端注册与封装具体包括以下步骤：
第一步，增加DataSourceWrapper接口的rest服务端方法，接口每个方法对应不同url资源；
第二步，服务端通过配置文件获取本机数据库相关信息，即在部署rest接口适配器时，配置好该配置文件；
第三步，rest服务端方法获取客户数据库数据数据相当于本地jdbc连接数据库，客户端将参数传给服务端，服务端本地jdbc连接并操作数据库将结果封装成资源返回给客户端，客户端做一定解析处理就可获得想要的数据或结果。
技术效果：
本方法通过将物理概念在逻辑层面加以总结和抽象，借助虚拟数据库、虚拟表两个逻辑概念和映射，就可以表示和操作多样数据源物理存在的应用服务器、数据库服务器、数据库和表。通过对这些数据源进行注册和封装，互连网的应用和资源的大量操作都可以直接基于这两个逻辑概念完成，屏蔽了以前必须关注的物理操作的烦琐细节，确保了对互连网平台使用的方便性，以及应用的可扩展、可移植、松耦合特性。
使用本发明的方法，用户在分布式网络环境透明的开发、部署、管理网络应用，不需要关注物理资源、运行应用服务器的具体位置，屏蔽了多样数据源复杂的物理操作细节；开发的应用程序具有全局可访问、可部署、可移植的特点，这样确保使用互连网平台的便捷性。
本发明提出数据库虚拟化微内核数据源注册与封装方法，针对当前各类主流 数据库MySQL、DB2、Oracle、SQLServer、SyBase可以直接接入，数据源通过封装器(wrapper)封装后，注册到系统中，形成具有统一形态的虚拟表。通过虚拟库和虚拟表创建向导可构建业务虚拟化封装数据视图，支持多用户在线对虚拟数据进行租用。
附图说明
下面结合附图对本发明进一步说明：
图1是本发明的数据库虚拟化微内核数据源注册与封装机制图；
图2是虚拟化数据源客户端与服务器注册与封装方法图。
具体实施方式
虚拟化微内核分布异构数据库、文件数据源注册与封装方法。将物理分布、异构的多样数据连通形成一个单一的虚拟数据库，为应用提供统一的数据标准和访问接口，支持对数据源的透明访问。通过多样数据的虚拟化，实现分布多样数据源的自主接入，屏蔽多样数据源的分布、异构特性。多样数据源通过封装器(wrapper)封装后，注册到系统中，形成具有统一形态的虚拟表，在不移动数据源的物理位置的前提下，形成一个整合的、统一数据标准的单一虚拟数据库。数据库虚拟化微内核基于SOA技术组织系统框架，以Web Service的形式发布服务，并供用户通过企业门户访问。平台通过虚拟库表管理员提供系统创建向导构建业务基础数据系统，支持用户在线添加删除系统功能模块。
虚拟库/表注册：虚拟表是在逻辑层面表示多样数据源、数据库、表的逻辑数据虚拟抽象概念，它所属一个特定的虚拟数据Federation。数据库虚拟化微内核要求虚拟库、虚拟表具有主动注册、自维护、自更新的能力。虚拟库/表属于虚拟数据Federation的子集，即可单独对外提供服务，也可通过封装后形成 新的虚拟视图对外提供服务。虚拟数据Federation把不同的虚拟元数据目录映射到不同的虚拟库/表，每个虚拟库/表都对应一个虚拟存储地址，存储虚拟库/表的主机有单独的虚拟存储池id。当存储虚拟库/表的主机启动时，它会按照虚拟存储池id向其对应的虚拟数据Federation汇报虚拟数据信息，进行自动注册，虚拟数据Federation收到汇报信息后，刷新虚拟元数据目录。当客户端访问到此虚拟库/表时，拥有该id的主机可以使用客户端挂载虚拟库/表。
虚拟库/表封装：封装关系反映了虚拟数据库、虚拟表/视图，与物理的应用服务器、数据库服务器、数据库、数据表(文件)之间的数据抽象映射。在逻辑层面对虚拟数据库、虚拟表的操作最终映射到对应的一个或多个物理概念实体的操作。这种映射是多对多的，同时还可以是交叉约束。首先，虚拟表可以建立到物理数据库和表(包括文件)的映射关系，这种映射关系是非一一对应的，一个虚拟表schema可以映射到网络上的一个或多个数据库服务器、数据库、表(文件)。
本发明数据库虚拟化微内核数据源注册与封装机制如图1所示。
第一步是客户端从数据源MySQL、DB2、Oracle、SQLServer、SyBase数据库/表、非关系型文件系统中，通过结构映射和内容映射，通过rest方式注册虚拟库/表，虚拟出虚拟数据库、虚拟数据表和虚拟文件。
第二步是虚拟数据Federation根据用户操作算子启动数据源封装接口实现从底层调整核心模块部分查询解析定向代码。
第三步是服务端根据虚拟数据Federation提供命令，调用rest服务端数据源封装方法，通过配置文件获取客户端数据库相关信息，部署rest封装接口适配器。
第四步是数据库虚拟化微内核对上层构建虚拟服务，该服务有虚拟表视图构成，数据库虚拟化微内核拥有一个虚拟用户空间，虚拟用户空间中的用户id，通过虚拟表操作算子，获得所需的虚拟表视图，由于虚拟表操作算子可以对不同的虚拟库/表进行运算，因而可以产生不同的虚拟表视图，实现虚拟库/表的多租户视图提取。
第五步根据数据库虚拟化微内核根据虚拟库/表的多租户视图提取数据，按照rest封装代码进行封装。
客户端注册与封装步骤如下：
第一步增加rest接口注册数据库界面，包括url资源地址，rest所需端口等信息，注册rest方式虚拟表，详情如图2所示。
第二步核心查询解析代码模块保持不变，底层jdbc连接数据库方式改成rest资源访问方式获取远端数据库数据
第三步修改库表设计，dbtype除了MySQL、DB2、Oracle、SQLServer、SyBase等数据库之外还加rest类型
第四步增加DataSourceWrapper接口实现类RestWrapper底层实现类调整核心模块部分查询解析定向代码，如当查询解析dbtype为rest时执行DataSourceWrapper实现类RestWrapper
服务端封装步骤如下：
第一步增加DataSourceWrapper接口的rest服务端方法，接口每个方法对应不同url资源
第二步服务端通过配置文件获取本机数据库相关信息，也就是在部署rest接口适配器时，配置好该配置文件
第三步rest服务端方法获取客户数据库数据数据相当于本地jdbc连接数据库，客户端将参数传给服务端，服务端本地jdbc连接并操作数据库将结果封装成资源返回给客户端，客户端做一定解析处理就可获得想要的数据或结果。

Claims (3)

  1. 一种数据库虚拟化微内核数据源的注册与封装方法，其特征在于：所述方法包括虚拟库/表注册和封装；
    虚拟库/表注册：虚拟数据Federation将不同的虚拟元数据目录映射到不同的虚拟库/表，每个虚拟库/表都对应一个虚拟存储地址，存储虚拟库/表的主机有单独的虚拟存储池id；当存储虚拟库/表的主机启动时，按照虚拟存储池id向其对应的虚拟数据Federation汇报虚拟数据信息，进行自动注册，虚拟数据Federation收到汇报信息后，刷新虚拟元数据目录；当客户端访问到此虚拟库/表时，拥有该id的主机可以使用客户端挂载虚拟库/表；
    虚拟库/表封装：封装关系反映虚拟数据库、虚拟表/视图与物理的应用服务器、数据库服务器、数据库、数据表或文件之间的数据抽象映射；在逻辑层面对虚拟数据库、虚拟表的操作最终映射到对应的一个或多个物理概念实体的操作；所述映射是多对多或交叉约束；虚拟表可以建立到物理数据库和表或文件的映射关系，这种映射关系是非一一对应的，一个虚拟表schema可以映射到网络上的一个或多个数据库服务器、数据库、表或文件。
  2. 根据权利要求1所述的方法，其特征在于：所述的注册与封装具体包括以下步骤：
    第一步是客户端从MySQL、DB2、Oracle、SQLServer、SyBase等数据源的数据库/表、非关系型文件系统中，通过结构映射和内容映射，通过rest方式注册虚拟库/表，虚拟出虚拟数据库、虚拟数据表和虚拟文件；
    第二步是虚拟数据Federation根据用户操作算子启动数据源封装接口实现 从底层调整核心模块部分查询解析定向代码；
    第三步是服务端根据虚拟数据Federation提供命令，调用rest服务端数据源封装方法，通过配置文件获取客户端数据库相关信息，部署rest封装接口适配器；
    第四步是数据库虚拟化微内核对上层构建虚拟服务，该服务由虚拟表视图构成，数据库虚拟化微内核拥有一个虚拟用户空间，虚拟用户空间中的用户id，通过虚拟表操作算子，获得所需的虚拟表视图；虚拟表操作算子可以对不同的虚拟库/表进行运算，可以产生不同的虚拟表视图，实现虚拟库/表的多租户视图提取；
    第五步数据库虚拟化微内核根据虚拟库/表的多租户视图提取数据，按照rest封装代码进行封装。
  3. 根据权利要求1或2所述的方法，其特征在于：所述方法的
    客户端注册与封装具体包括以下步骤：
    第一步，增加rest接口注册数据库界面，包括url资源地址，rest所需端口等信息，注册rest方式虚拟表；
    第二步，核心查询解析代码模块保持不变，底层jdbc连接数据库方式改成rest资源访问方式获取远端数据库数据；
    第三步，修改库表设计，dbtype除了MySQL、DB2、Oracle、SQLServer、SyBase等数据库之外还加rest类型；
    第四步，增加DataSourceWrapper接口实现类RestWrapper底层实现类调整核心模块部分查询解析定向代码，当查询解析dbtype为rest时执行DataSourceWrapper实现类RestWrapper；
    服务端注册与封装具体包括以下步骤：
    第一步，增加DataSourceWrapper接口的rest服务端方法，接口每个方法对应不同url资源；
    第二步，服务端通过配置文件获取本机数据库相关信息，即在部署rest接口适配器时，配置好该配置文件；
    第三步，rest服务端方法获取客户数据库数据数据相当于本地jdbc连接数据库，客户端将参数传给服务端，服务端本地jdbc连接并操作数据库将结果封装成资源返回给客户端，客户端做一定解析处理就可获得想要的数据或结果。
PCT/CN2014/094368 2014-07-30 2014-12-19 数据库虚拟化微内核数据源的注册与封装方法 WO2016015439A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410370200.6 2014-07-30
CN201410370200.6A CN104133889B (zh) 2014-07-30 2014-07-30 一种数据库虚拟化微内核数据源注册与封装方法

Publications (1)

Publication Number Publication Date
WO2016015439A1 true WO2016015439A1 (zh) 2016-02-04

Family

ID=51806567

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/094368 WO2016015439A1 (zh) 2014-07-30 2014-12-19 数据库虚拟化微内核数据源的注册与封装方法

Country Status (2)

Country Link
CN (1) CN104133889B (zh)
WO (1) WO2016015439A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111259006A (zh) * 2019-11-19 2020-06-09 中国科学院计算机网络信息中心 一种通用的分布式异构数据一体化物理汇聚、组织、发布与服务方法及系统
CN111274294A (zh) * 2020-01-09 2020-06-12 中国科学院计算机网络信息中心 一种通用的分布式异构数据一体化逻辑汇聚组织、发布与服务方法及系统
CN111708515A (zh) * 2020-04-28 2020-09-25 山东鲁能软件技术有限公司 基于分布式共享微模块的数据处理方法及薪档积分系统
CN112860420A (zh) * 2019-11-27 2021-05-28 阿里巴巴集团控股有限公司 基于硬件虚拟化的数据处理方法及装置
CN113112366A (zh) * 2021-04-09 2021-07-13 上海中汇亿达金融信息技术有限公司 分布式数据总线
US11366808B2 (en) 2017-04-25 2022-06-21 Huawei Technologies Co., Ltd. Query processing method, data source registration method, and query engine
CN116775310A (zh) * 2023-07-31 2023-09-19 北京中泰华电科技有限公司 一种实现多个工业实时数据库融合扩展的方法
CN112860420B (zh) * 2019-11-27 2024-06-11 阿里巴巴集团控股有限公司 基于硬件虚拟化的数据处理方法及装置

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104133889B (zh) * 2014-07-30 2017-04-19 国云科技股份有限公司 一种数据库虚拟化微内核数据源注册与封装方法
CN105786946A (zh) * 2015-12-24 2016-07-20 国云科技股份有限公司 一种虚拟化的数据集成和查询系统及其实现方法
CN107391688A (zh) * 2017-07-25 2017-11-24 郑州云海信息技术有限公司 一种基于web的虚拟化网络管理平台的构建方法与装置
CN109240996B (zh) * 2018-08-23 2024-02-27 平安科技（深圳）有限公司 附件导出方法、装置、计算机设备及计算机可读存储介质
CN115563075B (zh) * 2022-10-09 2023-05-30 电子科技大学 基于微内核的虚拟文件系统实现方法
CN116383238B (zh) * 2023-06-06 2023-08-29 湖南红普创新科技发展有限公司 基于图结构的数据虚拟化系统、方法、装置、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060041527A1 (en) * 2004-08-03 2006-02-23 Aaron Fessler Virtual file system
CN101799826A (zh) * 2010-03-04 2010-08-11 中国电子科技集团公司第二十八研究所 一种基于虚拟视图的网络化数据共享系统及方法
CN101969475A (zh) * 2010-11-15 2011-02-09 张军 基于云计算的商业数据可控分发与融合应用系统
CN103617175A (zh) * 2013-11-04 2014-03-05 广东电子工业研究院有限公司 一种用于大规模分布异构数据的虚拟化方法
CN104133889A (zh) * 2014-07-30 2014-11-05 国云科技股份有限公司 一种数据库虚拟化微内核数据源注册与封装方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101187937A (zh) * 2007-10-30 2008-05-28 北京航空航天大学 网格环境下模式复用的异构数据库访问和集成方法
US20130304738A1 (en) * 2012-05-11 2013-11-14 International Business Machines Corporation Managing multimedia information using dynamic semantic tables

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060041527A1 (en) * 2004-08-03 2006-02-23 Aaron Fessler Virtual file system
CN101799826A (zh) * 2010-03-04 2010-08-11 中国电子科技集团公司第二十八研究所 一种基于虚拟视图的网络化数据共享系统及方法
CN101969475A (zh) * 2010-11-15 2011-02-09 张军 基于云计算的商业数据可控分发与融合应用系统
CN103617175A (zh) * 2013-11-04 2014-03-05 广东电子工业研究院有限公司 一种用于大规模分布异构数据的虚拟化方法
CN104133889A (zh) * 2014-07-30 2014-11-05 国云科技股份有限公司 一种数据库虚拟化微内核数据源注册与封装方法

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11366808B2 (en) 2017-04-25 2022-06-21 Huawei Technologies Co., Ltd. Query processing method, data source registration method, and query engine
US11907213B2 (en) 2017-04-25 2024-02-20 Huawei Technologies Co., Ltd. Query processing method, data source registration method, and query engine
CN111259006A (zh) * 2019-11-19 2020-06-09 中国科学院计算机网络信息中心 一种通用的分布式异构数据一体化物理汇聚、组织、发布与服务方法及系统
CN112860420A (zh) * 2019-11-27 2021-05-28 阿里巴巴集团控股有限公司 基于硬件虚拟化的数据处理方法及装置
CN112860420B (zh) * 2019-11-27 2024-06-11 阿里巴巴集团控股有限公司 基于硬件虚拟化的数据处理方法及装置
CN111274294A (zh) * 2020-01-09 2020-06-12 中国科学院计算机网络信息中心 一种通用的分布式异构数据一体化逻辑汇聚组织、发布与服务方法及系统
CN111274294B (zh) * 2020-01-09 2023-03-10 中国科学院计算机网络信息中心 一种通用的分布式异构数据一体化逻辑汇聚组织、发布与服务方法及系统
CN111708515A (zh) * 2020-04-28 2020-09-25 山东鲁能软件技术有限公司 基于分布式共享微模块的数据处理方法及薪档积分系统
CN113112366A (zh) * 2021-04-09 2021-07-13 上海中汇亿达金融信息技术有限公司 分布式数据总线
CN116775310A (zh) * 2023-07-31 2023-09-19 北京中泰华电科技有限公司 一种实现多个工业实时数据库融合扩展的方法

Also Published As

Publication number Publication date
CN104133889A (zh) 2014-11-05
CN104133889B (zh) 2017-04-19

Similar Documents

Publication Publication Date Title
WO2016015439A1 (zh) 数据库虚拟化微内核数据源的注册与封装方法
US11120042B2 (en) Accelerator based data integration
US11308079B2 (en) Maintaining data stream history for generating materialized views
US9715536B2 (en) Virtualization method for large-scale distributed heterogeneous data
CN100573528C (zh) 数字博物馆网格及其构造方法
WO2016101301A1 (zh) 关系型数据库表模式对象化和虚拟化机制
US20110010379A1 (en) Database system with query interception and redirection
CN107533569B (zh) 用于多维数据库环境中的沙箱支持的系统和方法
CN104008135A (zh) 一种多源异构数据库融合系统及其数据查询方法
Samwel et al. F1 query: Declarative querying at scale
US11188555B2 (en) Isolating a network stack for pluggable databases
EP3867767B1 (en) Isolated hierarchical runtime environments for multi-tenant databases
US11074248B2 (en) Map of operations for ingesting external data
KR20170053013A (ko) 빅데이터 분석을 위한 데이터 가상화 시스템
US11113110B2 (en) Intelligent pooling of isolated hierarchical runtimes for cloud scale databases in a multi-tenant environment
US11194773B2 (en) Integration of existing databases into a sharding environment
Liu et al. A general model of data service in spatial information grid
Ahuja et al. An approach for virtualization and integration of heterogeneous cloud databases
Hsu et al. The study of application and evaluation with NoSQL databases in cloud computing
Han et al. Design and Implementation of Big Data Management Platform for Android Applications
US20200311086A1 (en) High-performance implementation of sharing of read-only data in a multi-tenant environment
CN115408414A (zh) 一种处理结构化查询语言sql语句的方法及相应设备
Wang et al. Design and Study of Website Architecture Model
Shukla et al. Empirical Performance Comparison of OODBMS over RDBMS
Htwe A Framework for Heterogeneous Data Integration Using Grid Technology

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14899030

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14899030

Country of ref document: EP

Kind code of ref document: A1