CN113254519B - Access method, device, equipment and storage medium of multi-source heterogeneous database - Google Patents
Access method, device, equipment and storage medium of multi-source heterogeneous database Download PDFInfo
- Publication number
- CN113254519B CN113254519B CN202110589454.7A CN202110589454A CN113254519B CN 113254519 B CN113254519 B CN 113254519B CN 202110589454 A CN202110589454 A CN 202110589454A CN 113254519 B CN113254519 B CN 113254519B
- Authority
- CN
- China
- Prior art keywords
- data
- interface
- data access
- access request
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Abstract
The embodiment of the invention discloses a method, a device, equipment and a storage medium for accessing a multi-source heterogeneous database. The method comprises the following steps: acquiring a first data access request input by a user through an access interface of a uniform interface layer; determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request; performing, by a data store layer, the second data access request to a target database, the database comprising: relational databases and non-relational databases. By the technical scheme of the embodiment of the invention, the universality of the multi-source heterogeneous database can be improved, and the development cost and the operation and maintenance cost of database access software can be reduced.
Description
Technical Field
The embodiment of the invention relates to a computer data processing technology, in particular to a method, a device, equipment and a storage medium for accessing a multi-source heterogeneous database.
Background
In the process of space digitization, many enterprises construct many heterogeneous storage schemes which are isolated from each other according to their own business requirements and technical characteristics. Due to the lack of uniform data model abstraction and no top-down architecture design, development platforms are different, and all service systems are independent.
The existing relational database (such as a PostgreSQL database) and the non-relational database (such as Aliyun tablescore) have different interaction instructions in storage schemes, ecological fragmentation is serious, the storage schemes are isolated, so that developers cannot use other storage schemes quickly by using existing knowledge and code bases, the learning and code migration cost is huge, and the operation and maintenance cost is high.
Disclosure of Invention
The embodiment of the invention provides an access method, device, equipment and storage medium of a multi-source heterogeneous database, which can improve the universality of the multi-source heterogeneous database and reduce the development cost and operation and maintenance cost of database access software.
In a first aspect, an embodiment of the present invention provides an access method for a multi-source heterogeneous database, including:
acquiring a first data access request input by a user through an access interface of a uniform interface layer, wherein the first data access request at least comprises: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein field attribute information is stored in the field array;
determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request;
performing, by a data store layer, the second data access request to a target database, the database comprising: relational databases and non-relational databases.
In a second aspect, an embodiment of the present invention further provides an apparatus for accessing a multi-source heterogeneous database, where the apparatus includes:
the unified interface layer is used for acquiring a first data access request input by a user through an access interface, wherein the first data access request at least comprises: a root node name and table attributes, the table attributes comprising: the system comprises a table name and a field array, wherein field attributes are stored in the field array;
the data driving layer is used for determining a target driver corresponding to the access interface and converting the first data access request through the target driver to obtain a second data access request;
a data store layer for performing the second data access request to a target database, the database comprising: relational databases and non-relational databases.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the method for accessing the multi-source heterogeneous database according to any one of the embodiments of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the accessing method for the multi-source heterogeneous database according to any one of the embodiments of the present invention.
The embodiment of the invention obtains a first data access request input by a user through an access interface of a unified interface layer; determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request; performing, by a data store layer, the second data access request to a target database, the database comprising: the relational database and the non-relational database solve the problems that developers cannot use other storage schemes quickly by using the existing knowledge and code base due to different interaction instructions, serious ecological fragmentation and isolated storage schemes of the multi-source heterogeneous database, and the learning and maintenance cost is high, improve the universality of the multi-source heterogeneous database, and reduce the development cost and the operation and maintenance cost of database access software.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart of an access method for a multi-source heterogeneous database according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an access method for a multi-source heterogeneous database according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a multi-source heterogeneous database according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of an access device for a multi-source heterogeneous database according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example one
Fig. 1 is a flowchart of an access method for multi-source heterogeneous data according to an embodiment of the present invention, where the embodiment is applicable to accessing a multi-source heterogeneous database to perform addition, deletion, and check and modification on data in the multi-source heterogeneous database, and the method may be executed by an access device for multi-source heterogeneous data according to an embodiment of the present invention, and the access device may be implemented in a software and/or hardware manner. As shown in fig. 1, the method specifically includes the following steps:
s110, a first data access request input by a user is obtained through an access interface of the unified interface layer.
The unified interface layer can be a user interface layer, interface service for accessing the heterogeneous database is provided for a user through a visualization technology, access operation of the user on the multi-source heterogeneous database is achieved, and a returned result is displayed.
The access interface provided by the unified interface layer may include: interfaces with various functions such as creating interfaces, deleting interfaces, inserting interfaces, updating interfaces, inquiring interfaces and the like. The creation interface is used for creating data in the multi-source heterogeneous database according to an access request of a user; the deleting interface is used for deleting data in the multi-source heterogeneous database according to an access request of a user; the inserting and removing interface is used for inserting data into the multi-source heterogeneous database according to an access request of a user; the updating interface is used for updating data stored in the multi-source heterogeneous database according to the access request of the user, and the query interface is used for querying the data stored in the multi-source heterogeneous database according to the access request of the user.
In the embodiment of the invention, the access to the heterogeneous database is abstracted into the access operation to the table data, namely, the first data access request is the access request to the table structure storage data object. Wherein the first data access request at least comprises: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein the field array stores field attribute information.
The root node name may be used to characterize the storage scheme of the multi-source heterogeneous database, for example, the root node name "cluster" represents the Aliyun tablescore database, and the root node name "edge" represents the PostgreSQL database.
The table attribute information may be used to characterize attribute information of each data table stored in the database, including a table name and a field array, where the field array is an array formed by the attribute information of each field in the data table. Each field has a plurality of attribute information.
The attribute information of each field in the field array may include: name, field type, field primary key attribute, field primary, field null, field number, field value, and parameter value corresponding to the field attribute. The field primary key attribute field. isprimary is used to indicate whether a field is a primary key, and the field null attribute field. isnullabel is used to indicate whether a field is allowed to be null. It is understood that the attribute information included in the field array may be different according to the access request of the user. The field array can further comprise: a query filter condition field, a sort operation field, an aggregate operation field, and the like. The field array may also include: the data field is returned.
For example, the manner of obtaining the first data access request input by the user through the access interface of the unified interface layer may be to input, for the user, the access parameters such as the name of the root node, the name of the table, and the field array, which need to be accessed, in the access interface of the unified access layer according to the access requirement and the access attribute provided by the access interface. Or may input a data access parameter for an access attribute provided in a Software Development Kit (SDK) at an access interface, and construct the first data access request according to the field attribute and the data access parameter. For example, the SDK sets the access attribute as the name, the field type and the like of a certain field, restricts the numerical value input by the user, and generates the field attribute information in the field array according to the numerical value input by the user and the access attribute.
And the access authority can be set for different users, and the data content and the access operation accessed by the users are limited. The method comprises the steps of obtaining authority information and a first data access request input by a user in an access interface of a unified interface layer, sending the authority information to a server, responding to the first data access request of the user if the user has access authority, and sending prompt information to the user if the user does not have the access authority.
S120, determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request.
The data driving layer comprises a plurality of drivers respectively corresponding to the access interfaces; the access interface protocol of the access interface corresponding to different databases is preset in the driver, and based on the access interface protocol, a first data access request acquired by the access interface can be converted into a second data access request suitable for accessing various heterogeneous databases.
The first data access request may be an access request constructed according to access parameters input by a user. The second data access request may be an access request of the target database converted by the driver, and the second data access request may be a structured statement based on the target database language.
For example, the manner of determining the target driver corresponding to the access interface may be to establish a communication transport protocol between each access interface and the corresponding driver, and encapsulate the communication transport protocol in a Software Development Kit (SDK), where the communication transport protocol of the access interface and the corresponding driver may be HTTP or GPRC. GRPC is a high-performance and universal Remote Procedure Call (RPC) framework, is designed based on HTTP/2 Protocol standard, is developed based on Protocol Buffers serialization Protocol, and supports a plurality of development languages. The hypertext transfer protocol HTTP is a transport protocol for transferring hypertext from a web server to a local browser.
S130, executing the second data access request to a target database through a data storage layer, wherein the database comprises: relational databases and non-relational databases.
The data storage layer is used for storing multi-source heterogeneous databases, the databases are independent from one another, and data are stored respectively based on respective storage schemes. The databases in the data storage layer may include a Relational Database and a non-Relational Database, the Relational Database (RDBMS) storing data using a predefined data structure, such as PostgreSQL, using SQL as an interface. The non-relational database (Not Only SQL, NoSQL), which may be, for example, alli cloud table store, provides SDK and API for users.
Illustratively, the second data access request is input into the interface provided by the target database through the data storage layer to access the data in the target database. The data access operation comprises a data creating operation, a data deleting operation, a data inserting operation, a data updating operation and a data inquiring operation.
According to the technical scheme of the embodiment, a first data access request input by a user is acquired through an access interface of a unified interface layer, and the first data access request at least comprises the following steps: a root node name and table attributes, the table attributes comprising: the system comprises a table name and a field array, wherein field attributes are stored in the field array; determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request; performing, by a data store layer, the second data access request to a target database, the database comprising: the relational database and the non-relational database can be accessed, the universality of the multi-source heterogeneous database is improved, and the development cost and the operation and maintenance cost of database access software are reduced.
Example two
Fig. 2 is a flowchart of an access method for a multi-source heterogeneous database according to a second embodiment of the present invention, where the present embodiment is optimized based on the foregoing embodiment, and in the present embodiment, the converting the first data access request by the target driver to obtain a second data access request includes: inquiring names of databases pre-stored in the target driver according to the root node names to determine a target database; determining an access interface protocol corresponding to the first data access request according to the access interface and the target database; and converting the data access parameters in the first data access request based on the access interface protocol to obtain a second data access request.
As shown in fig. 2, the method of this embodiment specifically includes the following steps:
s210, a first data access request input by a user is obtained through an access interface of the unified interface layer.
Optionally, obtaining the first data access request input by the user through the access interface of the unified interface layer includes:
providing a software development kit at each access interface in the unified interface layer, wherein the software development kit encapsulates data access attributes corresponding to the access interfaces;
acquiring data access parameters input by a user under each data access attribute;
and constructing the first data access request according to the data access attribute and the input data access parameter.
Wherein the data access attribute may include: at least one of a root node name, a table name, a field type, a field primary key attribute, a field value, and a field null attribute. It will be appreciated that the data access attributes may differ depending on the data access request of the user.
Specifically, the user may input a corresponding data access parameter under the data access attribute encapsulated in the tool development kit of the corresponding access interface according to the access requirement, and construct the first data access request according to the data access attribute and the data access parameter. The mode of inputting the corresponding data access parameter by the user may be directly inputting the data access parameter, or may be selecting the corresponding data access parameter from a pull-down menu configured by the data access attribute.
Illustratively, the user selects a creation interface according to the requirements of the creation table, and the attributes of the access data encapsulated in the creation interface include: root node name root, table name tablame, and field array. Wherein, fieldarry = [ field.name, field.value, field.type, field.isprimary, field.isnullabel ]. If the user selects "group" under the root node name attribute, "enters" table1 "under the table name attribute," enters "ID" under the field name attribute field, enters "20200001" under the field value attribute field, and selects "true" under the field primary key attribute field, a first data access request is constructed according to the access data attribute and the access parameter entered by the user, the access content of the first data access request may be root = group, table = table1, field array = [ ID, 20020001, true ], and the request is to create data with ID number 2002001 in table 1.
S220, determining a target driver corresponding to the access interface through a data driving layer.
Specifically, in the data driving layer, each access interface corresponds to a driver one by one, for example, a creation interface corresponds to a creation driver, an inquiry interface corresponds to an inquiry driver, and a deletion interface corresponds to a deletion driver.
Illustratively, a target driver corresponding to an access interface for inputting a first data access request by a user is determined based on a communication transmission protocol between the access interface and the corresponding driver, and the data access request is sent to the target driver.
And S230, determining a target database according to the root node name.
The target database may be a database in which the data requested to be accessed in the first data access request is stored.
Specifically, according to the root node name input by the user, a mapping relationship table between the root node name and a database pre-stored in the target database may be queried, and the target database corresponding to the root node name may be determined.
Illustratively, a root node name in a first data creation request input by a user is root = cloud, and then the target database is determined to be Aliyun tabletree according to Aliyun tabletree corresponding to the prestored root node name cloud.
S240, obtaining an access interface protocol corresponding to the first data access request according to the access interface and the target database.
The access interface protocol corresponding to each access interface is pre-configured in each driver, and the access interface protocol may be a conversion protocol between each access interface and a target database, and is used for converting a first data access request received by each access interface into a second data access request corresponding to each target database language.
Illustratively, according to an access interface corresponding to a first data access request input by a user and a target database of a root node name in the first data access request, determining an access interface protocol corresponding to the first data access request in a target driver. For example, the user may input a first creation data request at the creation interface, where the root node name root = cloud in the first creation data request, and then determine, in the creation driver, an access interface protocol corresponding to Aliyun tablescore.
And S250, converting the data access parameters in the first data access request based on the access interface protocol to obtain a second data access request.
Specifically, for the multi-source heterogeneous database, the interactive instructions and the user interfaces of the databases are different, the data input by the user are converted into a second data access request through the access interface protocol, and the second data access request can accord with the interactive language of each target database or the preset rule corresponding to the user interface, so that the target database is accessed.
S260, executing the second data access request to a target database through a data storage layer, wherein the database comprises: relational databases and non-relational databases.
According to the technical scheme of the embodiment, a first data access request input by a user is acquired through an access interface of a unified interface layer, and the first data access request at least comprises the following steps: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein field attribute information is stored in the field array; determining a target database according to the root node name; acquiring an access interface protocol corresponding to the first data access request according to the access interface and the target database; based on the access interface protocol, converting the data access parameters in the first data access request to obtain a second data access request, and executing the second data access request to a target database through a data storage layer, wherein the database comprises: the relational database and the non-relational database can access the multi-source heterogeneous database, improve the universality of the database and reduce the development cost and the operation and maintenance cost of database access software.
Optionally, executing the second data access request on the target database through the data storage layer includes:
querying, by the data store layer, second target data contained in a target database;
and reading second target data based on the second data access request and obtaining return data, or writing the second target data based on the second data access request.
And the second target data is queryable data stored in the target database.
Specifically, in the data storage layer, access requests such as creation, deletion, update, query, and the like are performed on the second target data included in the target database based on the second data access request.
When the target database is changed, only the data driving mode and the data access parameters need to be modified, and the access request instruction for accessing the target database does not need to be modified again, so that the universality of the database is improved, and the development cost and the operation and maintenance cost of database access software are reduced.
Optionally, the access interface includes: creating an interface and deleting an interface, and correspondingly, converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request comprises:
determining a first primary key array according to field attribute information in the first data access request;
and determining a second data access request corresponding to the first primary key array based on an access interface protocol corresponding to the access interface.
Specifically, if a first data access request input by a user is acquired at a creation interface or a deletion interface, a first primary key array is determined according to a first primary key attribute corresponding to each field in a field array of the first data access request. And inputting the first main key array into an access interface protocol corresponding to the access interface, and determining a second data access request corresponding to the first main key array according to the interface protocol.
In a specific example, if the field pk in the first data access request obtained by the creation interface and input by the user is a primary key, it is determined that the first primary key array { pk } = { pk1.name, pk.2.name, … … }, and the root node name edge corresponds to the target database being PostgreSQL, then the second access request may be:
in another specific example, if the primary key attribute of a field pk in the first data access request obtained by the creation interface and input by the user is a primary key, it is determined that the first primary key array { pk } = { pk1.name, pk2.name, … … }, and the root name cluster corresponds to the target database being Aliyun tablescore, then the second access request may be:
optionally, the access interface includes: correspondingly, converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request includes:
if the target database is a relational database, determining a first data insertion request corresponding to the data access parameter based on a first access interface protocol corresponding to an insertion interface;
and if the target database is a non-relational database, determining a second main key array and a first row array according to the field attribute information in the first data access request, and determining a second data insertion request corresponding to the second main key array and the first row array based on a second access interface protocol corresponding to the insertion interface.
Wherein the field array in the first data access request comprises: the data access method comprises an insertion data field array and a return data field array, wherein the insertion data field array is used for storing data fields needing to be inserted by users, and the return data field array is used for storing data fields obtained after access requests.
Specifically, if a first data access request input by a user is acquired at an insertion interface, determining the type of a target database corresponding to a root node name of the first data access request, and determining a primary key array and a column array according to a field primary key attribute field. isPrimary is false and is denoted as a general column.
In a specific example, when the first data access request input by the user is acquired at the insertion interface, and the target database corresponding to the root node name cluster is PostgreSQL and belongs to the relational database, the determining, based on the first access interface protocol corresponding to the insertion interface, that the data insertion request corresponding to the data access parameter may be:
in another specific example, when a first data access request input by a user is acquired at an insertion interface, and a target database corresponding to a root node name cluster is Aliyun table score and belongs to a non-relational database, a second primary key array { pk } is determined according to a second primary key attribute stored in the field array, a first column array { column } is determined according to a field value attribute stored in the field array, and a data insertion request corresponding to the second primary key array and the first column array may be determined as follows:
optionally, the access interface includes: correspondingly, the converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request includes:
determining a third main key array and a second column array according to the field attribute information of the first data access request;
if the target database is a relational database, judging whether first target data exist or not; if so, determining a first data updating request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the updating interface; if not, determining a third data insertion request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the insertion interface;
and if the target database is a non-relational database, determining a third data updating request corresponding to the third main key array and the second column array based on a second access interface protocol corresponding to the updating interface.
The first target data may be operation data corresponding to the first data access request, and if the first target data exists, the data update request is executed; if the first target data does not exist, a data insertion request is executed. The first target data may be constrained by a primary key array having a field primary key attribute field.
Specifically, a third primary key array and a column array are determined according to a third primary key attribute stored in a field array in the first data access request; and inputting the third main key array and the second column array into an access interface protocol corresponding to the access interface, and determining a data updating request corresponding to the third main key array and the column array.
In a specific example, a first data access request input by a user is obtained at an update interface, and a third main key array and a second column array are determined according to field attribute information of the first data access request; the target database corresponding to the root node name group in the first data access request is PostgreSQL, and belongs to a relational database, and whether the first target data exists is determined according to a preset expectExist function, if an execution result of the expectExist function is true, which indicates that the first target data exists, the first data update request corresponding to the third primary key array and the second column array may be:
if the execution result of the expectExist function is false, which indicates that the first target data does not exist, the third data insertion request corresponding to the third primary key array and the second column array may be:
the method has the advantages that the operation data corresponding to the updating request can be judged to exist, if the operation data do not exist, the operation of converting the data access parameters in the first data access request based on the updating interface fails, and then the data access parameters in the first data access request are automatically converted based on the inserting interface to obtain the second data access request.
In another specific example, when a first data access request input by a user is acquired at an update interface, a target database corresponding to a root node name cluoud is Aliyun tablescore and belongs to a non-relational database, a third primary key array { pk } and a second column array { column } are determined according to a third primary key attribute stored in a field array in the first data access request, and a third data update request corresponding to the second primary key array and the second column array may be determined as follows:
optionally, the access interface includes: the query interface correspondingly converts the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request, and includes:
if the target database is a relational database, determining a first data query request corresponding to the data access parameter based on a first access interface protocol corresponding to a query interface;
and if the target database is a non-relational database, determining an index value according to the table name in the first data access request, and determining a second data query request corresponding to the index value based on a second access interface protocol corresponding to the query interface.
Wherein the field array in the first data access request comprises: query fields, query filter criteria, aggregation operations, sort operations, and row number limits.
The query filter may be composed of three parts, a subject, a predicate, and an object. The subject may be a field name, or a filter condition combined by a subject predicate and an object, the object may be a field value or a filter condition combined by a subject predicate and an object, and the predicate specifies a mathematical operation and a logical operation, the mathematical operation may be, for example, "=, | =, > =, <, =" the logical operation may be, for example, and or.
For example, the query filter condition may be represented by a logical expression, such as ((((enclosure _ id = encl _ v) AND (start _ time > =1619269622)) AND (start _ time <1619273222)) AND ((entry _ id = eid _ v _1) OR (entry _ id = eid _ v _ 2))). Query filter terms may also be represented as interfaces, such as:
in a specific example, a first data access request input by a user is acquired in a query interface, where a target database corresponding to a root node name cluster in the first data access request is PostgreSQL and belongs to a relational database, and the first data query request determined according to data access parameters such as a query field selection, a query filter condition filter, an aggregation operation aggregation, a sort operation sorter, and a line limit may be:
in another specific example, a first data access request input by a user is acquired at a query interface, where a target database corresponding to a root node name cloud in the first data access request is Aliyun tabletree and belongs to a non-relational database, and an index value searchIndex is determined according to a table name in the first data access request. The second data query request corresponding to the index value searchIndex may be:
as shown in fig. 3, in the embodiment of the present invention, access to a multi-source heterogeneous database is implemented by a multi-source heterogeneous database access system, where the multi-source heterogeneous database access system includes a unified interface layer, a data driving layer, and a data storage layer, and the unified interface layer includes: the method comprises the following steps of creating an interface, deleting an interface, inserting an interface, updating an interface and inquiring an interface, wherein the data driving layer comprises the following steps: creating a driver, deleting the driver, inserting the driver, updating the driver and a driver interface, wherein each driver corresponds to each access interface in the unified interface layer one by one, a data storage layer comprises each database, and the databases comprise: and the relational database and the non-relational database convert the first data access request through each target driver in the data driving layer to obtain a second data access request, so that the relational database and the non-relational database are accessed, the universality of the multi-source heterogeneous database is improved, and the development cost and the operation and maintenance cost of database access software are reduced.
EXAMPLE III
Fig. 4 is a schematic structural diagram of an access apparatus for a multi-source heterogeneous database according to a third embodiment of the present invention. The embodiment is applicable to accessing a multi-source heterogeneous database, and performing addition, deletion, check and modification on data in the multi-source heterogeneous database, the apparatus may be implemented in a software and/or hardware manner, and the apparatus may be integrated in any device providing an access function of the multi-source heterogeneous database, as shown in fig. 4, the apparatus for accessing the multi-source heterogeneous database specifically includes: unified interface layer 310, data driving layer 320, and data storage layer 330.
The unified interface layer 310 is configured to obtain, through an access interface, a first data access request input by a user, where the first data access request at least includes: a root node name and table attributes, the table attributes comprising: the system comprises a table name and a field array, wherein field attributes are stored in the field array;
the data driving layer 320 is configured to determine a target driver corresponding to the access interface, and convert the first data access request through the target driver to obtain a second data access request;
a data store layer 330 for performing the second data access request on a target database, the database comprising: relational databases and non-relational databases.
Optionally, the unified interface layer 310 is specifically configured to:
providing a software development kit at each access interface in the unified interface layer, wherein the software development kit encapsulates data access attributes corresponding to the access interfaces;
acquiring data access parameters input by a user under each data access attribute;
and constructing the first data access request according to the data access attribute and the input data access parameter.
Optionally, the data driving layer 320 includes:
the determining module is used for determining a target database according to the root node name;
the acquisition module is used for acquiring an access interface protocol corresponding to the first data access request according to the access interface and the target database;
and the conversion module is used for converting the data access parameters in the first data access request based on the access interface protocol to obtain a second data access request.
Optionally, the access interface includes: creating an interface and deleting an interface, wherein correspondingly, the conversion module is specifically configured to:
converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request comprises:
determining a first primary key array according to field attribute information in the first data access request;
and determining a second data access request corresponding to the first primary key array based on an access interface protocol corresponding to the access interface.
Optionally, the access interface includes: and correspondingly, the conversion module is specifically configured to:
if the target database is a relational database, determining a first data insertion request corresponding to the data access parameter based on a first access interface protocol corresponding to an insertion interface;
if the target database is a non-relational database, determining a second main key array and a first row array according to field attribute information in the first data access request, and determining a second data insertion request corresponding to the second main key array and the first row array based on a second access interface protocol corresponding to the insertion interface;
wherein the field array in the first data access request comprises: an array of insert data fields and an array of return data fields.
Optionally, the access interface includes: the conversion module is specifically configured to:
determining a third main key array and a second column array according to the field attribute information of the first data access request;
if the target database is a relational database, judging whether first target data exist, if so, determining a first data updating request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the updating interface; if not, determining a third data insertion request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the insertion interface;
and if the target database is a non-relational database, determining a third data updating request corresponding to the third main key array and the second column array based on a second access interface protocol corresponding to the updating interface.
Optionally, the access interface includes: and a query interface, correspondingly, the conversion module is specifically configured to:
if the target database is a relational database, determining a first data query request corresponding to the data access parameter based on a first access interface protocol corresponding to a query interface;
if the target database is a non-relational database, determining an index value according to the table name in the first data access request, and determining a second data query request corresponding to the index value based on a second access interface protocol corresponding to a query interface;
wherein the field array in the first data access request comprises: query fields, query filter criteria, aggregation operations, sort operations, and row number limits.
Optionally, the data storage layer 330 is specifically configured to:
querying, by the data store layer, second target data contained in a target database;
and reading second target data based on the second data access request and obtaining return data, or writing the second target data based on the second data access request.
The product can execute the access method of the multi-source heterogeneous database provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 5 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention. FIG. 5 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 5 is only an example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.
As shown in FIG. 5, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processors 16, a system memory 28, and a bus 18 that connects the various system components (including the system memory 28 and the processors 16).
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The processor 16 executes programs stored in the system memory 28 to execute various functional applications and data processing, for example, to implement the access method of the multi-source heterogeneous database provided by the embodiment of the present invention: acquiring a first data access request input by a user through an access interface of a uniform interface layer, wherein the first data access request at least comprises: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein field attribute information is stored in the field array; determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request; performing, by a data store layer, the second data access request to a target database, the database comprising: relational databases and non-relational databases.
EXAMPLE five
An embodiment five of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for accessing a multi-source heterogeneous database according to the embodiments of the present invention: acquiring a first data access request input by a user through an access interface of a uniform interface layer, wherein the first data access request at least comprises: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein field attribute information is stored in the field array; determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request; performing, by a data store layer, the second data access request to a target database, the database comprising: relational databases and non-relational databases.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A multi-source heterogeneous database access method is characterized by comprising the following steps:
acquiring a first data access request input by a user through an access interface of a uniform interface layer, wherein the first data access request at least comprises: a root node name and table attribute information, the table attribute information comprising: the system comprises a table name and a field array, wherein field attribute information is stored in the field array;
determining a target driver corresponding to the access interface through a data driving layer, and converting the first data access request through the target driver to obtain a second data access request;
performing, by a data store layer, the second data access request to a target database, the database comprising: a relational database and a non-relational database;
wherein the access interface comprises: creating an interface, deleting an interface, inserting an interface, updating an interface and inquiring an interface; the data driving layer includes: creating a driver, deleting the driver, inserting the driver, updating the driver and inquiring the driver, wherein the access interfaces correspond to the drivers in the data driving layer one by one;
converting, by the target drive, the first data access request to obtain a second data access request, comprising:
determining a target database according to the root node name;
acquiring an access interface protocol corresponding to the first data access request according to the access interface and the target database;
and converting the data access parameters in the first data access request based on the access interface protocol to obtain a second data access request.
2. The method of claim 1, wherein obtaining the first data access request input by the user through the access interface of the unified interface layer comprises:
providing a software development kit at each access interface in the unified interface layer, wherein the software development kit encapsulates data access attributes corresponding to the access interfaces;
acquiring data access parameters input by a user under each data access attribute;
and constructing the first data access request according to the data access attribute and the input data access parameter.
3. The method of claim 1, wherein the access interface comprises: creating an interface and deleting an interface, and correspondingly, converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request comprises:
determining a first primary key array according to field attribute information in the first data access request;
and determining a second data access request corresponding to the first primary key array based on an access interface protocol corresponding to the access interface.
4. The method of claim 1, wherein the access interface comprises: correspondingly, converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request includes:
if the target database is a relational database, determining a first data insertion request corresponding to the data access parameter based on a first access interface protocol corresponding to an insertion interface;
if the target database is a non-relational database, determining a second main key array and a first row array according to field attribute information in the first data access request, and determining a second data insertion request corresponding to the second main key array and the first row array based on a second access interface protocol corresponding to the insertion interface;
wherein the field array in the first data access request comprises: an array of insert data fields and an array of return data fields.
5. The method of claim 1, wherein the access interface comprises: correspondingly, the converting the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request includes:
determining a third main key array and a second column array according to the field attribute information of the first data access request;
if the target database is a relational database, judging whether first target data exist or not; if so, determining a first data updating request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the updating interface; if not, determining a third data insertion request corresponding to the third main key array and the second column array based on a first access interface protocol corresponding to the insertion interface;
and if the target database is a non-relational database, determining a third data updating request corresponding to the third main key array and the second column array based on a second access interface protocol corresponding to the updating interface.
6. The method of claim 1, wherein the access interface comprises: the query interface correspondingly converts the data access parameter in the first data access request based on the access interface protocol to obtain a second data access request, and includes:
if the target database is a relational database, determining a first data query request corresponding to the data access parameter based on a first access interface protocol corresponding to a query interface;
if the target database is a non-relational database, determining an index value according to the table name in the first data access request, and determining a second data query request corresponding to the index value based on a second access interface protocol corresponding to a query interface;
wherein the field array in the first data access request comprises: query fields, query filter criteria, aggregation operations, sort operations, and row number limits.
7. The method of claim 1, wherein executing the second data access request to the target database through the data store layer comprises:
querying, by the data store layer, second target data contained in a target database;
and reading second target data based on the second data access request and obtaining return data, or writing the second target data based on the second data access request.
8. An apparatus for accessing a multi-source heterogeneous database, comprising:
the unified interface layer is used for acquiring a first data access request input by a user through an access interface, wherein the first data access request at least comprises: a root node name and table attributes, the table attributes comprising: the system comprises a table name and a field array, wherein field attributes are stored in the field array;
the data driving layer is used for determining a target driver corresponding to the access interface and converting the first data access request through the target driver to obtain a second data access request;
a data store layer for performing the second data access request to a target database, the database comprising: a relational database and a non-relational database;
wherein the access interface comprises: creating an interface, deleting an interface, inserting an interface, updating an interface and inquiring an interface; the data driving layer includes: creating a driver, deleting the driver, inserting the driver, updating the driver and inquiring the driver, wherein the access interfaces correspond to the drivers in the data driving layer one by one;
the data driving layer includes:
the determining module is used for determining a target database according to the root node name;
the acquisition module is used for acquiring an access interface protocol corresponding to the first data access request according to the access interface and the target database;
and the conversion module is used for converting the data access parameters in the first data access request based on the access interface protocol to obtain a second data access request.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of accessing the multi-source heterogeneous database of any of claims 1-7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for accessing a multi-source heterogeneous database according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110589454.7A CN113254519B (en) | 2021-05-28 | 2021-05-28 | Access method, device, equipment and storage medium of multi-source heterogeneous database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110589454.7A CN113254519B (en) | 2021-05-28 | 2021-05-28 | Access method, device, equipment and storage medium of multi-source heterogeneous database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113254519A CN113254519A (en) | 2021-08-13 |
| CN113254519B true CN113254519B (en) | 2021-10-08 |
Family
ID=77185196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110589454.7A Active CN113254519B (en) | 2021-05-28 | 2021-05-28 | Access method, device, equipment and storage medium of multi-source heterogeneous database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113254519B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821554B (en) * | 2021-08-17 | 2023-10-13 | 太原市高远时代科技有限公司 | Method for realizing heterogeneous database data acquisition |
| CN114723565B (en) * | 2022-06-02 | 2022-08-23 | 高盈国际创新科技(深圳)有限公司 | Multi-source multi-language information processing method and system based on golang |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656943A (en) * | 2018-12-12 | 2019-04-19 | 国网辽宁省电力有限公司信息通信分公司 | Shield the data service method of Query in Heterogeneous Databases complexity |
| CN110633292A (en) * | 2019-09-19 | 2019-12-31 | 上海依图网络科技有限公司 | Query method, device, medium, equipment and system for heterogeneous database |
| US20200175018A1 (en) * | 2018-11-29 | 2020-06-04 | Capital One Services, Llc | Real-time processing of event-based streaming with nosql databases |
| CN111581231A (en) * | 2020-04-20 | 2020-08-25 | 北京明略软件系统有限公司 | Query method and device based on heterogeneous database |
| CN112328623A (en) * | 2020-11-06 | 2021-02-05 | 昆山数字城市信息技术有限公司 | Multi-source heterogeneous data management method based on mixed ontology mode |
| CN112487075A (en) * | 2020-12-29 | 2021-03-12 | 中科院计算技术研究所大数据研究院 | Operator for integrating data conversion of relational database and non-relational database |
| CN112527876A (en) * | 2020-12-08 | 2021-03-19 | 国网四川省电力公司信息通信公司 | Unified database access system based on multi-source heterogeneous data analysis |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020032775A1 (en) * | 2000-08-28 | 2002-03-14 | Ramesh Venkataramaiah | System and method for transmitting and retrieving data via a distributed persistence framework |
| CN101754156A (en) * | 2008-12-05 | 2010-06-23 | 中国移动通信集团辽宁有限公司 | Office data management method, device and charging system |
| CN109614427A (en) * | 2018-10-23 | 2019-04-12 | 平安科技（深圳）有限公司 | The access method and device of Various database, storage medium and electronic equipment |
| CN110032604B (en) * | 2019-02-02 | 2021-12-07 | 北京奥星贝斯科技有限公司 | Data storage device, translation device and database access method |
| CN110889053B (en) * | 2019-11-11 | 2022-07-19 | 北京齐尔布莱特科技有限公司 | Interface data caching method and device and computing equipment |
| CN111061757B (en) * | 2019-12-19 | 2023-03-31 | 用友网络科技股份有限公司 | Language conversion method and device of database, electronic equipment and storage medium |
| CN112667662B (en) * | 2020-12-25 | 2022-11-08 | 银盛支付服务股份有限公司 | Common method for analyzing sql and nosql based on json key values |
-
2021
- 2021-05-28 CN CN202110589454.7A patent/CN113254519B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200175018A1 (en) * | 2018-11-29 | 2020-06-04 | Capital One Services, Llc | Real-time processing of event-based streaming with nosql databases |
| CN109656943A (en) * | 2018-12-12 | 2019-04-19 | 国网辽宁省电力有限公司信息通信分公司 | Shield the data service method of Query in Heterogeneous Databases complexity |
| CN110633292A (en) * | 2019-09-19 | 2019-12-31 | 上海依图网络科技有限公司 | Query method, device, medium, equipment and system for heterogeneous database |
| CN111581231A (en) * | 2020-04-20 | 2020-08-25 | 北京明略软件系统有限公司 | Query method and device based on heterogeneous database |
| CN112328623A (en) * | 2020-11-06 | 2021-02-05 | 昆山数字城市信息技术有限公司 | Multi-source heterogeneous data management method based on mixed ontology mode |
| CN112527876A (en) * | 2020-12-08 | 2021-03-19 | 国网四川省电力公司信息通信公司 | Unified database access system based on multi-source heterogeneous data analysis |
| CN112487075A (en) * | 2020-12-29 | 2021-03-12 | 中科院计算技术研究所大数据研究院 | Operator for integrating data conversion of relational database and non-relational database |
Non-Patent Citations (1)
| Title |
|---|
| 多源异构数据库数据查询处理研究;张靖，等;《研究与开发》;20091031;第27-29页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113254519A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| US8898147B2 (en) | Method and system for a transparent application of multiple queries across multiple data sources | |
| US6405209B2 (en) | Transparent object instantiation/initialization from a relational store | |
| JP5986195B2 (en) | Data enrichment recommendations | |
| CN113254519B (en) | Access method, device, equipment and storage medium of multi-source heterogeneous database | |
| CN111221791A (en) | Method for importing multi-source heterogeneous data into data lake | |
| US20180307692A1 (en) | Software application interface for mediating access to services of a centralized data store | |
| CN110704476A (en) | Data processing method, device, equipment and storage medium | |
| CA3017523C (en) | Technologies for auto discover and connect to a rest interface | |
| CN113051268A (en) | Data query method, data query device, electronic equipment and storage medium | |
| CN111026931A (en) | Data query method, device, equipment and medium | |
| CN112949269A (en) | Method, system, equipment and storage medium for generating visual data analysis report | |
| US20110078569A1 (en) | Value help user interface system and method | |
| US10915541B2 (en) | Generic API | |
| US10599728B1 (en) | Metadata agent for query management | |
| US11556316B2 (en) | Distributed extensible dynamic graph | |
| US10908924B2 (en) | System and methods for loading objects from hash chains | |
| CN114153438A (en) | API automatic generation system | |
| AU2019386603B2 (en) | Systems and methods for storing object state on hash chains | |
| CN112163029A (en) | Virtual database system and query method | |
| CN112800054A (en) | Data model determination method, device, equipment and storage medium | |
| US11874840B2 (en) | Table discovery service | |
| CN112100207B (en) | Sub-query processing method, device, equipment and storage medium | |
| US20230244661A1 (en) | Intelligent parity service with database query optimization | |
| CN113901071A (en) | Storage medium-oriented query method, device, medium, equipment and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |