WO2019223601A1 - Database system, and establishment method and apparatus therefor - Google Patents

Abstract

The present application relates to the field of communications, and relates to a database system, and an establishment method and apparatus therefor. The system comprises: a storage layer and a source data layer, the storage layer comprising a first storage system and a second storage system, and the source data layer comprising multiple databases, each database comprising at least one database table, each database table being used for storing data; a first storage system used for storing data stored in each database table comprised in each database; and a second storage system used for storing structural information corresponding to a target database table, and storing a correspondence between an identifier of the structural information and position information, the position information comprising a storage position, in the first storage system, of data comprised in the target database table, and the structural information comprising each data item in the target database table, the target database table being a database table in any one of multiple databases. The present application does not affect the original service access database corresponding to each database.

Description

Database system and establishment method and device thereof

This application claims priority from a Chinese patent application filed on May 23, 2018 with an application number of 201810502231.0 and an invention name of "a database system and a method and device for establishing the same", the entire contents of which are incorporated herein by reference. .

Technical field

The present application relates to the field of communications, and in particular, to a database system and a method and device for establishing the database system.

Background technique

In the process of informatization construction, many enterprise departments have established many isolated heterogeneous databases according to their own business and functional needs. Due to the lack of unified information standardization, the development time of each database is inconsistent, resulting in different ways to access each database, each database is independent of each other, information cannot be exchanged and integrated, and an information island is formed.

In order to solve this problem, a database system has appeared. The system includes a database layer and a fusion layer. The database layer includes multiple databases and database agents. When multiple databases need to be read at the same time, a read request can be sent to the fusion layer, and the fusion layer forwards the read request to the database agent. The database agent reads data from the multiple databases and sends the data read from each database to the fusion layer. The fusion layer integrates the data from each database before sending it out.

In the process of implementing the present application, the inventor found that the above method has at least the following defects:

Each database in the database has its own corresponding service. When the fusion layer accesses each database through a database agent, it may affect the services corresponding to the database to access the database.

Summary of the Invention

The embodiments of the present application provide a database system and a method and device for establishing the database system, so as to avoid affecting the original service corresponding to the database from accessing the database. The technical solution is as follows:

In one aspect, an embodiment of the present application provides a database system, including:

A storage layer and a source data layer, the storage layer including a first storage system and a second storage system, the source data layer including a plurality of databases, each of the plurality of databases including at least one database table, the Each database table included in each database is used to store data;

The first storage system is configured to store data stored in each database table included in each database;

The second storage system is configured to store structure information corresponding to a target database table and store a correspondence between an identifier of the structure information and location information, where the location information includes data included in the target database table in the first A storage location in a storage system, the structure information includes each data item in the target database table, and the target database table is a database table in any one of the plurality of databases.

Optionally, the first storage system is HDFS, and the second storage system is RDMS.

Optionally, the database type included in the source data layer is one or more of Oracle, MySQL, and HDFS.

On the other hand, the embodiment of the present application provides a method for establishing a database system, where the database system is one of the optional systems described above on the one hand or on the other hand;

The method includes:

Reading structure information of the target database table and data in the target database table;

Storing the structural information in the second storage system, and storing data in the target database table in the first storage system;

The identifier of the structural information and the location information of the data in the target database table in the first storage system are correspondingly stored in a correspondence between the identifier and the location information in the second storage system.

Optionally, after the data in the target database table is stored in the first storage system, the method further includes:

Periodically or irregularly read data in the target database table, and obtain position information corresponding to the target database table from the correspondence between the identifier and location information according to the identifier of the structure information of the target database table, according to The location information determines data included in the target database table stored in the first storage system, and replaces data included in the target database table stored in the first storage system with the read target Data in a database table.

Optionally, the method further includes:

According to the search information to be queried, query data corresponding to the search information in the storage layer.

Optionally, the querying data corresponding to the search information in the storage layer according to the search information to be queried includes:

Determining the target data item to which the search information to be queried belongs;

Obtaining structural information including the target data item from the second storage system;

Obtaining corresponding location information from the second storage system according to the identification of the structure information;

Determining a corresponding database table in the first storage system according to the location information;

Query data corresponding to the search information in the database table according to the search information.

On the other hand, the embodiment of the present application provides a device for establishing a database system, where the database system is an optional system on the one hand or any one of the aspects;

The device includes:

A reading module, configured to read the structure information of the target database table and the data in the target database table;

A storage module, configured to store the structure information in the second storage system, store data in the target database table in the first storage system, and store the identifier of the structure information and the target database table The position information of the data in the first storage system is correspondingly stored in the correspondence between the identifier and the position information in the second storage system.

Optionally, the device further includes:

The query module is configured to query data corresponding to the search information in the storage layer according to the search information to be queried.

Optionally, the query module includes: a determination unit, an acquisition unit, and a query unit;

The determining unit is configured to determine a target data item to which the search information to be queried belongs;

The obtaining unit is configured to obtain structure information including the target data item from the second storage system; and obtain corresponding position information from the second storage system according to an identifier of the structure information;

The determining unit is further configured to determine a corresponding database table according to the location information;

The query unit is configured to query data corresponding to the search information in the database table according to the search information.

The technical solutions provided in the embodiments of the present application may include the following beneficial effects:

By storing the data in each data table in each database table in the active data layer in the first storage system and storing the structural information corresponding to each database table in the second storage system, each database is queried When the data is stored, the first storage system can be queried according to the structural information in the second storage system, so that it is not necessary to query the databases in the source data layer, and it will not affect the original business access database corresponding to each database.

It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and should not limit the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description serve to explain the principles of the application.

FIG. 1-1 is a schematic structural diagram of a database system according to an embodiment of the present application; FIG.

FIG. 1-2 is a schematic structural diagram of another database system according to an embodiment of the present application;

FIG. 2-1 is a flowchart of a method for establishing a database system according to an embodiment of the present application; FIG.

2-2 is another flowchart of a method for establishing a database system according to an embodiment of the present application;

3 is a schematic structural diagram of an apparatus for establishing a database system according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.

Through the above drawings, the specific embodiments of the present application have been shown, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to explain the concepts of the present application to those skilled in the art by referring to specific embodiments.

Detailed ways

Exemplary embodiments will be described in detail here, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of devices and methods consistent with certain aspects of the application as detailed in the appended claims.

Referring to FIG. 1-1, an embodiment of the present application provides a database system, including:

Storage tier 1 and source data tier 2. Storage tier 1 includes a first storage system 11 and a second storage system 12. Source data tier 2 includes multiple databases. Each of the multiple databases includes at least one database table. Each database table included in each database is used to store data;

The first storage system 11 is configured to store data stored in each database table included in each database;

The second storage system 12 is configured to store structure information corresponding to the target database table and store a correspondence between an identifier of the structure information and location information, where the location information includes data included in the target database table in the first storage system 11. Storage location, the structure information includes each data item in the target database table, and the target database table is a database table in any one of the multiple databases.

The storage location may include a start storage location and an end storage location of the data of the target database table in the first storage system 11. Alternatively, the storage location may be a starting storage location of data of the target database table in the first storage system 11, but the location information may further include an amount of data in the target database.

For each database table included in each database in the source data layer 2, the database table includes multiple data items, each data item in the database table corresponds to a column, and the column corresponding to the data item is used to store the data item The data. For example, see the database table shown in Table 1 below. The database table is a household registration information table for storing household registration information of a user. The household registration information table includes four data items, which are "identity card number", "name", "hukou address" and "nation", and the corresponding column of the data item "identity card number" in the household registration information table is used to store each The user's ID number, the column corresponding to the data item "name" is used to store the name of each user, the column corresponding to the data item "account address" is used to store the account address of each user, and the column corresponding to the data item "nation" For storing the nationalities of each user.

Table 1 is the household registration information form

identity number	Name	Account Address	Nation
No1	Name1	Address1	Chinese
No2	Name2	Address2	White
...	...	...	...

For another example, refer to the database table shown in Table 2 below. The database table is an education information table for storing the user's education information. The education information table includes four data items, which are "ID number", "Name", "Education" and "School". The corresponding column of the data item "ID number" in the education information table is used to store each user. ID number, the column corresponding to the data item "name" is used to store the name of each user, the column corresponding to the data item "education" is used to store the user's education, and the column corresponding to the data item "school" is used to store each user The user's school.

Table 2 is the academic information table

identity number	Name	Education	school
No1	Name1	Undergraduate	Beijing University
No2	Name2	Postgraduate	Beijing University
...	...	...	...

Optionally, the first storage system 11 is configured to store data in the household registration information table shown in Table 1 and to store data in the academic information table shown in Table 2. The first storage system 11 may not save each data item stored in the first row of the household registration information table, and may not save each data item stored in the first row of the academic information table.

Optionally, see Table 3 below. The second storage system 12 is used to store the structural information corresponding to the household registration information table. The structural information includes data items "ID card number", "name", and "hukou" in the household registration information table. Address "and" nation ", as well as structure information corresponding to the education information table, the structure information includes data items" ID card number "," name "," education "and" school "in the education information table.

table 3

Structural information
"ID Number", "Name", "Hukou Address" and "Nation"
"ID", "Name", "Education" and "School"
...

And, referring to Table 4 below, the second storage system 12 is further configured to store the correspondence between the identifier and the location information, where the correspondence includes the first record corresponding to the household registration information table and the second record corresponding to the academic information table. The first record includes the identification ID1 of the structural information corresponding to the household registration information table and the position information Position1 of the data included in the household registration information table in the first storage system 11; and the second record includes the identification ID2 of the structural information corresponding to the academic information table and the academic qualifications Position information Position2 of the data included in the information table in the first storage system 11.

Table 4

Logo	location information
ID1	Position1
ID2	Position2

...

...

Optionally, referring to FIGS. 1-2, the embodiment of the present application further provides: an acquisition conversion layer 3, and the acquisition conversion layer 3 includes at least one establishment device;

The establishment device is configured to read the structure information of the target database table and the data in the target database table, store the structure information in the second storage system 12, and store the data in the target database table in the first storage system 11. , The identifier of the structural information and the position information of the data in the target database table in the first storage system 11 are correspondingly stored in the correspondence between the identifier and the position information in the second storage system 12.

The establishment device is further configured to read the data in the target database table from the source data layer 2 periodically or irregularly, and according to the identification of the structure information of the target database table, the correspondence between the identification in the second storage system 12 and the location information The position information corresponding to the target database table is obtained in the relationship, and the data included in the target database table stored in the first storage system 11 is determined according to the position information, and the data included in the target database table stored in the first storage system 11 is replaced by reading Take the data from the target database table.

Because the end position of the data in the replaced target database table in the first storage system 11 may change, or the amount of data in the target database table changes, the establishment device may also obtain the data in the target data table again. The location information in the first storage system 11 is replaced with the newly acquired location information in the correspondence between the identifier and the location information stored in the second storage system 11.

Optionally, the databases in the source data layer 2 may belong to different database types. For example, the source database layer 2 may have at least one database whose database type is Oracle, there may be at least one database whose database type is MySQL, and / or, there may be at least one database whose database type is Hadoop Distributed File System (Hadoop Distributed File System) File System, HDFS).

The collection and conversion layer 3 may include a building device corresponding to each database type in at least one database type; for the building device corresponding to each database type, the building device is configured to determine a database belonging to the database type from the source data layer 2, For any database table in the database (referred to as the target database table), obtain the structure information of the target database table, and store the structure information in the second storage system 12, obtain the data in the target database table, and save the target database. The data in the table is stored in the first storage system 11 and the position information of the data in the first storage system 11 is obtained, and the identifier of the structural information and the position information are correspondingly stored in the identifier and the position information of the second storage system 12 Corresponding relationship.

Because the structures of databases of different database types are different, different establishment devices are needed to analyze the structure of the database and read data. Each establishment in the acquisition and conversion layer 3 presets a scheme for reading the structure of the analysis database and a scheme for reading the data. The establishment apparatus may use the scheme to analyze the structure of the database belonging to its corresponding database type and from the database. Read data from the database.

Each establishment device in the acquisition conversion layer 3 may be set in advance by a technician. When a new database type database appears in the source data layer 2, a technician can design a building device corresponding to the database type, and add the building device to the acquisition conversion layer 3.

Optionally, the first storage system 11 may be HDFS, and the second storage system 12 may be a relational database management system (Relational Data (base) Management System (RDMS)).

Optionally, the database type included in the source data layer 2 may be one or more of Oracle, MySQL, and HDFS.

In the embodiment of the present application, data in each data table in each database table in the active data layer is stored in the first storage system, and storage corresponding to each database table is stored in the second storage system. Items, so that when querying data in each database, you can query the first storage system according to each storage item in the second storage system, so there is no need to query each database in the source data layer, so that each database in the source data layer The corresponding original business access to each database in the source data layer will not affect the original business access database corresponding to each database.

An embodiment of the present application provides a method for establishing a database system. The system includes a storage layer and a source data layer. The storage layer includes a first storage system and a second storage system. The source data layer includes multiple databases and multiple databases. Each database in the database includes at least one database table, and each database table included in each database is used to store data.

Referring to FIG. 2-1, the execution body of the method may be an establishment device for any one of the acquisition conversion layers shown in FIG. 1-2, including:

Step 201: Obtain structural information corresponding to the target database table, where the structural information includes each data item in the target database table, and the target database table is a database table in a database in the source data layer.

For any establishment device in the acquisition and transformation layer, the establishment device corresponds to at least one database type, determines a database belonging to its corresponding database type from the databases included in the source data layer, and selects any database table in the database as the target data Table, and then proceed from this step.

Optionally, each database table included in each database in the source data layer includes multiple data items. Each data item in the database table corresponds to a column, and the corresponding column of the data item is used to store data belonging to the data item. . For example, refer to the database table shown in Table 1 above. The database table is a household registration information table for storing household registration information of a user. The household registration information table includes four data items, which are "identity card number", "name", "hukou address" and "nation", and the corresponding column of the data item "identity card number" in the household registration information table is used to store each The user's ID number, the column corresponding to the data item "name" is used to store the name of each user, the column corresponding to the data item "account address" is used to store the account address of each user, and the column corresponding to the data item "nation" For storing the nationalities of each user.

For another example, refer to the database table shown in Table 2 above. The database table is a degree information table for storing the user's degree information. The household registration information table includes four data items, which are "ID number", "Name", "Education" and "School". The corresponding column of the data item "ID number" in the education information table is used to store each user. ID number, the column corresponding to the data item "name" is used to store the name of each user, the column corresponding to the data item "education" is used to store the user's education, and the column corresponding to the data item "school" is used to store each user The user's school.

For example, if the household registration information table shown in Table 1 is selected, in this step, the first structure information corresponding to the household registration information table can be obtained from the household registration information table shown in Table 1, and the first structure information includes the household registration information table. The data items are "identity card number", "name", "hukou address" and "nation".

For another example, assuming that the academic information table shown in Table 2 is selected, in this step, the second structure information corresponding to the academic information table is obtained. The second structure information includes data items in the academic information table, which are “identity card numbers”. , "Name", "Education" and "School".

Step 202: Store the structure information in a second storage system, and obtain an identifier of the structure information.

Optionally, after storing the structure information in the second storage system, an identifier for uniquely identifying the structure information may be allocated.

Optionally, the identifier of the structure information may be sequentially assigned, or the name of the target database table and the name of the target database where the target database table is located may be obtained. According to the structure information, the name of the target database table, and the At least one, generating the identification of the structure letter.

For example, a hash value may be obtained by hashing at least one of the structure information, the name of the target database table, and the name of the target database, or at least one of the structure information, the name of the target database table, and the name of the target database. One performs a Message-Digest Algorithm (MD5) operation to obtain an MD5 value, and uses the hash value or the MD5 value as an identifier of the structural information.

For example, as shown in Table 3 above, the "identity number", "name", "hukou address", and "nation" included in the first structure information are stored in the second storage system, and the identification ID1 of the first structure information is assigned. . The "identification number", "name", "education", and "school" included in the second structure information are stored in the second storage system, and the identification ID2 of the second structure information is assigned.

Step 203: Obtain data in the target database table, store the data in the target database table in the first storage system, and obtain position information of the data in the first storage system.

For example, the data in the household registration information table shown in Table 1 is acquired, the data in the household registration information table shown in Table 1 is stored in the first storage system, and the data in the household registration information table in the first storage system is obtained. A piece of position information. The first position information includes a storage position Position1.

For another example, the data in the academic information table shown in Table 2 is acquired, the data in the academic information table shown in Table 2 is stored in the first storage system 11, and the data in the secondary school information table in the first storage system 11 is obtained The second position information of the data, and the second position information includes a storage position Position2.

Step 204: The identifier of the structural information of the target database table and the location information are correspondingly stored in a correspondence between the identifier and the location information in the second storage system.

For example, referring to Table 4 above, the identifier ID1 of the first structure information and Position1 included in the first location information may be stored in the correspondence between the identifier and the location information in the second storage system, and the second structure information may be The position ID included in the identification ID2 and the second position information corresponds to the correspondence between the identification and the position information stored in the second storage system.

Optionally, the data in the target database table can be read from the source data layer periodically or irregularly, and the target can be obtained from the correspondence between the identifier in the second storage system and the location information according to the identifier of the structure information of the target database table The location information corresponding to the database table, and based on the location information, determine the data included in the target database table stored in the first storage system, and replace the data included in the target database table stored in the first storage system with the read target database table The data.

Because the end position of the data in the replaced target database table in the first storage system may change, or the amount of data in the target database table changes, the establishment device may also obtain the data in the target data table again. For the location information in the first storage system, the location information corresponding to the identifier of the structure information of the target database table is replaced with the newly acquired location information in the correspondence between the identifier and the location information stored in the second storage system.

Optionally, after the database system is established, when it is necessary to access multiple databases in the source data layer, a query request can be sent to the system. See Figure 2-2. The system can perform the query by the following operations.

Step 205: Receive a query request message, where the query request message carries search information.

The search information belongs to at least one data item. For example, assuming that the search information is No1, the data item to which the search information belongs is an ID number.

Step 206: Query data in the storage layer according to the retrieval information.

This step can be implemented by the following operations from 2061 to 2064:

2061: Determine a target data item to which the search information belongs.

For example, assuming that the search information is No1, the target data item to which the search information belongs is an ID number.

2062: Query structure information including the target data item from the second storage system.

Optionally, the column corresponding to the target data item may also be determined according to the position of the target data item in the structural information.

For example, the second storage system shown in Table 3 is used to query the structural information including the ID number, including the first structural information and the second structural information. The structure information included in the first structure information is “ID card number”, “Name”, “Account address” and “nation”, and the position of “ID card number” in the first structure information is the first position, and it is determined accordingly. The column corresponding to "ID Card Number" is the first column; the structure information included in the second structure information is "ID Card Number", "Name", "Education" and "School", and "ID Card Number" is in the first structure The position of the information is the first position, and accordingly, the column corresponding to the "ID card number" is determined as the first column.

2063: Obtain the location information of the database table corresponding to the structure information according to the identifier of the structure information, and determine the database table corresponding to the location information in the first storage system. The database table includes at least the retrieval information and the retrieval information. Corresponding data.

Optionally, since the data item of the first row in the database table does not exist, the corresponding column of the target data item in the database table may also be determined according to the position of the target data item in the structural information.

For example, according to the identifier ID1 of the first structure information, the corresponding position information Position1 is obtained from the correspondence between the identifier and the position information shown in Table 4, and the household registration information table corresponding to the position information Position1 is determined in the first storage system. The household registration information table includes the ID number No1 and data corresponding to No1, and the data includes the name "Name1", the household address "Address1", and the ethnic group "Han".

The target data item "ID card number" is ranked at the first position in the first structure information, and the column corresponding to the target data item "ID card number" in the household registration information table is determined as the first column according to the first position.

According to the identifier ID2 of the second structure information, the corresponding position information Position2 is obtained from the correspondence between the identifier and the position information shown in Table 4, and the academic information table corresponding to the position information Position2 is determined in the first storage system. The table includes ID number No2 and data corresponding to No2, which includes the name "Name1", education "undergraduate", and the school "Beijing ### 大学".

The target data item "ID card number" is ranked at the first position in the second structure information, and the column corresponding to the target data item "ID card number" in the academic information table is determined as the first column according to the first position.

2064: Query data corresponding to the search information in the database table according to the search information, and send the queried data.

Optionally, find the same data as the search information in the column corresponding to the target data item in the database table, and obtain other data located in the row where the data is located, and the other data is the query information corresponding to the search information. data.

For example, according to the ID number "No1", look up the ID number "No1" in the first column in the household registration information table, and obtain other data in the row where the ID number "No1" is located, including the name "Name1", household registration Address "Address1" and ethnic "Han"; according to the ID number "No1", look up the ID number "No1" in the first column of the academic information table, and get other data in the row where the ID number "No1" is located , Including name "Name1", education "undergraduate", and school "北京 ### 大学". Then, the data obtained from each database table can be fused and the fused data can be sent.

In the embodiment of the present application, each database in the source data layer has its own corresponding service. For each service, data belonging to the service can be queried from the database corresponding to the service; when multiple databases need to be queried, Since the data in each database table in each database table in the active data layer is stored in the first storage system, and the structural information corresponding to each database table is stored in the second storage system, each database is queried in this way. When the data is stored, the first storage system can be queried according to the structural information in the second storage system, so that it is not necessary to query the databases in the source data layer, and it will not affect the original business access database corresponding to each database.

The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Referring to FIG. 3, an embodiment of the present application provides a device 300 for establishing a database system, where the system includes a storage layer and a source data layer, the storage layer includes a first storage system and a second storage system, and the source data layer includes a plurality of A database, each of the plurality of databases includes at least one database table, and each database table included in each database is used to store data.

The device 300 includes:

The reading module 301 is used for structural information of a target database table and data in the target database table;

A storage module 302, configured to store the structure information in the second storage system, store data in the target database table in the first storage system, and store the identifier of the structure information and the target database The position information of the data in the table in the first storage system is correspondingly stored in the correspondence between the identifier and the position information in the second storage system.

Optionally, the apparatus 300 further includes:

The query module is configured to query data corresponding to the search information in the storage layer according to the search information to be queried.

Optionally, the query module includes: a determination unit, an acquisition unit, and a query unit;

The determining unit is configured to determine a target data item to which the search information to be queried belongs;

The obtaining unit is configured to obtain structure information including the target data item from the second storage system; and obtain corresponding position information from the second storage system according to an identifier of the structure information;

The determining unit is further configured to determine a corresponding database table according to the location information;

The query unit is configured to query data corresponding to the search information in the database table according to the search information.

In the embodiment of the present application, since data in each data table in each database table in the active data layer is stored in the first storage system, and a structure corresponding to each database table is stored in the second storage system Information, so when querying the data in each database, you can query the first storage system based on the structural information in the second storage system, so there is no need to query each database in the source data layer, and the original data corresponding to each database will not be Business access to the database has an impact. In addition, the storage system has a redundant function. When a database table in the source data layer fails, the first storage system can restore the database table because the database table is stored in the first storage system.

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

FIG. 4 shows a structural block diagram of a terminal 400 provided by an exemplary embodiment of the present invention. The terminal 400 may be used to run the establishment device in the acquisition conversion layer 3 or the terminal 400 may be the establishment device in the acquisition conversion layer 3, and the terminal 400 may be a laptop computer or a desktop computer. The terminal 400 may also be called other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and the like.

Generally, the terminal 400 includes a processor 401 and a memory 402.

The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 401 may use at least one hardware form among DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). achieve. The processor 401 may also include a main processor and a co-processor. The main processor is a processor for processing data in the wake-up state, also called a CPU (Central Processing Unit). The co-processor is Low-power processor for processing data in standby. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 401 may further include an AI (Artificial Intelligence) processor, and the AI processor is configured to process computing operations related to machine learning.

The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 401 to implement the multiple sources provided by the method embodiments in this application. Method for establishing structured database system.

In some embodiments, the terminal 400 may optionally include a peripheral device interface 403 and at least one peripheral device. The processor 401, the memory 402, and the peripheral device interface 403 may be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 403 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 404, a touch display screen 405, a camera 406, an audio circuit 407, a positioning component 408, and a power source 409.

The peripheral device interface 403 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 401 and the memory 402. In some embodiments, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 401, the memory 402, and the peripheral device interface 403 or Both can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The radio frequency circuit 404 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuit 404 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 404 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, mobile communication networks (2G, 3G, 4G, and 5G) of various generations, wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 404 may further include circuits related to Near Field Communication (NFC), which is not limited in this application.

The display screen 405 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to collect touch signals on or above the surface of the display screen 405. The touch signal can be input as a control signal to the processor 401 for processing. At this time, the display screen 405 may also be used to provide a virtual button and / or a virtual keyboard, which is also called a soft button and / or a soft keyboard. In some embodiments, the display screen 405 may be one, and the front panel of the terminal 400 is provided; in other embodiments, the display screen 405 may be at least two, which are respectively disposed on different surfaces of the terminal 400 or have a folded design; In still other embodiments, the display screen 405 may be a flexible display screen disposed on a curved surface or a folded surface of the terminal 400. Moreover, the display screen 405 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 405 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

The camera component 406 is used for capturing images or videos. Optionally, the camera component 406 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, each of which is a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the integration of the main camera and the depth-of-field camera. Integrate with a wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other fusion shooting functions. In some embodiments, the camera assembly 406 may further include a flash. The flash can be a monochrome temperature flash or a dual color temperature flash. A dual color temperature flash is a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 401 for processing, or input them to the radio frequency circuit 404 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively disposed at different parts of the terminal 400. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for ranging purposes. In some embodiments, the audio circuit 407 may further include a headphone jack.

The positioning component 408 is used to locate the current geographic position of the terminal 400 to implement navigation or LBS (Location Based Service). The positioning component 408 may be a positioning component based on a US-based GPS (Global Positioning System), a Beidou system in China, or a Galileo system in Russia.

The power supply 409 is used to power various components in the terminal 400. The power source 409 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power source 409 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the terminal 400 further includes one or more sensors 410. The one or more sensors 410 include, but are not limited to, an acceleration sensor 411, a gyroscope sensor 412, a pressure sensor 413, a fingerprint sensor 414, an optical sensor 415, and a proximity sensor 416.

The acceleration sensor 411 can detect the magnitude of acceleration on three coordinate axes of the coordinate system established by the terminal 400. For example, the acceleration sensor 411 may be used to detect components of the acceleration of gravity on three coordinate axes. The processor 401 may control the touch display screen 405 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 411. The acceleration sensor 411 may also be used for collecting motion data of a game or a user.

The gyro sensor 412 can detect the body direction and rotation angle of the terminal 400, and the gyro sensor 412 can cooperate with the acceleration sensor 411 to collect a 3D motion of the user on the terminal 400. Based on the data collected by the gyro sensor 412, the processor 401 can implement the following functions: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 413 may be disposed on a side frame of the terminal 400 and / or a lower layer of the touch display screen 405. When the pressure sensor 413 is disposed on the side frame of the terminal 400, a user's holding signal to the terminal 400 can be detected, and the processor 401 can perform left-right hand recognition or quick operation according to the holding signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed on the lower layer of the touch display screen 405, the processor 401 operates according to the user's pressure operation on the touch display screen 405 to control the operable controls on the UI interface. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 414 is used to collect a user's fingerprint, and the processor 401 recognizes the identity of the user based on the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the identity of the user based on the collected fingerprint. When the user's identity is identified as trusted, the processor 401 authorizes the user to perform related sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 414 may be provided on the front, back, or side of the terminal 400. When a physical button or a manufacturer's logo is set on the terminal 400, the fingerprint sensor 414 can be integrated with the physical button or the manufacturer's logo.

The optical sensor 415 is used to collect ambient light intensity. In one embodiment, the processor 401 may control the display brightness of the touch display screen 405 according to the ambient light intensity collected by the optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 405 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 405 is decreased. In another embodiment, the processor 401 may also dynamically adjust the shooting parameters of the camera component 406 according to the ambient light intensity collected by the optical sensor 415.

The proximity sensor 416, also called a distance sensor, is usually disposed on the front panel of the terminal 400. The proximity sensor 416 is used to collect the distance between the user and the front side of the terminal 400. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front of the terminal 400 becomes gradually smaller, the processor 401 controls the touch display screen 405 to switch from the bright screen state to the closed screen state; when the proximity sensor 416 detects When the distance between the user and the front side of the terminal 400 gradually becomes larger, the processor 401 controls the touch display screen 405 to switch from the on-screen state to the on-screen state.

Those skilled in the art can understand that the structure shown in FIG. 4 does not constitute a limitation on the terminal 400, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

Those skilled in the art will readily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application. These variations, uses, or adaptations follow the general principles of this application and include common general knowledge or conventional technical means in the technical field not disclosed in this application. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the application is limited only by the accompanying claims.

Claims (10)

1. A database system, comprising:

   A storage layer and a source data layer, the storage layer including a first storage system and a second storage system, the source data layer including a plurality of databases, each of the plurality of databases including at least one database table, the Each database table included in each database is used to store data;

   The first storage system is configured to store data stored in each database table included in each database;

   The second storage system is configured to store structure information corresponding to a target database table and store a correspondence between an identifier of the structure information and location information, where the location information includes data included in the target database table in the first A storage location in a storage system, the structure information includes each data item in the target database table, and the target database table is a database table in any one of the plurality of databases.
2. The system of claim 1, wherein the first storage system is a Hadoop distributed file system (HDFS), and the second storage system is a relational database management system (RDMS).
3. The system according to claim 1, wherein the source data layer comprises one or more of Oracle, MySQL, and HDFS.
4. A method for establishing a database system, wherein the database system is a system according to any one of claims 1 to 3, and the method includes:

   Reading structure information of the target database table and data in the target database table;

   Storing the structural information in the second storage system, and storing data in the target database table in the first storage system;

   The identifier of the structural information and the location information of the data in the target database table in the first storage system are correspondingly stored in a correspondence between the identifier and the location information in the second storage system.
5. The method according to claim 4, wherein after storing the data in the target database table in the first storage system, further comprising:

   Periodically or irregularly read data in the target database table, and obtain position information corresponding to the target database table from the correspondence between the identifier and location information according to the identifier of the structure information of the target database table, according to The location information determines data included in the target database table stored in the first storage system, and replaces data included in the target database table stored in the first storage system with the read target Data in a database table.
6. The method according to claim 4, further comprising:

   According to the search information to be queried, query data corresponding to the search information in the storage layer.
7. The method according to claim 6, wherein the querying data corresponding to the search information in the storage layer according to the search information to be queried comprises:

   Determining the target data item to which the search information to be queried belongs;

   Obtaining structural information including the target data item from the second storage system;

   Obtaining corresponding location information from the second storage system according to the identification of the structure information;

   Determining a corresponding database table in the first storage system according to the location information;

   Query data corresponding to the search information in the database table according to the search information.
8. A device for establishing a database system, wherein the database system is the system according to any one of claims 1 to 3, and the device includes:

   A reading module, configured to read the structure information of the target database table and the data in the target database table;

   A storage module, configured to store the structure information in the second storage system, store data in the target database table in the first storage system, and store the identifier of the structure information and the target database table The position information of the data in the first storage system is correspondingly stored in the correspondence between the identifier and the position information in the second storage system.
9. The apparatus according to claim 8, further comprising:

   The query module is configured to query data corresponding to the search information in the storage layer according to the search information to be queried.
10. The apparatus according to claim 9, wherein the query module comprises: a determination unit, an acquisition unit, and a query unit;

    The determining unit is configured to determine a target data item to which the search information to be queried belongs;

    The obtaining unit is configured to obtain structure information including the target data item from the second storage system; and obtain corresponding position information from the second storage system according to an identifier of the structure information;

    The determining unit is further configured to determine a corresponding database table according to the location information;

    The query unit is configured to query data corresponding to the search information in the database table according to the search information.

Images