CN114564501A

CN114564501A - Database data storage and query methods, devices, equipment and medium

Info

Publication number: CN114564501A
Application number: CN202210191740.2A
Authority: CN
Inventors: 刘紫千; 余启明; 常力元; 曹然; 佟欣哲; 李金伟; 陈林; 刘长波
Original assignee: Tianyi Safety Technology Co Ltd
Current assignee: Tianyi Safety Technology Co Ltd
Priority date: 2022-02-28
Filing date: 2022-02-28
Publication date: 2022-05-31

Abstract

In the embodiment of the application, each data table in a database is split into a plurality of vertical sub-tables according to each preset field, a database with huge data volume is split into a plurality of vertical sub-databases according to the category of each data table, so that the data stored in each vertical sub-database are different, the data volume of a single data table can be reduced, an index is created for each vertical sub-table, and the vertical sub-database in which the field to be queried is searched can be determined through the index during querying, so that the efficiency of data querying is improved.

Description

Database data storage and query methods, devices, equipment and medium

Technical Field

The present application relates to the field of database technologies, and in particular, to a method, an apparatus, a device, and a medium for storing and querying database data.

Background

With the rapid development of information technology, more and more business platforms for providing daily services for a large number of users, such as communication business service platforms, e-commerce commodity purchasing platforms, government affair service platforms, etc., emerge. Usually, a large amount of service data is stored in the database of these service platforms, the data volume of the service data recorded in a single data table may be as many as ten thousand rows, the occupied memory may reach 100G, and the increase of the service volume may also cause the data volume of the database to continuously expand. Meanwhile, due to the diversity and coupling of services, the association relationship between data is complex, and the above situation often reduces the data query performance of the database, which becomes a bottleneck of service development, for example, when data query is performed, the data needs to be searched in thousands of service data of one data table, the calculation amount of a CPU is increased, and the scanning of data rows greatly reduces the efficiency of data query.

The existing method for improving the data query efficiency of the database comprises the following steps:

the hardware capacity is improved to improve the database processing capacity, for example, the storage capacity of a server in which the database is located, the number of CPUs, and the like are increased. The method is high in cost and cannot improve the database data query efficiency fundamentally.

The database data query efficiency is improved by adopting the technical means of database caching, read-write separation and the like. In this way, database query efficiency still drops significantly, typically as the amount of data proliferates.

Disclosure of Invention

The application provides a database data storage and query method, device, equipment and medium, and aims to solve the problem of low database query efficiency in the prior art.

In a first aspect, the present application provides a database data storage method, including:

aiming at each data table in a database, acquiring the category to which the business data recorded in the data table belongs, searching first data corresponding to each preset field in the data table, storing the searched first data, the identifier of the business data to which the first data belongs and the preset field in a vertical sub-table of the data table, and deleting the first data in the data table;

judging whether a first target vertical sub-library corresponding to the category is created or not, if so, storing the vertical sub-table into the first target vertical sub-library, and recording each preset field in the vertical sub-table and index information of the vertical sub-table stored into the first target vertical sub-library; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded.

In a second aspect, the present application provides a data query method, including:

receiving a query instruction of data to be queried, wherein the query instruction comprises a field to be queried;

determining a target vertical sub-table in which the fields to be inquired are stored and a target vertical sub-base in which the target vertical sub-table is stored according to each preset field in the stored vertical sub-table and index information of the vertical sub-base in which the vertical sub-table is stored;

and searching data corresponding to the field to be inquired in the target vertical branch table of the target vertical branch library.

In a third aspect, the present application provides a database data storage apparatus, the apparatus comprising:

the vertical table dividing module is used for acquiring the attributive type of the service data recorded in the data table aiming at each data table in the database, searching the first data corresponding to each preset field in the data table, storing the searched first data, the identifier of the service data to which the first data belongs and the preset field in the vertical table dividing of the data table, and deleting the first data in the data table;

the vertical sub-library module is used for judging whether a first target vertical sub-library corresponding to the category is established or not, if so, storing the vertical sub-table into the first target vertical sub-library, and recording each preset field in the vertical sub-table and index information of the first target vertical sub-library stored by the vertical sub-table; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded.

In a fourth aspect, the present application provides a data query apparatus, including:

the device comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query instruction of data to be queried, and the query instruction comprises a field to be queried;

the query module is used for determining a target vertical sub-table in which the fields to be queried are stored and a target vertical sub-base in which the target vertical sub-table is stored according to each preset field in the stored vertical sub-table and the index information of the vertical sub-base in which the vertical sub-table is stored; and searching data corresponding to the field to be inquired in the target vertical branch table of the target vertical branch library.

In a fifth aspect, the present application further provides an electronic device, which at least includes a processor and a memory, and the processor is configured to implement the steps of the database data storage method or the data query method as described above when executing the computer program stored in the memory.

In a sixth aspect, the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of any of the database data storage methods described above or the data query methods described above.

The embodiment of the application provides a database data storage and query method, a device, equipment and a medium, wherein the method comprises the steps of aiming at each data table in a database, obtaining the attributive type of service data recorded in the data table, searching first data corresponding to each preset field in the data table, storing the searched first data, the identifier of the service data to which the first data belongs and the preset field in a vertical sub-table of the data table, deleting the first data in the data table, judging whether a first target vertical sub-table corresponding to the type is established or not, if so, storing the vertical sub-table in the first target vertical sub-table, and recording each preset field in the vertical sub-table and index information stored in the first target vertical sub-table by the vertical sub-table; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded. In the embodiment of the application, each data table in the database is split into a plurality of vertical sub-tables according to each preset field, and one database with huge data volume is split into a plurality of vertical sub-databases according to the category of each data table, so that the data stored in each vertical sub-database are different, the data volume of a single data table can be reduced, an index is created for each vertical sub-table, and the vertical sub-database in which the field to be queried is searched can be determined through the index during query, so that the efficiency of data query is improved.

Drawings

In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of a database data storage process provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a vertical sub-library provided in an embodiment of the present application;

FIG. 3 is a schematic diagram of a vertical sub-table provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a horizontal sub-library provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of a horizontal sub-library provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a data query process provided in an embodiment of the present application;

fig. 7 is a schematic structural diagram of a database data storage device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a data query device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived from the embodiments given herein by a person of ordinary skill in the art are intended to be within the scope of the present disclosure.

The embodiment of the application provides a database data storage and query method, a device, equipment and a medium, wherein the method comprises the steps of aiming at each data table in a database, obtaining the attributive type of service data recorded in the data table, searching first data corresponding to each preset field in the data table, storing the searched first data, the identifier of the service data to which the first data belongs and the preset field in a vertical sub-table of the data table, deleting the first data in the data table, judging whether a first target vertical sub-table corresponding to the type is established or not, if so, storing the vertical sub-table in the first target vertical sub-table, and recording each preset field in the vertical sub-table and index information stored in the first target vertical sub-table by the vertical sub-table; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded. In the embodiment of the application, each data table in the database is split into a plurality of vertical sub-tables according to each preset field, and one database with huge data volume is split into a plurality of vertical sub-databases according to the category of each data table, so that the data stored in each vertical sub-database is different, the data volume of a single data table can be reduced, an index is created for each vertical sub-table, and a field to be queried can be determined in which vertical sub-database to be searched through the index during query, so that the efficiency of data query is improved.

Example 1:

fig. 1 is a schematic diagram of a database data storage process provided in an embodiment of the present application, where the process specifically includes the following steps:

s101: the method comprises the steps of obtaining the attributive type of the business data recorded in a data table aiming at each data table in a database, searching first data corresponding to each preset field in the data table, storing the searched first data, the identification of the business data to which the first data belongs and the preset field in a vertical sub-table of the data table, and deleting the first data in the data table.

The database data storage process provided by the embodiment of the application is suitable for electronic equipment, and the electronic equipment can be equipment such as a server and a PC.

In the embodiment of the present application, the database may be a database of a platform of a communication service, an e-commerce commodity purchase, a government service, and the like, and the type of the database includes, but is not limited to, oracle, mysql, ftp, sftp, hive, and the like. In order to improve the efficiency of database data query, each data table in the database can be split, and the data volume stored in each data table is reduced, so that the scanned data volume is reduced.

The database may store a plurality of data tables, the category to which the business data described in each data table belongs is fixed, one data table describes one category of business data, and the categories to which the business data described in different data tables belongs may be the same but generally different, wherein the category to which the business data belongs may be determined based on the business type of the business data or may be determined according to different enterprises to which the business data belongs.

In the embodiment of the application, the category recorded in each data table in the database can be acquired, so that the data table can be conveniently and subsequently determined to be split according to the category. The category to which the service data stored in the data table belongs may be stored in advance for each data table, and the category to which the service data described in the data table belongs may be directly determined based on the stored information, or may be determined based on the content or format of each service data recorded in a certain field in the data table.

In addition, each data table records a plurality of pieces of service data, and each piece of service data is recorded in the data table in a row form, that is, each piece of service data occupies one row in the data table, and each piece of service data includes a plurality of fields. Because the number of fields in the data table is large, the occupied space of the data table is large, a large number of IO operations need to be executed when the data in the database is queried, and the efficiency of database data query is seriously reduced.

Therefore, in the embodiment of the application, for each data table in the database, the first data corresponding to each preset field may be searched in the data table, and the searched first data and the preset field may be stored in the vertical sub-table of the data table, that is, the data table with a large number of fields is split into a plurality of data tables. The first data found according to each preset field, the identifier of the service data to which the first data belongs, and the preset field may be stored in a vertical sublist of the data table, specifically, each preset field may be stored in a different vertical sublist of the data table, that is, one preset field, data corresponding to the preset field, and the identifier of the service data to which the data belongs may be stored in one vertical sublist.

The preset field may be one or more, and may be multiple in order to greatly reduce the data size of the data table. In the embodiment of the application, each field in the data table may be arranged according to a certain sequence, for example, the fields are arranged according to the number of times of being queried, a preset number of field names are selected as preset fields according to the sequence after the arrangement, and the preset fields may also be determined according to the data size of each data recorded corresponding to each field. And the preset fields can be grouped in advance, each group comprises at least one preset field, preferably a plurality of preset fields, the preset fields contained in one group and the searched corresponding first data are stored in the same vertical sub-table, and different groups correspond to different vertical sub-tables.

In order to avoid excessive redundant data, the first data and the preset field can be deleted in the data table after the searched first data and the preset field are stored in the vertical sub-table of the data table.

S102: judging whether a first target vertical sub-library corresponding to the category is created or not, if so, storing the vertical sub-table into the first target vertical sub-library, and recording each preset field in the vertical sub-table and index information of the vertical sub-table stored into the first target vertical sub-library; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded.

After the first data of each data table in the database, the identifier of the service data to which the first data belongs, and the preset field are stored in the vertical sub-table of each data table according to the preset field, although the efficiency of data query is improved to a certain extent, the efficiency of data query is also affected by an excessive amount of data or data in one database. In this embodiment, a database may include a plurality of vertical sub-libraries, and each vertical sub-library corresponds to a data table in which a category of service data is stored.

In this embodiment of the present application, whether the first target vertical sub-base corresponding to the category has been created may be determined according to the category to which the acquired service data belongs, and if the first target vertical sub-base has been created, the vertical sub-table of the data table storing the service data of the category may be stored in the first target vertical sub-base. Since the index can increase the speed of searching and retrieving data, the efficiency of processing query requests can be effectively increased, and in order to increase the efficiency of data query, in the embodiment of the present application, each preset field in the vertical sublist and the index information stored in the first target vertical sublist by the vertical sublist can be recorded, wherein, each preset field in the vertical sublist is each field contained in the vertical sublist, the vertical sublist is stored in the index information of the first target vertical sublist and can record the database name of the first target vertical sublist, in order to further improve the efficiency of data query, the index information of the vertical sublist stored in the first target vertical sublibrary can be more detailed, the index information may further include a data table name of the vertical sublist and an identifier corresponding to each piece of service data recorded in the vertical sublist.

And if the target vertical sub-base corresponding to the category is not created, creating a second target vertical sub-base corresponding to the category, storing the category corresponding to the second target vertical sub-base, storing a vertical sub-table of a data table storing the service data of the category into the second target vertical sub-base, and recording each preset field in the vertical sub-table and index information stored in the second target vertical sub-base by the vertical sub-table.

Specifically, for example, in a database of the communication service platform, different users sign different communication services, and the different communication services correspond to different service types, then the category to which the service data recorded in the database of the communication service platform belongs may be a service type corresponding to the different communication services, where the service type may include: prepaid service, postpaid service, global services, etc. In the embodiment of the application, the vertical sublist corresponding to the service type of the prepaid service can be stored in the vertical sublist corresponding to the prepaid service, and the corresponding index information is recorded, the vertical sublist corresponding to the service type of the postpaid service is stored in the vertical sublist corresponding to the postpaid service, and the corresponding index information is recorded, and the vertical sublist corresponding to the service type of the global service is stored in the vertical sublist corresponding to the global service, and the corresponding index information is recorded.

For example, in a database of an enterprise communication management platform, an enterprise identifier corresponding to business data recorded in each vertical sub-table can be acquired for each vertical sub-table, the same enterprise with the same enterprise identifier is used as the same category, the vertical sub-tables corresponding to the same enterprise are stored in the same vertical sub-table, and the data of different enterprises are stored in different vertical sub-tables, and the data of each enterprise are not interfered with each other.

Because the traditional data storage method is to store all the service data in one server in a centralized way, and the requirement of storing the service data in a large scale cannot be met, in the embodiment of the application, a plurality of servers can be used for sharing the storage load, and each vertical sub-library is deployed in different servers, so that the reliability, the availability and the access efficiency of the system are improved, and the expansion is easy.

Fig. 2 is a schematic diagram of a vertical sub-library provided in an embodiment of the present application, as shown in the figure, a vertical sub-table 1, a vertical sub-table 2, and a vertical sub-table 3 exist in a database, and two vertical sub-libraries exist, which are the vertical sub-library 1 and the vertical sub-library 2, respectively. The service type of the service data recorded in the vertical sublist 1 and the vertical sublist 2 is service type a, and the service type of the service data recorded in the vertical sublist 3 is service type B, and the vertical sublist 1 and the vertical sublist 2 can be stored in the vertical sublist 1 which is responsible for storing the service data of the service type a, and the vertical sublist 3 can be stored in the vertical sublist 2 which is responsible for storing the service data of the service type B.

In the embodiment of the application, each data table in the database is split into a plurality of vertical sub-tables according to each preset field, and one database with huge data volume is split into a plurality of vertical sub-databases according to the category of each data table, so that the data stored in each vertical sub-database is different, the data volume of a single data table can be reduced, an index is created for each vertical sub-table, and a field to be queried can be determined in which vertical sub-database to be searched through the index during query, so that the efficiency of data query is improved.

Example 2:

in order to further improve the efficiency of data query, on the basis of the foregoing embodiment, in this embodiment of the application, the determining of the preset field includes:

acquiring an operation log of the database within a first preset time period, and identifying a condition field in each piece of database operation information recorded in the operation log;

counting the number of times of using the condition field aiming at each condition field, and if the number of times of using the condition field is greater than a preset threshold value, determining the condition field as a first preset field; otherwise, determining the condition field as a second preset field;

the step of storing the searched first data and the preset field into the vertical sub-table of the data table comprises:

and storing the searched first preset field and the corresponding first data into a first vertical sub-table of the data table, and storing the searched second preset field and the corresponding first data into a second vertical sub-table of the data table.

In order to further improve the efficiency of data query, in the embodiment of the present application, the preset field may be manually pre-stored in the electronic device, and may also be automatically identified by the electronic device with respect to different data tables.

In the embodiment of the application, the field names corresponding to the fields can be determined as the preset fields according to each piece of database operation information recorded in the operation log of the database, and the preset fields may be one or multiple. In the embodiment of the present application, it is not necessary to obtain all database operation information recorded in all operation logs of the database, and the operation logs of the database within a first preset time period may be obtained, and a condition field in each piece of database operation information recorded in the obtained operation logs is identified. The first preset time period may be a time period corresponding to a preset time length after the current time, or a time period corresponding to a preset time length before the current time.

For example, a certain piece of database operation information recorded in the operation log is: and inquiring the average medical expense of the user with the age of more than 50 years from the medical expense list of the user, wherein the inquiry condition is that the age is more than 50 years, and the field name corresponding to the medical expense list of the user with the age of 50 is 'age', and then the field 'age' is the condition field in the piece of database operation information. Since each database operation information of the database operation command record is used in the operation log, the condition field in each instruction can be identified according to the database operation command specification.

After acquiring the condition field in each piece of database operation information of the operation log record in the first preset time period, in the embodiment of the present application, the number of times that the condition field is used may be counted for each condition field, for example, if the condition field of "age" in the acquired condition field is present in 10 pieces of database operation information in total, then the number of times that the condition field "age" is used may be determined to be 10. If the number of times of use is greater than a preset threshold, it indicates that the condition field is frequently used, and the condition field may be determined as a first preset field, which may be referred to as an active field; if the number of times of use is not greater than the predetermined threshold, which indicates that the condition field is used less frequently, the condition field may be determined as a second predetermined field, which may be referred to as an inactive field.

Specifically, when a user browses a commodity list through an e-commerce platform, the user usually views detailed information of the commodity only when the user is interested in the commodity, so that fields of basic information of the commodity, such as a commodity name, a commodity unit price, a commodity picture and the like, belong to active fields with high query frequency when the user browses the commodity list, the basic information can be determined as first preset fields, fields of detailed information of the commodity, such as a commodity description, a commodity specification and the like, belong to inactive fields with low query frequency, and the detailed information can be determined as second preset fields.

After the first preset field and the second preset field are determined, the first data corresponding to each preset field may be searched in a data table, where multiple pieces of service data are recorded in the data table, and each piece of service data corresponds to multiple fields, and if 10 pieces of service data are recorded in a certain data table, each preset field corresponds to 10 pieces of data, where the preset fields may be the first preset field and the second preset field. Storing the searched first preset field and corresponding first data into a first vertical sub-table of a data table, wherein the first data is data corresponding to each service data recorded in the data table by the first preset field, the first preset field corresponds to an active field, and each field stored in the first vertical sub-table is an active field; and storing the searched second preset field and the corresponding first data in a second vertical sublist of the data table, wherein the second preset field corresponds to an inactive field, and each field stored in the second vertical sublist is an inactive field.

Specifically, the first data corresponding to the basic information of the commodity can be searched in the data table, and since the basic information of the commodity is the active field, the searched basic information and the corresponding first data can be stored in the first vertical sub-table which is in charge of storing the active field. And searching first data corresponding to the detailed information of the commodity in the data table, wherein the searched detailed information and the corresponding first data can be stored in a second vertical sub-table which is used for storing the inactive fields because the detailed information of the commodity is the inactive fields.

Fig. 3 is a schematic diagram of a vertical sublist provided in this embodiment, as shown in the figure, a data table 1 exists in a database, where the data table 1 includes fields such as a field a, a field B, a field C, and a field D, and it is assumed that the field a and the field C are active fields, the field B and the field D are inactive fields, the vertical sublist 1 is responsible for storing active fields, and the vertical sublist 2 is responsible for storing inactive fields, and then the field a, the field C, and corresponding first data may be stored in the vertical sublist 1 of the data table, and the field B, the field D, and corresponding first data are stored in the vertical sublist 2 of the data table.

In the embodiment of the present application, before storing the found first preset field or second preset field and the corresponding first data in the vertical sub-tables of the data table, the field corresponding to the identifier corresponding to each piece of service data may be obtained, and the field and the identifier corresponding to each piece of service data are stored in each vertical sub-table of the data table, so as to ensure that the field names of at least one field existing between the vertical sub-tables of each data table are the same, and each vertical sub-table may be associated with each other through the same field.

Specifically, in the process of splitting a certain data table to obtain a vertical sub-table, a field corresponding to the identifier of the service data of the data table may be obtained, and the field is stored in each vertical sub-table of the data table.

In the embodiment of the application, whether the inquired active field or the inquired inactive field is determined according to the inquired condition field, and different fields and corresponding data are stored in different vertical tables, so that the inquiry in which data table is to be inquired can be determined during inquiry, the scanned data volume is reduced, the operation of inquiring the inactive field and the operation of inquiring the active field are not influenced mutually, and the occurrence probability of table locking caused by the fact that database reading and writing operations compete for IO resources is reduced.

Example 3:

in order to further improve the efficiency of data query, on the basis of the foregoing embodiments, in an embodiment of the present application, the method includes:

acquiring each piece of second data corresponding to a third preset field in each target vertical branch table aiming at each vertical branch table in each target vertical branch base; acquiring a numerical value corresponding to the second data aiming at each acquired second data, determining a target horizontal sub-library corresponding to the second data according to the numerical value corresponding to the second data and a target numerical value corresponding to each established horizontal sub-library, judging whether a first data table for recording service data of a vertical sub-table where the second data is located exists in the target horizontal sub-library, if so, storing the service data of the second data into the first data table, deleting the second data from the vertical sub-table where the second data is located, and recording an index relation between an identifier of the service data of the second data in a third preset field and the first data table in the target horizontal sub-library in index information; if not, creating a second data table corresponding to the vertical sublist where the second data is located, storing the service data where the second data is located into the second data table, deleting the second data from the vertical sublist where the second data is located, and recording the index relationship between the identifier of the service data where the second data is located in a third preset field and the second data table in the target horizontal sublist in the index information; and deleting the index information stored in the target vertical sublibrary by the vertical sublibrary.

Although each vertical sub-table is stored in different target vertical sub-banks, the efficiency of data query is improved to a certain extent, as time goes on, the service data continuously increases, the data volume of a single vertical sub-bank is also larger and larger, in order to further improve the efficiency of data query, sub-banks can be performed for each vertical sub-bank, that is, sub-banks are performed again for data recorded in each vertical sub-table in each target vertical sub-bank, data in each target vertical sub-bank is split into a plurality of horizontal sub-banks, and the operation of performing sub-banks again can be called performing horizontal sub-banks for convenience of description.

In this embodiment of the application, second data corresponding to a third preset field in each vertical sublist may be obtained for each vertical sublist in each vertical sublist, where the third preset field may be any field in the vertical sublist, and preferably, a field corresponding to an identifier of service data to which each piece of data belongs in each vertical sublist may be determined as the third preset field.

After each second data is acquired, a numerical value corresponding to the second data is acquired for each second data, if the second data is a number, subsequent operation is directly performed according to the numerical value of the number, if the second data is not a numerical value, a corresponding relationship between the data and the numerical value can be preset, and the numerical value corresponding to the second data is searched according to the corresponding relationship and subsequent operation is performed. Alternatively, the identifier of the service data to which the second data belongs may be obtained, and since the identifier of the service data has uniqueness and non-null property, and in order to make the database storage data have clear orderliness, the identifiers of the service data recorded in the general database are all set according to a specific rule, for example, the service identifier of each service data is set according to an incremental method. Even if the identification of the service data is not a number but a character string, the corresponding numerical value of the second data can be determined according to the corresponding relation between each character and the number of the character string.

For example, the fields stored in a vertical sublist are the product ID, product name, product amount, and product quantity, respectively. Assuming that the third preset field is a commodity ID, each commodity ID recorded in the vertical sublist can be obtained according to the third preset field, where each obtained commodity ID is the second data, and for each commodity ID, because the commodity ID is generally a number, the commodity ID can be determined as a numerical value corresponding to the second data, and then the target horizontal sublist corresponding to the second data is determined according to the commodity ID.

In this embodiment of the present application, in order to perform horizontal sub-base according to data stored in each vertical sub-table, after obtaining each second data and a value corresponding to the second data, a target horizontal sub-base corresponding to the second data is determined according to the value corresponding to the second data and a target value corresponding to each created horizontal sub-base.

In this embodiment, the target level sub-bank corresponding to the second data may be determined according to the parity of a preset position in the numerical value corresponding to the second data or the parity of the entire numerical value, for example, according to the parity of the last digit in the numerical value corresponding to the second data, the target level sub-bank corresponding to the second data is determined.

In this embodiment, a value range where the value is located may also be determined according to the value corresponding to the obtained second data, and the level sub-base corresponding to the value range is determined as the target level sub-base corresponding to the second data. Determining a value range of the value, determining a target value range where the value is located directly according to the value and a preset value range corresponding to each horizontal sub-library, and determining a target horizontal sub-library corresponding to the second data according to the determined target value range, wherein the value ranges corresponding to each horizontal sub-library are not overlapped; the value may also be calculated by using a certain algorithm, a final value is determined, and a value range where the final value is located is determined, for example, a hash value of a value corresponding to the second data may be determined according to algorithms such as a numerical analysis method, a square-based method, a folding method, and the like, and a target level sub-library corresponding to the second data may be determined according to the determined hash value.

In this embodiment of the present application, after determining the target horizontal sub-base corresponding to the second data, it may be determined whether a first data table recording data of a vertical sub-table in which the second data is located exists in the target horizontal sub-base, that is, it is required to determine whether a certain data table exists in the target horizontal sub-base, a field name included in the data table is consistent with a field name of the vertical sub-table in which the second data is located, and the data table in the target horizontal sub-base is a data table only used for storing data of the vertical sub-table in which the second data is located.

If the first data table exists, the service data where the second data belongs can be searched in the corresponding data table according to the identifier of the service data where the second data belongs, the searched service data is stored in the first data table in the target horizontal sub-database, the second data and the identifier of the service data where the second data belongs are deleted in the vertical sub-table where the second data belongs, the identifier of the service data where the second data belongs in a third preset field is recorded in the index information, and the database name of the second data in the stored target horizontal sub-database and the data table name of the data table where the second data is stored; if the first data table does not exist, a second data table corresponding to the vertical sub-table where the second data is located needs to be created in the target horizontal sub-base, according to the identifier of the service data to which the second data belongs, the service data where the second data is located is searched in the corresponding data table, the searched service data is stored in the created second data table, the second data and the identifier of the service data to which the second data belongs are deleted in the vertical sub-table where the second data is located, the identifier of the service data where the second data is located is recorded in the index information, and the database name of the second data in the stored horizontal sub-base and the data table name of the data table where the second data is stored are obtained.

For example, if the third preset field is a commodity ID, each piece of second data corresponding to the third preset field in the vertical sublist is the commodity ID of the commodity corresponding to each piece of business data stored in the vertical sublist, a target horizontal sublibrary corresponding to each piece of business data recorded in the vertical sublist can be determined according to the parity of the commodity ID in the vertical sublibrary, the business data with the odd-numbered commodity ID is stored in a target horizontal sublibrary responsible for storing the odd-numbered commodity ID, and the business data with the even-numbered commodity ID is stored in a target horizontal sublibrary responsible for storing the even-numbered commodity ID. For another example, the product ID is not a number, and in this embodiment of the application, a numerical value corresponding to the second data may be obtained according to each obtained product ID, a value range corresponding to the data is determined, and the target level sub-library corresponding to the second data is determined according to the determined value range.

Specifically, when the corresponding system of the second data and the numerical value is set, if the second data is the ID of the commodity, the amount of the commodity may be set to the numerical value corresponding to the second data, and the target horizontal sub-library corresponding to the service data of each commodity in the vertical sub-table is determined according to the numerical value and the preset value range. For example, the business data of the commodity with the sum of less than 10 yuan is stored in the target level sub-library which is responsible for storing the commodity with the sum of less than 10 yuan, and the business data of the commodity with the sum of not less than 10 yuan and not more than 50 yuan is stored in the target level sub-library which is responsible for storing the commodity with the sum of not less than 10 yuan and not more than 50 yuan. If the database has a data table for recording the identity information of the person, the corresponding service data in the vertical sub-table can be stored in different horizontal sub-tables according to different value ranges in the age field or the gender corresponding to the gender field.

In the embodiment of the application, the field names contained in the data tables which are split into the same vertical sub-table and correspond to different horizontal sub-databases are the same, but the service data stored between the data tables in the vertical sub-table and the horizontal sub-databases are different, the data table in each horizontal sub-database stores the service data where part of the second data in the third preset field is located, and all the second data in the third preset field stored in the vertical sub-table can further reduce the data volume of a single database through the horizontal sub-databases.

Fig. 4 is a schematic diagram of the horizontal sub-library provided in the embodiment of the present application, as shown in the figure, for the vertical sub-library 1, the vertical sub-library 1 includes a vertical sub-table 1 and a vertical sub-table 2, each second data corresponding to a second preset field in each vertical sub-table is obtained, obtaining a numerical value corresponding to the second data according to each obtained second data, and obtaining a value range corresponding to each established horizontal sub-library according to the numerical value corresponding to the second data and the established value range corresponding to each horizontal sub-library, part of the service data recorded in the vertical sub-table 1 may be stored in the data table a in the horizontal sub-base 1, another part of the service data recorded in the vertical sub-table 1 is stored in the data table a' in the horizontal sub-base 2, part of the service data recorded in the vertical sublist 2 is stored into the data table B in the horizontal sublist 1, and storing another part of the service data recorded in the vertical sub-table 2 into a data table B' in the horizontal sub-base 2.

As the traffic volume increases with the passage of time, the data volume recorded in each data table will be larger and larger, and in order to further improve the efficiency of data query, in this embodiment of the present application, the method includes:

acquiring each third data corresponding to a fourth preset field in each data table aiming at each data table in each horizontal sub-library;

acquiring a numerical value corresponding to the third data aiming at each acquired third data, determining a target level sub-table corresponding to the third data according to the numerical value corresponding to the third data and a target numerical value corresponding to each established level sub-table, storing service data where the third data is located in the target level sub-table, and deleting the third data in a data table where the third data is located; recording the index relation between the identifier of the service data of the third data of the fourth preset field and the target level sublist in the index information; and deleting the index information stored in the horizontal sub-library by the data table.

In order to further improve the efficiency of data query, the table splitting may be performed again according to the service data recorded in each data table in the horizontal sub-library, the same data table is split into multiple data tables, the data structures of the data tables are the same and the data contents are different between each data table obtained by splitting, and the operation of performing table splitting again may be referred to as performing the operation of performing horizontal table splitting on each data table for convenience of description.

In this embodiment of the present application, each piece of third data corresponding to a fourth preset field in each horizontal sub-library may be acquired for each data table in each horizontal sub-library, where the fourth preset field and the third preset field may be the same or different.

After each third data is obtained, a value corresponding to the third data is obtained for each third data, and specifically, a method for obtaining a value corresponding to the third data is similar to a method for obtaining a value corresponding to the second data.

In this embodiment of the present application, in order to perform horizontal division according to data stored in each data table in the horizontal division library, after obtaining each third data and a value corresponding to the third data, a target horizontal division table corresponding to the third data is determined according to the value corresponding to the third data and a target value corresponding to each created horizontal division table.

In this embodiment, the target level sub-table corresponding to the third data may be determined according to the parity of a preset position in the numerical value corresponding to the third data or the parity of the entire numerical value, for example, the target level sub-table corresponding to the second data may be determined according to the parity of the last digit in the numerical value corresponding to the third data.

In this embodiment of the application, a value range where the value is located may also be determined according to the value corresponding to the acquired third data, and the level sub-table corresponding to the value range is determined as the target level sub-table corresponding to the third data. Determining a value range of the value, determining a target value range where the value is located directly according to the value and a preset value range corresponding to each horizontal sub-table, and determining a target horizontal sub-table corresponding to the third data according to the determined target value range, wherein the value ranges corresponding to the horizontal sub-tables are not overlapped; the value may also be calculated by using a certain algorithm, a final value is determined, and a value range where the final value is located is determined, for example, a hash value of a value corresponding to the third data may be determined according to algorithms such as a numerical analysis method, a square-based mid-set method, a folding method, and the like, and a target level sublist corresponding to the third data may be determined according to the determined hash value.

After the target level sub-table corresponding to the third data is determined, the service data where the third data is located may be searched in the corresponding data table according to the identifier of the service data to which the third data belongs, and the searched service data may be stored in the target level sub-table.

And recording the index relation between the identifier of the service data where the third data is located and the target level sublist in the index information, namely recording which target level sublist in the level sublist the service data where the third data is located is stored in. Since the third data is already deleted in the data table in which the third data is located, the index information in which the data table is stored in the horizontal sub-base should also be deleted in the index information.

Specifically, it is worth explaining that if the horizontal sub-base in which the parity of the numerical value corresponding to each second data is completed is obtained after each second data is obtained according to the third preset field, the service data recorded in a certain horizontal sub-base is either all the service data corresponding to an even number or all the service data corresponding to an odd number, therefore, when performing horizontal sub-table division, if the fourth preset field is the same as the third preset field, the horizontal sub-table operation after the parity of the numerical value corresponding to each third data is completed cannot be performed, and the target horizontal sub-table corresponding to the service data in which each third data is located can be determined according to the value range of the numerical value corresponding to each third data.

Fig. 5 is a schematic diagram of the horizontal sublibrary provided in the embodiment of the present application, as shown in the drawing, for the reason of the horizontal sublibrary 1, a data table with a data table name of data table a is stored in the horizontal sublibrary 1, where the data table a includes service data a, service data B, service data C, service data D, and other service data, each third data corresponding to a fourth preset field in the data table may be obtained, and for each obtained third data, according to a value corresponding to each third data and a value range corresponding to each created horizontal sublibrary, service data B, service data C, and other service data are stored in the horizontal sublibrary 1, and service data a, service data D, and other service data are stored in the horizontal sublibrary 2.

Each data table in each horizontal sub-library is split into different data tables to obtain a plurality of horizontal sub-tables, so that the data volume of the same data table can be further reduced, and the rapid increase of the data volume can be effectively coped with.

Example 4:

in order to further improve the efficiency of data query, on the basis of the foregoing embodiments, in the embodiment of the present application, the storing the service data where the second data is located in the first data table includes:

judging whether the first data table is fully written or not;

if so, and when the target horizontal sub-database is fully written, according to the stored quantity threshold value of the horizontal sub-database, if the quantity of the created horizontal sub-database does not reach the quantity threshold value of the horizontal sub-database, creating a horizontal sub-database, activating the horizontal sub-database, taking the target value corresponding to the target horizontal sub-database as the target value corresponding to the created horizontal sub-database, creating a first data table for recording the service data of the vertical sub-table where the second data is located in the created horizontal sub-database, and storing the service data where the second data is located in the first data table;

if not, directly storing the service data of the second data into the first data table.

In order to avoid that the data amount in a single horizontal sub-library is too much and the efficiency of data query is affected, in the embodiment of the application, the maximum capacity of each horizontal sub-library can be configured in advance, and when one horizontal sub-library is full, another horizontal sub-library is created. However, the horizontal sub-libraries cannot be created without limitation, in the embodiment of the present application, a number threshold of the horizontal sub-libraries is stored, if the number of the created horizontal sub-libraries does not reach the number threshold, a new horizontal sub-library may be created, and if the number of the created horizontal sub-libraries reaches the number threshold, a prompt message may be displayed to remind the user that the number of the horizontal sub-libraries has reached the number threshold.

Specifically, in this embodiment of the present application, if a data table in a database is not fully written, the database is certainly not fully written, and therefore, in this embodiment of the present application, when storing service data in which second data is located in a first data table, it may first be determined whether the first data table is fully written, and when determining whether the first data table is fully written, a line number of the data written in the first data table may be obtained, the line number is compared with a preset maximum line number that can be written in the first data, it is determined whether the first data table is fully written, and a data amount of the data written in the first data table may also be obtained, and compared with a preset maximum data amount that can be stored in the first data table, it is determined whether the first data table is fully written.

If the first data table is written to full and the target horizontal sub-base is written to full at this time, the number of created horizontal sub-bases may be obtained, and if the number of created horizontal sub-bases does not reach the threshold value of the number of created horizontal sub-bases, it is indicated that the horizontal sub-bases may be created again.

If the first data table is not fully written, the target level sub-library in which the first data table is located is not fully written, and the service data in which the second data is located can be directly stored in the first data table which is not fully written.

Specifically, it may be predefined that the maximum data amount of the data allowed to be written in the first data table is 5G, the maximum data amount of the data allowed to be written in the target horizontal sub-bank is 100G, and the threshold value of the number of horizontal sub-banks is 10. When the service data of the second data is stored in the first data table, if the data volume of the first data table is 5G and the data volume of the target horizontal sub-base of the first data table is 100G, the number of the created horizontal sub-bases can be obtained, if the number of the created horizontal sub-bases is 3, that is, the number of the created horizontal sub-bases does not reach the threshold value of the number of the horizontal sub-bases, a new horizontal sub-base can be created, the horizontal sub-base is activated, the target value corresponding to the target horizontal sub-base of the first data table is used as the target value corresponding to the newly created horizontal sub-base, a first data table for recording the service data of the vertical sub-base of the second data is created in the horizontal sub-base, and the service data of the second data is stored in the first data table.

In order to further improve the efficiency of data query, on the basis of the foregoing embodiments, in the embodiment of the present application, the storing the service data where the third data is located in the target level sublist includes:

judging whether the target level sublist is fully written or not;

if so, and the number of the created horizontal sub-tables does not reach the number threshold of the horizontal sub-tables, creating the horizontal sub-tables, activating the created horizontal sub-tables, taking the target values corresponding to the target horizontal sub-tables as the target values corresponding to the created horizontal sub-tables, and storing the service data where the third data is located in the created horizontal sub-tables;

if not, directly storing the service data of the third data into the target vertical sub-table.

In order to avoid that the data amount in a single target horizontal sublist is too much and affects the efficiency of data query, in the embodiment of the application, the maximum capacity of each horizontal sublist can be configured in advance, and when one horizontal sublist is full, another horizontal sublist is created. However, the level sublist cannot be created without limitation, in the embodiment of the present application, a number threshold of the level sublist is stored, if the number of the already created level sublists does not reach the number threshold, a new level sublist may be created, and if the number of the already created level sublists reaches the number threshold, a prompt message may be displayed to remind the user that the number of the level sublists has reached the number threshold.

Specifically, in this embodiment of the application, when the service data where the third data is located is stored in the target level sub table, it may be determined whether the target level sub table is full, and when the target level sub table is determined to be full, the line number of the data written in the target level sub table may be obtained, the line number is compared with a preset maximum line number that can be written in the target level sub table, it is determined whether the target level sub table is full, and a data amount of the data written in the target level sub table may also be obtained, and is compared with a preset maximum data amount that can be stored in the target level sub table, so as to determine whether the target level sub table is full.

If the target level sublist is fully written and the number of the created level sublists does not reach the threshold value of the number of the level sublists, it is indicated that the level sublists can be created again.

If the target level sublist is not fully written, the service data where the third data is located can be directly stored in the target level sublist which is not fully written.

Example 5:

in order to further improve the efficiency of data query, on the basis of the above embodiments, in the embodiment of the present application, the threshold value of the number of created horizontal sub-libraries or horizontal sub-tables is determined according to the following manner:

acquiring the data capacity and the average daily increase of the database;

determining the accumulated increment of the database in a second preset time period according to the average daily increment and the second preset time period;

determining a sum of the data capacity and the cumulative growth amount, and determining the sum as a predicted capacity of the database;

and determining the number of the horizontal sub-libraries to be created or the number threshold of the horizontal sub-tables according to the predicted capacity, the preset maximum capacity of each horizontal sub-library or the preset maximum capacity of each horizontal sub-table.

In order to further improve the efficiency of data query, the number of the vertical sub-tables and the number of the vertical sub-tables are determined because the number of the fields of the data tables and the category to which the business data recorded in each data table belongs are fixed, but the business data is continuously increased along with the time, and the data amount of the horizontal sub-tables are also continuously increased. In order to ensure the efficiency of data query, in the embodiment of the present application, a certain number of horizontal sub-libraries and horizontal sub-tables may be created in advance, so as to ensure that the recorded service data is not excessive and the efficiency of data query is not affected. Before the database is vertically divided, the average daily increment of the database can be obtained, and the number threshold of the horizontal sub-database and the horizontal sub-table which need to be created in the second preset time period is calculated.

In the embodiment of the application, the data capacity and the average daily growth of the database may be obtained, where the data capacity of the database may be understood as the amount of data already stored in the database, and may be obtained through a preset database query statement or script, or may be obtained through a preset query menu. The average daily gain may be determined by obtaining daily data gains for the database and calculating the average daily gain for the database over a period of time.

After the data capacity and the average daily increment of the database are obtained, the accumulated increment of the database in the second preset time period may be determined according to the average daily increment and the second preset time period, and the obtained data capacity of the database and the determined accumulated increment are added to obtain the predicted capacity of the database in the second preset time period.

After the preset capacity of the database is determined, the ratio of the predicted capacity to the preset maximum capacity of each horizontal sublibrary can be calculated and subjected to rounding to obtain the number threshold of the horizontal sublibraries to be created, and the ratio of the predicted capacity to the preset maximum capacity of each horizontal sublibrary is calculated and subjected to rounding to obtain the number threshold of the horizontal sublibraries to be created.

Specifically, the preset maximum capacity of each horizontal sub-base and the preset maximum capacity of each horizontal sub-table may be configured according to actual traffic and performance of the server, where the preset maximum capacity of each horizontal sub-base may be 100G, and the preset maximum capacity of each horizontal sub-table may be 5000 rows, and for some databases with large traffic or user volume, it is necessary to regularly monitor the data volume increase condition of the database, and to take measures for planning in advance, so as to ensure that the service is stably provided to the outside.

Example 6:

fig. 6 is a schematic diagram of a data query process provided in the embodiment of the present application, where the process specifically includes the following steps:

s601: receiving a query instruction of data to be queried, wherein the query instruction comprises a field to be queried.

The data query process provided by the embodiment of the application is suitable for electronic equipment, and the electronic equipment can be equipment such as a server and a PC.

When an inquiry instruction for inquiring data is received, a field to be inquired contained in the inquiry instruction can be obtained, and the field to be inquired can be one or a plurality of fields.

S602: and determining a target vertical sub-table in which the fields to be inquired are stored and a target vertical sub-base in which the target vertical sub-table is stored according to each preset field in the saved vertical sub-table and the index information of the vertical sub-base in which the vertical sub-table is stored.

After the field to be queried included in the query instruction is acquired, each preset field in the stored vertical sub-table and the index information of the vertical sub-base in which the vertical sub-table is stored can be acquired, the index information matched with the field to be queried is searched in the stored index information, if the field to be queried is consistent with all or part of the preset fields of the vertical sub-table in one of the stored index information, the index information can be considered to be matched with the field to be queried, the position of the vertical sub-base in which the vertical sub-table recorded in the index information is stored can be acquired, and then the target vertical sub-table in which the field to be queried is stored and the target vertical sub-base in which the target vertical sub-table is stored can be determined.

S603: and searching data corresponding to the field to be inquired in the target vertical branch table of the target vertical branch library.

After determining the target vertical sub-base in which the field to be queried is stored, the target vertical sub-base may be searched for the target vertical sub-table, and data corresponding to the field to be queried may be searched for in the target vertical sub-table according to a query instruction.

In the embodiment of the application, each preset field of each vertical sub-table and the index information of the vertical sub-base in which the vertical sub-table is stored are stored, so that corresponding index information can be searched in each stored index information according to the field to be queried contained in the query instruction during data query, a target vertical sub-table in which the field to be queried is stored and a target vertical sub-base in which the target vertical sub-table is stored can be quickly determined, and the efficiency of data query is improved.

Example 7:

in order to further improve the efficiency of data query, on the basis of the foregoing embodiments, in this embodiment of the present application, the query instruction further includes an identifier of service data to which the field to be queried belongs, and the method further includes:

and determining a target data table of the target level sub-database corresponding to the identifier of the service data to which the field to be queried belongs according to the index relationship between the identifier of the service data in which each second data of the third preset field is stored and the data table in the target level sub-database, and searching the data corresponding to the field to be queried in the target data table of the target level sub-database.

In order to further improve the efficiency of data query, in this embodiment of the application, the received query instruction may further include an identifier of service data to which a field to be queried belongs, when performing data query, an index relationship between an identifier of service data in which each second data in a stored third preset field is located and a data table in a target level sub-library may also be obtained, index information matching the identifier of service data to which the field to be queried belongs is searched in the stored index information, if the identifier of service data to which the field to be queried belongs is consistent with the identifier of service data in which the second data in the third preset field in a certain stored index information is located, the index information may be considered to match the identifier of service data to which the field to be queried belongs, and which data table in the target level sub-library corresponds to the identifier recorded in the index information may be obtained, and determining a target data table of the target level sub-base corresponding to the identifier of the service data to which the field to be queried belongs, searching the target data table in the determined target level sub-base, and searching the data corresponding to the field to be queried in the target data table according to a query instruction.

In order to further improve the efficiency of data query, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:

and determining a target level sublist of the target level sublist corresponding to the identifier of the service data to which the field to be queried belongs according to the stored index relationship between the identifier of the service data where each third data of the fourth preset field is located and the target level sublist in the level sublist, and searching the data corresponding to the field to be queried in the target level sublist of the target level sublist.

In order to further improve the efficiency of data query, when data query is performed, an index relationship between an identifier of service data in which each third data of a fourth stored preset field is located and a target horizontal sublist in a horizontal sublist may be further obtained, index information matching the identifier of the service data to which the field to be queried belongs is searched in each stored index information, if the identifier of the service data to which the field to be queried belongs is consistent with the identifier of the service data in which the third data of the fourth stored field in one of the stored index information is located, the index information may be considered to be matched with the identifier of the service data to which the field to be queried belongs, and a horizontal sublist corresponding to the identifier recorded in the index information may be obtained, so as to determine which horizontal sublist in the horizontal sublist the identifier to which the identifier of the field to be queried belongs, and search the target data table in the determined horizontal sublist, and searching data corresponding to the field to be queried in the target data table according to the query instruction.

Example 8:

fig. 7 is a schematic structural diagram of a database data storage device according to an embodiment of the present application, and as shown in fig. 7, the database data storage device includes:

a vertical table dividing module 701, configured to obtain, for each data table in the database, a category to which service data recorded in the data table belongs, search first data corresponding to each preset field in the data table, store the searched first data, an identifier of the service data to which the first data belongs, and the preset field in a vertical table dividing of the data table, and delete the first data in the data table;

a vertical sub-library module 702, configured to determine whether a first target vertical sub-library corresponding to the category has been created, if so, store the vertical sub-table in the first target vertical sub-library, and record each preset field in the vertical sub-table and index information of the first target vertical sub-library stored in the vertical sub-table; if not, a second target vertical sub-library corresponding to the category is created, the vertical sub-table is stored in the second target vertical sub-library, and each preset field in the vertical sub-table and index information stored in the second target vertical sub-library by the vertical sub-table are recorded.

In a possible implementation manner, the vertical table splitting module 701 is specifically configured to obtain an operation log of the database within a first preset time period, and identify a condition field in each piece of database operation information recorded in the operation log; counting the number of times of using the condition field aiming at each condition field, and if the number of times of using the condition field is greater than a preset threshold value, determining the condition field as a first preset field; otherwise, determining the condition field as a second preset field; and storing the searched first preset field and the corresponding first data in a first vertical sub-table of the data table, and storing the searched second preset field and the corresponding first data in a second vertical sub-table of the data table.

In a possible embodiment, the apparatus further comprises:

the horizontal sub-library module 703 is configured to obtain, for each vertical sub-table in each target vertical sub-library, each second data corresponding to a third preset field in the vertical sub-table; aiming at each acquired second data, acquiring a numerical value corresponding to the second data, determining a target horizontal sub-database corresponding to the second data according to the numerical value corresponding to the second data and a target numerical value corresponding to each established horizontal sub-database, judging whether a first data table recording service data of a vertical sub-table where the second data is located exists in the target horizontal sub-database, if so, storing the service data where the second data is located in the first data table, deleting the second data in the vertical sub-table where the second data is located, and recording an index relation between an identifier of the service data where the second data is located in a third preset field and the first data table in the target horizontal sub-database in index information; if not, creating a second data table corresponding to the vertical sublist where the second data is located, storing the service data where the second data is located into the second data table, deleting the second data from the vertical sublist where the second data is located, and recording the index relationship between the identifier of the service data where the second data is located in a third preset field and the second data table in the target horizontal sublist in the index information; and deleting the index information stored in the target vertical sublibrary by the vertical sublibrary.

In one possible embodiment, the apparatus further comprises:

the horizontal sub-table module 704 is configured to obtain, for each data table in each horizontal sub-base, each third data corresponding to a fourth preset field in the data table; acquiring a numerical value corresponding to the third data aiming at each acquired third data, determining a target level sub-table corresponding to the third data according to the numerical value corresponding to the third data and a target numerical value corresponding to each established level sub-table, storing service data where the third data is located in the target level sub-table, and deleting the third data in a data table where the third data is located; recording the index relation between the identifier of the service data of the third data of the fourth preset field and the target level sublist in the index information; and deleting the index information stored in the horizontal sub-library by the data table.

In a possible implementation manner, the horizontal library dividing module 703 is specifically configured to determine whether the first data table is full; if so, and when the target horizontal sub-database is fully written, according to the stored quantity threshold value of the horizontal sub-database, if the quantity of the created horizontal sub-database does not reach the quantity threshold value of the horizontal sub-database, creating a horizontal sub-database, activating the horizontal sub-database, taking the target value corresponding to the target horizontal sub-database as the target value corresponding to the created horizontal sub-database, creating a first data table for recording the service data of the vertical sub-table where the second data is located in the created horizontal sub-database, and storing the service data where the second data is located in the first data table; if not, directly storing the service data of the second data into the first data table.

In a possible implementation manner, the level sub-table module 704 is specifically configured to determine whether the target level sub-table is full; if so, and the number of the created horizontal sub-tables does not reach the number threshold of the horizontal sub-tables, creating the horizontal sub-tables, activating the created horizontal sub-tables, taking the target values corresponding to the target horizontal sub-tables as the target values corresponding to the created horizontal sub-tables, and storing the service data where the third data is located in the created horizontal sub-tables; if not, directly storing the service data of the third data into the target level sublist.

In a possible implementation, the apparatus further includes a quantity determining module 705, configured to obtain a data capacity and an average daily increase amount of the database; determining the accumulated increment of the database in a second preset time period according to the average daily increment and the second preset time period; determining a sum of the data capacity and the accumulated growth amount, and determining the sum as a predicted capacity of the database; and determining the number of the horizontal sub-libraries to be created or the number threshold of the horizontal sub-tables according to the predicted capacity, the preset maximum capacity of each horizontal sub-library or the preset maximum capacity of each horizontal sub-table.

Example 9:

fig. 8 is a schematic structural diagram of a data query apparatus according to an embodiment of the present application, and as shown in fig. 8, the apparatus includes:

a receiving module 801, configured to receive a query instruction of data to be queried, where the query instruction includes a field to be queried;

the query module 802 is configured to determine, according to each preset field in the saved vertical sub-table and index information of a vertical sub-base in which the vertical sub-table is stored, a target vertical sub-table in which the field to be queried is stored and a target vertical sub-base in which the target vertical sub-table is stored; and searching data corresponding to the field to be inquired in the target vertical branch table of the target vertical branch library.

In a possible implementation manner, the query module 802 is further configured to determine, according to an index relationship between an identifier of the stored service data where each second data of the third preset field is located and a data table in a target level sub-library, the target data table of the target level sub-library corresponding to the identifier of the service data to which the field to be queried belongs, and search, in the target data table of the target level sub-library, data corresponding to the field to be queried.

In a possible implementation manner, the query module 802 is further configured to determine, according to an index relationship between an identifier of service data where each third data of a fourth preset field is stored and a target level sublist in a level sublist, a target level sublist of the target level sublist corresponding to the identifier of the service data to which the field to be queried belongs, and search for data corresponding to the field to be queried in the target level sublist of the target level sublist.

Example 10:

on the basis of the foregoing embodiments, the present application provides an electronic device, and fig. 9 is a schematic structural diagram of the electronic device provided in the embodiments of the present application, as shown in fig. 9, including: a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902 and the memory 903 are communicated with each other through the communication bus 904;

the memory 903 has stored therein a computer program which, when executed by the processor 901, causes the processor 901 to perform the steps of:

In a possible embodiment, the determining of the preset field includes:

In one possible embodiment, the method comprises:

aiming at each vertical sublist in each target vertical sublist, acquiring each second data corresponding to a third preset field in the vertical sublist; acquiring a numerical value corresponding to the second data aiming at each acquired second data, determining a target horizontal sub-library corresponding to the second data according to the numerical value corresponding to the second data and a target numerical value corresponding to each established horizontal sub-library, judging whether a first data table for recording service data of a vertical sub-table where the second data is located exists in the target horizontal sub-library, if so, storing the service data of the second data into the first data table, deleting the second data from the vertical sub-table where the second data is located, and recording an index relation between an identifier of the service data of the second data in a third preset field and the first data table in the target horizontal sub-library in index information; if not, creating a second data table corresponding to the vertical sublist where the second data is located, storing the service data where the second data is located into the second data table, deleting the second data from the vertical sublist where the second data is located, and recording the index relationship between the identifier of the service data where the second data is located in a third preset field and the second data table in the target horizontal sublist in the index information; and deleting the index information stored in the target vertical sublibrary by the vertical sublibrary.

In one possible embodiment, the method comprises:

In a possible implementation manner, the storing the service data in which the second data is located in the first data table includes:

judging whether the first data table is full;

In a possible implementation manner, the storing the service data in which the third data is located in the target level sublist includes:

judging whether the target level sublist is fully written or not;

if not, directly storing the service data of the third data into the target level sublist.

In one possible embodiment, the threshold number of created level bins or level tables is determined according to the following:

acquiring the data capacity and the average daily increase of the database;

Because the principle of solving the problem of the electronic device is similar to the database data storage method, the implementation of the electronic device may refer to the above embodiments, and repeated parts are not described again.

The processor 901 may further perform the following steps:

In a possible implementation manner, the query instruction further includes an identifier of service data to which the field to be queried belongs, and the method further includes:

In one possible embodiment, the method further comprises:

Because the principle of the electronic device for solving the problem is similar to the data query method, the implementation of the electronic device may refer to the above embodiments, and repeated details are not repeated.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus. The communication interface 902 is used for communication between the electronic apparatus and other apparatuses. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor. The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

Example 11:

on the basis of the foregoing embodiments, the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored, and when the program is run on the processor, the processor is caused to execute the following steps:

In a possible embodiment, the determining of the preset field includes:

In one possible embodiment, the method comprises:

aiming at each vertical sublist in each target vertical sublist, acquiring each second data corresponding to a third preset field in the vertical sublist; acquiring a numerical value corresponding to the second data aiming at each acquired second data, determining a target horizontal sub-library corresponding to the second data according to the numerical value corresponding to the second data and a target numerical value corresponding to each established horizontal sub-library, judging whether a first data table for recording service data of a vertical sub-table where the second data is located exists in the target horizontal sub-library, if so, storing the service data of the second data into the first data table, deleting the second data from the vertical sub-table where the second data is located, and recording an index relation between an identifier of the service data of the second data in a third preset field and the first data table in the target horizontal sub-library in index information; if not, creating a second data table corresponding to the vertical sublist where the second data is located, storing the service data where the second data is located into the second data table, deleting the second data from the vertical sublist where the second data is located, and recording the index relationship between the identifier of the service data where the second data is located in a third preset field and the second data table in the target horizontal sublist in the index information; and deleting the index information stored in the target vertical sublist.

In one possible embodiment, the method comprises:

judging whether the first data table is fully written or not;

judging whether the target level sublist is fully written or not;

acquiring the data capacity and the average daily increase of the database;

and determining the number of the horizontal sub-banks to be created or the number threshold of the horizontal sub-tables according to the predicted capacity, the preset maximum capacity of each horizontal sub-bank or the preset maximum capacity of each horizontal sub-table.

Since the principle of solving the problem of the computer readable medium is similar to the database data storage method, after the processor executes the computer program in the computer readable medium, the steps implemented may refer to the above embodiments, and repeated parts are not described again.

Example 12:

In one possible embodiment, the method further comprises:

Since the principle of solving the problem of the computer readable medium is similar to that of the data query method, after the processor executes the computer program in the computer readable medium, the steps implemented can be referred to the above embodiments, and repeated parts are not described again.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

For the system/apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A database data storage method, the method comprising:

2. The method of claim 1, wherein the predetermined field is determined by:

3. The method of claim 1, wherein the method comprises:

4. The method of claim 3, wherein the method comprises:

5. The method of claim 3, wherein storing the service data in which the second data is located in the first data table comprises:

judging whether the first data table is fully written or not;

6. The method of claim 4, wherein storing the service data in which the third data is located in the target level sublist comprises:

judging whether the target level sublist is fully written or not;

if yes, and the number of the created horizontal sublists does not reach the number threshold of the horizontal sublists, creating the horizontal sublists, activating the created horizontal sublists, taking the target value corresponding to the target horizontal sublist as the target value corresponding to the created horizontal sublist, and storing the service data where the third data is located into the created horizontal sublists;

7. The method of claim 5 or 6, wherein the threshold number of created level bins or level sub-tables is determined according to the following:

acquiring the data capacity and the average daily increase of the database;

8. A method for data query, the method comprising:

9. The method as claimed in claim 8, wherein the query instruction further includes an identifier of service data to which the field to be queried belongs, and the method further comprises:

10. The method of claim 9, wherein the method further comprises:

11. A database data storage apparatus, the apparatus comprising:

12. The apparatus according to claim 11, wherein the vertical table splitting module is specifically configured to obtain an operation log of the database within a first preset time period, and identify a condition field in each piece of database operation information recorded in the operation log; counting the number of times of using the condition field aiming at each condition field, and determining the condition field as a first preset field if the number of times of using is greater than a preset threshold value; otherwise, determining the condition field as a second preset field; and storing the searched first preset field and the corresponding first data in a first vertical sub-table of the data table, and storing the searched second preset field and the corresponding first data in a second vertical sub-table of the data table.

13. The apparatus of claim 11, wherein the apparatus further comprises:

the horizontal sub-library module is used for acquiring each piece of second data corresponding to a third preset field in each target vertical sub-library aiming at each vertical sub-table in each target vertical sub-library; acquiring a numerical value corresponding to the second data aiming at each acquired second data, determining a target horizontal sub-library corresponding to the second data according to the numerical value corresponding to the second data and a target numerical value corresponding to each established horizontal sub-library, judging whether a first data table for recording service data of a vertical sub-table where the second data is located exists in the target horizontal sub-library, if so, storing the service data of the second data into the first data table, deleting the second data from the vertical sub-table where the second data is located, and recording an index relation between an identifier of the service data of the second data in a third preset field and the first data table in the target horizontal sub-library in index information; if not, creating a second data table corresponding to the vertical sublist where the second data is located, storing the service data where the second data is located into the second data table, deleting the second data from the vertical sublist where the second data is located, and recording the index relationship between the identifier of the service data where the second data is located in a third preset field and the second data table in the target horizontal sublist in the index information; and deleting the index information stored in the target vertical sublibrary by the vertical sublibrary.

14. The apparatus of claim 13, wherein the apparatus further comprises:

the horizontal sub-table module is used for acquiring each third data corresponding to a fourth preset field in each data table according to each data table in each horizontal sub-base; acquiring a numerical value corresponding to the third data aiming at each acquired third data, determining a target level sub-table corresponding to the third data according to the numerical value corresponding to the third data and a target numerical value corresponding to each established level sub-table, storing service data where the third data is located in the target level sub-table, and deleting the third data in a data table where the third data is located; recording the index relation between the identifier of the service data of the third data of the fourth preset field and the target level sublist in the index information; and deleting the index information stored in the horizontal sub-library by the data table.

15. The apparatus of claim 13, wherein the horizontal banking module is specifically configured to determine whether the first data table is full; if so, and when the target horizontal sub-database is fully written, according to the stored quantity threshold value of the horizontal sub-database, if the quantity of the created horizontal sub-database does not reach the quantity threshold value of the horizontal sub-database, creating a horizontal sub-database, activating the horizontal sub-database, taking the target value corresponding to the target horizontal sub-database as the target value corresponding to the created horizontal sub-database, creating a first data table for recording the service data of the vertical sub-table where the second data is located in the created horizontal sub-database, and storing the service data where the second data is located in the first data table; if not, directly storing the service data of the second data into the first data table.

16. The apparatus of claim 14, wherein the level sub-table module is specifically configured to determine whether the target level sub-table is full; if so, and the number of the created horizontal sub-tables does not reach the number threshold of the horizontal sub-tables, creating the horizontal sub-tables, activating the created horizontal sub-tables, taking the target values corresponding to the target horizontal sub-tables as the target values corresponding to the created horizontal sub-tables, and storing the service data where the third data is located in the created horizontal sub-tables; if not, directly storing the service data of the third data into the target level sublist.

17. The apparatus of claim 15 or 16, wherein the apparatus further comprises:

the quantity determining module is used for acquiring the data capacity and the average daily increase quantity of the database; determining the accumulated increment of the database in a second preset time period according to the average daily increment and the second preset time period; determining a sum of the data capacity and the accumulated growth amount, and determining the sum as a predicted capacity of the database; and determining the number of the horizontal sub-libraries to be created or the number threshold of the horizontal sub-tables according to the predicted capacity, the preset maximum capacity of each horizontal sub-library or the preset maximum capacity of each horizontal sub-table.

18. A data query apparatus, characterized in that the apparatus comprises:

19. The apparatus of claim 18, wherein the query module is further configured to determine, according to an index relationship between an identifier of the stored service data where each second data of the third preset field is located and a data table in a target level sub-base, a target data table of the target level sub-base corresponding to the identifier of the service data to which the field to be queried belongs, and search, in the target data table of the target level sub-base, data corresponding to the field to be queried.

20. The apparatus of claim 18, wherein the query module is further configured to determine a target level sublist of the target level sublist corresponding to the identifier of the service data to which the field to be queried belongs according to an index relationship between the identifier of the service data where each third data of the fourth preset field is stored and a target level sublist in the level sublist, and search for data corresponding to the field to be queried in the target level sublist of the target level sublist.

21. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being adapted to implement the steps of the database data storage method of any of claims 1-7 or the data query method of any of claims 8-10 when executing a computer program stored in the memory.

22. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, carries out the steps of the database data storage method according to any one of claims 1 to 7 or the data query method according to any one of claims 8 to 10.