CN115168409A - Data query method and device for database sub-tables and computer equipment - Google Patents
Data query method and device for database sub-tables and computer equipment Download PDFInfo
- Publication number
- CN115168409A CN115168409A CN202211076071.0A CN202211076071A CN115168409A CN 115168409 A CN115168409 A CN 115168409A CN 202211076071 A CN202211076071 A CN 202211076071A CN 115168409 A CN115168409 A CN 115168409A
- Authority
- CN
- China
- Prior art keywords
- fragment
- attribute value
- index
- attribute
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data query method and device for database sub-tables and computer equipment. The method comprises the following steps: determining an attribute value input for data query in a data query stage; if the input attribute value is the index attribute value with the index attribute, searching a target fragment identifier which is stored corresponding to the input attribute value; the index attribute is an attribute which is represented by an index attribute column in the original data table and can be subjected to index query; the index attribute value has a fragment identifier which is correspondingly stored; the data of the index attribute value in the row where the original data table is located is written in the fragment table represented by the fragment identification which is correspondingly stored; determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target row data is written; the target line data is data of a line where the attribute value input in the original data table is located; and performing data query in the target fragment table according to the input attribute value. By adopting the method, the data query efficiency of the horizontal sub-table can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for querying data in a database sub-table, and a computer device.
Background
When the system is used to a certain stage, a large amount of data can be accumulated in the data table of the database, so that the number of rows in the data table can be very large, and the use of the data table is influenced. Therefore, in practical applications, the data table is often subjected to horizontal table splitting, that is, data in the original data table is split in units of rows according to attribute values of some columns in the original data table, and the original data table is split into a plurality of fragment tables. After obtaining a plurality of fragment tables, how to query the data in the fragment tables becomes an important issue.
In the conventional method, when data is queried for a segment table, if data input by data query is a segment attribute value of a segment attribute according to which the data belongs to a horizontal segment table, the input data can be calculated according to a segment policy to obtain a segment identifier, so that the corresponding segment table is directly located according to the segment identifier, and if the input data is not the segment attribute value belonging to the segment attribute, all the segment tables need to be searched for data query, so that the required data is difficult to quickly find, and the efficiency is very low.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus, a computer device, a computer readable storage medium, and a computer program product for querying data in a database sub-table, which can improve the efficiency of querying data in the database sub-table.
In a first aspect, the application provides a data query method for database sub-tables. The method comprises the following steps:
in the data query stage, determining an attribute value input for data query;
if the input attribute value is an index attribute value with index attribute, searching a target fragment identifier stored corresponding to the input attribute value; the index attribute is an attribute which is represented by an index attribute column in an original data table and can be used for index query; the index attribute value has a corresponding stored fragment identifier; the data of the index attribute value in the row where the original data table is located is written in the fragment table represented by the fragment identification which is correspondingly stored;
determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line where the input attribute value is located in an original data table;
and according to the input attribute value, performing data query in the target fragment table.
In a second aspect, the application further provides a data query device for the sub-tables of the database. The device comprises:
the data acquisition module is used for determining an attribute value input for data query in a data query stage;
the fragment identifier determining module is used for searching a target fragment identifier which is stored corresponding to the input attribute value if the input attribute value is the index attribute value with the index attribute; the index attribute is an attribute which is represented by an index attribute column in an original data table and can be used for index query; the index attribute value has a corresponding stored fragment identifier; the data of the index attribute value in the row where the original data table is located is written in the fragment table represented by the fragment identification which is correspondingly stored;
the fragment table determining module is used for determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line where the input attribute value is located in an original data table;
and the data query module is used for performing data query in the target fragment table according to the input attribute value.
In one embodiment, the fragment identifier determining module is further configured to, if the input attribute value is an index attribute value with an index attribute, look up, from a dynamic index table, a fragment identifier recorded in a row where the input attribute value is located, where the fragment identifier is used as a target fragment identifier; the dynamic index table is obtained by filling index attribute values corresponding to index attribute columns in each row in the original data table and recording fragment identifications corresponding to the row in each row in a horizontal table division stage in advance.
In one embodiment, the data query device for the database sub-table further includes:
the fragment identification calculation module is used for determining fragment identifications corresponding to all rows in the original data table to be divided in the horizontal table division stage;
the index attribute value recording module is used for correspondingly storing the index attribute values of all rows in the original data table under the index attribute column and the fragment identifications corresponding to the row;
and the data writing module is used for writing the data of each row in the original data table into the fragment table represented by the fragment identifier corresponding to the row respectively so as to complete horizontal division.
In one embodiment, the fragment identifier calculation module is further configured to determine fragment attribute values of rows in the original data table under a fragment attribute column; the fragmentation attribute column is a column corresponding to the fragmentation attribute in the original data table; the fragment attribute is an attribute according to which the original data table is divided to obtain the fragment table in the horizontal table division stage; and calculating the fragment attribute values of each row according to a fragment strategy to obtain fragment identifications respectively corresponding to each row.
In one embodiment, the index attribute value recording module is further configured to correspondingly fill the index attribute values of each row in the original data table under the index attribute column into each row in the dynamic index table respectively; and recording the fragment identification corresponding to the index attribute value of the line in each line of the dynamic index table.
In one embodiment, the fragment identifier determining module is further configured to calculate the input attribute value according to a fragment policy in a horizontal table division stage to obtain a target fragment identifier if the input attribute value is a fragment attribute value with a fragment attribute.
In one embodiment, the data query module is further configured to query data in each segment table according to the input attribute value if the input attribute value is neither a segment attribute value nor an index attribute value.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the steps in the data query method of the database sub-table according to the embodiments of the present application.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the processor is caused to execute the steps in the data query method of the database sub-table according to the embodiments of the present application.
In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the processor is enabled to execute the steps in the data query method for the database sub-table according to the embodiments of the present application.
According to the data query method, the data query device, the computer equipment, the storage medium and the computer program product of the database sub-tables, in the data query stage, the attribute value input for data query is determined, if the input attribute value is the index attribute value with the index attribute, the target fragment identifier corresponding to the input attribute value is searched, then the target fragment table corresponding to the target fragment identifier is determined, and data query is performed in the target fragment table according to the input attribute value. The method and the device have the advantages that the target fragment identification can be rapidly determined under the condition that the input attribute value is the index attribute value, the target fragment table is further determined, and data query is carried out from the target fragment table, so that the problem that data query is carried out from each fragment table as long as the input attribute value is not the fragment attribute value is solved, and the efficiency of carrying out data query on the database fragment table under the condition of horizontal fragment table is improved.
Drawings
FIG. 1 is a diagram of an exemplary implementation of a database sub-table data query method;
FIG. 2 is a flowchart illustrating a method for querying data in a database table under an embodiment;
FIG. 3 is a schematic diagram illustrating the overall flow of the data query phase in one embodiment;
FIG. 4 is a schematic diagram of the overall flow of the horizontal tabbing phase in one embodiment;
FIG. 5 is a diagram illustrating an overall flowchart of a data query method for database sub-tables according to an embodiment;
FIG. 6 is a block diagram of an apparatus for querying data in a database table according to an embodiment;
FIG. 7 is a block diagram showing an alternative embodiment of a data query apparatus for database sub-tables;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data query method for the database sub-tables provided by the embodiment of the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104, or may be located on the cloud or other network server. In the data query phase, a user may input an attribute value through the terminal 102, the terminal 102 may transmit the input attribute value to the server 104, and the server 104 may execute the method in the embodiments of the data query phase in the present application to perform data query according to the input attribute value. In the horizontal table splitting stage, a user may set a fragmentation attribute and a fragmentation policy through the terminal 102, the terminal 102 may send the fragmentation attribute and the fragmentation policy to the server 104, and the server 104 may execute the methods in the embodiments of the horizontal table splitting stage according to the fragmentation attribute and the fragmentation policy to split the original data table. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart car-mounted devices, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers.
In one embodiment, as shown in fig. 2, a method for querying data in a database sub-table is provided, which is described by taking the method as an example applied to the server 104 in fig. 1, and includes the following steps:
in the data query phase, the attribute values input for performing the data query are determined, step 202.
The data query stage refers to a processing stage for querying data corresponding to the input attribute values from the fragment table. The sharding table (i.e., database sharding) refers to a data table obtained by dividing an original data table in a horizontal sharding manner in a horizontal sharding stage. The horizontal table splitting refers to a process of splitting data in the original data table by taking rows as units according to attribute values of some columns in the original data table, and splitting the original data table into a plurality of fragment tables.
Specifically, in the data query phase, the user may input the attribute value through the terminal, the terminal may transmit the input attribute value to the server, and the server may receive the input attribute value transmitted by the terminal.
Step 204, if the input attribute value is an index attribute value with index attribute, searching a target fragment identifier stored corresponding to the input attribute value; the index attribute is an attribute which is represented by an index attribute column in the original data table and can be used for index query; the index attribute value has a fragment identifier which is correspondingly stored; and writing the data of the index attribute value in the row in which the original data table is positioned in the fragment table characterized by the corresponding stored fragment identification.
The fragment identifier is an identifier for uniquely characterizing the fragment table.
In one embodiment, the index attribute may be one or more, and accordingly, the index attribute column in the original data table may be one or more, and the index attribute value in each row in the original data table may be one or more.
In one embodiment, an index attribute may be an attribute that can uniquely identify an object.
In one embodiment, the index attribute may be the attribute of the primary key. Wherein the primary key is used to uniquely identify a row in the data table.
In another embodiment, the index attributes may include one or more attributes other than the primary key, such as: order number or job number, etc. Wherein the order number is used to uniquely identify the order. The job number is used to uniquely identify the employee.
In one embodiment, the user may set the index attribute during the horizontal tabbing phase.
Specifically, in the stage of horizontal table division, the server may write the data in each row into the table represented by the segment identifier corresponding to the row, respectively, according to the segment identifier corresponding to each row of the original data table, and store the index attribute value in each row corresponding to the segment identifier corresponding to the row. In the data query phase, if the input attribute value belongs to the index attribute value, the server may search the fragment identifier stored corresponding to the input attribute value as the target fragment identifier.
In one embodiment, if the input attribute value belongs to the index attribute value, the server may look up a segment identifier corresponding to the input attribute value from the dynamic index table as the target segment identifier. The dynamic index table is obtained by correspondingly recording the index attribute value and the fragment identifier corresponding to the row of the index attribute value in the original data table in the dynamic index table at the stage of horizontal table division.
For example: if the input attribute value is the order number 123456, the server may determine that the input attribute value is an index attribute value, the server may search for the fragment identifier stored corresponding to the order number 123456, assuming that the fragment identifier stored corresponding to the order number 123456 is found to be 5, the target fragment identifier is 5, and the server may perform data query from the fragment table represented by the target fragment identifier 5.
Step 206, determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line in which the attribute value input in the original data table is located.
And step 208, performing data query in the target fragment table according to the input attribute value.
The target fragment table is determined from a plurality of fragment tables obtained by dividing the original data table in a horizontal fragment table mode.
Specifically, the server may determine a target fragment table represented by the target fragment identifier according to the target fragment identifier, then search a position of the input attribute value in the target fragment table according to the input attribute value, and determine data corresponding to the input attribute value in the target fragment table.
In the data query method of the database sub-tables, in a data query stage, an attribute value input for data query is determined, if the input attribute value is an index attribute value with index attributes, a target fragment identifier stored corresponding to the input attribute value is searched, then a target fragment table corresponding to the target fragment identifier is determined, and data query is performed in the target fragment table according to the input attribute value. The method and the device have the advantages that the target fragment identification can be rapidly determined under the condition that the input attribute value is the index attribute value, the target fragment table is further determined, and data query is carried out from the target fragment table, so that the problem that data query is carried out from each fragment table as long as the input attribute value is not the fragment attribute value is solved, and the efficiency of carrying out data query on the database fragment table under the condition of horizontal fragment table is improved. The method breaks through the use scene after the horizontal table division, and can not only quickly query the data through the attribute values of the fragments, but also quickly query the data through the attribute values of the indexes.
In one embodiment, if the input attribute value belongs to the index attribute value, finding the target fragment identifier stored corresponding to the input attribute value includes: if the input attribute value is the index attribute value with the index attribute, searching the fragment identifier recorded in the row of the input attribute value from the dynamic index table as the target fragment identifier; the dynamic index table is obtained by filling index attribute values corresponding to index attribute columns in each row in an original data table and recording fragment identifications corresponding to the row in each row in a horizontal table division stage in advance.
Specifically, in the stage of horizontal table division, the server may correspondingly fill the index attribute values of each row in the original data table under the index attribute column into each row in the dynamic index table, and record the segment identifier corresponding to the index attribute value of the row in each row of the dynamic index table. In the data query phase, if the input attribute value is an index attribute value with an index attribute, the server may search the fragment identifier recorded in the row where the input attribute value is located from the dynamic index table, as the target fragment identifier.
As shown in fig. 3, which is an overall flowchart diagram of the data query phase, it can be seen that, when the input attribute value is an index attribute value with an index attribute, the server may query the dynamic index table to obtain a target segment identifier, and then perform data query from the target segment table. The dynamic index table in fig. 3 includes an index attribute column for recording index attribute values other than the primary key, a column for recording slice identification, and a pk column for recording the primary key (pk). The server may look up the input attribute value from the dynamic index table, and then determine the fragment identifier corresponding to the input attribute value from the dynamic index table, such as: the input attribute value is located in the 2 nd row in the dynamic index table, and the server may determine the segment identifier in the 2 nd row in the dynamic index table as the target segment identifier.
In the above embodiment, if the input attribute value is an index attribute value with an index attribute, the fragment identifier recorded in the row where the input attribute value is located is searched from the dynamic index table, and is used as the target fragment identifier, so that when the input attribute value is an index attribute value, the target fragment identifier can be quickly determined through the dynamic index table, the target fragment table is further determined, and data query is performed from the target fragment table, the problem that data query is performed from each fragment table as long as the input attribute value is not a fragment attribute value is avoided, and the efficiency of performing data query on the database fragment table under the condition of horizontal fragment table is improved.
In one embodiment, the method further comprises: in the horizontal table dividing stage, determining the corresponding fragment identification of each row in the original data table to be divided; correspondingly storing the index attribute values of all rows in the original data table under the index attribute column and the fragment identifications corresponding to the rows; and respectively writing the data of each row in the original data table into the fragment table represented by the fragment identifier corresponding to the row so as to complete horizontal division.
Specifically, in the horizontal tabbing stage, the user may set the index attribute through the terminal, and the terminal may notify the set index attribute to the server. The server can determine the fragment identifier corresponding to each row according to the fragment attribute value of each row in the original data table to be divided, determine the index attribute value of each row in the original data table under the index attribute column, correspondingly store the index attribute value of each row and the fragment identifier corresponding to the row, and finally write the data of each row in the original data table into the fragment table represented by the fragment identifier corresponding to the row respectively to complete horizontal division.
In the above embodiment, in the horizontal table splitting stage, the fragment identifier corresponding to each row in the original data table to be split is determined, and the index attribute value of each row in the original data table under the index attribute column is stored in correspondence with the fragment identifier corresponding to the row, so that in the data query stage, when the input attribute value is the index attribute value, the target fragment identifier can be quickly determined according to the input attribute value, the target fragment table is further determined, and data query is performed from the target fragment table, thereby avoiding the problem that data query is performed from each fragment table as long as the input attribute value is not the fragment attribute value, and improving the efficiency of performing data query on the database tables in the horizontal table splitting state.
In one embodiment, determining the fragment identifier corresponding to each row in the original data table to be partitioned includes: determining the fragment attribute values of each row in the original data table under the fragment attribute column; the fragment attribute column is a column corresponding to the fragment attribute in the original data table; the fragment attribute is an attribute according to which the original data table is divided to obtain the fragment table in the horizontal table division stage; and calculating the fragment attribute values of each row according to the fragment strategy to obtain fragment identifications respectively corresponding to each row.
Specifically, in the stage of horizontal table division, a user may set a fragmentation attribute and a fragmentation policy through a terminal, and the terminal may notify the server of the set fragmentation attribute and the fragmentation policy. The server may determine the fragmentation attribute values of each row in the original data table under the fragmentation attribute column corresponding to the fragmentation attribute, and then calculate the fragmentation attribute values of each row according to the fragmentation policy to obtain fragmentation identifications corresponding to each row.
In one embodiment, the shard attribute column may be one or more.
In the above embodiment, in the horizontal table splitting stage, the segment attribute values of each row in the original data table under the segment attribute column are determined, the segment attribute values of each row are calculated according to the segment policy, segment identifiers corresponding to each row are obtained, and then the index attribute values and the segment identifiers can be stored correspondingly, so that in the data query stage, when the input attribute value is the index attribute value, the target segment identifier can be quickly determined according to the input attribute value.
In one embodiment, the storing the index attribute values of each row in the original data table under the index attribute column and the fragment identifier corresponding to the row correspondingly comprises: respectively and correspondingly filling index attribute values of each row in the original data table under the index attribute column into each row in the dynamic index table; and recording the fragment identification corresponding to the index attribute value of the line in each line of the dynamic index table.
Specifically, in the stage of horizontal table division, after determining the segment identifiers corresponding to each row, the server may correspondingly fill the index attribute values of each row in the original data table under the index attribute column into each row in the dynamic index table, and record the segment identifier corresponding to the index attribute value of the row in each row of the dynamic index table. In the data query phase, if the input attribute value is the index attribute value, the server may search the fragment identifier corresponding to the input attribute value from the dynamic index table.
In one embodiment, the index attribute may be one or more, and accordingly, the index attribute column in the original data table may be one or more, and the index attribute value in each row in the original data table may be one or more.
In an embodiment, if the index attribute is multiple, the server may correspondingly fill the index attribute values of each row in the original data table under the index attribute columns into corresponding positions of the corresponding row in the dynamic index table under the index attribute columns in the dynamic index table, and record the fragment identifier corresponding to the index attribute value of the row in each row of the dynamic index table.
Fig. 4 is a schematic diagram of the overall flow in the horizontal tabbing stage. The server calculates the fragment attribute values of each row under each fragment attribute column in the original data table according to the fragment strategy to obtain fragment identifiers, determines the index attribute values under each index attribute column in the original data table, then correspondingly records the index attribute values of each row except the primary key, the primary key (namely, the data under pk column in the figure) and the corresponding fragment identifiers in the dynamic index table, and finally writes the data of each row in the original data table into the fragment table represented by the corresponding fragment identifiers.
In the above embodiment, in the horizontal table splitting stage, the index attribute values of each row in the original data table under the index attribute column are respectively and correspondingly filled into each row in the dynamic index table, and the segment identifier corresponding to the index attribute value of the row is recorded in each row in the dynamic index table, so that in the data query stage, under the condition that the input attribute value is the index attribute value, the target segment identifier can be quickly determined through the dynamic index table, the target segment table is further determined, and data query is performed from the target segment table, thereby avoiding the problem that data query is performed from each segment table as long as the input attribute value is not the segment attribute value, and improving the efficiency of performing data query on the database table splitting under the horizontal table splitting condition.
In one embodiment, the method further comprises: if the input attribute value is the fragment attribute value with the fragment attribute, calculating the input attribute value according to the fragment strategy of the horizontal spreadsheet stage to obtain a target fragment identifier; and executing the target fragment table corresponding to the target fragment identifier and the subsequent steps.
Specifically, in the data query stage, after the input attribute value is determined, if the input attribute value is the fragment attribute value with the fragment attribute, the server may calculate the input attribute value according to the fragment policy in the horizontal table splitting stage to obtain a target fragment identifier, then determine a target fragment table corresponding to the target fragment identifier, and perform data query in the target fragment table according to the input attribute value.
In one embodiment, as shown in fig. 3, in the data query phase, the server may first determine whether the input attribute value is a fragment attribute value with a fragment attribute, and if the input attribute value is a fragment attribute value with a fragment attribute, the server may perform calculation and subsequent steps on the input attribute value according to a fragment policy in the horizontal table splitting phase. If the input attribute value is not the fragment attribute value with the fragment attribute, the server can judge whether the input attribute value is the index attribute value with the index attribute, and if the input attribute value is the index attribute value with the index attribute, the server can search the target fragment identifier stored corresponding to the input attribute value and perform the subsequent steps.
In the above embodiment, if the input attribute value is a fragment attribute value, the input attribute value is calculated according to a fragment policy in the horizontal table splitting stage to obtain a target fragment identifier, and the target fragment identifier can be quickly obtained by calculating the input attribute value in the data query stage under the condition that the input attribute value is the fragment attribute value, so that the target fragment table is quickly located, and the data query efficiency is improved.
In one embodiment, the method further comprises: and if the input attribute value is neither the fragment attribute value nor the index attribute value, performing data query in each fragment table according to the input attribute value.
Specifically, as shown in fig. 3, in the data query phase, after determining the input attribute value, if the input attribute value is neither the fragment attribute value nor the index attribute value, the server may perform full-table scanning, that is, searching the position of the input attribute value in each fragment table according to the input attribute value, and determining data corresponding to the input attribute value.
In the above embodiment, only when the input attribute value is neither a fragment attribute value nor an index attribute value, data query is performed in each fragment table according to the input attribute value, so that the target fragment identifier can be quickly determined and further positioned to the target fragment table when the input attribute value is either a fragment attribute value or an index attribute value, and data query can be performed when the input attribute value is neither a fragment attribute value nor an index attribute value.
Fig. 5 is a schematic overall flow chart of a data query method for database sub-tables in the embodiments of the present application. In fig. 5, in the steps of the horizontal table splitting stage, as in fig. 4, data of each row in the original data table is finally written into the slice table characterized by the slice identifier in fig. 5 (i.e., slice table 1 \8230; slice table N), so as to complete the horizontal table splitting. In fig. 5, in the data query phase, the steps are the same as those in fig. 3, and finally, a corresponding target fragment table is queried according to the target fragment identifier.
It should be understood that, although the steps in the flowcharts related to the embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the application also provides a data query device of the database sub-tables, which is used for realizing the data query method of the database sub-tables. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method, so the specific limitations in the embodiment of the data query device for one or more sub-tables of the database provided below can refer to the limitations on the data query method for the sub-tables of the database, and are not described again here.
In one embodiment, as shown in fig. 6, there is provided a data query apparatus 600 for database sub-tables, including: a data acquisition module 602, a fragment identification determination module 604, a fragment table determination module 606, and a data query module 608, wherein:
the data obtaining module 602 is configured to determine, in a data query phase, an attribute value input for performing a data query.
A fragment identifier determining module 604, configured to search, if the input attribute value is an index attribute value with an index attribute, a target fragment identifier stored in correspondence with the input attribute value; the index attribute is an attribute which is represented by an index attribute column in the original data table and can be used for index query; the index attribute value has a fragment identifier which is correspondingly stored; and writing the data of the index attribute value in the row in which the original data table is positioned in the fragment table characterized by the corresponding stored fragment identification.
A fragment table determining module 606, configured to determine a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line in which the attribute value input in the original data table is located.
And the data query module 608 is configured to perform data query in the target fragment table according to the input attribute value.
In one embodiment, the fragment identifier determining module 604 is further configured to, if the input attribute value is an index attribute value with an index attribute, search, from the dynamic index table, a fragment identifier recorded in a row where the input attribute value is located, as a target fragment identifier; the dynamic index table is obtained by filling index attribute values corresponding to index attribute columns in each row in an original data table and recording fragment identifications corresponding to the row in each row in a horizontal table division stage in advance.
In one embodiment, as shown in fig. 7, the data query apparatus 600 for database sub-tables further includes:
the fragment identifier calculating module 610 is configured to determine, in the horizontal table division stage, a fragment identifier corresponding to each row in the original data table to be divided.
And an index attribute value recording module 612, configured to correspondingly store the index attribute values of the rows in the index attribute column in the original data table and the fragment identifiers corresponding to the row.
And a data writing module 614, configured to write data in each row in the original data table into the fragment table represented by the fragment identifier corresponding to the row, respectively, so as to complete horizontal table division.
In one embodiment, the fragment identifier calculating module 610 is further configured to determine fragment attribute values of rows in the original data table under the fragment attribute column; the fragmentation attribute column is a column corresponding to the fragmentation attribute in the original data table; the fragment attribute is an attribute according to which the original data table is divided to obtain a fragment table in the horizontal table division stage; and calculating the fragment attribute values of each row according to the fragment strategy to obtain fragment identifications respectively corresponding to each row.
In an embodiment, the index attribute value recording module 612 is further configured to correspondingly fill the index attribute values of each row in the original data table under the index attribute column into each row in the dynamic index table, respectively; and recording the fragment identification corresponding to the index attribute value of the line in each line of the dynamic index table.
In an embodiment, the fragment identifier determining module 604 is further configured to, if the input attribute value is a fragment attribute value with a fragment attribute, calculate the input attribute value according to a fragment policy in a horizontal table segment to obtain a target fragment identifier.
In one embodiment, the data query module 608 is further configured to perform data query in each segment table according to the input attribute value if the input attribute value is neither the segment attribute value nor the index attribute value.
The data query device of the database sub-tables determines an attribute value input for data query in a data query stage, searches a target fragment identifier corresponding to the input attribute value if the input attribute value is an index attribute value with index attribute, then determines a target fragment table corresponding to the target fragment identifier, and performs data query in the target fragment table according to the input attribute value. The method and the device have the advantages that the target fragment identification can be rapidly determined under the condition that the input attribute value is the index attribute value, the target fragment table is further determined, and data query is carried out from the target fragment table, so that the problem that data query is carried out from each fragment table as long as the input attribute value is not the fragment attribute value is solved, and the efficiency of carrying out data query on the database fragment table under the condition of horizontal fragment table is improved.
All or part of each module in the data query device of the database sub-table can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The database of the computer device is used for storing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of data query of a database sub-table.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above-described method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, carries out the steps in the method embodiments described above.
It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include a Read-Only Memory (ROM), a magnetic tape, a floppy disk, a flash Memory, an optical Memory, a high-density embedded nonvolatile Memory, a resistive Random Access Memory (ReRAM), a Magnetic Random Access Memory (MRAM), a Ferroelectric Random Access Memory (FRAM), a Phase Change Memory (PCM), a graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the various embodiments provided herein may be, without limitation, general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, or the like.
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
Claims (10)
1. A data query method for database sub-tables is characterized by comprising the following steps:
in the data query stage, determining an attribute value input for data query;
if the input attribute value is an index attribute value with index attribute, searching a target fragment identifier stored corresponding to the input attribute value; the index attribute is an attribute which is represented by an index attribute column in an original data table and can be used for index query; the index attribute value has a corresponding stored fragment identifier; writing data of the index attribute value in a row in the original data table into a fragment table characterized by the fragment identification stored correspondingly;
determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line where the input attribute value is located in the original data table;
and according to the input attribute value, performing data query in the target fragment table.
2. The method according to claim 1, wherein if the input attribute value is an index attribute value with an index attribute, finding the target segment identifier stored corresponding to the input attribute value comprises:
if the input attribute value is an index attribute value with index attributes, searching the fragment identifier recorded in the row of the input attribute value from a dynamic index table as a target fragment identifier;
the dynamic index table is obtained by filling index attribute values corresponding to index attribute columns in each row in the original data table and recording fragment identifications corresponding to the row in each row in a horizontal table dividing stage in advance.
3. The method of claim 1, further comprising:
in the horizontal table dividing stage, determining the corresponding fragment identification of each row in the original data table to be divided;
correspondingly storing the index attribute values of all rows in the original data table under the index attribute column and the fragment identifications corresponding to the rows;
and respectively writing the data of each row in the original data table into the fragment table represented by the fragment identifier corresponding to the row so as to complete horizontal division.
4. The method according to claim 3, wherein the determining the fragmentation identification corresponding to each row in the original data table to be partitioned comprises:
determining fragment attribute values of each row in the original data table under a fragment attribute column; the fragmentation attribute column is a column corresponding to the fragmentation attribute in the original data table; the fragment attribute is an attribute according to which the original data table is divided to obtain the fragment table in the horizontal table division stage;
and calculating the fragment attribute values of each row according to a fragment strategy to obtain fragment identifications respectively corresponding to each row.
5. The method according to claim 3, wherein the correspondingly storing the index attribute values of the rows in the original data table under the index attribute column with the slice identifier corresponding to the row comprises:
correspondingly filling index attribute values of all rows in the original data table under the index attribute columns into all rows in a dynamic index table respectively;
and recording the fragment identification corresponding to the index attribute value of the line in each line of the dynamic index table.
6. The method of claim 1, further comprising:
if the input attribute value is the fragment attribute value with the fragment attribute, calculating the input attribute value according to the fragment strategy of the horizontal spreadsheet stage to obtain a target fragment identifier;
and executing the target fragment table corresponding to the target fragment identifier and the subsequent steps.
7. The method according to any one of claims 1 to 6, further comprising:
and if the input attribute value is neither a fragment attribute value nor an index attribute value, performing data query in each fragment table according to the input attribute value.
8. An apparatus for querying data in a database, the apparatus comprising:
the data acquisition module is used for determining an attribute value input for data query in a data query stage;
the fragment identifier determining module is used for searching a target fragment identifier which is stored corresponding to the input attribute value if the input attribute value is the index attribute value with the index attribute; the index attribute is an attribute which is represented by an index attribute column in an original data table and can be used for index query; the index attribute value has a corresponding stored fragment identifier; the data of the index attribute value in the row where the original data table is located is written in the fragment table represented by the fragment identification which is correspondingly stored;
the fragment table determining module is used for determining a target fragment table corresponding to the target fragment identifier; the target fragment table is a fragment table into which target line data is written; the target line data is data of a line where the input attribute value is located in an original data table;
and the data query module is used for performing data query in the target fragment table according to the input attribute value.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076071.0A CN115168409B (en) | 2022-09-05 | 2022-09-05 | Data query method and device for database sub-tables and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076071.0A CN115168409B (en) | 2022-09-05 | 2022-09-05 | Data query method and device for database sub-tables and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115168409A true CN115168409A (en) | 2022-10-11 |
| CN115168409B CN115168409B (en) | 2023-02-28 |
Family
ID=83481466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211076071.0A Active CN115168409B (en) | 2022-09-05 | 2022-09-05 | Data query method and device for database sub-tables and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115168409B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060262976A1 (en) * | 2004-10-01 | 2006-11-23 | Hart Peter E | Method and System for Multi-Tier Image Matching in a Mixed Media Environment |
| CN110019211A (en) * | 2017-11-27 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The methods, devices and systems of association index |
| CN111488340A (en) * | 2019-01-29 | 2020-08-04 | 菜鸟智能物流控股有限公司 | Data processing method and device and electronic equipment |
| CN112380276A (en) * | 2021-01-15 | 2021-02-19 | 四川新网银行股份有限公司 | Method for querying data by non-fragment key fields after database division and table division of distributed system |
| CN113312355A (en) * | 2021-06-15 | 2021-08-27 | 北京沃东天骏信息技术有限公司 | Data management method and device |
| CN113742343A (en) * | 2021-09-01 | 2021-12-03 | 上海浦东发展银行股份有限公司 | Data splitting method, device and storage medium based on large amount of service data scenes |
| CN113849499A (en) * | 2021-08-25 | 2021-12-28 | 青岛海尔科技有限公司 | Data query method and device, storage medium and electronic device |
| CN114090638A (en) * | 2022-01-20 | 2022-02-25 | 支付宝(杭州)信息技术有限公司 | Combined data query method and device based on privacy protection |
-
2022
- 2022-09-05 CN CN202211076071.0A patent/CN115168409B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060262976A1 (en) * | 2004-10-01 | 2006-11-23 | Hart Peter E | Method and System for Multi-Tier Image Matching in a Mixed Media Environment |
| CN110019211A (en) * | 2017-11-27 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The methods, devices and systems of association index |
| CN111488340A (en) * | 2019-01-29 | 2020-08-04 | 菜鸟智能物流控股有限公司 | Data processing method and device and electronic equipment |
| CN112380276A (en) * | 2021-01-15 | 2021-02-19 | 四川新网银行股份有限公司 | Method for querying data by non-fragment key fields after database division and table division of distributed system |
| CN113312355A (en) * | 2021-06-15 | 2021-08-27 | 北京沃东天骏信息技术有限公司 | Data management method and device |
| CN113849499A (en) * | 2021-08-25 | 2021-12-28 | 青岛海尔科技有限公司 | Data query method and device, storage medium and electronic device |
| CN113742343A (en) * | 2021-09-01 | 2021-12-03 | 上海浦东发展银行股份有限公司 | Data splitting method, device and storage medium based on large amount of service data scenes |
| CN114090638A (en) * | 2022-01-20 | 2022-02-25 | 支付宝(杭州)信息技术有限公司 | Combined data query method and device based on privacy protection |
Non-Patent Citations (2)
| Title |
|---|
| FELIPE CASTRO-MEDINA ET AL.: "FRAGMENT: A Web Application for Database Fragmentation, Allocation and Replication over a Cloud Environment", 《IEEE LATIN AMERICA TRANSACTIONS》 * |
| 张国柱: "云平台架构下的银行系统数据分片策略分析", 《中国金融电脑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115168409B (en) | 2023-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11132346B2 (en) | Information processing method and apparatus | |
| TWI603211B (en) | Construction of inverted index system based on Lucene, data processing method and device | |
| CN103577440B (en) | A kind of data processing method and device in non-relational database | |
| CN111177302A (en) | Business document processing method and device, computer equipment and storage medium | |
| CN105373541A (en) | Processing method and system for data operation request of database | |
| CN110968585B (en) | Storage method, device, equipment and computer readable storage medium for alignment | |
| CN112445889A (en) | Method for storing data and retrieving data and related equipment | |
| CN111475105A (en) | Monitoring data storage method, device, server and storage medium | |
| CN114138907A (en) | Data processing method, computer device, storage medium, and computer program product | |
| CN115168499B (en) | Database table fragmentation method and device, computer equipment and storage medium | |
| CN115168409B (en) | Data query method and device for database sub-tables and computer equipment | |
| CN111666302A (en) | User ranking query method, device, equipment and storage medium | |
| CN116414854A (en) | Data asset query method, device, computer equipment and storage medium | |
| CN114579617A (en) | Data query method and device, computer equipment and storage medium | |
| CN105224596A (en) | A kind of method of visit data and device | |
| CN117236298A (en) | Report data generation method, report data generation device, computer equipment, report data generation medium and program product | |
| CN115221249B (en) | Cross-database data query method and device, computer equipment and storage medium | |
| CN114328755B (en) | Data writing method, data reading device and electronic equipment | |
| CN115118616B (en) | Display result testing method, device, computer equipment and storage medium | |
| CN116303405B (en) | Data duplicate checking method and device and computer equipment | |
| CN117708380A (en) | LSM-tree time sequence database-based query method, equipment and storage medium | |
| CN116910069A (en) | Database updating method, device, computer equipment and storage medium | |
| CN114647630A (en) | File synchronization method, information generation method, file synchronization device, information generation device, computer equipment and storage medium | |
| CN117648336A (en) | Data query method, device, computer equipment and storage medium | |
| CN117076476A (en) | Object information processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |