CN114579561A - Data processing method and device, and storage medium - Google Patents

Data processing method and device, and storage medium Download PDF

Info

Publication number
CN114579561A
CN114579561A CN202011394374.8A CN202011394374A CN114579561A CN 114579561 A CN114579561 A CN 114579561A CN 202011394374 A CN202011394374 A CN 202011394374A CN 114579561 A CN114579561 A CN 114579561A
Authority
CN
China
Prior art keywords
index
value
tree
field
data record
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011394374.8A
Other languages
Chinese (zh)
Inventor
邱海港
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN202011394374.8A priority Critical patent/CN114579561A/en
Publication of CN114579561A publication Critical patent/CN114579561A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • G06F16/2246Trees, e.g. B+trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24553Query execution of query operations

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the disclosure relates to a data processing method and device and a storage medium. The method comprises the following steps: inserting a first data record into a data table, wherein the data table comprises a first field and a second field; creating an index element for the first data record in the B-tree index according to the first index data; the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record; determining a first insertion position according to a first index value according to a B-tree insertion algorithm; creating an index element for the first data record at the first insertion location if the index element does not exist at the first insertion location; in the case where an index element already exists at the first insertion location, a second insertion location is determined according to a B-tree insertion algorithm based on the second index value and a secondary index of the index element already at the first insertion location, the index element being created for the first data record at the second insertion location.

Description

Data processing method and device, and storage medium
Technical Field
The present disclosure relates to computer technology, and more particularly, to a data processing method and apparatus, and a storage medium.
Background
The database stores a large amount of data, and in order to improve the query capability, a certain field of the data in the data table is usually used as a key word to establish an index for the data table. However, some fields in the data table are special, a large amount of repeated data exists, a large amount of repeated elements exist in the index established by using the fields, and traversal search is needed when the index is updated, so that the updating time is too long.
Disclosure of Invention
An object of the present disclosure is to provide a new data processing method and apparatus, storage medium.
According to a first aspect of the present disclosure, there is provided a data processing method, including:
inserting a first data record into a data table, wherein the data table comprises a first field and a second field;
creating an index element for the first data record in a B-tree index according to first index data; wherein the B-tree index is preconfigured such that each index element comprises a primary index corresponding to the first field and a secondary index corresponding to the second field; the first index data comprises a first index value and a second index value, the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record;
creating an index element for the first data record in a B-tree index according to the first index data, comprising:
determining a first insertion position corresponding to the first index value according to the first index value and a first-level index of an index element in the B-tree index and a B-tree insertion algorithm;
creating an index element for the first data record at a first insertion location if the index element is not present at the first insertion location;
in the case that an index element already exists at a first insertion location, a second insertion location corresponding to the second index value is determined according to a B-tree insertion algorithm based on the second index value and a secondary index of the index element already at the first insertion location, the index element being created for the first data record at the second insertion location.
Optionally, the second field is a primary key field of the data table or a ROWID field of the data table.
Optionally, the method further comprises:
deleting the first data record from the data table;
deleting the index element corresponding to the first data record from the B-tree index after the first data record is deleted;
the deleting the index element corresponding to the first data record from the B-tree index includes:
according to a B-tree searching algorithm, searching out an index element with a primary index having the same value as the first index from the B-tree index as a candidate index element;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the second index from the candidate index elements as a target index element;
deleting the target index element from the B-tree index.
Optionally, the method further comprises:
deleting the first data record from the data table;
deleting the index element corresponding to the first data record from the B-tree index after the first data record is deleted;
the deleting the index element corresponding to the first data record from the B-tree index includes:
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the second index from the B-tree index as a target index element;
deleting the target index element from the B-tree index.
Optionally, the method further comprises:
receiving a query request sent by a client, wherein the query request contains a third index value, and the third index value is a value of a first field of a query object;
according to a B-tree searching algorithm, searching out an index element with a primary index having the same value as the third index from the B-tree index as a target index element;
and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
Optionally, the method further comprises:
receiving a query request sent by a client, wherein the query request comprises a fourth index value and a fifth index value, the fourth index value is the value of a first field of a query object, and the fifth index value is the value of a second field of the query object;
according to a B-tree searching algorithm, searching out an index element with the same value of the first-level index and the fourth index from the B-tree index as a candidate index element;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the fifth index from the candidate index elements as a target index element;
and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
Optionally, the method further comprises:
receiving a query request sent by a client, wherein the query request comprises a sixth index value, and the sixth index value is a value of a second field of a query object;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the sixth index from the B-tree index as a target index element;
and acquiring a data record corresponding to the target index element and returning the data record to the client according to the pointer corresponding to the target index element.
According to a second aspect of the present disclosure, there is provided a data processing apparatus comprising:
the data record inserting module is used for inserting a first data record into a data table, and the data table comprises a first field and a second field;
the index element creating module is used for creating an index element for the first data record in the B-tree index according to the first index data; wherein the B-tree index is preconfigured such that each index element comprises a primary index corresponding to the first field and a secondary index corresponding to the second field; the first index data includes a first index value and a second index value, the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record;
the index element creation module comprises:
a first insertion position determining submodule, configured to determine, according to the first index value and a first-level index of an index element in the B-tree index, a first insertion position corresponding to the first index value according to a B-tree insertion algorithm;
a first creating submodule, configured to create an index element for the first data record at a first insertion position if the index element does not exist at the first insertion position;
the second insertion position determining submodule is used for determining a second insertion position corresponding to a second index value according to a B-tree insertion algorithm according to the second index value and a secondary index of an existing index element at a first insertion position under the condition that the index element already exists at the first insertion position;
a second creating submodule for creating an index element for said first data record at said second insertion position if an index element already exists at the first insertion position.
According to a third aspect of the present disclosure, there is provided a data processing apparatus comprising a processor and a memory; the memory has stored therein computer-executable instructions that, when executed by the processor, implement the method described in the first aspect of the present disclosure.
According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method described in the first aspect of the present disclosure.
The data processing method and device and the storage medium provided by the embodiment of the disclosure can establish the B-tree index for the data table in a mode of combining the primary index and the secondary index, and can further search the secondary index when the primary index has a repeated condition, thereby reducing the search times and improving the index updating speed.
Other features of the present disclosure and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a schematic diagram of a hardware configuration of a database server that may be used to implement embodiments of the present disclosure;
FIG. 2 is a flow chart of a data processing method of an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of building a B-tree index using a first field of a data table;
FIG. 4 is a B-tree index schematic of an embodiment of the disclosure;
FIG. 5 is a schematic diagram of inserting index elements into a B-tree index according to an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of inserting index elements into a B-tree index, according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of a data processing apparatus of an embodiment of the present disclosure.
Fig. 8 is a schematic diagram of a data processing apparatus of an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
< hardware configuration >
FIG. 1 illustrates a database system that can be used to implement embodiments of the present disclosure. The client 103 may access the database server 101, send an operation instruction to the database, and the database server 101 performs corresponding operations, such as adding, searching, modifying, and deleting operations, on the data table according to the operation instruction.
The configuration of database server machine 101 may include, but is not limited to: processor 1011, memory 1012, interface 1013, communication device 1014, input device 1015, output device 1016. The processor 1011 may include, but is not limited to, a central processing unit CPU, a microprocessor MCU, or the like. The memory 1012 may include, but is not limited to, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. Interface device 1013 may include, but is not limited to, a USB interface, a serial interface, a parallel interface, and the like. The communication device 1014 is capable of wired or wireless communication, for example, and may specifically include WiFi communication, bluetooth communication, 2G/3G/4G/5G communication, and the like. Input devices 1015 include, but are not limited to, a keyboard, a mouse, and the like. Output device 1016 includes, but is not limited to, a display screen or the like. The configuration of the database server machine 101 may include only some of the above devices.
In embodiments applicable to this description, memory 1012 of database server machine 101 is configured to store instructions for controlling processor 1011 to operate in support of implementing a method according to any of the embodiments of this description. The skilled person can design the instructions according to the solution disclosed in the present specification. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.
It should be understood by those skilled in the art that although a plurality of devices of the database server 101 are shown in fig. 1, the database server 101 of the present embodiment may refer to only some of the devices, for example, only the processor 1011, the memory 1012 and the communication device 1014.
The hardware configuration shown in fig. 1 is merely illustrative and is in no way intended to limit the present disclosure, its application, or uses.
< method example >
The embodiment provides a data processing method. The method may be implemented, for example, by database server 101 shown in fig. 1.
Referring to fig. 2, the data processing method includes steps S102-S104.
Step S102, inserting a first data record into a data table, wherein the data table comprises a first field and a second field.
And step S104, creating an index element for the first data record in the B-tree index according to the first index data. The first index data includes a first index value and a second index value, the first index value being a value of a first field of the first data record, the second index value being a value of a second field of the first data record. The B-tree index is preconfigured such that each index element includes a primary index corresponding to the first field and a secondary index corresponding to the second field.
The data table typically includes a primary key or ROWID. In a data table, a primary key corresponds to a combination of one or more columns in the data table, i.e., a primary key field may be a field or a combination of fields, and the primary key is used for uniquely identifying a row of data (i.e., a data record) in the data table. The ROWID column is a pseudo-column of the data table that uniquely identifies a row of data (i.e., a data record) and is automatically generated when a data record is inserted into the database. The data in the primary key field or ROWID field of the data table has no duplication, and the other fields have the possibility of data duplication.
An example of a data table is given below:
ROWID field Field F1 Field F2 Field F3
1 21 31 147
2 25 32 146
3 17 33 133
4 18 34 134
5 18 35 135
6 20 36 136
7 22 37 137
8 23 38 138
9 23 39 139
10 23 40 132
11 26 41 141
12 27 42 142
13 27 43 131
14 28 44 144
Data table 1
In this data table 1, 4 fields are included, which are a ROWID field, a field F1, a field F2, and a field F3, respectively. It can be seen that in field F1 of this data table 1, there are two data "18", 3 data "23", and 2 data "27".
In the disclosed embodiment, the B-tree index is an index created for a data table. The B-tree is a tree suitable for searching proposed by r.bayer and e.mccreeight, also called a B-tree.
If only the data of field F1 is used as an index for a data record, the B-tree index created from data Table 1 is shown in FIG. 3. There are 14 data records in the data table 1, and the B-tree index correspondingly includes 14 index elements, and a pointer (not shown in fig. 3) should be included in the B-tree index. It can be seen that, since there are 3 "23" in the field F1 of the data table 1 and there are 3 index elements "23" in the B-tree index, in this case, if the data record with the ROWID of "10" is deleted from the data table 1, 3 times of traversal are required to find the index element corresponding to the deleted data record, and then the index element is deleted from the B-tree index. The process results in a slow update speed of the B-tree index due to the large number of queries.
In the embodiment of the present disclosure, the index of the data record is set to a combination type, and includes a primary index and a secondary index, the primary index of the data record is data of a first field of the data record, and the secondary index of the data record is data of a second field of the data record. That is, the index type of the data table is pre-specified as a combination type, and the B-tree index is configured such that each index element includes a primary index corresponding to the first field and a secondary index corresponding to the second field.
In one example of the present disclosure, the first field may be a search field that the user sets according to his needs. The second field may be a primary key field of the data table or a ROWID field of the data table, in which case there is no duplication of data in the second field of the data table.
For example, for this data table 1, the first field is field F1 and the second field is the ROWID field. By inserting an index element into a blank B-tree index configured in advance according to the data records in the data table 1, the B-tree index shown in fig. 4 can be obtained. There are 14 data records in the data table 1, and the B-tree index correspondingly includes 14 index elements, and a pointer (not shown in fig. 4) should be included in the B-tree index.
The following describes the process of creating an index element for the first data record in the B-tree index, i.e., step S104. Step S104 includes steps S1042-S1048.
Step S1042, determining a first insertion position corresponding to the first index value according to the first index value and the first-level index of the index element in the B-tree index and according to a B-tree insertion algorithm.
In this step, a first insertion position is determined according to a comparison relationship between the first index value and the primary index of the index element in the B-tree index according to a B-tree insertion algorithm.
Step S1044, under the condition that the index element does not exist in the first insertion position, creating an index element for the first data record at the first insertion position.
Step S1046, in a case that the index element already exists in the first insertion position, determining a second insertion position corresponding to the second index value according to the second index value and the secondary index of the index element already existing in the first insertion position and according to a B-tree insertion algorithm.
In this step, the second insertion position is determined according to the B-tree insertion algorithm according to the comparison relationship of the second index value and the secondary index of the index element existing at the first insertion position.
Step S1048, creating an index element for the first data record at the second insertion position.
Two specific examples are given below based on the B-tree index shown in data Table 1 and FIG. 4:
< example 1>
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. Assume that the first data record inserted into data table 1 is as follows, the data of the ROWID field is "15", the data of the field F1 is "30", the data of the field F2 is "35", and the value of the field F3 is "145". The first index data includes a first index value of "30" and a second index value of "15".
ROWID field Field F1 Field F2 Field F3
15 30 35 145
Referring to fig. 5, according to the B-tree insertion algorithm, the first insertion position may be determined to be position a according to a comparison relationship of the first index value "30" and the primary index of the index element in the B-tree index. If no index element exists at location A, an index element is created for the first data record at location A, the index element having a primary index of "30" and a secondary index of "15".
< example 2>
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. Assume that the first data record inserted into data table 1 is as follows, the data of the ROWID field is "15", the data of the field F1 is "23", the data of the field F2 is "32", and the data of the field F3 is "143". The first index data includes a first index value '23' and a second index value '15'.
ROWID field Field F1 Field F2 Field F3
15 23 32 143
Referring to fig. 6, according to the B-tree insertion algorithm, the first insertion position is determined to be position B according to the comparison relationship between the first index value "23" and the primary index of the index element in the B-tree index. And 3 index elements already exist in the position B, and according to the B-tree insertion algorithm, the second insertion position can be determined to be the position C according to the comparison relationship between the second index value '15' and the secondary indexes of the 3 index elements already existing in the position B. An index element is created for the first data record at position C, with a primary index of "23" and a secondary index of "15".
When a data record is deleted from the data table, the index element in the B-tree index corresponding to the deleted data record should be deleted.
In the embodiment of the present disclosure, the method may further include steps S202-S204.
Step S202, deleting the first data record from the data table.
And step S204, after the first data record is deleted, deleting the index element corresponding to the first data record from the B-tree index.
The following describes a process of deleting an index element corresponding to the first data record from the B-tree index, that is, the step S204, with 2 examples.
< example 1>
Deleting the index element corresponding to the first data record from the B-tree index includes steps S2041-S2043.
Step S2041, according to the B-tree searching algorithm, searching out the index element with the first index value being the same as the first index value in the B-tree index as a candidate index element.
Step S2042, according to a B-tree searching algorithm, searching out an index element with the same secondary index value as the second index value from the candidate index elements as a target index element;
and step S2043, deleting the target index element from the B-tree index.
Taking data table 1 as an example, the first field is field F1, and the second field is a ROWID field. Assume that the first data record inserted into data table 1 is as follows, the data of the ROWID field is "15", the data of the field F1 is "23", the data of the field F2 is "32", and the data of the field F3 is "143". The first index data includes a first index value '23' and a second index value '15'.
ROWID field Field F1 Field F2 Field F3
15 23 32 143
After deleting the data record with ROWID of "15" from the data table 1, 4 candidate index elements with primary index of "23" can be found according to the B-tree lookup algorithm. The first candidate index element has a level two index of "8", the second candidate index element has a level one index of "9", the third candidate index element has a level one index of "10", and the fourth candidate index element has a level one index of "15". According to the B-tree searching algorithm, binary searching is carried out on 4 candidate index elements by utilizing the B-tree characteristic to search the index element with the secondary index of '15' as a target index element, and then the target index element is deleted.
< example 2>
If the second field is a primary key field of the data table or a ROWID field of the data table, there is no duplication of data of the second field in the data table in this case. The index element corresponding to the first data record is deleted from the B-tree index, which may include steps S2045-S2046.
Step S2045, according to the B-tree searching algorithm, searching out an index element with the same secondary index value as the second index value in the B-tree index as a target index element.
And step S2046, deleting the target index element from the B-tree index.
Because the data of the second field in the data table has no duplication, the secondary index in the B-tree index also has no duplication, the index element corresponding to the deleted data record can be directly determined by using the secondary index as the target index element, and then the target index element is deleted.
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. Assume that the first data record inserted into data table 1 is as follows, the data of the ROWID field is "15", the data of the field F1 is "23", the data of the field F2 is "32", and the data of the field F3 is "143". The first index data includes a first index value '23' and a second index value '15'.
ROWID field Field F1 Field F2 Field F3
15 23 32 143
After deleting the data record with ROWID of "15" from the data table 1, according to the B-tree lookup algorithm, the index element with the secondary index of "15" is found out as the target index element, and then the target index element is deleted.
The following 3 examples illustrate how to use the B-tree index to find a data record.
< example 1>
The process of using the B-tree index to lookup data records may include steps S302-S306.
Step S302, receiving a query request sent by a client, wherein the query request contains a third index value, and the third index value is a value of a first field of a query object.
And step S304, according to the B-tree searching algorithm, searching out an index element with the same first-level index and third index value in the B-tree index as a target index element.
And S306, acquiring the data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
In this way, all data records that meet the query request can be found.
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. The value of the first field of the query object is "18", that is, the third index value is 18, there are 2 target index elements found from the B-tree index, and there are 2 corresponding data records, specifically:
ROWID field Field F1 Field F2 Field F3
4 18 34 134
5 18 35 135
< example 2>
The process of using the B-tree index to lookup data records may include steps S402-S408.
Step S402, receiving a query request sent by a client, wherein the query request comprises a fourth index value and a fifth index value, the fourth index value is a value of a first field of a query object, and the fifth index value is a value of a second field of the query object.
And S404, according to the B-tree searching algorithm, searching out an index element with the same value of the first-level index and the fourth index from the B-tree index as a candidate index element.
Step S406, according to the B-tree searching algorithm, searching out an index element with the secondary index having the same value as the fifth index from the candidate index elements as a target index element.
And step S408, acquiring the data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
In this way, only one data record can be found which meets the query request.
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. If the first field of the query object has a value of "18" and the second field of the query object has a value of "4", i.e., the fourth index value is "18" and the fifth index value is "4", then there are 2 candidate index elements found from the B-tree index. Searching index elements with the same secondary index value as the fifth index value by using the B-tree characteristic in the 2 candidate index elements in a binary search manner, wherein the number of the finally determined target index elements is 1, and the number of corresponding data records is 1, specifically:
ROWID field Field F1 Field F2 Field F3
4 18 34 134
< example 3>
If the second field is a primary key field of the data table or a ROWID field of the data table, there is no duplication of data of the second field in the data table in this case. If the second field is a primary key field of a data table or a ROWID field of a data table, there is no duplication of data of the second field in this case. The process of using the B-tree index to lookup the data record may include steps S502-S506.
Step S502, receiving a query request sent by the client, wherein the query request contains a sixth index value, and the sixth index value is a value of a second field of the query object.
Step S504, according to the B-tree searching algorithm, searching out the index element with the same secondary index value as the sixth index value in the B-tree index as the target index element.
And S506, acquiring the data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
Taking data table 1 as an example, the first field is field F1, and the second field is the ROWID field. The value of the second field of the query object is "12", that is, the sixth index value is "12", there are 1 target index element found from the B-tree index, and there are 1 corresponding data record, specifically:
ROWID field Field F1 Field F2 Field F3
12 27 42 142
The data processing method, the B-tree insertion algorithm and the B-tree search algorithm provided by the embodiment of the disclosure are realized based on a binary search principle, and when N repeated data exist in a first field, the search times of a second field can be reduced from N times to log2And (5) N times.
According to the data processing method provided by the embodiment of the disclosure, the B-tree index can be established for the data table in a mode of combining the first-level index and the second-level index, and when the first-level index has repeated conditions, the second-level index can be further searched, so that the searching times are reduced, and the index updating speed is increased.
The data processing method provided by the embodiment of the disclosure optimizes the storage format of the B-tree index of the database, and improves the performance of database data updating.
< apparatus embodiment >
Referring to fig. 7, an embodiment of the present disclosure provides a data processing apparatus. The data processing device 30 comprises the following modules:
a data record inserting module 10, configured to insert a first data record into a data table, where the data table includes a first field and a second field.
An index element creating module 20, configured to create an index element for the first data record in the B-tree index according to the first index data. The B-tree index is preconfigured such that each index element includes a primary index corresponding to the first field and a secondary index corresponding to the second field. The first index data includes a first index value and a second index value, the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record.
The index element creation module 20 includes:
the first insertion position determining submodule 21 is configured to determine, according to the first index value and a first-level index of an index element in the B-tree index, a first insertion position corresponding to the first index value according to a B-tree insertion algorithm.
A first creating submodule 22 for creating an index element for said first data record at a first insertion position in case no index element is present at the first insertion position.
The second insertion position determining submodule 23 is configured to, in a case where an index element already exists at a first insertion position, determine, according to the B-tree insertion algorithm, a second insertion position corresponding to the second index value according to the second index value and a secondary index of the index element already existing at the first insertion position.
A second creating submodule 24 for creating an index element for said first data record at said second insertion position, in case an index element already exists at the first insertion position.
Optionally, the second field is a primary key field of the data table or a ROWID field of the data table.
In one example, the data processing apparatus 30 may further include a data record deletion module and an index element deletion module.
And the data record deleting module is used for deleting the first data record from the data table.
And the index element deleting module is used for deleting the index element corresponding to the first data record from the B-tree index after the first data record is deleted.
In one example, the index element deletion module includes a first lookup submodule, a second lookup submodule, and a first deletion submodule.
The first searching submodule is used for searching out an index element with a primary index having the same value as the first index from the B-tree index as a candidate index element according to a B-tree searching algorithm;
the second searching submodule is used for searching out an index element with a secondary index having the same value as the second index from the candidate index elements as a target index element according to a B-tree searching algorithm;
a first deletion submodule, configured to delete the target index element from the B-tree index.
In one example, the index element deletion module includes a third search submodule and a second deletion submodule.
The third searching submodule is used for searching out an index element with the same secondary index value as the second index value in the B-tree index as a target index element according to a B-tree searching algorithm;
and the second deleting submodule is used for deleting the target index element from the B-tree index.
In one example, the data processing apparatus 30 may further include a first data query module.
The first data query module is used for receiving a query request sent by a client, wherein the query request contains a third index value, and the third index value is a value of a first field of a query object; according to a B-tree searching algorithm, searching out an index element with the same primary index and the third index value in the B-tree index as a target index element; and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
In one example, the data processing device 30 may also include a second data query module.
The second data query module is used for receiving a query request sent by a client, wherein the query request comprises a fourth index value and a fifth index value, the fourth index value is a value of a first field of a query object, and the fifth index value is a value of a second field of the query object; according to a B-tree searching algorithm, searching out index elements with the same first-level index and fourth index values in the B-tree index as candidate index elements; according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the fifth index from the candidate index elements as a target index element; and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
In one example, the data processing apparatus 30 may further include a third data query module.
A third data query module, configured to receive a query request sent by a client, where the query request includes a sixth index value, and the sixth index value is a value of a second field of a query object; according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the sixth index from the B-tree index as a target index element; and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
Referring to fig. 8, an embodiment of the present disclosure provides a data processing apparatus. The data processing device 50 comprises a processor 51 and a memory 52. The memory 52 stores computer-executable commands, which when executed by the processor 51, implement the data processing method described in the embodiments of the method of the present disclosure.
< readable storage Medium embodiment >
The present embodiment provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the data processing methods described in the method embodiments of the present disclosure.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
Computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims (10)

1. A data processing method, comprising:
inserting a first data record into a data table, wherein the data table comprises a first field and a second field;
creating an index element for the first data record in a B-tree index according to first index data; wherein the B-tree index is pre-configured such that each index element comprises a primary index corresponding to the first field and a secondary index corresponding to the second field; the first index data comprises a first index value and a second index value, the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record;
creating an index element for the first data record in a B-tree index according to the first index data, comprising:
determining a first insertion position corresponding to the first index value according to the first index value and a first-level index of an index element in the B-tree index and a B-tree insertion algorithm;
creating an index element for the first data record at a first insertion location if the index element is not present at the first insertion location;
in the case that an index element already exists at a first insertion location, a second insertion location corresponding to the second index value is determined according to a B-tree insertion algorithm based on the second index value and a secondary index of the index element already at the first insertion location, the index element being created for the first data record at the second insertion location.
2. The method of claim 1, wherein the second field is a primary key field of the data table or a ROWID field of the data table.
3. The method of claim 1, further comprising:
deleting the first data record from the data table;
deleting the index element corresponding to the first data record from the B-tree index after the first data record is deleted;
the deleting the index element corresponding to the first data record from the B-tree index includes:
according to a B-tree searching algorithm, searching out index elements with the same primary index value as the first index value in the B-tree index as candidate index elements;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the second index from the candidate index elements as a target index element;
deleting the target index element from the B-tree index.
4. The method of claim 2, further comprising:
deleting the first data record from the data table;
deleting the index element corresponding to the first data record from the B-tree index after the first data record is deleted;
the deleting the index element corresponding to the first data record from the B-tree index includes:
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the second index from the B-tree index as a target index element;
deleting the target index element from the B-tree index.
5. The method of claim 1, further comprising:
receiving a query request sent by a client, wherein the query request contains a third index value, and the third index value is a value of a first field of a query object;
according to a B-tree searching algorithm, searching out an index element with a primary index having the same value as the third index from the B-tree index as a target index element;
and acquiring a data record corresponding to the target index element and returning the data record to the client according to the pointer corresponding to the target index element.
6. The method of claim 1, further comprising:
receiving a query request sent by a client, wherein the query request comprises a fourth index value and a fifth index value, the fourth index value is a value of a first field of a query object, and the fifth index value is a value of a second field of the query object;
according to a B-tree searching algorithm, searching out index elements with the same first-level index and fourth index values in the B-tree index as candidate index elements;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the fifth index from the candidate index elements as a target index element;
and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
7. The method of claim 2, further comprising:
receiving a query request sent by a client, wherein the query request comprises a sixth index value, and the sixth index value is a value of a second field of a query object;
according to a B-tree searching algorithm, searching out an index element with a secondary index having the same value as the sixth index from the B-tree index as a target index element;
and acquiring a data record corresponding to the target index element according to the pointer corresponding to the target index element, and returning the data record to the client.
8. A data processing apparatus, comprising:
the data record inserting module is used for inserting a first data record into a data table, and the data table comprises a first field and a second field;
the index element creating module is used for creating an index element for the first data record in the B-tree index according to the first index data; wherein the B-tree index is preconfigured such that each index element comprises a primary index corresponding to the first field and a secondary index corresponding to the second field; the first index data comprises a first index value and a second index value, the first index value is a value of a first field of the first data record, and the second index value is a value of a second field of the first data record;
the index element creation module comprises:
a first insertion position determining submodule, configured to determine, according to the first index value and a first-level index of an index element in the B-tree index, a first insertion position corresponding to the first index value according to a B-tree insertion algorithm;
a first creating submodule, configured to create an index element for the first data record at a first insertion position if the index element does not exist at the first insertion position;
the second insertion position determining submodule is used for determining a second insertion position corresponding to a second index value according to a B-tree insertion algorithm according to the second index value and a secondary index of an existing index element at a first insertion position under the condition that the index element already exists at the first insertion position;
a second creating submodule for creating an index element for said first data record at said second insertion position if an index element already exists at the first insertion position.
9. A data processing apparatus comprising a processor and a memory; the memory has stored therein computer-executable instructions that, when executed by the processor, implement the method of any of claims 1-7.
10. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, perform the method of any of claims 1-7.
CN202011394374.8A 2020-12-02 2020-12-02 Data processing method and device, and storage medium Pending CN114579561A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011394374.8A CN114579561A (en) 2020-12-02 2020-12-02 Data processing method and device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011394374.8A CN114579561A (en) 2020-12-02 2020-12-02 Data processing method and device, and storage medium

Publications (1)

Publication Number Publication Date
CN114579561A true CN114579561A (en) 2022-06-03

Family

ID=81768699

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011394374.8A Pending CN114579561A (en) 2020-12-02 2020-12-02 Data processing method and device, and storage medium

Country Status (1)

Country Link
CN (1) CN114579561A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115994145A (en) * 2023-02-09 2023-04-21 中国证券登记结算有限责任公司 Method and device for processing data
WO2024016594A1 (en) * 2022-07-19 2024-01-25 天翼云科技有限公司 Pseudo column implementation method and apparatus, electronic device, and storage medium
WO2024027464A1 (en) * 2022-08-05 2024-02-08 华为技术有限公司 Method and device for processing global secondary index

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024016594A1 (en) * 2022-07-19 2024-01-25 天翼云科技有限公司 Pseudo column implementation method and apparatus, electronic device, and storage medium
WO2024027464A1 (en) * 2022-08-05 2024-02-08 华为技术有限公司 Method and device for processing global secondary index
CN115994145A (en) * 2023-02-09 2023-04-21 中国证券登记结算有限责任公司 Method and device for processing data
CN115994145B (en) * 2023-02-09 2023-08-22 中国证券登记结算有限责任公司 Method and device for processing data

Similar Documents

Publication Publication Date Title
US11194779B2 (en) Generating an index for a table in a database background
CN107526777B (en) Method and equipment for processing file based on version number
CN111247518B (en) Method and system for database sharding
CN114579561A (en) Data processing method and device, and storage medium
CN108228646B (en) Method and electronic device for accessing data
CN107704202B (en) Method and device for quickly reading and writing data
CN103810212A (en) Automated database index creation method and system
US20160239538A1 (en) Method for processing a database query
CN111858577B (en) Method, apparatus and computer program product for storage management
US10984050B2 (en) Method, apparatus, and computer program product for managing storage system
US20180181581A1 (en) Systems and methods for implementing object storage and fast metadata search using extended attributes
CN113760971A (en) Method, computing device and storage medium for retrieving data of a graph database
CN110502317B (en) Transaction management method and device
KR20200094074A (en) Method, apparatus, device and storage medium for managing index
CN111190899B (en) Buried data processing method, buried data processing device, server and storage medium
CN111488341B (en) Database index management method and device and electronic equipment
CN108694172B (en) Information output method and device
CN108959294B (en) Method and device for accessing search engine
CN113553458A (en) Data export method and device in graph database
CN113742332A (en) Data storage method, device, equipment and storage medium
CN108255486B (en) View conversion method and device for form design and electronic equipment
CN113590543A (en) Method, apparatus and computer program product for information processing
CN111666278A (en) Data storage method, data retrieval method, electronic device and storage medium
CN112905587A (en) Database data management method and device and electronic equipment
CN112307061A (en) Method and device for querying data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination