CN109739857B - Data distributed writing method and device under high concurrency, terminal and storage medium - Google Patents
Data distributed writing method and device under high concurrency, terminal and storage medium Download PDFInfo
- Publication number
- CN109739857B CN109739857B CN201811629694.XA CN201811629694A CN109739857B CN 109739857 B CN109739857 B CN 109739857B CN 201811629694 A CN201811629694 A CN 201811629694A CN 109739857 B CN109739857 B CN 109739857B
- Authority
- CN
- China
- Prior art keywords
- key
- value
- virtual
- index table
- reverse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a data distributed writing method under high concurrency, which is applied to a terminal, wherein a forward index table and a reverse index table are stored in the terminal, and the method comprises the following steps: when a write-in instruction of the key-value pair is received, calculating a reverse position index in a first virtual key corresponding to a first virtual value of a corresponding value in a reverse index table where the key is located; when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index; calculating a forward position index in a corresponding second virtual value in a forward index table in which the value is to be located; and when the version number of the forward index table is not changed, writing the value into the array corresponding to the sub-key with the forward position index. The invention also discloses a data distributed writing device under high concurrency, a terminal and a storage medium. The invention can solve the technical problem of distributed writing of many-to-many data and improve the efficiency of accessing data under the condition of high concurrency.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a data distributed writing method, a device, a terminal and a storage medium under high concurrency.
Background
With the deep development of the internet technology and the rise of the distributed computing technology, the amount of data to be processed by a database management system is more and more huge, and although the consistency of indexes can be ensured by a traditional relational database based on single-machine operation, the reading and writing performance of the database is too low, and a large amount of computing power of a database cluster is occupied. For this reason, various technical attempts have been made. Various distributed database systems have been developed including BigTable, HBase, MongoDB, Cassandra, and Redis, among others. These distributed database systems may include hundreds or thousands of nodes with the ability to process megabytes (TB) to beats (PB) of data.
However, the existing distributed databases are all based on a single data table, and cannot effectively break the bottleneck of the traditional relational database in data writing efficiency under high concurrency.
Therefore, it is necessary to provide a technique capable of solving the problem of distributed writing of many-to-many type data so as to improve the efficiency of accessing data in high concurrency.
Disclosure of Invention
The invention mainly aims to provide a data distributed writing method, a device, a terminal and a storage medium under high concurrency, and aims to solve the technical problem of distributed writing of many-to-many data so as to improve the efficiency of accessing data under the high concurrency condition.
In order to achieve the above object, a first aspect of the present invention provides a data distributed writing method under high concurrency, which is applied to a terminal, where a forward index table and a reverse index table are stored in advance in the terminal, and the method includes:
when a write-in instruction of a key-value pair is received, recording the version number of the reverse index table and the version number of the forward index table;
obtaining a first virtual value in the reverse index table corresponding to a value in the key-value pair, and calculating a reverse position index in a first virtual key corresponding to the first virtual value in which a key in the key-value pair should be located;
when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index in the first virtual key;
acquiring a second virtual key corresponding to a key in the key-value pair in the forward index table, and calculating a forward position index in a second virtual value corresponding to the second virtual key in which the value in the key-value pair should be located;
and when the version number of the forward index table is not changed, writing the value into an array corresponding to the sub-key with the forward position index in the second virtual value.
Preferably, the calculating the reverse position index in the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located comprises:
calculating a first hash value of the key;
and determining a reverse position index in the first virtual key where the key is to be located according to a corresponding relation between a preset first position sequence number and a first hash value, wherein the reverse position index comprises the first position sequence number.
Preferably, the calculating a forward position index in a second virtual value corresponding to the second virtual key at which a value in the key-value pair should be located comprises:
calculating a second hash value of the value;
and determining a forward position index in the second virtual value in which the value is to be located according to a corresponding relation between a preset second position sequence number and a second hash value, wherein the forward position index comprises the second position sequence number.
Preferably, when the writing of the key-value pair into the reverse index table is successful but the writing of the key-value pair into the forward index table fails, the method further comprises:
rolling back the reverse index table to maintain data consistency in the forward index table and the reverse index table;
the rolling back the reverse index table comprises: and deleting all the sub-values and the corresponding keys written into the first virtual key when the current version number of the reverse index table is the same as the version number when the reverse index table is successfully written.
Preferably, before the obtaining the first virtual value corresponding to the value in the key-value pair in the inverted index table, the method further comprises:
determining whether a first virtual value corresponding to a value in the key-value pair exists in the inverted index table;
when it is determined that there is no first virtual value corresponding to a value in the key-value pair, creating the first virtual value, where the first virtual value includes a first number of sub-values into which the value and a preset value are split.
Preferably, after the calculating of the inverted position index in the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located, the method further comprises:
determining whether a sub-value having the reverse position index exists in the first virtual key;
when it is determined that there is no sub-value having the reverse position index, a sub-value having the reverse position index is created while adding 1 to the first number.
Preferably, before the writing the key into the array corresponding to the sub-value having the inverted position index in the first virtual key, the method further comprises:
judging whether the key exists in an array corresponding to the sub-value with the reverse position index in the first virtual key;
when the key is determined to be present, no operation is performed and a result of successful writing is returned.
Preferably, before the obtaining a second virtual key in the forward index table corresponding to a key in the key-value pair, the method further includes:
judging whether a second virtual key corresponding to the key in the key-value pair exists in the forward index table;
and when determining that a second virtual key corresponding to the key in the key-value pair does not exist, creating the second virtual key, wherein the second virtual key comprises a second number of sub-keys obtained by splitting the key and a preset key.
Preferably, after said calculating a forward position index in a second virtual value corresponding to the second virtual key in which a value in the key-value pair should be located, the method further comprises:
judging whether a sub-key with the forward position index exists in the second virtual value;
when it is determined that there is no sub-key with the forward position index, creating a sub-key with the forward position index while adding 1 to the second number.
Preferably, before the writing the value into the array corresponding to the sub-key having the forward position index in the second virtual value, the method further includes:
judging whether the value exists in an array corresponding to the sub-key with the forward position index in the second virtual value;
when the value is determined to exist, no operation is performed and a result of a write success is returned.
In order to achieve the above object, a second aspect of the present invention provides a data distributed writing apparatus with high concurrency, which operates in a terminal, where a forward index table and a reverse index table are stored in advance, and the apparatus includes:
a receiving module, configured to record a version number of the reverse index table and a version number of the forward index table when a write instruction of a key-value pair is received;
a first obtaining module, configured to obtain a first virtual value corresponding to a value in the key-value pair in the reverse index table, and calculate a reverse position index in a first virtual key corresponding to the first virtual value where a key in the key-value pair should be located;
a first writing module, configured to write the key into an array corresponding to a sub-value having the reverse position index in the first virtual key when the version number of the reverse index table is not changed;
a second obtaining module, configured to obtain a second virtual key in the forward index table corresponding to a key in the key-value pair, and calculate a forward position index in a second virtual value corresponding to the second virtual key in which a value in the key-value pair should be located;
and a second writing module, configured to write the value into an array corresponding to the sub-key with the forward position index in the second virtual value when the version number of the forward index table is not changed.
In order to achieve the above object, a third aspect of the present invention provides a terminal, where the terminal includes a memory and a processor, the memory stores a data distributed write program under high concurrency that is executable on the processor, and the data distributed write program under high concurrency realizes the data distributed write method under high concurrency when executed by the processor.
To achieve the above object, a fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a data distributed write program under high concurrency, where the data distributed write program under high concurrency is executable by one or more processors to implement the data distributed write method under high concurrency.
According to the data distributed writing method, the data distributed writing device, the data distributed writing terminal and the data distributed writing storage medium under high concurrency, when a writing instruction of a key-value pair is received, a reverse position index in a first virtual key corresponding to a first virtual value of a corresponding value in a reverse index table where the key is located is calculated; when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index; calculating a forward position index in a second virtual value corresponding to a second virtual key of a key in the key-value pair in a forward index table in which the value should be located; and when the version number of the forward index table is not changed, writing the value into the array corresponding to the sub-key with the forward position index. By writing the key-value pair to be written into the reverse index table first and then into the forward index table, the writing efficiency of high-concurrency, many-to-many data can be effectively ensured, and the consistency of the forward index and the reverse index can be ensured.
Drawings
FIG. 1 is a flowchart illustrating a data distributed writing method under high concurrency according to a first embodiment of the present invention;
FIG. 2 is a diagram of an inverted index table according to an embodiment of the present invention;
FIG. 3 is a diagram of a forward index table according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data distributed writing apparatus under high concurrency according to a second embodiment of the present invention;
fig. 5 is a schematic diagram of an internal structure of a terminal according to a fourth embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first" and "second" in the description and claims of the present application and the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
To facilitate a clearer understanding of the technical solution of the present invention, the following terms will now be briefly introduced:
the key (key) in the database is also called a keyword, the key (key) is used as an index of an element, and the key (key) is represented by a Serial Number (SN) in the embodiment;
the value (value) indicates the stored and read data, and in this embodiment, the value (value) corresponding to the SN is indicated by a Globally Unique Identifier (GUID).
One line of data in the database is a corresponding relation, SN- > GUID is a forward index, and GUID- > SN is a reverse index. In many-to-many databases, the same SN may correspond to multiple different GUIDs, and different SNs may correspond to multiple same GUIDs.
Example one
Fig. 1 is a flowchart illustrating a data distributed writing method under high concurrency according to a first embodiment of the present invention.
The data distributed writing method under the high concurrency can be applied to a terminal and can also be applied to a database server. For a terminal needing to create the distributed database index, the function of creating the distributed database index provided by the method of the invention can be directly integrated on the terminal. Or an interface of the function of creating the distributed database index is provided in the form of a Software Development Kit (SDK), and the terminal realizes data distributed writing under high concurrency through the provided interface.
As shown in fig. 1, the data distributed writing method under high concurrency specifically includes the following steps, and the order of the steps in the flowchart may be changed and some steps may be omitted according to different requirements.
S11: and recording the version number of the reverse index table and the version number of the forward index table when a write instruction of the key-value pair is received.
In this embodiment, a forward index table and a reverse index table are pre-stored in the terminal, and data in the pre-stored forward index table and data in the pre-stored reverse index table are both in the form of virtual key value pairs. The forward index table facilitates operations related to forward indexing and the reverse index table facilitates operations related to reverse indexing.
In this embodiment, the key-value pairs to be written are described by taking many-to-many relational data as an example, and in the many-to-many relational data, the same key may correspond to a plurality of different values, and the same value may correspond to a plurality of different keys.
The user can request to write at least one key-value pair through the upper layer logic call interface or the man-machine interaction interface or other modes. And when the terminal receives a write-in instruction of the key-value pair, firstly recording the version number of the reverse index table and the version number of the forward index table when the write-in instruction is received.
The version number is a self-increment that is incremented after each write is successful. Since there may be many persons requesting to write the key-value pair at the same time in the case of high concurrency, in order to solve the problem of data contention in the high concurrency, it is necessary to read the data in the entire reverse index table and forward index table to avoid the situation of repeatedly writing data, and therefore before writing the key-value pair, it is checked whether the version numbers of the reverse index table and forward index table are changed. If the version number of the reverse index table and/or the forward index table is changed, indicating that other requests have completed writing, the index table will not be updated, but retried.
S12: and acquiring a first virtual value corresponding to the value in the key-value pair in the reverse index table, and calculating a reverse position index in a first virtual key corresponding to the first virtual value in which the key in the key-value pair is to be located.
In this embodiment, because the requirement on the correctness of the data of the forward index is higher, the key-value pair is written into the reverse index table at first, and then the key-value pair is written into the forward index table, so that it can be ensured that the feedback of the data is more sensitive once an error occurs when the key-value pair is written into the forward index table.
The data in the inverted index table is in the form of a first virtual key-value pair, the first virtual value is composed of a value and a first number of sub-values into which the value is split, the first virtual key is composed of the sub-values and keys corresponding to each of the sub-values, and the number of the sub-values corresponds to the number of the keys. That is, the value in the relational database, at this time, is virtualized in the inverted index table into a first number of values and sub-values corresponding to the value (the first number of values and sub-values of values taken together may be referred to as a first info); the key of the value in the relational database is virtualized into the sub-value (the sub-value may be referred to as the first shard) and the key corresponding to the sub-value in the inverted index table, and the key of the sub-value is stored in the form of an array, and the inverted index table is shown in fig. 2.
Preferably, the calculating the reverse position index in the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located comprises:
1) calculating a first hash value of the key;
in this embodiment, the first hash value of the key may be calculated by using a hash function. The hash function is prior art, and the present invention is not described in detail herein.
2) And determining a reverse position index in the first virtual key where the key is to be located according to a corresponding relation between a preset first position sequence number and a first hash value, wherein the reverse position index comprises the first position sequence number.
The corresponding relation between the first position serial number in the reverse position index and the first hash value is pre-established, after the first hash value of the key to be written is obtained through calculation, the first position serial number corresponding to the first hash value can be matched, and then the corresponding reverse position index is matched according to the first position serial number. The sub-value has a reverse position index, the key corresponding to the sub-value is stored in a list of position numbers corresponding to the reverse position index, and the keys in the list are stored in an array form.
Preferably, before the obtaining the first virtual value corresponding to the value in the key-value pair in the inverted index table, the method further comprises:
determining whether a first virtual value corresponding to a value in the key-value pair exists in the inverted index table;
when it is determined that there is no first virtual value corresponding to a value in the key-value pair, creating the first virtual value, where the first virtual value includes a first number of sub-values into which the value and a preset value are split.
In this embodiment, to write a key-value pair, it is first determined whether a first virtual value corresponding to a value in the key-value pair exists. If not, a first virtual value corresponding to a value in the key-value pair is created. The created first virtual value includes a first number of values split into sub-values with a preconfigured value.
Preferably, after the calculating of the inverted position index in the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located, the method further comprises:
determining whether a sub-value having the reverse position index exists in the first virtual key;
when it is determined that there is no sub-value having the reverse position index, a sub-value having the reverse position index is created while adding 1 to the first number.
In this embodiment, before writing the key into the array corresponding to the sub-value having the reverse position index, it is determined whether the sub-value having the reverse position index exists in the first virtual key. And if the position index does not exist, creating a sub-value according to the reverse position index.
S13: and when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index in the first virtual key.
In this embodiment, before writing the key into the array corresponding to the sub-value having the reverse position index in the first virtual key, the version number of the reverse index table is obtained again, the obtained version number is compared with the version number obtained when the write instruction is received before, and if the obtained version number is the same as the version number obtained when the write instruction is received before, it is indicated that the version number is not changed, and a new key may be inserted. If the version number obtained at this time is different from the version number obtained when the write instruction is received before, the version number is changed, namely the version number is operated and updated by other concurrent requests, and then the operation is not carried out and retries are carried out.
In this embodiment, if writing into the reverse index table fails and the number of retries exceeds the preset first threshold, an error is returned and the operation is exited.
Preferably, before the writing the key into the array corresponding to the sub-value having the inverted position index in the first virtual key, the method further comprises:
judging whether the key exists in an array corresponding to the sub-value with the reverse position index in the first virtual key;
when the key is determined to be present, no operation is performed and a result of successful writing is returned.
In order to perform deduplication checking on data in the inverted index table, before writing the key, it is determined whether the key exists in the array of sub-values having the inverted position index in the first virtual key, and when the key exists, the key-value pair to be written is considered to be already in the inverted index table, i.e., the writing is successful.
S14: and acquiring a second virtual key corresponding to the key in the key-value pair in the forward index table, and calculating a forward position index in a second virtual value corresponding to the second virtual key, where the value in the key-value pair should be located.
In this embodiment, after the key-value pairs are written into the reverse index table, the key-value pairs are also written into the forward index table.
The data in the forward index table is in the form of a second virtual key-value pair, the second virtual value is composed of a key and a second number obtained by splitting the key into subkeys, the first virtual value is composed of the subkeys and values corresponding to the subkeys, and the number of the subkeys corresponds to the number of the values. That is, the key in the relational database is virtualized in the forward index table into a second number of keys and corresponding sub-keys for the key (the second number of keys and sub-keys for the key together may be referred to as a second info); the values of the keys in the relational database are virtualized into the sub-key (the sub-key may be referred to as a second shard) and the values corresponding to the sub-key in the forward index table, and the values of the sub-key are stored in an array form, and the forward index table is shown in fig. 3.
Preferably, the calculating a forward position index in a second virtual value corresponding to the second virtual key at which a value in the key-value pair should be located comprises:
1) calculating a second hash value of the value;
in this embodiment, the second hash value of the value may be calculated by using a hash function. The hash function is prior art, and the present invention is not described in detail herein.
2) And determining a forward position index in the second virtual value in which the value is to be located according to a corresponding relation between a preset second position sequence number and a second hash value, wherein the forward position index comprises the second position sequence number.
And after the second hash value of the value to be written is obtained through calculation, the second position serial number corresponding to the second hash value can be matched, and then the corresponding forward position index is matched according to the second position serial number. The sub-key has a forward position index, the value corresponding to the sub-key is stored in a list of position numbers corresponding to the forward position index, and the values in the list are stored in an array form.
Preferably, before the obtaining a second virtual key in the forward index table corresponding to a key in the key-value pair, the method further includes:
judging whether a second virtual key corresponding to the key in the key-value pair exists in the forward index table;
creating a second virtual key when it is determined that there is no second virtual key corresponding to a key in the key-value pair, the second virtual key including a second number of sub-keys into which the key and a preset key are split.
In this embodiment, to write a key-value pair, it is first determined whether a second virtual key corresponding to a key in the key-value pair exists. And if not, creating a first virtual key corresponding to the key. The created second virtual key includes a second number of keys split into subkeys with the preconfigured keys.
Preferably, after said calculating a forward position index in a second virtual value corresponding to the second virtual key in which a value in the key-value pair should be located, the method further comprises:
judging whether a sub-key with the forward position index exists in the second virtual value;
when it is determined that there is no sub-key with the forward position index, creating a sub-key with the forward position index while adding 1 to the second number.
In this embodiment, before writing the value into the array corresponding to the sub-key with the forward position index, it is determined whether the sub-key with the forward position index exists in the second virtual value. And if not, creating a sub-key according to the forward position index.
S15: and when the version number of the forward index table is not changed, writing the value into an array corresponding to the sub-key with the forward position index in the second virtual value.
In this embodiment, before writing the value into the array corresponding to the sub-key having the forward position index in the second virtual value, the version number of the forward index table is obtained again, the obtained version number is compared with the version number obtained when the write instruction is received before, and if the obtained version number is the same as the version number obtained when the write instruction is received before, it indicates that the version number is not changed, and a new value may be inserted. If the version number obtained at this time is different from the version number obtained when the write instruction is received before, the version number is changed, namely the version number is operated and updated by other concurrent requests, and then the operation is not carried out and retries are carried out.
In this embodiment, if the writing of the forward index table fails and the number of retries exceeds the preset second threshold, an error is returned and a rollback operation is performed.
Preferably, before the writing the value into the array corresponding to the sub-key having the forward position index in the second virtual value, the method further includes:
judging whether the value exists in an array corresponding to the sub-key with the forward position index in the second virtual value;
when the value is determined to exist, no operation is performed and a result of a write success is returned.
In order to perform deduplication checking on the data in the forward index table, before writing the value, it is determined whether the value exists in the array of sub-keys having the forward position index in the second virtual value, and when the value exists, the key-value pair to be written is considered to be already in the forward index table, that is, the writing is successful.
Preferably, when the writing of the key-value pair to the reverse index table is successful but the writing to the forward index table is failed, the method further comprises:
rolling back the reverse index table to maintain data consistency in the forward index table and the reverse index table.
The rolling back the reverse index table comprises: and deleting all the sub-values and the corresponding keys written into the first virtual key when the current version number of the reverse index table is the same as the version number when the reverse index table is successfully written.
In this embodiment, in order to ensure consistency of the forward index table and the reverse index table, when data to be written is successfully written into the reverse index table and fails to be written into the forward index table, a deletion operation needs to be performed on a key-value pair written into the reverse index table. And when the current version number of the reverse index table is judged to be different from the version number when the write is successful, the operation is not carried out and retry is carried out. After several retries fail, the rollback is considered to fail. If the previously written sub-value and the key corresponding to the sub-value do not exist, the rollback is considered to have been successful. Rollback may fail in the case of index overload, where the consistency of the database is broken, reliability of service is not guaranteed, and the upper layer logic is alerted and required to retry after a certain period of time.
According to the data distributed writing method under high concurrency, when a writing instruction of a key-value pair is received, a reverse position index in a first virtual key corresponding to a first virtual value of a corresponding value in a reverse index table where the key is located is calculated; when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index; calculating a forward position index in a second virtual value corresponding to a second virtual key of a key in the key-value pair in a forward index table in which the value should be located; and when the version number of the forward index table is not changed, writing the value into the array corresponding to the sub-key with the forward position index. By writing the key-value pair to be written into the reverse index table first and then into the forward index table, the writing efficiency of high-concurrency, many-to-many data can be effectively ensured, and the consistency of the forward index and the reverse index can be ensured.
Fig. 1 describes the data distributed writing method under high concurrency in detail, and the functional modules and hardware device architecture of the software system implementing the data distributed writing method under high concurrency are described below with reference to fig. 4 to 5.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
Example two
Fig. 4 is a functional block diagram of a data distributed writing apparatus with high concurrency according to an embodiment of the disclosure.
In some embodiments, the data distributed writing means 40 at high concurrency is running in the terminal. The data distributed writing apparatus 40 under high concurrency may include a plurality of functional modules composed of program code segments. The program codes of the program segments in the data distributed writing device 40 under high concurrency can be stored in the memory of the terminal and executed by the at least one processor to perform (see detailed description of fig. 1) distributed writing of data under high concurrency.
In this embodiment, the data distributed writing apparatus 40 under high concurrency may be divided into a plurality of functional modules according to the functions executed by the apparatus. The functional module may include: the device comprises a receiving module 401, a first obtaining module 402, a first judging module 403, a first creating module 404, a first writing module 405, a second obtaining module 406, a second judging module 407, a second creating module 408, a second writing module 409 and a rollback module 410. The module referred to herein is a series of computer program segments capable of being executed by at least one processor and capable of performing a fixed function and is stored in memory. In the present embodiment, the functions of the modules will be described in detail in the following embodiments.
A receiving module 401, configured to record a version number of the reverse index table and a version number of the forward index table when receiving a write instruction of a key-value pair.
In this embodiment, a forward index table and a reverse index table are pre-stored in the terminal, and data in the pre-stored forward index table and data in the pre-stored reverse index table are both in the form of virtual key value pairs. The forward index table facilitates operations related to forward indexing and the reverse index table facilitates operations related to reverse indexing.
In this embodiment, the key-value pairs to be written are described by taking many-to-many relational data as an example, and in the many-to-many relational data, the same key may correspond to a plurality of different values, and the same value may correspond to a plurality of different keys.
The user can request to write at least one key-value pair through the upper layer logic call interface or the man-machine interaction interface or other modes. And when the terminal receives a write-in instruction of the key-value pair, firstly recording the version number of the reverse index table and the version number of the forward index table when the write-in instruction is received.
The version number is a self-increment that is incremented after each write is successful. Since there may be many persons requesting to write the key-value pair at the same time in the case of high concurrency, in order to solve the problem of data contention in the high concurrency, it is necessary to read the data in the entire reverse index table and forward index table to avoid the situation of repeatedly writing data, and therefore before writing the key-value pair, it is checked whether the version numbers of the reverse index table and forward index table are changed. If the version number of the reverse index table and/or the forward index table is changed, indicating that other requests have completed writing, the index table will not be updated, but retried.
A first obtaining module 402, configured to obtain a first virtual value corresponding to a value in the key-value pair in the reverse index table, and calculate a reverse position index in a first virtual key corresponding to the first virtual value where a key in the key-value pair should be located.
In this embodiment, because the requirement on the correctness of the data of the forward index is higher, the key-value pair is written into the reverse index table at first, and then the key-value pair is written into the forward index table, so that it can be ensured that the feedback of the data is more sensitive once an error occurs when the key-value pair is written into the forward index table.
The data in the inverted index table is in the form of a first virtual key-value pair, the first virtual value is composed of a value and a first number of sub-values into which the value is split, the first virtual key is composed of the sub-values and keys corresponding to each of the sub-values, and the number of the sub-values corresponds to the number of the keys. That is, the value in the relational database, at this time, is virtualized in the inverted index table into a first number of values and sub-values corresponding to the value (the first number of values and sub-values of values taken together may be referred to as a first info); the key of the value in the relational database is virtualized into the sub-value (the sub-value may be referred to as the first shard) and the key corresponding to the sub-value in the inverted index table, and the key of the sub-value is stored in the form of an array, and the inverted index table is shown in fig. 2.
Preferably, the calculating, by the first obtaining module 402, the reverse position index in the first virtual key corresponding to the first virtual value where the key in the key-value pair should be located includes:
1) calculating a first hash value of the key;
in this embodiment, the first hash value of the key may be calculated by using a hash function. The hash function is prior art, and the present invention is not described in detail herein.
2) And determining a reverse position index in the first virtual key where the key is to be located according to a corresponding relation between a preset first position sequence number and a first hash value, wherein the reverse position index comprises the first position sequence number.
The corresponding relation between the first position serial number in the reverse position index and the first hash value is pre-established, after the first hash value of the key to be written is obtained through calculation, the first position serial number corresponding to the first hash value can be matched, and then the corresponding reverse position index is matched according to the first position serial number. The sub-value has a reverse position index, the key corresponding to the sub-value is stored in a list of position numbers corresponding to the reverse position index, and the keys in the list are stored in an array form.
A first determining module 403, configured to determine whether a first virtual value corresponding to a value in the key-value pair exists in the reverse index table before the first obtaining module 402 obtains the first virtual value corresponding to the value in the key-value pair in the reverse index table;
a first creating module 404, configured to create a first virtual value when the first determining module 403 determines that there is no first virtual value corresponding to a value in the key-value pair, where the first virtual value includes a first number of sub-values obtained by splitting the value and a preset value.
In this embodiment, to write a key-value pair, it is first determined whether a first virtual value corresponding to a value in the key-value pair exists. If not, a first virtual value corresponding to a value in the key-value pair is created. The created first virtual value includes a first number of values split into sub-values with a preconfigured value.
The first determining module 403 is further configured to determine, after the first obtaining module 402 calculates the reverse position index in the first virtual key corresponding to the first virtual value where the key in the key-value pair should be located, whether a sub-value with the reverse position index exists in the first virtual key;
the first creating module 404 is further configured to create a sub-value with the reverse position index when the first determining module 403 determines that there is no sub-value with the reverse position index, and add 1 to the first number.
In this embodiment, before writing the key into the array corresponding to the sub-value having the reverse position index, it is determined whether the sub-value having the reverse position index exists in the first virtual key. And if the position index does not exist, creating a sub-value according to the reverse position index.
A first writing module 405, configured to write the key into an array corresponding to the sub-value having the reverse position index in the first virtual key when the reverse index table version number is not changed.
In this embodiment, before writing the key into the array corresponding to the sub-value having the reverse position index in the first virtual key, the version number of the reverse index table is obtained again, the obtained version number is compared with the version number obtained when the write instruction is received before, and if the obtained version number is the same as the version number obtained when the write instruction is received before, it is indicated that the version number is not changed, and a new key may be inserted. If the version number obtained at this time is different from the version number obtained when the write instruction is received before, the version number is changed, namely the version number is operated and updated by other concurrent requests, and then the operation is not carried out and retries are carried out.
In this embodiment, if writing into the reverse index table fails and the number of retries exceeds the preset first threshold, an error is returned and the operation is exited.
Preferably, the first determining module 403 is further configured to determine whether the key exists in the array corresponding to the sub-value having the reverse position index in the first virtual key before the first writing module 405 writes the key into the array corresponding to the sub-value having the reverse position index in the first virtual key; when the key is determined to be present, no operation is performed and a result of successful writing is returned.
In order to perform deduplication checking on data in the inverted index table, before writing the key, it is determined whether the key exists in the array of sub-values having the inverted position index in the first virtual key, and when the key exists, the key-value pair to be written is considered to be already in the inverted index table, i.e., the writing is successful.
A second obtaining module 406, configured to obtain a second virtual key in the forward index table corresponding to a key in the key-value pair, and calculate a forward position index in a second virtual value corresponding to the second virtual key where a value in the key-value pair should be located.
In this embodiment, after the key-value pairs are written into the reverse index table, the key-value pairs are also written into the forward index table.
The data in the forward index table is in the form of a second virtual key-value pair, the second virtual value is composed of a key and a second number obtained by splitting the key into subkeys, the first virtual value is composed of the subkeys and values corresponding to the subkeys, and the number of the subkeys corresponds to the number of the values. That is, the key in the relational database is virtualized in the forward index table into a second number of keys and corresponding sub-keys for the key (the second number of keys and sub-keys for the key together may be referred to as a second info); the values of the keys in the relational database are virtualized into the sub-key (the sub-key may be referred to as a second shard) and the values corresponding to the sub-key in the forward index table, and the values of the sub-key are stored in an array form, and the forward index table is shown in fig. 3.
Preferably, the calculating, by the second obtaining module 406, a forward position index in the second virtual value corresponding to the second virtual key, where the value in the key-value pair should be located includes:
1) calculating a second hash value of the value;
in this embodiment, the second hash value of the value may be calculated by using a hash function. The hash function is prior art, and the present invention is not described in detail herein.
2) And determining a forward position index in the second virtual value in which the value is to be located according to a corresponding relation between a preset second position sequence number and a second hash value, wherein the forward position index comprises the second position sequence number.
And after the second hash value of the value to be written is obtained through calculation, the second position serial number corresponding to the second hash value can be matched, and then the corresponding forward position index is matched according to the second position serial number. The sub-key has a forward position index, the value corresponding to the sub-key is stored in a list of position numbers corresponding to the forward position index, and the values in the list are stored in an array form.
A second determining module 407, configured to determine whether a second virtual key corresponding to a key in the key-value pair exists in the forward index table before the second obtaining module 406 obtains the second virtual key corresponding to the key in the key-value pair in the forward index table;
a second creating module 408, configured to create a second virtual key when the second determining module 407 determines that there is no second virtual key corresponding to a key in the key-value pair, where the second virtual key includes a second number of sub-keys into which the key and a preset key are split.
In this embodiment, to write a key-value pair, it is first determined whether a second virtual key corresponding to a key in the key-value pair exists. And if not, creating a first virtual key corresponding to the key. The created second virtual key includes a second number of keys split into subkeys with the preconfigured keys.
A second determining module 407, configured to determine, after the second obtaining module 406 calculates a forward position index in a second virtual value corresponding to the second virtual key where a value in the key-value pair should be located, whether a sub-key having the forward position index exists in the second virtual value;
a second creating module 408, configured to create a sub-key with the forward position index when the second determining module 407 determines that there is no sub-key with the forward position index, and add 1 to the second number.
In this embodiment, before writing the value into the array corresponding to the sub-key with the forward position index, it is determined whether the sub-key with the forward position index exists in the second virtual value. And if not, creating a sub-key according to the forward position index.
A second writing module 409, configured to write the value into an array corresponding to the sub-key with the forward position index in the second virtual value when the forward index table version number is not changed.
In this embodiment, before writing the value into the array corresponding to the sub-key having the forward position index in the second virtual value, the version number of the forward index table is obtained again, the obtained version number is compared with the version number obtained when the write instruction is received before, and if the obtained version number is the same as the version number obtained when the write instruction is received before, it indicates that the version number is not changed, and a new value may be inserted. If the version number obtained at this time is different from the version number obtained when the write instruction is received before, the version number is changed, namely the version number is operated and updated by other concurrent requests, and then the operation is not carried out and retries are carried out.
In this embodiment, if the writing of the forward index table fails and the number of retries exceeds the preset second threshold, an error is returned and a rollback operation is performed.
The second determining module 407 is further configured to determine whether the value exists in the array corresponding to the sub-key with the forward position index in the second virtual value before the second writing module 409 writes the value in the array corresponding to the sub-key with the forward position index in the second virtual value; when the value is determined to exist, no operation is performed and a result of a write success is returned.
In order to perform deduplication checking on the data in the forward index table, before writing the value, it is determined whether the value exists in the array of sub-keys having the forward position index in the second virtual value, and when the value exists, the key-value pair to be written is considered to be already in the forward index table, that is, the writing is successful.
Preferably, the apparatus further comprises:
a rolling back module 410, configured to roll back the reverse index table when the first writing module 405 successfully writes the key-value pair into the reverse index table but the second writing module 409 fails to write the key-value pair into the forward index table, so as to maintain data consistency in the forward index table and the reverse index table.
The rolling back the reverse index table comprises: and deleting all the sub-values and the corresponding keys written into the first virtual key when the current version number of the reverse index table is the same as the version number when the reverse index table is successfully written.
In this embodiment, in order to ensure consistency of the forward index table and the reverse index table, when data to be written is successfully written into the reverse index table and fails to be written into the forward index table, a deletion operation needs to be performed on a key-value pair written into the reverse index table. And when the current version number of the reverse index table is judged to be different from the version number when the write is successful, the operation is not carried out and retry is carried out. After several retries fail, the rollback is considered to fail. If the previously written sub-value and the key corresponding to the sub-value do not exist, the rollback is considered to have been successful. Rollback may fail in the case of index overload, where the consistency of the database is broken, reliability of service is not guaranteed, and the upper layer logic is alerted and required to retry after a certain period of time.
According to the data distributed writing device under high concurrency, when a writing instruction of a key-value pair is received, a reverse position index in a first virtual key corresponding to a first virtual value of a corresponding value in a reverse index table where the key is located is calculated; when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index; calculating a forward position index in a second virtual value corresponding to a second virtual key of a key in the key-value pair in a forward index table in which the value should be located; and when the version number of the forward index table is not changed, writing the value into the array corresponding to the sub-key with the forward position index. By writing the key-value pair to be written into the reverse index table first and then into the forward index table, the writing efficiency of high-concurrency, many-to-many data can be effectively ensured, and the consistency of the forward index and the reverse index can be ensured.
EXAMPLE III
Fig. 5 is a schematic diagram of an internal structure of a terminal according to an embodiment of the disclosure.
In this embodiment, the terminal 5 may be a fixed terminal or a mobile terminal.
The terminal 5 may include a memory 51, a processor 52 and a bus 53.
The memory 51 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 51 may in some embodiments be an internal storage unit of the terminal 5, such as a hard disk of the terminal 5. The memory 51 may also be an external storage device of the terminal 5 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a flash memory Card (FlashCard), and the like, provided on the terminal 5. Further, the memory 51 may also comprise both internal memory units and external memory devices of said terminal 5. The memory 51 may be used not only to store application software installed in the terminal 5 and various types of data, such as codes of the data distributed writing apparatus 40 under high concurrency and the like and various modules, but also to temporarily store data that has been output or is to be output.
The bus 53 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.
Further, the terminal 5 may further include a network interface, which may optionally include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), and is generally used to establish a communication connection between the terminal 5 and another terminal.
Optionally, the terminal 5 may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and an optional user interface which may also comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. Wherein a display, also appropriately referred to as a display screen or display unit, is used for displaying messages processed in said terminal 5 and for displaying a visual user interface.
Fig. 5 shows only the terminal 5 with the components 51-53 and the high-concurrency data distributed writing means 40, and it will be understood by those skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the terminal 5, and may be a bus-type structure or a star-type structure, and the terminal 5 may also include fewer or more components than those shown, or combine certain components, or arrange different components. Other electronic products, now existing or hereafter developed, that may be adapted to the present invention, are also included within the scope of the present invention and are hereby incorporated by reference.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.
The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (13)
1. A data distributed writing method under high concurrency is applied to a terminal, and is characterized in that a forward index table and a reverse index table are stored in the terminal in advance, and the method comprises the following steps:
when a write-in instruction of a key-value pair is received, recording the version number of the reverse index table and the version number of the forward index table;
obtaining a first virtual value in the reverse index table corresponding to a value in the key-value pair, and calculating a reverse position index in a first virtual key corresponding to the first virtual value in which a key in the key-value pair should be located;
when the version number of the reverse index table is not changed, writing the key into an array corresponding to the sub-value with the reverse position index in the first virtual key;
acquiring a second virtual key corresponding to a key in the key-value pair in the forward index table, and calculating a forward position index in a second virtual value corresponding to the second virtual key in which the value in the key-value pair should be located;
and when the version number of the forward index table is not changed, writing the value into an array corresponding to the sub-key with the forward position index in the second virtual value.
2. The method of claim 1, wherein the calculating the inverted position index into the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located comprises:
calculating a first hash value of the key;
and determining a reverse position index in the first virtual key where the key is to be located according to a corresponding relation between a preset first position sequence number and a first hash value, wherein the reverse position index comprises the first position sequence number.
3. The method of claim 1, wherein the calculating a forward position index into a second virtual value corresponding to the second virtual key in which a value in the key-value pair should be located comprises:
calculating a second hash value of the value;
and determining a forward position index in the second virtual value in which the value is to be located according to a corresponding relation between a preset second position sequence number and a second hash value, wherein the forward position index comprises the second position sequence number.
4. The method of claim 1, wherein upon a successful write of the key-value pair to the reverse index table but a failure to write the key-value pair to the forward index table, the method further comprises:
rolling back the reverse index table to maintain data consistency in the forward index table and the reverse index table;
the rolling back the reverse index table comprises: and deleting all the sub-values and the corresponding keys written into the first virtual key when the current version number of the reverse index table is the same as the version number when the reverse index table is successfully written.
5. The method of any of claims 1-4, wherein prior to said obtaining a first virtual value in the inverted index table corresponding to a value in the key-value pair, the method further comprises:
determining whether a first virtual value corresponding to a value in the key-value pair exists in the inverted index table;
when it is determined that there is no first virtual value corresponding to a value in the key-value pair, creating the first virtual value, where the first virtual value includes a first number of sub-values into which the value and a preset value are split.
6. The method of claim 5, wherein after said calculating the inverted position index in the first virtual key corresponding to the first virtual value in which the key of the key-value pair should be located, the method further comprises:
determining whether a sub-value having the reverse position index exists in the first virtual key;
when it is determined that there is no sub-value having the reverse position index, a sub-value having the reverse position index is created while adding 1 to the first number.
7. The method of claim 6, wherein prior to said writing the key into the array corresponding to the sub-value of the first virtual key having the inverted position index, the method further comprises:
judging whether the key exists in an array corresponding to the sub-value with the reverse position index in the first virtual key;
when the key is determined to be present, no operation is performed and a result of successful writing is returned.
8. The method of any of claims 1-4, wherein prior to said obtaining a second virtual key in the forward index table corresponding to a key in the key-value pair, the method further comprises:
judging whether a second virtual key corresponding to the key in the key-value pair exists in the forward index table;
and when determining that a second virtual key corresponding to the key in the key-value pair does not exist, creating the second virtual key, wherein the second virtual key comprises a second number of sub-keys obtained by splitting the key and a preset key.
9. The method of claim 8, wherein after said computing the forward position index in the second virtual value corresponding to the second virtual key in which the value in the key-value pair should be located, the method further comprises:
judging whether a sub-key with the forward position index exists in the second virtual value;
when it is determined that there is no sub-key with the forward position index, creating a sub-key with the forward position index while adding 1 to the second number.
10. The method of claim 9, wherein prior to said writing the value into the array corresponding to the sub-key in the second virtual value having the forward position index, the method further comprises:
judging whether the value exists in an array corresponding to the sub-key with the forward position index in the second virtual value;
when the value is determined to exist, no operation is performed and a result of a write success is returned.
11. A data distributed writing device under high concurrency is operated in a terminal, and is characterized in that a forward index table and a reverse index table are stored in the terminal in advance, and the device comprises:
a receiving module, configured to record a version number of the reverse index table and a version number of the forward index table when a write instruction of a key-value pair is received;
a first obtaining module, configured to obtain a first virtual value corresponding to a value in the key-value pair in the reverse index table, and calculate a reverse position index in a first virtual key corresponding to the first virtual value where a key in the key-value pair should be located;
a first writing module, configured to write the key into an array corresponding to a sub-value having the reverse position index in the first virtual key when the version number of the reverse index table is not changed;
a second obtaining module, configured to obtain a second virtual key in the forward index table corresponding to a key in the key-value pair, and calculate a forward position index in a second virtual value corresponding to the second virtual key in which a value in the key-value pair should be located;
and a second writing module, configured to write the value into an array corresponding to the sub-key with the forward position index in the second virtual value when the version number of the forward index table is not changed.
12. A terminal, characterized in that the terminal comprises a memory and a processor, the memory stores thereon a data distributed write program under high concurrency which can be run on the processor, and the data distributed write program under high concurrency realizes the data distributed write method under high concurrency according to any one of claims 1 to 10 when executed by the processor.
13. A computer-readable storage medium having stored thereon a high-concurrency data-distributed writer program executable by one or more processors to implement the high-concurrency data-distributed writing method of any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811629694.XA CN109739857B (en) | 2018-12-28 | 2018-12-28 | Data distributed writing method and device under high concurrency, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811629694.XA CN109739857B (en) | 2018-12-28 | 2018-12-28 | Data distributed writing method and device under high concurrency, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109739857A CN109739857A (en) | 2019-05-10 |
| CN109739857B true CN109739857B (en) | 2020-09-01 |
Family
ID=66362065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811629694.XA Active CN109739857B (en) | 2018-12-28 | 2018-12-28 | Data distributed writing method and device under high concurrency, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109739857B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579003B (en) * | 2020-12-15 | 2022-06-14 | 浙江大华技术股份有限公司 | Key value pair adjustment method, key value pair adjustment device, storage medium and electronic device |
| CN113611383B (en) * | 2021-08-20 | 2024-02-02 | 深圳平安智慧医健科技有限公司 | Medical information acquisition method, device, electronic equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8335776B2 (en) * | 2008-07-02 | 2012-12-18 | Commvault Systems, Inc. | Distributed indexing system for data storage |
| CN103051664B (en) * | 2012-08-14 | 2016-04-27 | 深圳市朗科科技股份有限公司 | A kind of file management method of cloud storage system, device and this cloud storage system |
| CN104794177B (en) * | 2015-04-02 | 2016-10-12 | 广州神马移动信息科技有限公司 | A kind of date storage method and device |
| US11030262B2 (en) * | 2015-08-25 | 2021-06-08 | Verizon Media Inc. | Recyclable private memory heaps for dynamic search indexes |
| CN107766478A (en) * | 2017-10-11 | 2018-03-06 | 复旦大学 | A kind of design method of concurrent index structure towards high competition scene |
| CN108628942B (en) * | 2018-03-27 | 2020-07-31 | 深圳市网心科技有限公司 | Block chain node equipment and data reading and writing method of distributed database |
-
2018
- 2018-12-28 CN CN201811629694.XA patent/CN109739857B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109739857A (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108388599B (en) | Electronic device, data migration and calling method and storage medium | |
| US11003437B2 (en) | Automated transparent distribution of updates to server computer systems in a fleet | |
| CN111541753A (en) | Distributed storage system, method, computer device and medium for block chain data | |
| CN103248667A (en) | Resource access method and system for distributed system | |
| US10282120B2 (en) | Method, apparatus and system for inserting disk | |
| CN110555150B (en) | Data monitoring method, device, equipment and storage medium | |
| CN106294387B (en) | Data migration method and device | |
| CN109739857B (en) | Data distributed writing method and device under high concurrency, terminal and storage medium | |
| CN111338834B (en) | Data storage method and device | |
| CN109857579B (en) | Data processing method and related device | |
| CN109753506B (en) | Data distributed storage method, device, terminal and storage medium | |
| CN114637611A (en) | Information processing method and device based on message queue and computer equipment | |
| CN111694684A (en) | Abnormal construction method and device of storage equipment, electronic equipment and storage medium | |
| WO2019169771A1 (en) | Electronic device, access instruction information acquisition method and storage medium | |
| CN111193804B (en) | Distributed storage method and device, network node and storage medium | |
| CN111756618B (en) | Group chat processing method and device, electronic equipment and readable storage medium | |
| CN111885184A (en) | Method and device for processing hot spot access keywords in high concurrency scene | |
| CN113986921A (en) | Blacklist query method, system, electronic equipment and storage medium | |
| CN109740027B (en) | Data exchange method, device, server and storage medium | |
| CN111680334A (en) | Disk security access method, device, equipment and medium | |
| CN108959405B (en) | Strong consistency reading method of data and terminal equipment | |
| CN112632192A (en) | Node maintenance method and device, computer equipment and medium | |
| CN110727639A (en) | Fragment data reading method, electronic device, system and medium | |
| CN114546705B (en) | Operation response method, operation response device, electronic apparatus, and storage medium | |
| CN111614675B (en) | Request execution method, device, system and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |