CN111104403A - LSM tree data processing method, system, equipment and computer medium - Google Patents
LSM tree data processing method, system, equipment and computer medium Download PDFInfo
- Publication number
- CN111104403A CN111104403A CN201911208758.3A CN201911208758A CN111104403A CN 111104403 A CN111104403 A CN 111104403A CN 201911208758 A CN201911208758 A CN 201911208758A CN 111104403 A CN111104403 A CN 111104403A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- layer
- read
- lsm tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
Abstract
The application discloses a method, a system, equipment and a computer medium for processing LSM tree data, which receive a data reading instruction; acquiring a pre-stored data storage relation of a target LSM tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree; determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation; and reading the data to be read in the target data layer. According to the LSM tree data processing method, the target data layer can be quickly determined in the LSM tree based on the data storage relation, so that the data to be read can be quickly read, and compared with the existing technology of searching data by one layer, the data reading efficiency is high. The LSM tree data processing system, the LSM tree data processing device and the computer readable storage medium solve the corresponding technical problems.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a method, a system, a device, and a computer medium for processing LSM tree data.
Background
In a storage system, an LSM (Log Structured Merge Tree) Tree is often used as a storage structure in a time sequence database, such as infiluxdb, OpenTSdb, and the like, compared with B + Tree storage in a conventional database, the LSM Tree employs a hierarchical sequential disk writing method when writing data, which greatly improves the writing speed of data. Because the data combination is asynchronous, the current real-time data writing-in is not influenced, the IO is greatly improved, and meanwhile, the data is written into the disk block according to the time sequence, so that the time-related data query is facilitated, and the method is very suitable for the real-time performance data in the monitoring system.
When data is read, the data is read from the uppermost layer, and if the data is not found, the data is tested to the lower layer until the data is found or the last layer is found, so that the data query efficiency is low.
In summary, how to improve the data reading efficiency in the LSM tree is a problem to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide an LSM tree data processing method which can solve the technical problem of improving the data reading efficiency in an LSM tree to a certain extent. The application also provides an LSM tree data processing system, an LSM tree data processing device and a computer readable storage medium.
In order to achieve the above purpose, the present application provides the following technical solutions:
an LSM tree data processing method, comprising:
receiving a data reading instruction;
acquiring a pre-stored data storage relation of a target LSM (least significant Square) tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree;
determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and reading the data to be read in the target data layer.
Preferably, before receiving the data reading instruction, the method further includes:
for each data to be written, determining a data layer for storing the data to be written in the target LSM tree;
and establishing and storing a data storage relation between the data to be written and the corresponding data layer.
Preferably, the determining, in the target LSM tree, a data layer storing the data to be written includes:
determining the writing time of the data to be written in the target LSM tree;
acquiring a data time period of each data layer in the target LSM tree, wherein the data time period comprises writing time of each data in the data layer;
determining a data layer corresponding to a data time period containing the writing time of the data to be written as a data layer corresponding to the data to be written;
wherein the data storage relationship includes the data time period of the data layer.
Preferably, the data time period includes a start writing time and an end writing time of data in the data layer.
Preferably, the determining, according to the data storage relationship, a target data layer corresponding to the data to be read corresponding to the data reading instruction includes:
judging whether the number of the data to be read is one or not;
and if the number of the data to be read is one, determining the data writing time of the data to be read, and determining a data layer containing the data writing time as the target data layer according to the data storage relation.
Preferably, the determining, according to the data storage relationship, a target data layer corresponding to the data to be read corresponding to the data reading instruction includes:
judging whether the number of the data to be read is one or not;
and if the quantity of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as the target data layer according to the data storage relation.
Preferably, after acquiring the data time period of each data layer in the target LSM tree, the method further includes:
and judging whether the data of the data layer changes or not, and if so, updating the data time period of the data layer.
An LSM tree data processing system, comprising:
the first receiving module is used for receiving a data reading instruction;
the data storage system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a pre-stored data storage relationship of a target LSM (least significant Square) tree, the data storage relationship comprises stored data and a corresponding relationship between corresponding data layers, and the data layers comprise data layers in the target LSM tree, which store the stored data;
the first determining module is used for determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and the first reading module is used for reading the data to be read in the target data layer.
An LSM tree data processing apparatus comprising:
a memory for storing a computer program;
a processor for implementing the steps of the LSM tree data processing method as described above when executing the computer program.
A computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the LSM tree data processing method as described in any one of the above.
The application provides an LSM tree data processing method, which includes receiving a data reading instruction; acquiring a pre-stored data storage relation of a target LSM tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree; determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation; and reading the data to be read in the target data layer. According to the LSM tree data processing method, the target data layer can be quickly determined in the LSM tree based on the data storage relation, so that the data to be read can be quickly read, and compared with the existing technology of searching data by one layer, the data reading efficiency is high. The LSM tree data processing system, the LSM tree data processing device and the computer readable storage medium solve the corresponding technical problems.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of an LSM tree data processing method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a data layer structure of an LSM tree;
fig. 3 is a schematic structural diagram of an LSM tree data processing system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an LSM tree data processing apparatus according to an embodiment of the present application;
fig. 5 is another schematic structural diagram of an LSM tree data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart of an LSM tree data processing method according to an embodiment of the present application, fig. 3 is a schematic structural diagram of an LSM tree data processing system according to an embodiment of the present application, and fig. 4 is a schematic structural diagram of an LSM tree data processing apparatus according to an embodiment of the present application.
The LSM tree data processing method provided in the embodiment of the present application may include the following steps:
step S101: a data read instruction is received.
In practical application, a data reading instruction may be received first, and the type of the data reading instruction may be determined according to actual needs, for example, the data reading instruction may be an instruction for reading one data, may be an instruction for reading a group of data, may be an instruction for reading text data, may be an instruction for reading video data, and the like.
Step S102: and acquiring a pre-stored data storage relation of the target LSM tree, wherein the data storage relation comprises the stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree.
In practical application, after receiving the data reading instruction, the data storage relationship of the target LSM tree stored in advance may be obtained. The data storage relationship reflects which data layer in the target LSM tree each data is stored in, as can be seen from the data storage relationship including the stored data and the corresponding relationship between the corresponding data layers, and the data layer including the description of the data layer in the target LSM tree where the stored data is stored. It should be noted that the correspondence between the stored data and the corresponding data layer is not necessarily a direct correspondence between the stored data and the corresponding data layer, and may be an indirect correspondence, for example, the stored data and the corresponding data layer may be corresponded by a data writing time.
Step S103: and determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation.
Step S104: and reading the data to be read in the target data layer.
In practical application, after the data storage relationship is obtained, the target data layer corresponding to the data to be read corresponding to the data reading instruction can be determined according to the data storage relationship, the target data layer is also the data layer with the minimum level for storing the data to be read, and the data to be read is read in the target data layer.
The application provides an LSM tree data processing method, which includes receiving a data reading instruction; acquiring a pre-stored data storage relation of a target LSM tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree; determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation; and reading the data to be read in the target data layer. According to the LSM tree data processing method, the target data layer can be quickly determined in the LSM tree based on the data storage relation, so that the data to be read can be quickly read, and compared with the existing technology of searching data by one layer, the data reading efficiency is high.
According to the LSM tree data processing method provided by the embodiment of the application, before a data reading instruction is received, a data layer for storing data to be written can be determined in a target LSM tree for each data to be written; and establishing and storing a data storage relation between the data to be written and the corresponding data layer. That is, in the process of writing data into the target LSM tree, the write data layer of each data to be written may be determined, and a data storage relationship between the data to be written and the corresponding data layer is established and stored.
According to the LSM tree data processing method provided by the embodiment of the application, when the data layer for storing the data to be written is determined in the target LSM tree, the writing time of the data to be written in the target LSM tree can be determined; acquiring a data time period of each data layer in a target LSM tree, wherein the data time period comprises the writing time of each data in the data layer; determining a data layer corresponding to a data time period containing the writing time of the data to be written as a data layer corresponding to the data to be written; and the data storage relation comprises the data time period of the data layer. In practical applications, since each data layer in the target LSM tree may contain a plurality of data, if the corresponding relationship between each data layer and each data layer is directly established, the structure of the corresponding relationship is huge, and in order to reduce the structure of the corresponding relationship, the data periods of the data layer may be added in the data storage relationship, describing by means of the data periods the write moments of the data in the data layer, in this way, the storage information of the data to be written in the data layer can be determined only by comparing the data writing time of the data to be written with the data time period of the data layer, which is described with reference to fig. 2, for example, the data time period of the data layer L1 is 10 to 11 points, the data time period of the data layer L2 is 11 to 12 points, and the data writing time of one to-be-written data is 10 o' clock and 30 min, it can be determined that the to-be-written data is written in the data layer L1.
In the LSM tree data processing method provided in the embodiment of the present application, in order to further reduce the structure of the data storage relationship, the data time period may include only the start write time and the end write time of the data in the data layer.
In practical applications, in order to facilitate saving the data storage relationship, a data locator may be defined in the target LSM tree, the data storage relationship is saved in the data locator in the form of a hierarchy descriptor, the hierarchy descriptor may include a hierarchy size, a data start writing time, a data end writing time, and the like, and a data structure of the hierarchy description may be defined as follows:
according to the LSM tree data processing method provided by the embodiment of the application, when a target data layer corresponding to data to be read corresponding to a data reading instruction is determined according to a data storage relation, whether the number of the data to be read is one or not can be judged; if the number of the data to be read is one, determining the data writing time of the data to be read, and determining a data layer containing the data writing time as a target data layer according to the data storage relation; and if the number of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as a target data layer according to the data storage relation. Still taking the data layer structure shown in fig. 2 as an example, assuming that the data period of the data layer L1 is 10 to 12 points, the data period of the data layer L2 is 12 to 14 points, and the data writing period of the data to be read is 11 to 13 points, the data layer L1 and the data layer L2 need to be determined together as the target data layer.
According to the LSM tree data processing method provided by the embodiment of the application, in the application process of the target LSM tree, data movement between data layers is accompanied, and the data movement between the data layers can cause a data structure of the data layer to change, so that after the data time period of each data layer in the target LSM tree is acquired, whether the data of the data layer changes can be further judged, and if yes, the data time period of the data layer is updated.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an LSM tree data processing system according to an embodiment of the present disclosure.
An LSM tree data processing system provided in an embodiment of the present application may include:
a first receiving module 101, configured to receive a data reading instruction;
a first obtaining module 102, configured to obtain a pre-stored data storage relationship of a target LSM tree, where the data storage relationship includes a corresponding relationship between stored data and corresponding data layers, and the data layer includes a data layer in the target LSM tree, where the stored data is stored;
the first determining module 103 is configured to determine, according to the data storage relationship, a target data layer corresponding to the data to be read corresponding to the data reading instruction;
the first reading module 104 is configured to read data to be read in a target data layer.
The LSM tree data processing system provided in the embodiment of the present application may further include:
the second determining module is used for determining a data layer for storing the data to be written in the target LSM tree for each data to be written before the first receiving module receives the data reading instruction;
and the first storage module is used for establishing and storing a data storage relation between the data to be written and the corresponding data layer.
In an LSM tree data processing system provided in an embodiment of the present application, the second determining module may include:
the first determining submodule is used for determining the writing time of the data to be written in the target LSM tree;
the first obtaining submodule is used for obtaining a data time period of each data layer in the target LSM tree, and the data time period comprises the writing time of each data in the data layer;
the second determining submodule is used for determining the data layer corresponding to the data time period containing the writing time of the data to be written as the data layer corresponding to the data to be written;
and the data storage relation comprises the data time period of the data layer.
In the LSM tree data processing system provided in the embodiment of the present application, the data time period may include a start writing time and an end writing time of data in the data layer.
In an LSM tree data processing system provided in an embodiment of the present application, the first determining module may include:
the first judging unit is used for judging whether the number of the data to be read is one or not; if the number of the data to be read is one, determining the data writing time of the data to be read, and determining a data layer containing the data writing time as a target data layer according to the data storage relation; and if the number of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as a target data layer according to the data storage relation.
The LSM tree data processing system provided in the embodiment of the present application may further include:
and the first judgment module is used for judging whether the data of the data layer changes after the first acquisition submodule acquires the data time period of each data layer in the target LSM tree, and if so, updating the data time period of the data layer.
The application also provides an LSM tree data processing device and a computer readable storage medium, which have the corresponding effects of the LSM tree data processing method provided by the embodiment of the application. Referring to fig. 4, fig. 4 is a schematic structural diagram of an LSM tree data processing apparatus according to an embodiment of the present disclosure.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program:
receiving a data reading instruction;
acquiring a pre-stored data storage relation of a target LSM tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree;
determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and reading the data to be read in the target data layer.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: before receiving a data reading instruction, determining a data layer for storing data to be written in a target LSM tree for each data to be written; and establishing and storing a data storage relation between the data to be written and the corresponding data layer.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: determining the writing time of the data to be written in the target LSM tree; acquiring a data time period of each data layer in a target LSM tree, wherein the data time period comprises the writing time of each data in the data layer; determining a data layer corresponding to a data time period containing the writing time of the data to be written as a data layer corresponding to the data to be written; and the data storage relation comprises the data time period of the data layer.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: the data time period includes the initial write time and the end write time of the data in the data layer.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: judging whether the number of data to be read is one or not; and if the number of the data to be read is one, determining the data writing time of the data to be read, and determining the data layer containing the data writing time as a target data layer according to the data storage relation.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: judging whether the number of data to be read is one or not; and if the number of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as a target data layer according to the data storage relation.
The LSM tree data processing device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: and after the data time period of each data layer in the target LSM tree is acquired, judging whether the data of the data layer is changed, if so, updating the data time period of the data layer.
Referring to fig. 5, another LSM tree data processing apparatus provided in the embodiment of the present application may further include: an input port 203 connected to the processor 202, for transmitting externally input commands to the processor 202; a display unit 204 connected to the processor 202, for displaying the processing result of the processor 202 to the outside; and the communication module 205 is connected with the processor 202 and is used for realizing the communication between the LSM tree data processing device and the outside. The display unit 204 may be a display panel, a laser scanning display, or the like; the communication method adopted by the communication module 205 includes, but is not limited to, mobile high definition link technology (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), and wireless connection: wireless fidelity technology (WiFi), bluetooth communication technology, bluetooth low energy communication technology, ieee802.11s based communication technology.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps:
receiving a data reading instruction;
acquiring a pre-stored data storage relation of a target LSM tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree;
determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and reading the data to be read in the target data layer.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: before receiving a data reading instruction, determining a data layer for storing data to be written in a target LSM tree for each data to be written; and establishing and storing a data storage relation between the data to be written and the corresponding data layer.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: determining the writing time of the data to be written in the target LSM tree; acquiring a data time period of each data layer in a target LSM tree, wherein the data time period comprises the writing time of each data in the data layer; determining a data layer corresponding to a data time period containing the writing time of the data to be written as a data layer corresponding to the data to be written; and the data storage relation comprises the data time period of the data layer.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: the data time period includes the initial write time and the end write time of the data in the data layer.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: judging whether the number of data to be read is one or not; and if the number of the data to be read is one, determining the data writing time of the data to be read, and determining the data layer containing the data writing time as a target data layer according to the data storage relation.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: judging whether the number of data to be read is one or not; and if the number of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as a target data layer according to the data storage relation.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: and after the data time period of each data layer in the target LSM tree is acquired, judging whether the data of the data layer is changed, if so, updating the data time period of the data layer.
The computer-readable storage media to which this application relates include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage media known in the art.
For a description of relevant parts in an LSM tree data processing system, an LSM tree data processing device, and a computer readable storage medium provided in the embodiments of the present application, reference is made to detailed descriptions of corresponding parts in an LSM tree data processing method provided in the embodiments of the present application, and details are not repeated here. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. An LSM tree data processing method, comprising:
receiving a data reading instruction;
acquiring a pre-stored data storage relation of a target LSM (least significant Square) tree, wherein the data storage relation comprises stored data and a corresponding relation between corresponding data layers, and the data layers comprise data layers for storing the stored data in the target LSM tree;
determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and reading the data to be read in the target data layer.
2. The method of claim 1, wherein prior to receiving the data read instruction, further comprising:
for each data to be written, determining a data layer for storing the data to be written in the target LSM tree;
and establishing and storing a data storage relation between the data to be written and the corresponding data layer.
3. The method of claim 2, wherein determining, in the target LSM tree, a data layer storing the data to be written comprises:
determining the writing time of the data to be written in the target LSM tree;
acquiring a data time period of each data layer in the target LSM tree, wherein the data time period comprises writing time of each data in the data layer;
determining a data layer corresponding to a data time period containing the writing time of the data to be written as a data layer corresponding to the data to be written;
wherein the data storage relationship includes the data time period of the data layer.
4. The method of claim 3, wherein the data period comprises a start writing time and an end writing time of data in the data layer.
5. The method according to claim 3, wherein the determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relationship comprises:
judging whether the number of the data to be read is one or not;
and if the number of the data to be read is one, determining the data writing time of the data to be read, and determining a data layer containing the data writing time as the target data layer according to the data storage relation.
6. The method according to claim 3, wherein the determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relationship comprises:
judging whether the number of the data to be read is one or not;
and if the quantity of the data to be read is not one, determining a data writing time period of the data to be read, and determining a data layer included in the data writing time period as the target data layer according to the data storage relation.
7. The method of claim 3, wherein after obtaining the data time periods of the data layers in the target LSM tree, further comprising:
and judging whether the data of the data layer changes or not, and if so, updating the data time period of the data layer.
8. An LSM tree data processing system, comprising:
the first receiving module is used for receiving a data reading instruction;
the data storage system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a pre-stored data storage relationship of a target LSM (least significant Square) tree, the data storage relationship comprises stored data and a corresponding relationship between corresponding data layers, and the data layers comprise data layers in the target LSM tree, which store the stored data;
the first determining module is used for determining a target data layer corresponding to the data to be read corresponding to the data reading instruction according to the data storage relation;
and the first reading module is used for reading the data to be read in the target data layer.
9. An LSM tree data processing apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the LSM tree data processing method of any of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the LSM tree data processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911208758.3A CN111104403B (en) | 2019-11-30 | 2019-11-30 | LSM tree data processing method, system, equipment and computer medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911208758.3A CN111104403B (en) | 2019-11-30 | 2019-11-30 | LSM tree data processing method, system, equipment and computer medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111104403A true CN111104403A (en) | 2020-05-05 |
| CN111104403B CN111104403B (en) | 2022-06-07 |
Family
ID=70421146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911208758.3A Active CN111104403B (en) | 2019-11-30 | 2019-11-30 | LSM tree data processing method, system, equipment and computer medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104403B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140188870A1 (en) * | 2012-12-28 | 2014-07-03 | Dhrubajyoti Borthakur | Lsm cache |
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN105117415A (en) * | 2015-07-30 | 2015-12-02 | 西安交通大学 | Optimized SSD data updating method |
| CN105787093A (en) * | 2016-03-17 | 2016-07-20 | 清华大学 | Log file system construction method based on LSM-Tree structure |
| CN106156070A (en) * | 2015-03-31 | 2016-11-23 | 华为技术有限公司 | A kind of querying method, Piece file mergence method and relevant apparatus |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
| CN110168532A (en) * | 2017-05-09 | 2019-08-23 | 华为技术有限公司 | Data-updating method and storage device |
| CN110377227A (en) * | 2019-06-13 | 2019-10-25 | 阿里巴巴集团控股有限公司 | A kind of data block storage method, apparatus and electronic equipment |
| CN110515897A (en) * | 2019-07-17 | 2019-11-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method and system for optimizing reading performance of LSM storage system |
-
2019
- 2019-11-30 CN CN201911208758.3A patent/CN111104403B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140188870A1 (en) * | 2012-12-28 | 2014-07-03 | Dhrubajyoti Borthakur | Lsm cache |
| CN104142958A (en) * | 2013-05-10 | 2014-11-12 | 华为技术有限公司 | Storage method for data in Key-Value system and related device |
| CN106156070A (en) * | 2015-03-31 | 2016-11-23 | 华为技术有限公司 | A kind of querying method, Piece file mergence method and relevant apparatus |
| CN105117415A (en) * | 2015-07-30 | 2015-12-02 | 西安交通大学 | Optimized SSD data updating method |
| CN105787093A (en) * | 2016-03-17 | 2016-07-20 | 清华大学 | Log file system construction method based on LSM-Tree structure |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
| CN110168532A (en) * | 2017-05-09 | 2019-08-23 | 华为技术有限公司 | Data-updating method and storage device |
| CN110377227A (en) * | 2019-06-13 | 2019-10-25 | 阿里巴巴集团控股有限公司 | A kind of data block storage method, apparatus and electronic equipment |
| CN110515897A (en) * | 2019-07-17 | 2019-11-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method and system for optimizing reading performance of LSM storage system |
Non-Patent Citations (1)
| Title |
|---|
| 付佳: ""基于LSM树的NoSQL数据库索引研究"", 《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》, 15 November 2016 (2016-11-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111104403B (en) | 2022-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11068441B2 (en) | Caseless file lookup in a distributed file system | |
| CN102508784B (en) | Data storage method of flash memory card in video monitoring equipment, and system thereof | |
| CN109542911B (en) | Metadata organization method, system, equipment and computer readable storage medium | |
| CN108205577B (en) | Array construction method, array query method, device and electronic equipment | |
| US10649905B2 (en) | Method and apparatus for storing data | |
| CN111309687A (en) | Object storage small file processing method, device, equipment and storage medium | |
| CN111737221B (en) | Data read-write method, device and equipment of cluster file system and storage medium | |
| CN100507921C (en) | Words display process in embedded system and system thereof | |
| CN103049261A (en) | Mobile terminal font changing method | |
| CN104268229A (en) | Resource obtaining method and device based on multi-process browser | |
| CN111143290A (en) | Memory management method, system, equipment and computer readable storage medium | |
| CN106776795B (en) | Data writing method and device based on Hbase database | |
| CN111125008B (en) | Dynamic repairing method and system of abnormal inode and related components | |
| CN111241071A (en) | Data migration method, system, equipment and computer readable storage medium | |
| CN109213450B (en) | Associated metadata deleting method, device and equipment based on flash memory array | |
| CN112000971B (en) | File permission recording method, system and related device | |
| CN111104403B (en) | LSM tree data processing method, system, equipment and computer medium | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN103020186A (en) | File searching method, device and equipment based on embedded device | |
| CN104636079A (en) | File access method and system thereof | |
| CN115017098A (en) | File system management method and device | |
| CN114861003A (en) | Object enumeration method, device and medium under specified directory | |
| CN103678008A (en) | Data reading method and corresponding data reading device | |
| CN111143293A (en) | Metadata acquisition method, device, equipment and computer readable storage medium | |
| CN112084141A (en) | Full-text retrieval system capacity expansion method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |