CN112732191A - Method, system, device and medium for merging tree merging data based on log structure - Google Patents
Method, system, device and medium for merging tree merging data based on log structure Download PDFInfo
- Publication number
- CN112732191A CN112732191A CN202110025147.6A CN202110025147A CN112732191A CN 112732191 A CN112732191 A CN 112732191A CN 202110025147 A CN202110025147 A CN 202110025147A CN 112732191 A CN112732191 A CN 112732191A
- Authority
- CN
- China
- Prior art keywords
- data
- blocks
- index
- merging
- data block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000004044 response Effects 0.000 claims abstract description 32
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a system, equipment and a storage medium for merging tree merging data based on a log structure, wherein the method comprises the following steps: dividing the disk into data blocks with preset sizes, storing the data in the data blocks and storing the information of the data in the index blocks; combining the current index blocks with a preset number at intervals of preset time to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks; in response to the fact that the effective data of the multiple data blocks are smaller than the threshold value, combining the effective data of the multiple data blocks, and releasing the data blocks without the effective data; and in response to the valid data of only one data block being less than the threshold, merging the valid data of the data block into the data block with the lowest valid data except the data block. The invention separates the data index from the data, stores the data in the pre-distributed data block, and merges and recovers the data according to the index, thereby reducing the read-write data volume of the disk.
Description
Technical Field
The present invention relates to the field of data processing, and more particularly, to a method, system, computer device, and readable medium for merging tree merged data based on a log structure.
Background
LSM-Tree (Log-Structured Merge-Tree) was originally proposed in a paper "The Log-Structured Merge-Tree (LSM-Tree"), which is not a product or system per se, but an algorithm and idea for solving The problem. Based on the idea, a plurality of storage products, such as NoSQL storage systems like HBase and LevelDB, are realized. The core idea is to convert random writing into sequential writing to improve writing performance, and then perform data merging (compact) periodically to improve reading performance. Based on this paper, a common implementation is to layer data separately as L0-Ln and then merge the data of L layer to L +1 layer periodically, which results in a large amount of disk reads and writes during the merging process and ultimately affects the performance of the upper layer application.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a method, a system, a computer device, and a computer-readable storage medium for merging data based on a log structure merge tree, in which data indexes are separated from data, the data are stored in pre-allocated data blocks, the indexes are periodically merged, and the data are merged and recycled according to the indexes, so as to reduce the read-write data volume of a disk.
Based on the above object, an aspect of the embodiments of the present invention provides a method for merging tree data based on a log structure, including the following steps: dividing a disk into data blocks with preset sizes, storing data in the data blocks and storing information of the data in an index block; combining the current index blocks with a preset number at intervals of preset time to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks; in response to the fact that the valid data of the plurality of data blocks is smaller than the threshold value, combining the valid data of the plurality of data blocks and releasing the data blocks without valid data; and in response to the valid data of only one data block being less than the threshold, merging the valid data of the data block into a data block with the lowest valid data except the data block.
In some embodiments, said merging the current predetermined number of index chunks at predetermined intervals to form a new index chunk comprises: and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
In some embodiments, the method further comprises: in response to a data block being released, a reclamation flag is placed on the data block to allow new data to be saved.
In some embodiments, the method further comprises: in response to the valid data for the non-existing data block being less than the threshold, suspending receiving new data until a next cycle.
In another aspect of the embodiments of the present invention, a merged data system based on a log structure merged tree is further provided, including: the cutting module is configured to divide a disk into data blocks with preset sizes, store data in the data blocks and store information of the data in the index blocks; the judging module is configured to combine the index blocks of the current preset number at intervals of preset time to form new index blocks and judge whether the effective data of the corresponding data block is smaller than a threshold value according to the information of the new index blocks; the first merging module is configured to merge valid data of a plurality of data blocks and release a data block without valid data in response to the valid data of the plurality of data blocks being smaller than the threshold; and a second merging module, configured to, in response to that valid data of only one data block is smaller than the threshold, merge valid data of the data block into a data block with the lowest valid data except the data block.
In some embodiments, the determining module is configured to: and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
In some embodiments, the system further comprises: a marking module configured to mark back a data block to allow new data to be saved in response to the data block being released.
In some embodiments, the system further comprises: a suspending module configured to suspend receiving new data until a next cycle in response to no valid data of the data block being less than the threshold.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method as above.
In a further aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, in which a computer program for implementing the above method steps is stored when the computer program is executed by a processor.
The invention has the following beneficial technical effects: the data index is separated from the data, the data is stored in the pre-distributed data blocks, the indexes are combined periodically, and the data are combined and recycled according to the indexes, so that the read-write data volume of the disk is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a diagram illustrating an embodiment of a method for merging tree data based on a log structure according to the present invention;
FIG. 2 is a schematic diagram of data indexing and data separation in an embodiment of the present invention;
FIG. 3 is a diagram illustrating index merging according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a hardware structure of an embodiment of a computer apparatus for merging data based on a log structure merged tree provided in the present invention;
FIG. 5 is a diagram of an embodiment of a computer storage medium for merging data based on a log structure merge tree provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the above, a first aspect of the embodiments of the present invention provides an embodiment of a method for merging tree data based on a log structure. FIG. 1 is a diagram illustrating an embodiment of a method for merging tree data based on a log structure according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s1, dividing the disk into data blocks with preset sizes, storing the data in the data blocks and storing the information of the data in the index blocks;
s2, combining the index blocks with the preset number at preset time intervals to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks;
s3, in response to the fact that the valid data of the multiple data blocks are smaller than the threshold value, combining the valid data of the multiple data blocks, and releasing the data blocks without valid data; and
and S4, in response to the fact that the valid data of only one data block is smaller than the threshold value, combining the valid data of the data block into the data block with the lowest valid data except the data block.
The embodiment of the invention divides the disk into data blocks with fixed sizes in advance for storing data, and the data blocks are not deleted and only can be recycled for storing data. The detailed information of the data stored in the data block is used as index information of the data, and the index information is stored separately from the data. And merging the indexes periodically, merging a plurality of old index blocks into new index blocks, merging and recovering data in the data blocks according to the merged index information, and releasing the data blocks in time for repeated use.
And dividing the disk into data blocks with preset sizes, storing the data in the data blocks and storing the information of the data in the index blocks. The index blocks and the data blocks can be stored by using different storage media, and the index blocks use a fast read-write medium. The information that the data is saved in the data block is saved to the index block as index information, such as start position, length, time, compression flag, etc. FIG. 2 is a diagram illustrating data indexing and data separation according to an embodiment of the present invention. As shown in FIG. 2, the data1 data in data chunk 1 has its index information saved to the sub-index 1 in index chunk 1.
And combining the index blocks of the current preset number at preset time intervals to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks.
In some embodiments, said merging the current predetermined number of index chunks at predetermined intervals to form a new index chunk comprises: and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved. FIG. 3 is a diagram illustrating index merging according to an embodiment of the present invention. As shown in fig. 3, index chunks 1 and 2 are merged into a new index chunk, index1 and index2 remain the newest, index3 and index4 remain directly, a new index chunk is formed, and the old index chunk is discarded.
And in response to the fact that the valid data of the plurality of data blocks is smaller than the threshold value, combining the valid data of the plurality of data blocks and releasing the data blocks without valid data. In response to there being only one data block with valid data less than the threshold, merging the valid data of the data block into a data block with the lowest valid data except the data block.
In some embodiments, the method further comprises: in response to a data block being released, a reclamation flag is placed on the data block to allow new data to be saved. And recovering and sorting the data block information according to the index block information, wherein if any valid data does not exist in a certain data block (the data which can be associated in the index block is called valid data), the recovery of the data block is marked, and the data block can be reused for storing new data. 1. If the effective data in a certain data block is less than a certain threshold value, carrying out migration, combining a plurality of data blocks lower than the threshold value, and releasing the data blocks without effective data.
In some embodiments, the method further comprises: in response to the valid data for the non-existing data block being less than the threshold, suspending receiving new data until a next cycle. If there is no valid data for the data block that is less than the threshold, indicating that all data blocks are occupied, reception of new data may be suspended to ensure that the data blocks have good performance.
According to the invention, the index block and the data block are separated, the merging process only needs to merge indexes, and the index block and the data block are respectively stored in different storage media, so that the data volume read and written in the merging process is reduced; the data blocks are pre-distributed and can be recycled, so that the deletion operation of the data only needs to be marked for recycling, the real deletion action is not needed, and the reading and writing of the disk are reduced. And finally, the read-write of the disk is reduced, and the storage performance is improved.
It should be particularly noted that, the steps in the embodiments of the method for merging data based on log structure merge tree can be mutually intersected, replaced, added, and deleted, so that these reasonable permutation and combination transformations for the method for merging data based on log structure merge tree also belong to the scope of the present invention, and should not limit the scope of the present invention to the embodiments.
In view of the above object, a second aspect of the embodiments of the present invention provides a system for merging tree data based on a log structure, including: the cutting module is configured to divide a disk into data blocks with preset sizes, store data in the data blocks and store information of the data in the index blocks; the judging module is configured to combine the index blocks of the current preset number at intervals of preset time to form new index blocks and judge whether the effective data of the corresponding data block is smaller than a threshold value according to the information of the new index blocks; the first merging module is configured to merge valid data of a plurality of data blocks and release a data block without valid data in response to the valid data of the plurality of data blocks being smaller than the threshold; and a second merging module, configured to, in response to that valid data of only one data block is smaller than the threshold, merge valid data of the data block into a data block with the lowest valid data except the data block.
In some embodiments, the determining module is configured to: and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
In some embodiments, the system further comprises: a marking module configured to mark back a data block to allow new data to be saved in response to the data block being released.
In some embodiments, the system further comprises: a suspending module configured to suspend receiving new data until a next cycle in response to no valid data of the data block being less than the threshold.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions being executable by the processor to perform the steps of: s1, dividing the disk into data blocks with preset sizes, storing the data in the data blocks and storing the information of the data in the index blocks; s2, combining the index blocks with the preset number at preset time intervals to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks; s3, in response to the fact that the valid data of the multiple data blocks are smaller than the threshold value, combining the valid data of the multiple data blocks, and releasing the data blocks without valid data; and S4, in response to the valid data of only one data block being less than the threshold, merging the valid data of the data block into the data block with the lowest valid data except the data block.
In some embodiments, said merging the current predetermined number of index chunks at predetermined intervals to form a new index chunk comprises: and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
In some embodiments, the steps further comprise: in response to a data block being released, a reclamation flag is placed on the data block to allow new data to be saved.
In some embodiments, the steps further comprise: in response to the valid data for the non-existing data block being less than the threshold, suspending receiving new data until a next cycle.
Fig. 4 is a schematic hardware structural diagram of an embodiment of the computer device for merging data based on a log structure merge tree provided in the present invention.
Taking the apparatus shown in fig. 4 as an example, the apparatus includes a processor 201 and a memory 202, and may further include: an input device 203 and an output device 204.
The processor 201, the memory 202, the input device 203 and the output device 204 may be connected by a bus or other means, and fig. 4 illustrates the connection by a bus as an example.
The memory 202, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method for merging tree data based on a log structure in the embodiment of the present application. The processor 201 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 202, that is, the method for merging data based on a log structure of the above-described method embodiment is implemented.
The memory 202 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the method of merging tree merging data based on the log structure, and the like. Further, the memory 202 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 202 may optionally include memory located remotely from processor 201, which may be connected to local modules via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 203 may receive information such as a user name and a password that are input. The output device 204 may include a display device such as a display screen.
Program instructions/modules corresponding to one or more methods for merging data based on a log structure are stored in the memory 202, and when executed by the processor 201, perform the method for merging data based on a log structure in any of the above-described method embodiments.
Any embodiment of a computer device executing the method for merging tree data based on a log structure can achieve the same or similar effects as any corresponding embodiment of the method.
The invention also provides a computer readable storage medium storing a computer program which, when executed by a processor, performs the method as above.
FIG. 5 is a schematic diagram of an embodiment of a computer storage medium for merging data based on a log structure according to the present invention. Taking the computer storage medium as shown in fig. 5 as an example, the computer readable storage medium 3 stores a computer program 31 which, when executed by a processor, performs the above method.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the method for merging tree data based on a log structure can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for merging tree data based on log structure is characterized by comprising the following steps:
dividing a disk into data blocks with preset sizes, storing data in the data blocks and storing information of the data in an index block;
combining the current index blocks with a preset number at intervals of preset time to form new index blocks, and judging whether the effective data of the corresponding data blocks are smaller than a threshold value according to the information of the new index blocks;
in response to the fact that the valid data of the plurality of data blocks is smaller than the threshold value, combining the valid data of the plurality of data blocks and releasing the data blocks without valid data; and
in response to the valid data of only one data block being less than the threshold, merging the valid data of the data block into a data block with the lowest valid data except the data block.
2. The method of claim 1, wherein said merging a current predetermined number of index chunks at predetermined intervals to form a new index chunk comprises:
and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
3. The method of claim 1, further comprising:
in response to a data block being released, a reclamation flag is placed on the data block to allow new data to be saved.
4. The method of claim 1, further comprising:
in response to the valid data for the non-existing data block being less than the threshold, suspending receiving new data until a next cycle.
5. A method for merging tree data based on log structure is characterized in that the method comprises the following steps:
the cutting module is configured to divide a disk into data blocks with preset sizes, store data in the data blocks and store information of the data in the index blocks;
the judging module is configured to combine the index blocks of the current preset number at intervals of preset time to form new index blocks and judge whether the effective data of the corresponding data block is smaller than a threshold value according to the information of the new index blocks;
the first merging module is configured to merge valid data of a plurality of data blocks and release a data block without valid data in response to the valid data of the plurality of data blocks being smaller than the threshold; and
and the second merging module is used for merging the effective data of the data block into the data block with the lowest effective data except the data block in response to the condition that the effective data of only one data block is smaller than the threshold value.
6. The system of claim 5, wherein the determination module is configured to:
and all the different sub indexes in the index blocks with the preset number are reserved, and only the latest one of the same sub indexes is reserved.
7. The system of claim 5, further comprising:
a marking module configured to mark back a data block to allow new data to be saved in response to the data block being released.
8. The system of claim 5, further comprising:
a suspending module configured to suspend receiving new data until a next cycle in response to no valid data of the data block being less than the threshold.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 4.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110025147.6A CN112732191B (en) | 2021-01-08 | 2021-01-08 | Method, system, device and medium for merging tree merging data based on log structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110025147.6A CN112732191B (en) | 2021-01-08 | 2021-01-08 | Method, system, device and medium for merging tree merging data based on log structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112732191A true CN112732191A (en) | 2021-04-30 |
| CN112732191B CN112732191B (en) | 2023-01-10 |
Family
ID=75589896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110025147.6A Active CN112732191B (en) | 2021-01-08 | 2021-01-08 | Method, system, device and medium for merging tree merging data based on log structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112732191B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342274A (en) * | 2021-06-10 | 2021-09-03 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN113568581A (en) * | 2021-07-29 | 2021-10-29 | 武汉天喻信息产业股份有限公司 | Multi-application resource recovery method and system for embedded equipment |
| CN114063935A (en) * | 2022-01-17 | 2022-02-18 | 阿里云计算有限公司 | Method and device for processing data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| CN106886375A (en) * | 2017-03-27 | 2017-06-23 | 百度在线网络技术（北京）有限公司 | The method and apparatus of data storage |
| CN110188108A (en) * | 2019-06-10 | 2019-08-30 | 北京平凯星辰科技发展有限公司 | Date storage method, device, system, computer equipment and storage medium |
| CN110532347A (en) * | 2019-09-02 | 2019-12-03 | 北京博睿宏远数据科技股份有限公司 | A kind of daily record data processing method, device, equipment and storage medium |
| US20200372005A1 (en) * | 2019-05-22 | 2020-11-26 | Druva Inc. | Space- efficient change journal for a storage system |
-
2021
- 2021-01-08 CN CN202110025147.6A patent/CN112732191B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| CN106886375A (en) * | 2017-03-27 | 2017-06-23 | 百度在线网络技术（北京）有限公司 | The method and apparatus of data storage |
| US20200372005A1 (en) * | 2019-05-22 | 2020-11-26 | Druva Inc. | Space- efficient change journal for a storage system |
| CN110188108A (en) * | 2019-06-10 | 2019-08-30 | 北京平凯星辰科技发展有限公司 | Date storage method, device, system, computer equipment and storage medium |
| CN110532347A (en) * | 2019-09-02 | 2019-12-03 | 北京博睿宏远数据科技股份有限公司 | A kind of daily record data processing method, device, equipment and storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342274A (en) * | 2021-06-10 | 2021-09-03 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN113342274B (en) * | 2021-06-10 | 2024-01-30 | 北京火山引擎科技有限公司 | Data processing method and device |
| CN113568581A (en) * | 2021-07-29 | 2021-10-29 | 武汉天喻信息产业股份有限公司 | Multi-application resource recovery method and system for embedded equipment |
| CN113568581B (en) * | 2021-07-29 | 2023-08-01 | 武汉天喻信息产业股份有限公司 | Multi-application resource recovery method and system for embedded equipment |
| CN114063935A (en) * | 2022-01-17 | 2022-02-18 | 阿里云计算有限公司 | Method and device for processing data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112732191B (en) | 2023-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112732191B (en) | Method, system, device and medium for merging tree merging data based on log structure | |
| CN107832406B (en) | Method, device, equipment and storage medium for removing duplicate entries of mass log data | |
| CN102567427B (en) | Method and device for processing object data | |
| CN111198856B (en) | File management method, device, computer equipment and storage medium | |
| CN108932271B (en) | File management method and device | |
| CN104778222A (en) | Media library establishing and updating method on basis of USB (Universal Serial Bus) storage equipment | |
| CN103077166A (en) | Spatial multiplexing method and device for small file storage | |
| CN105068885A (en) | JPG fragmented file recovery and reconstruction method | |
| CN104965835A (en) | Method and apparatus for reading and writing files of a distributed file system | |
| CN103593447A (en) | Data processing method and device applied to database table | |
| CN111625531A (en) | Merging device based on programmable device, data merging method and database system | |
| CN106528703A (en) | Deduplication mode switching method and apparatus | |
| CN111176901B (en) | HDFS deleted file recovery method, terminal device and storage medium | |
| CN117369731A (en) | Data reduction processing method, device, equipment and medium | |
| CN110196786B (en) | Method and equipment for controlling memory in database rollback synchronization | |
| CN108090128B (en) | Recovery method and device for merged storage space and electronic equipment | |
| CN113625952A (en) | Object storage method, device, equipment and storage medium | |
| CN111698330B (en) | Data recovery method and device of storage cluster and server | |
| CN111061719B (en) | Data collection method, device, equipment and storage medium | |
| CN104991963B (en) | Document handling method and device | |
| CN113254273A (en) | Method, system, device and medium for real-time recovery of principal metadata | |
| CN106708831B (en) | FAT (file allocation table) image file processing method and device | |
| CN106933970A (en) | A kind of fast writing data to EXT subregions method and apparatus | |
| CN113467997A (en) | Data recovery method and device, mobile device and storage medium | |
| CN105224561B (en) | Cache storage method and device based on paging file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |