CN111722785A - Cache updating method and device - Google Patents
Cache updating method and device Download PDFInfo
- Publication number
- CN111722785A CN111722785A CN201910217386.4A CN201910217386A CN111722785A CN 111722785 A CN111722785 A CN 111722785A CN 201910217386 A CN201910217386 A CN 201910217386A CN 111722785 A CN111722785 A CN 111722785A
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- updated
- buffer
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Abstract
The disclosure relates to a cache updating method and device. The method comprises the following steps: receiving an IO request sent by a user through a foreground task; determining data to be updated by responding to the IO request through a foreground task, and sending the data to be updated to a first buffer area; and updating the cache through a background task based on the first buffer area. The method and the device can reduce the influence of cache updating on the IO critical path corresponding to the IO request, thereby reducing the delay of the IO critical path, effectively improving the IO performance of the IO critical path and improving the user experience.
Description
Technical Field
The present disclosure relates to the field of computers, and in particular, to a cache updating method and apparatus.
Background
In a general storage system, a storage medium with a small capacity and high performance is used as a cache layer, for example, a memory, a Solid State Drive (SSD), or the like. After data is written to or read from an underlying mass storage medium, such as a Hard Disk Drive (HDD), the written or read data needs to be updated into a cache for subsequent use. The cache update introduces additional operations such as data copy, metadata update, and old data elimination, and if the cache update occurs on a path of a current Input Output (IO) request of a user, IO delay is obviously increased, and user experience is reduced.
Disclosure of Invention
In view of this, the present disclosure provides a cache updating method and apparatus, so that the influence of cache updating on IO delay can be reduced, and the IO performance of an IO critical path is effectively improved.
According to a first aspect of the present disclosure, there is provided a cache update method, including: receiving an IO request sent by a user through a foreground task; determining data to be updated by responding to the IO request through a foreground task, and sending the data to be updated to a first buffer area; and updating the cache through a background task based on the first buffer area.
In one possible implementation, the first buffer is located at a first level of an LSM-Tree structure, and the cache is located at a second level of the LSM-Tree structure, where data in the first level has a read-write characteristic and data in the second level has a read-only characteristic.
In a possible implementation manner, updating the cache by the background task based on the first buffer includes: when the data amount in the first buffer area exceeds a threshold value, converting the first buffer area into a second buffer area, wherein the second buffer area is located in a third level of the LSM-Tree structure, and the data in the third level has a read-only characteristic; generating a new empty first buffer in the first tier; and performing data merging on the data in the second buffer area and the data in the cache to obtain an updated cache in the second level.
In one possible implementation, the data in the LSM-Tree structure includes metadata and data corresponding to the metadata.
In one possible implementation, the method further includes: and releasing the second buffer after data merging is carried out on the data in the second buffer and the data in the cache.
In a possible implementation manner, after the data to be updated is sent to the first buffer, the method further includes: and updating the metadata of the data to be updated in the cache through a foreground task.
In a possible implementation manner, updating the cache by the background task based on the first buffer includes: and updating the data to be updated in the cache through a background task.
According to a second aspect of the present disclosure, there is provided a cache updating apparatus, including: the foreground task processing module is used for receiving an IO request sent by a user through a foreground task; the foreground task processing module is further configured to determine data to be updated by responding to the IO request through a foreground task, and send the data to be updated to a first buffer area; and the background task processing module is used for updating the cache through a background task based on the first buffer area.
In one possible implementation, the first buffer is located at a first level of an LSM-Tree structure, and the cache is located at a second level of the LSM-Tree structure, where data in the first level has a read-write characteristic and data in the second level has a read-only characteristic.
In one possible implementation manner, the background task processing module includes: the conversion module is used for converting the first buffer area into a second buffer area when the data volume in the first buffer area exceeds a threshold value, the second buffer area is positioned at a third level of the LSM-Tree structure, and the data in the third level has read-only characteristic; a generation submodule for generating a new empty first buffer in the first hierarchy; and the cache updating submodule is used for carrying out data combination on the data in the second buffer area and the data in the cache and obtaining an updated cache in the second level.
In one possible implementation, the data in the LSM-Tree structure includes metadata and data corresponding to the metadata.
In one possible implementation manner, the background task processing module further includes: and the releasing submodule is used for releasing the second buffer area after the data in the second buffer area and the data in the cache are subjected to data combination.
In a possible implementation manner, the foreground task processing module is further configured to update the metadata of the data to be updated in the cache through the foreground task after the data to be updated is sent to the first buffer.
In a possible implementation manner, the background task processing module is specifically configured to: and updating the data to be updated in the cache through a background task.
According to a third aspect of the present disclosure, there is provided a cache updating apparatus, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the cache update method of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the cache update method of the first aspect described above.
Receiving an IO request sent by a user through a foreground task, responding the IO request to determine data to be updated, sending the data to be updated to a first buffer area, updating a cache based on the first buffer area through the background task, reducing the influence of cache updating on an IO critical path corresponding to the IO request, reducing the delay of the IO critical path, effectively improving the IO performance of the IO critical path, and improving the user experience.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a diagram illustrating a conventional cache update method in a storage system;
fig. 2 is a schematic flow chart illustrating a cache updating method according to an embodiment of the present disclosure;
FIG. 3 illustrates a schematic diagram of updating a cache according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating an update of a cache based on an LSM-Tree structure according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram illustrating updating a cache according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a cache updating apparatus according to an embodiment of the present disclosure;
fig. 7 shows a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
In a general storage system, a storage medium (such as a memory or an SSD) having a small capacity and high performance is used as a cache layer. After data is written to or read from the underlying high-capacity media (HDD), the written or read data needs to be updated into a cache for subsequent use.
Fig. 1 is a schematic diagram illustrating a conventional cache update method in a storage system. As shown in fig. 1, when the storage system receives an IO request sent by a user, a path of the IO request is an IO critical path. If the data corresponding to the IO request is not hit in the cache, cache updating needs to be performed on the IO key path. The cache update process includes at least one of: copying data to be updated corresponding to the IO request, which is written into or read from the HDD, into a cache; updating the metadata of the data to be updated in the cache; and deleting the old data corresponding to the data to be updated in the cache.
As stated in the background section, cache update occurs on the IO critical path, which may cause delay increase on the IO critical path, reduce IO performance of the IO critical path, and affect user experience.
The cache updating method provided by the disclosure can reduce the influence of cache updating on the IO critical path, thereby reducing the delay of the IO critical path, effectively improving the IO performance of the IO critical path, and improving the user experience.
Fig. 2 is a flowchart illustrating a cache update method according to an embodiment of the disclosure. As shown in fig. 2, the method may include:
step S21, receiving an IO request sent by a user through a foreground task.
Step S22, determining data to be updated by the foreground task responding to the IO request, and sending the data to be updated to the first buffer.
And step S23, updating the cache through the background task based on the first buffer area.
The cache update is divided into a foreground task part and a background task part. When the foreground task receives an IO request sent by a user and determines that data corresponding to the IO request is not hit in a cache, the foreground task determines to-be-updated data corresponding to the IO request and sends the to-be-updated data to the first buffer area. And the background task completes subsequent cache updating based on the first buffer area. Compared with the existing operation of performing cache updating on the IO key path, the cache updating is transferred to the background task processing, so that the influence of the cache updating on the IO key path can be reduced, and the delay of the IO key path is reduced.
The ways of updating the cache by the background task include, but are not limited to, the following two.
The first method comprises the following steps:
in one possible implementation, the first buffer is located at a first level of a log structured Merge Tree (LSM-Tree) structure, and the cache is located at a second level of the LSM-Tree structure, where data in the first level has a read-write property and data in the second level has a read-only property.
In one possible implementation manner, updating the cache by the background task based on the first buffer includes: when the data amount in the first buffer area exceeds a threshold value, converting the first buffer area into a second buffer area, wherein the second buffer area is positioned at a third level of the LSM-Tree structure, and the data in the third level has a read-only characteristic; generating a new empty first buffer in the first tier; and merging the data in the second buffer area and the data in the cache to obtain an updated cache in the second level.
In a possible implementation manner, the first Buffer area is an Active Buffer, and the second Buffer area is an Immutable Buffer.
Fig. 3 is a schematic diagram illustrating updating a cache according to an embodiment of the present disclosure. In fig. 3, for an IO request, after a foreground task sends data to be updated corresponding to the IO request to an Active Buffer in a first level of an LSM-Tree structure, an IO critical path of the IO request is ended. And subsequently, updating the cache in the second level of the LSM-Tree structure through the hierarchical storage and data merging characteristics of the LSM-Tree structure.
Fig. 4 is a schematic diagram illustrating an update of a cache based on an LSM-Tree structure according to an embodiment of the present disclosure. In the cache updating process shown in fig. 4, as the foreground task sends data to be updated to the Active Buffer increases, when the data amount of the data to be updated in the Active Buffer exceeds the threshold, the Active Buffer is converted into an executable Buffer in the third level in the LSM-Tree structure, and a new empty Active Buffer is generated in the first level, and the newly generated Active Buffer is used for subsequently and continuously receiving the data to be updated sent by the foreground task. The third level is positioned between the first level and the second level, and the data in the Immunable buffer in the third level has read-only characteristic. And merging the data in the Immunable buffer in the third level with the data in the cache in the second level to obtain the updated cache in the second level.
In the data storage system based on the LSM-Tree structure, the storage format of the data is a key-value pair (key-value) structure, the value is a data value of the data, and the key value (key) is an index parameter of the data. Data in the LSM-Tree structure is stored hierarchically, data with the same key can be stored in different hierarchical levels, and data stored in a higher hierarchical level is new data relative to data stored in a lower hierarchical level. That is, if the data of the same key word exists in the Immutable buffer and the cache, the data corresponding to the key word in the Immutable buffer is new data relative to the data corresponding to the key word in the cache, and when the Immutable buffer and the cache are subjected to data merging, the data corresponding to the key word in the Immutable buffer covers the data corresponding to the key word in the cache.
Still taking the above fig. 4 as an example, as shown in fig. 4, the Immutable buffer includes: data a to be updated and data b to be updated. The cache comprises the following steps: data c and data d. The data a to be updated and the data c are data with the same keyword, and the data a to be updated in the Immutable buffer is new data (the data c is old data) relative to the data c in the cache, so that the data a to be updated in the Immutable buffer covers the data c in the cache (that is, the old data c is deleted) in the data merging process. The obtained updated data includes: data a, data b, and data d.
In one possible implementation, the data in the LSM-Tree structure includes metadata and data corresponding to the metadata.
The data in the LSM-Tree structure comprises metadata and data corresponding to the metadata, and the method for updating the cache based on the LSM-Tree structure is that based on the LSM-Tree structure, the data to be updated corresponding to the IO critical path and the metadata of the data to be updated are cached and updated in a background task, so that the influence of cache updating on the IO critical path can be effectively reduced, and the delay of the IO critical path is reduced.
Because the data to be updated corresponding to the IO critical path and the metadata of the data to be updated are cached and updated in the background task, a more complex cache updating algorithm can be adopted to improve the updating accuracy, and the delay of the IO critical path cannot be increased.
In one possible implementation manner, the method further includes: and releasing the second buffer after data merging is carried out on the data in the second buffer and the data in the cache.
And after data in the Immunable buffer in the third level and the data in the cache in the second level are subjected to data combination in the LSM-Tree structure, releasing the Immunable buffer in the third level so as to improve the resource utilization rate.
And the second method comprises the following steps:
in a possible implementation manner, after sending the data to be updated to the first buffer, the method further includes: and updating the metadata of the data to be updated in the cache through the foreground task.
In one possible implementation manner, updating the cache by the background task based on the first buffer includes: and updating the data to be updated in the cache through the background task.
In one possible implementation, the first Buffer is a Buffer.
Fig. 5 is a schematic diagram illustrating updating a cache according to an embodiment of the present disclosure. In fig. 5, for an IO request, a foreground task sends data to be updated corresponding to the IO request to a Buffer of a storage system, and updates metadata of the data to be updated in a cache according to the Buffer, and an IO critical path of the IO request is ended. At this time, since only the metadata of the data to be updated is updated in the cache, but the data to be updated is not updated in the cache, an identifier needs to be added to the metadata of the data to be updated in the cache, and the identifier may be used to indicate that the data to be updated is not updated in the cache.
And the background task updates the data to be updated in the cache based on the Buffer, and updates the identifier of the metadata of the data to be updated in the cache again after the updating is finished, so that the updated identifier can be used for indicating that the updating of the data to be updated in the cache is finished.
In one possible implementation manner, the method further includes: and releasing the buffer after the metadata of the data to be updated and the data to be updated in the cache are updated.
And after the metadata of the data to be updated and the data to be updated in the cache are updated, releasing the buffer so as to improve the resource utilization rate.
According to the IO request, the metadata of the data to be updated corresponding to the IO request is cached and updated through the foreground task, the data to be updated corresponding to the IO request is cached and updated through the background task, the influence of the cache update on the IO critical path can be reduced to a partial extent, and the delay of the IO critical path is reduced.
Receiving an IO request sent by a user through a foreground task, responding the IO request to determine data to be updated, sending the data to be updated to a first buffer area, updating a cache based on the first buffer area through the background task, reducing the influence of cache updating on an IO critical path corresponding to the IO request, reducing the delay of the IO critical path, effectively improving the IO performance of the IO critical path, and improving the user experience.
Fig. 6 is a schematic structural diagram of a cache updating apparatus according to an embodiment of the present disclosure. The apparatus 60 shown in fig. 6 may be used to perform the steps of the above-described method embodiment shown in fig. 2, the apparatus 60 comprising:
a foreground task processing module 61, configured to receive an IO request sent by a user through a foreground task;
the foreground task processing module 61 is further configured to determine data to be updated by responding to the IO request through a foreground task, and send the data to be updated to the first buffer area;
and a background task processing module 62, configured to update the cache by the background task based on the first buffer.
In one possible implementation, the first buffer area is located at a first level of the LSM-Tree structure, the cache is located at a second level of the LSM-Tree structure, data in the first level has a read-write characteristic, and data in the second level has a read-only characteristic.
In one possible implementation, the background task processing module 62 includes:
the conversion module is used for converting the first buffer area into a second buffer area when the data volume in the first buffer area exceeds a threshold value, the second buffer area is positioned at a third level of the LSM-Tree structure, and the data in the third level has read-only characteristic;
a generation submodule for generating a new empty first buffer in the first hierarchy;
and the cache updating submodule is used for carrying out data combination on the data in the second buffer area and the data in the cache and obtaining the updated cache in the second level.
In one possible implementation, the data in the LSM-Tree structure includes metadata and data corresponding to the metadata.
In one possible implementation, the background task processing module 62 further includes:
and the releasing submodule is used for releasing the second buffer area after the data in the second buffer area and the data in the cache are subjected to data combination.
In a possible implementation manner, the foreground task processing module 61 is further configured to update metadata of the data to be updated in the cache through the foreground task.
In a possible implementation manner, the background task processing module 62 is specifically configured to:
and updating the data to be updated in the cache through the background task.
The apparatus 60 provided in the present disclosure can implement each step in the method embodiment shown in fig. 2, and implement the same technical effect, and is not described herein again to avoid repetition.
Fig. 7 shows a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 7, at the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a Non-Volatile Memory (NVM), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be interconnected by an internal bus, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an extended EISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.
And a memory for storing the program. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form a cache updating device on a logic level. The processor executes the program stored in the memory and specifically executes: receiving an IO request sent by a user through a foreground task; determining data to be updated by responding the IO request through a foreground task, and sending the data to be updated to a first buffer area; and updating the cache through the background task based on the first buffer area.
In one possible implementation, the first buffer area is located at a first level of the LSM-Tree structure, the cache is located at a second level of the LSM-Tree structure, data in the first level has a read-write characteristic, and data in the second level has a read-only characteristic.
In one possible implementation, the processor is specifically configured to perform: when the data amount in the first buffer area exceeds a threshold value, converting the first buffer area into a second buffer area, wherein the second buffer area is positioned at a third level of the LSM-Tree structure, and the data in the third level has a read-only characteristic; generating a new empty first buffer in the first tier; and merging the data in the second buffer area and the data in the cache to obtain an updated cache in the second level.
In one possible implementation, the data in the LSM-Tree structure includes metadata and data corresponding to the metadata.
In one possible implementation, the processor is specifically configured to perform: and releasing the second buffer after data merging is carried out on the data in the second buffer and the data in the cache.
In one possible implementation, the processor is specifically configured to perform: and after the data to be updated is sent to the first buffer area, updating the metadata of the data to be updated in the cache through a foreground task.
In one possible implementation, the processor is specifically configured to perform: and updating the data to be updated in the cache through the background task.
The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may execute the method executed in the method embodiment shown in fig. 2, and implement the functions of the method embodiment shown in fig. 2, which are not described herein again in this specification.
Embodiments of the present specification also propose a computer-readable storage medium storing one or more programs, the one or more programs including instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the caching method in the embodiment shown in fig. 2, and in particular to perform the steps of the embodiment of the method shown in fig. 2.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (16)
1. A cache update method, comprising:
receiving an input/output (IO) request sent by a user through a foreground task;
determining data to be updated by responding to the IO request through a foreground task, and sending the data to be updated to a first buffer area;
and updating the cache through a background task based on the first buffer area.
2. The method of claim 1, wherein the first buffer is located at a first level of a log-structured merged Tree (LSM-Tree) structure, and wherein the cache is located at a second level of the LSM-Tree structure, and wherein data in the first level has read-write characteristics and data in the second level has read-only characteristics.
3. The method of claim 2, wherein updating the cache by a background task based on the first buffer comprises:
when the data amount in the first buffer area exceeds a threshold value, converting the first buffer area into a second buffer area, wherein the second buffer area is located in a third level of the LSM-Tree structure, and the data in the third level has a read-only characteristic;
generating a new empty first buffer in the first tier;
and performing data merging on the data in the second buffer area and the data in the cache to obtain an updated cache in the second level.
4. The method of claim 2 or 3, wherein the data in the LSM-Tree structure comprises metadata and data corresponding to the metadata.
5. The method of claim 3, further comprising:
and releasing the second buffer after data merging is carried out on the data in the second buffer and the data in the cache.
6. The method of claim 1, wherein after sending the data to be updated to the first buffer, the method further comprises:
and updating the metadata of the data to be updated in the cache through a foreground task.
7. The method of claim 6, wherein updating the cache by a background task based on the first buffer comprises:
and updating the data to be updated in the cache through a background task.
8. A cache update apparatus, comprising:
the foreground task processing module is used for receiving an IO request sent by a user through a foreground task;
the foreground task processing module is further configured to determine data to be updated by responding to the IO request through a foreground task, and send the data to be updated to a first buffer area;
and the background task processing module is used for updating the cache through a background task based on the first buffer area.
9. The apparatus of claim 8, wherein the first buffer is located at a first level of an LSM-Tree structure, wherein the cache is located at a second level of the LSM-Tree structure, and wherein data in the first level has read-write characteristics and data in the second level has read-only characteristics.
10. The apparatus of claim 9, wherein the background task processing module comprises:
the conversion module is used for converting the first buffer area into a second buffer area when the data volume in the first buffer area exceeds a threshold value, the second buffer area is positioned at a third level of the LSM-Tree structure, and the data in the third level has read-only characteristic;
a generation submodule for generating a new empty first buffer in the first hierarchy;
and the cache updating submodule is used for carrying out data combination on the data in the second buffer area and the data in the cache and obtaining an updated cache in the second level.
11. The apparatus of claim 9 or 10, wherein the data in the LSM-Tree structure comprises metadata and data corresponding to the metadata.
12. The apparatus of claim 10, wherein the background task processing module further comprises:
and the releasing submodule is used for releasing the second buffer area after the data in the second buffer area and the data in the cache are subjected to data combination.
13. The apparatus of claim 8,
and the foreground task processing module is further configured to update the metadata of the data to be updated in the cache through a foreground task after the data to be updated is sent to the first buffer.
14. The apparatus according to claim 13, wherein the background task processing module is specifically configured to:
and updating the data to be updated in the cache through a background task.
15. A cache update apparatus, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the cache update method of any of claims 1-7.
16. A non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the cache update method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910217386.4A CN111722785A (en) | 2019-03-21 | 2019-03-21 | Cache updating method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910217386.4A CN111722785A (en) | 2019-03-21 | 2019-03-21 | Cache updating method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111722785A true CN111722785A (en) | 2020-09-29 |
Family
ID=72563193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910217386.4A Pending CN111722785A (en) | 2019-03-21 | 2019-03-21 | Cache updating method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111722785A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114385067A (en) * | 2020-10-19 | 2022-04-22 | 澜起科技股份有限公司 | Data updating method for memory system and memory controller |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| US20170212680A1 (en) * | 2016-01-22 | 2017-07-27 | Suraj Prabhakar WAGHULDE | Adaptive prefix tree based order partitioned data storage system |
| CN108733316A (en) * | 2017-04-17 | 2018-11-02 | 伊姆西Ip控股有限责任公司 | Method for managing storage system and manager |
| US20190073301A1 (en) * | 2017-09-05 | 2019-03-07 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an i/o operation over a bus interface |
-
2019
- 2019-03-21 CN CN201910217386.4A patent/CN111722785A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| US20170212680A1 (en) * | 2016-01-22 | 2017-07-27 | Suraj Prabhakar WAGHULDE | Adaptive prefix tree based order partitioned data storage system |
| CN108733316A (en) * | 2017-04-17 | 2018-11-02 | 伊姆西Ip控股有限责任公司 | Method for managing storage system and manager |
| US20190073301A1 (en) * | 2017-09-05 | 2019-03-07 | International Business Machines Corporation | Asynchronous update of metadata tracks in response to a cache hit generated via an i/o operation over a bus interface |
Non-Patent Citations (1)
| Title |
|---|
| 马留英;蔡杰明;刘浏;刘振军;: "一种加速广域文件系统读写访问的缓存策略", 计算机研究与发展, no. 1 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114385067A (en) * | 2020-10-19 | 2022-04-22 | 澜起科技股份有限公司 | Data updating method for memory system and memory controller |
| CN114385067B (en) * | 2020-10-19 | 2023-07-18 | 澜起科技股份有限公司 | Data updating method for memory system and memory controller |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107870728B (en) | Method and apparatus for moving data | |
| CN108228646B (en) | Method and electronic device for accessing data | |
| US11288210B2 (en) | Method, device and computer program product for managing storage system | |
| US10838857B2 (en) | Multi-section garbage collection | |
| US10817383B2 (en) | Method, apparatus and computer program product for managing data backup | |
| CN111857550A (en) | Method, apparatus and computer readable medium for data deduplication | |
| US11210003B2 (en) | Method, device and computer program product for restoring data based on replacing child node identifiers with parent node identifier | |
| CN110765036A (en) | Method, apparatus and computer program product for managing metadata at a control device | |
| CN112748863A (en) | Method, electronic device and computer program product for processing data | |
| JP2022159405A (en) | Method and device for appending data, electronic device, storage medium, and computer program | |
| US10747675B2 (en) | Method and device for managing caches | |
| CN111722785A (en) | Cache updating method and device | |
| CN111694992B (en) | Data processing method and device | |
| CN111625180B (en) | Data writing method and device and storage medium | |
| CN109614383B (en) | Data copying method and device, electronic equipment and storage medium | |
| US8533423B2 (en) | Systems and methods for performing parallel multi-level data computations | |
| CN113568561A (en) | Method for information processing, electronic device, and computer storage medium | |
| US11431349B2 (en) | Method, electronic device and computer program product for processing data | |
| US11494100B2 (en) | Method, device and computer program product for storage management | |
| US9697018B2 (en) | Synthesizing inputs to preserve functionality | |
| CN111459398B (en) | Data processing method and device of distributed system | |
| CN115994151B (en) | Data request changing method, device, electronic equipment and computer readable medium | |
| CN115297169B (en) | Data processing method, device, electronic equipment and medium | |
| CN117472556A (en) | Data processing method, electronic device and computer program product | |
| CN112486404A (en) | Method, apparatus and computer program product for managing memory blocks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |