CA3129982A1 - Method and system for accessing distributed block storage system in kernel mode - Google Patents
Method and system for accessing distributed block storage system in kernel mode Download PDFInfo
- Publication number
- CA3129982A1 CA3129982A1 CA3129982A CA3129982A CA3129982A1 CA 3129982 A1 CA3129982 A1 CA 3129982A1 CA 3129982 A CA3129982 A CA 3129982A CA 3129982 A CA3129982 A CA 3129982A CA 3129982 A1 CA3129982 A1 CA 3129982A1
- Authority
- CA
- Canada
- Prior art keywords
- data
- block storage
- distributed block
- read request
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000004590 computer program Methods 0.000 claims description 23
- 238000004891 communication Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 abstract description 8
- 238000012545 processing Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 230000000694 effects Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The present invention disclose a method and a system of accessing distributed block storage system in kernel state, a computer equipment and a storage medium, the system contains a computing node and a distributed block storage cluster, wherein the computing node is applied by the method, and the method comprises: receiving a data read request sent from a data accessor;
judging whether there is target data corresponding to the data read request in a cache of the computing node; if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, to obtain target data corresponding to the data read request and return the target data to the data accessor. By realizing cache mechanism of data at the block driver layer in kernel state, the present invention reduces rounds of switch between the kernel state and the user state, equalizes the use mode of the distributed block device with the use convention of the local disk, to reduce use cost and enhance efficiency of utilizing resources and access efficiency.
judging whether there is target data corresponding to the data read request in a cache of the computing node; if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, to obtain target data corresponding to the data read request and return the target data to the data accessor. By realizing cache mechanism of data at the block driver layer in kernel state, the present invention reduces rounds of switch between the kernel state and the user state, equalizes the use mode of the distributed block device with the use convention of the local disk, to reduce use cost and enhance efficiency of utilizing resources and access efficiency.
Description
METHOD AND SYSTEM FOR ACCESSING DISTRIBUTED BLOCK STORAGE
SYSTEM IN KERNEL MODE
BACKGROUND OF THE INVENTION
Technical Field [0001] The present invention relates to the field of distributed storage technology, and particularly to a method and a system of accessing distributed block storage system in kernel state, a computer equipment and a storage medium.
Description of Related Art
SYSTEM IN KERNEL MODE
BACKGROUND OF THE INVENTION
Technical Field [0001] The present invention relates to the field of distributed storage technology, and particularly to a method and a system of accessing distributed block storage system in kernel state, a computer equipment and a storage medium.
Description of Related Art
[0002] It is usual in the state of the art to provide operational access of a block device via user-state interface in the implementation of distributed block storage. Since such implementation is at the user state, coding implementation is facilitated, kernel stability is not affected, and it is hence favorably received. However, such implementation is disadvantageous in the fact that many rounds of switch between the user state and the kernel state are required for the user to read and write messages, and message delay is increased. At the same time, with regard to implementation at the client side, there are more simple transmission and reception of messages while there are few mechanisms for realizing cache at the client side.
[0003] Accordingly, there is an urgent need to propose a novel method of accessing distributed block storage system to address the aforementioned problems.
SUMMARY OF THE INVENTION
SUMMARY OF THE INVENTION
[0004] In order to solve prior-art problems, embodiments of the present invention provide a method and a system of accessing distributed block storage system in kernel state, a computer equipment and a storage medium, so as to solve such problems prevailing in the state of art as many rounds of switch between the user state and the kernel state are required for the user to read and write messages, message delay is increased, and there are more simple transmission and reception of messages in the implementation at the client side while there are few mechanisms for realizing cache at the client side.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0005] To solve one or more of the aforementioned technical problems, the present invention proposes the following technical solutions.
[0006] According to the first aspect, there is provided a method of accessing distributed block storage system in kernel state, which distributed block storage system contains a computing node and a distributed block storage cluster, and the method comprises the following steps:
[0007] receiving a data read request sent from a data accessor;
[0008] judging whether there is target data corresponding to the data read request in a cache of the computing node; and
[0009] if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0010] Further, the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor includes:
[0011] generating a corresponding thread in a preconfigured thread pool based on the data read request; and
[0012] requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
[0013] Further, the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor further includes:
[0014] writing the target data into the cache after the target data corresponding to the data read request has been requested from the distributed block storage cluster through execution of Date recue / Date received 2021-11-03 the thread.
[0015] Further, the method further comprises:
[0016] receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
[0017] writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and
[0018] writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
[0019] Further, prior to the step of generating a corresponding thread in the preconfigured thread pool, the method further comprises:
[0020] generating a corresponding data write task based on the data write request; and
[0021] periodically executing the data write task to preprocess the to-be-processed data;
[0022] the step of writing the to-be-processed data in the distributed block storage cluster through execution of the thread includes:
[0023] writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
[0024] Further, the computing node includes a computer equipment installed with a Linux system, and the method further comprises a process of realizing the cache, including:
[0025] loading a preconfigured kernel module at a device driver layer of the Linux system, wherein the kernel module contains a cache.
[0026] Further, the kernel module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
[0027] According to the second aspect, there is provided a distributed block storage system, the Date recue / Date received 2021-11-03 system comprises a computing node and a distributed block storage cluster, and the computing node includes:
[0028] a data receiving module, for receiving a data read request sent from a data accessor;
[0029] a data judging module, for judging whether there is target data corresponding to the data read request in a cache of the computing node;
[0030] a data returning module, for returning the target data to the data accessor; and
[0031] a request sending module, for sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0032] According to the third aspect, there is provided a computer equipment, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, and the following steps are realized when the processor executes the computer program:
[0033] receiving a data read request sent from a data accessor;
[0034] judging whether there is target data corresponding to the data read request in a cache of the computing node; and
[0035] if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0036] According to the fourth aspect, there is provided a computer-readable storage medium, storing thereon a computer program, and the following steps are realized when the computer program is executed by a processor:
[0037] receiving a data read request sent from a data accessor;
[0038] judging whether there is target data corresponding to the data read request in a cache of Date recue / Date received 2021-11-03 the computing node; and
[0039] if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0040] Technical solutions provided by the embodiments of the present invention bring out the following advantageous effects.
[0041] In the method and system of accessing distributed block storage system in kernel state, computer equipment and storage medium provided by the embodiments of the present invention, by receiving a data read request sent from a data accessor, judging whether there is target data corresponding to the data read request in a cache of the computing node, and if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor, cache mechanism of data is realized at the block driver layer in kernel state, rounds of switch between kernel state and user state are reduced, the use mode of the distributed block device is equalized with the use convention of the local disk, and use cost is lowered at the same time of effectively utilizing resources and effectively enhancing access efficiency.
[0042] In the method and system of accessing distributed block storage system in kernel state, computer equipment and storage medium provided by the embodiments of the present invention, through such functions as loading a preconfigured kernel module at a device driver layer of the Linux system, providing the kernel module with a cache, enabling the kernel module to be communicable with the distributed block storage cluster through a preset communication protocol, and directly realizing cache at the computing node (corresponding to the client side), the task originally processed by the server side is moved forward to the client side, whereby servicing capability and response speed of the entire cluster are enhanced.
Date recue / Date received 2021-11-03 BRIEF DESCRIPTION OF THE DRAWINGS
Date recue / Date received 2021-11-03 BRIEF DESCRIPTION OF THE DRAWINGS
[0043] To more clearly explain the technical solutions in the embodiments of the present invention, drawings required for use in the following explanation of the embodiments are briefly described below. Apparently, the drawings described below are merely directed to some embodiments of the present invention, while it is further possible for persons ordinarily skilled in the art to base on these drawings to acquire other drawings, and no creative effort will be spent in the process.
[0044] Fig. 1 is a view illustrating the architecture of a distributed block storage system shown according to an exemplary embodiment;
[0045] Fig. 2 is a flowchart of the method of accessing distributed block storage system in kernel state shown according to an exemplary embodiment;
[0046] Fig. 3 is another flowchart of the method of accessing distributed block storage system in kernel state shown according to an exemplary embodiment; and
[0047] Fig. 4 is a view schematically illustrating the internal structure of a computer equipment shown according to an exemplary embodiment.
DETAILED DESCRIPTION OF THE INVENTION
DETAILED DESCRIPTION OF THE INVENTION
[0048] To make more lucid and clear the objectives, technical solutions and advantages of the present invention, technical solutions in the embodiments of the present invention will be described more clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the embodiments described below are partial, rather than the entire, embodiments of the present invention. All other embodiments achievable by persons ordinarily skilled in the art on the basis of the embodiments in the present invention without creative effort shall all fall within the protection scope of the present invention.
[0049] Embodiment 1 Date recue / Date received 2021-11-03
[0050] As noted in the Description of Related Art, in providing operational access of a block device via user-state interface, many rounds of switch between the user state and the kernel state are required for the user to read and write messages, and message delay is increased, while distributed block storage puts a very high requirement on read/write delay, particularly under the scenario of use in database service.
[0051] In order to shorten the delay as far as practically possible, a method of accessing distributed block storage system in kernel state is creatively proposed in the embodiments of the present invention, this method adds support for the distributed block device at the block device driver layer in kernel state. At the same time, the method equalizes the mode of using the distributed block device with the mode of using a local disk, and lowers operational and maintenance expense and use cost. In the embodiments of the present invention, a self-defined kernel module of the distributed block device is mainly added at the device driver layer, and the kernel is enabled to identify the distributed device, directly performs read/write operations on the distributed block storage cluster, reduces the rounds of switch between user state and kernel state, and decreases delay.
[0052] Fig. 1 is a view illustrating the architecture of a distributed block storage system shown according to an exemplary embodiment. Referring to Fig. 1, in the embodiment of the present invention, the distributed block storage system at least comprises a computing node and a distributed block storage cluster, and the distributed block storage cluster includes a plurality of distributed block storage devices, taking for example the realization of distributed block storage device driver at Linux kernel state, i.e., when the computing node is a computer equipment installed with a Linux system, the computing node includes a block device layer and a block device driver layer, and a kernel module is configured in the device driver layer, through which kernel module can be realized such functions as caching, pre-reading and write merging, etc. at the device driver layer (namely the client side), and the task originally processed by the server side is moved forward to the client side, whereby servicing capability and response speed of the entire cluster are enhanced.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0053] Specifically, the foregoing solution can be realized via the following steps.
[0054] Step 1 ¨ realizing distributed block storage client side kernel module, which kernel module includes, but is not limited to include, such functions as link managing function, message transceiving function, data caching function, pre-reading function, write caching function, etc.
[0055] Specifically, in order to enhance the reading/writing performance of the distributed block device, development of the kernel module is performed at the device driver layer, and the kernel module externally provides operation interfaces with respect to the block storage device, including, but not limited to, turning on the block storage device, turning off the block storage device, reading operation, and writing operation, etc. The kernel module can at least realize the following functions.
[0056] The kernel module can establish multi-link communications with the various nodes of the distributed block storage cluster, whereby concurrent capacity is enhanced.
[0057] The kernel module can be realized as a multi-thread mode, and the threads are mainly divided into two types: 10 transceiving threads and TO processing threads. The two types of threads each constitute a thread pool, namely TO transceiving thread pool responsible for transceiving network data, and TO processing thread pool responsible for specifically processing data, such as controlling message analysis, message processing, EC
processing, and so on.
processing, and so on.
[0058] The kernel module can realize cache mechanism of local data. In the process of use, the reading operation preferentially hits local cache, and sends a read request to the cluster if hitting is not done. The writing operation is first cached in a local memory or SSD, write data is subsequently merged, aggregated, and removed of redundancy via timed tasks, and a write request is then sent to the cluster.
[0059] As should be noted here, as a preferred embodiment in the embodiments of the present Date recue / Date received 2021-11-03 invention, the cache mechanism can adopt B-tree storage during specific implementation, and hotspot data is cached in LRU mode.
[0060] Step 2 ¨ loading the distributed block storage device kernel module.
[0061] Specifically, modprobe command can be utilized in the embodiments of the present invention to load the distributed block storage device kernel module, and the well-defined kernel module is loaded to the block driver layer, so that support for the distributed block device is added in the block driver layer, and distributed block device module interfaces are invoked. After loading, kernel codes are compiled, and kernel is updated, the machine is rebooted, and the machine indicates a computing node required to use the block device in this context.
[0062] Modprobe is a command of linux, and can be loaded into a separate module, or loaded into a set of dependent modules. Modprobe will base on the dependency generated by depmod to decide into which modules to be loaded. Should error occur during the loading process, the whole set of modules will be unloaded at modprobe.
[0063] Step 3 ¨ mapping the foregoing distributed block storage device into a local system via a self-defined command.
[0064] Specifically, in the embodiments of the present invention, the kernel module realizes a map command to map the block device already existent in the distributed block storage cluster into the local computing node, and the device newly added in the disk list is precisely the distributed block device. After operation is completed, the distributed block device can be used just like a local disk is used, in other words, the distributed block device is read and written like the local disk is operated.
[0065] Embodiment 2
[0066] Figs. 2 and 3 are flowcharts of the method of accessing distributed block storage system in kernel state shown according to an exemplary embodiment. Referring to Figs.
2 and 3, the Date recue / Date received 2021-11-03 method comprises the following steps.
2 and 3, the Date recue / Date received 2021-11-03 method comprises the following steps.
[0067] Si - receiving a data read request sent from a data accessor.
[0068] Specifically, taking for example the computing node being a computer equipment installed with a Linux system, in the embodiment of the present invention, a kernel module is preconfigured in the computing node, and the kernel module can realize such functions as caching, pre-reading and write merging, etc. at the device driver layer (namely the client side). After the data accessor has sent the data read request, the data read request is transmitted via the block device layer to the block device driver layer, and received and processed by the kernel module.
[0069] S2 - judging whether there is target data corresponding to the data read request in a cache of the computing node.
[0070] Specifically, in the embodiment of the present invention, the kernel module is configured to have cache mechanism of local data. After receiving the data read request, the kernel module performs analysis processing on the data read request, and bases on the analysis result to judge whether there is target data corresponding to the data read request in a cache of the computing node.
[0071] S3 - if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0072] Specifically, if target data corresponding to the data read request is found in the cache, the target data is directly returned to the data accessor, whereby it is no longer required to request the target data from the distributed block storage cluster, reduces the rounds of switch between the kernel state and the user state, effectively utilizes resources, and enhances access efficiency. In the case target data corresponding to the data read request is not found in the cache, at this time the data read request is sent to the distributed block storage cluster, so as Date recue / Date received 2021-11-03 to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0073] As a preferred embodiment in the embodiments of the present invention, the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor includes:
[0074] generating a corresponding thread in a preconfigured thread pool based on the data read request; and
[0075] requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
[0076] Specifically, in the embodiments of the present invention, the kernel module can be realized as a multi-thread mode, and the threads are mainly divided into two types: TO
transceiving threads and TO processing threads. The two types of threads each constitute a thread pool, namely TO transceiving thread pool responsible for transceiving network data, and TO processing thread pool responsible for specifically processing data, such as controlling message analysis, message processing, EC processing, and so on.
When target data corresponding to the data read request is not found in the cache, the data read request is based on to generate a corresponding thread in a preconfigured thread pool, the target data corresponding to the data read request is then requested from the distributed block storage cluster through execution of the thread, and the target data returned by the distributed block storage cluster is sent to the data accessor.
transceiving threads and TO processing threads. The two types of threads each constitute a thread pool, namely TO transceiving thread pool responsible for transceiving network data, and TO processing thread pool responsible for specifically processing data, such as controlling message analysis, message processing, EC processing, and so on.
When target data corresponding to the data read request is not found in the cache, the data read request is based on to generate a corresponding thread in a preconfigured thread pool, the target data corresponding to the data read request is then requested from the distributed block storage cluster through execution of the thread, and the target data returned by the distributed block storage cluster is sent to the data accessor.
[0077] As a preferred embodiment in the embodiments of the present invention, the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor further includes:
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0078] writing the target data into the cache after the target data corresponding to the data read request has been requested from the distributed block storage cluster through execution of the thread.
[0079] Specifically, in the embodiments of the present invention, after the target data corresponding to the data read request has been requested from the distributed block storage cluster, it is further needed to write the target data into the cache, so that it is possible to directly hit the data from the cache during subsequent reception of this read request of the target data, whereby the rounds of switch between user state and kernel state are reduced.
During specific operation, it is possible to write the target data into the cache by processing the thread.
During specific operation, it is possible to write the target data into the cache by processing the thread.
[0080] As a preferred embodiment in the embodiments of the present invention, the method further comprises:
[0081] receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
[0082] writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and
[0083] writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
[0084] Specifically, likewise in the embodiments of the present invention, when the kernel module receives the data write request sent from the data accessor, the to-be-processed data carried with the write request is first written in the cache of the computing node (namely the cache of the kernel module), a corresponding thread is subsequently generated in the preconfigured thread pool, and the to-be-processed data is then written in the distributed block storage cluster through execution of the thread.
[0085] As a preferred embodiment in the embodiments of the present invention, prior to the step Date recue / Date received 2021-11-03 of generating a corresponding thread in the preconfigured thread pool, the method further comprises:
[0086] generating a corresponding data write task based on the data write request; and
[0087] periodically executing the data write task to preprocess the to-be-processed data;
[0088] and the step of writing the to-be-processed data in the distributed block storage cluster through execution of the thread includes:
[0089] writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
[0090] Specifically, in the embodiments of the present invention, preprocess of the to-be-processed data includes such operations on the to-be-processed data as merging, aggregating and removing of redundancy, etc., to which no repetition is made in this context.
[0091] As a preferred embodiment in the embodiments of the present invention, the computing node includes a computer equipment installed with a Linux system, and the method further comprises a process of realizing the cache, including:
[0092] loading a preconfigured kernel module at a device driver layer of the Linux system, wherein the kernel module contains a cache.
[0093] As a preferred embodiment in the embodiments of the present invention, the kernel module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
[0094] Embodiment 3
[0095] In the embodiments of the present invention, there is further provided a distributed block storage system, which comprises a computing node and a distributed block storage cluster, and the computing node includes:
[0096] a data receiving module, for receiving a data read request sent from a data accessor;
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0097] a data judging module, for judging whether there is target data corresponding to the data read request in a cache of the computing node;
[0098] a data returning module, for returning the target data to the data accessor; and
[0099] a request sending module, for sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0100] As a preferred embodiment in the embodiments of the present invention, the request sending module is specifically employed for:
[0101] generating a corresponding thread in a preconfigured thread pool based on the data read request; and
[0102] requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
[0103] As a preferred embodiment in the embodiments of the present invention, the request sending module is further employed for:
[0104] writing the target data into the cache after the target data corresponding to the data read request has been requested from the distributed block storage cluster through execution of the thread.
[0105] As a preferred embodiment in the embodiments of the present invention, the data receiving module is further employed for:
[0106] receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster.
[0107] The computing node further includes:
[0108] a data writing module, for writing the to-be-processed data in the cache of the computing Date recue / Date received 2021-11-03 node, generating a corresponding thread in the preconfigured thread pool, and writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
[0109] As a preferred embodiment in the embodiments of the present invention, the computing node further includes:
[0110] a task generating module, for generating a corresponding data write task based on the data write request; and
[0111] a preprocessing module, for periodically executing the data write task to preprocess the to-be-processed data;
[0112] the data writing module is further employed for:
[0113] writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
[0114] As a preferred embodiment in the embodiments of the present invention, the computing node includes a computer equipment installed with a Linux system, the device driver layer of the Linux system is loaded with a preconfigured kernel module, and the kernel module contains a cache.
[0115] As a preferred embodiment in the embodiments of the present invention, the kernel module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
[0116] Embodiment 4
[0117] Fig. 4 is a view schematically illustrating the internal structure of a computer equipment shown according to an exemplary embodiment. Referring to Fig. 4, the computer equipment comprises a processor, a memory, and a network interface connected to each other via a system bus. The processor of the computer equipment is employed to provide computing and controlling capabilities. The memory of the computer equipment includes a nonvolatile Date recue / Date received 2021-11-03 storage medium, and an internal memory. The nonvolatile storage medium stores therein an operating system, a computer program and a database. The internal memory provides environment for the running of the operating system and the computer program in the nonvolatile storage medium. The network interface of the computer equipment is employed to connect to an external terminal via network for communication. The computer program realizes a method of accessing distributed block storage system in kernel state when it is executed by a processor.
[0118] As understandable to persons skilled in the art, the structure illustrated in Fig. 4 is merely a block diagram of partial structure relevant to the solution of the present invention, and does not constitute any restriction to the computer equipment on which the solution of the present invention is applied, as the specific computer equipment may comprise component parts that are more than or less than those illustrated in Fig. 4, or may combine certain component parts, or may have different layout of component parts.
[0119] As a preferred embodiment in the embodiments of the present invention, the computer equipment comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the following steps are realized when the processor executes the computer program:
[0120] receiving a data read request sent from a data accessor;
[0121] judging whether there is target data corresponding to the data read request in a cache of the computing node; and
[0122] if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0123] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the processor executes the computer program:
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0124] generating a corresponding thread in a preconfigured thread pool based on the data read request; and
[0125] requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
[0126] As a preferred embodiment in the embodiments of the present invention, the following step is further realized when the processor executes the computer program:
[0127] writing the target data into the cache after the target data corresponding to the data read request has been requested from the distributed block storage cluster through execution of the thread.
[0128] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the processor executes the computer program:
[0129] receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
[0130] writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and
[0131] writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
[0132] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the processor executes the computer program:
[0133] generating a corresponding data write task based on the data write request; and
[0134] periodically executing the data write task to preprocess the to-be-processed data; and
[0135] writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0136] As a preferred embodiment in the embodiments of the present invention, the computing node includes a computer equipment installed with a Linux system, the device driver layer of the Linux system is loaded with a preconfigured kernel module, and the kernel module contains a cache.
[0137] As a preferred embodiment in the embodiments of the present invention, the kernel module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
[0138] Embodiment 5
[0139] In the embodiments of the present invention, there is further provided a computer-readable storage medium, storing thereon a computer program, and the following steps are realized when the computer program is executed by a processor:
[0140] receiving a data read request sent from a data accessor;
[0141] judging whether there is target data corresponding to the data read request in a cache of the computing node; and
[0142] if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
[0143] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the computer program is executed by a processor:
[0144] generating a corresponding thread in a preconfigured thread pool based on the data read request; and
[0145] requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
[0146] As a preferred embodiment in the embodiments of the present invention, the following step is further realized when the computer program is executed by a processor:
[0147] writing the target data into the cache after the target data corresponding to the data read request has been requested from the distributed block storage cluster through execution of the thread.
[0148] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the computer program is executed by a processor:
[0149] receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
[0150] writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and
[0151] writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
[0152] As a preferred embodiment in the embodiments of the present invention, the following steps are further realized when the computer program is executed by a processor:
[0153] generating a corresponding data write task based on the data write request; and
[0154] periodically executing the data write task to preprocess the to-be-processed data; and
[0155] writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
[0156] As a preferred embodiment in the embodiments of the present invention, the computing node includes a computer equipment installed with a Linux system, the device driver layer of the Linux system is loaded with a preconfigured kernel module, and the kernel module contains a cache.
[0157] As a preferred embodiment in the embodiments of the present invention, the kernel Date recue / Date received 2021-11-03 module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
[0158] In summary, the advantageous effects brought about by the technical solutions provided by the embodiments of the present invention are as follows.
[0159] In the method and system of accessing distributed block storage system in kernel state, computer equipment and storage medium provided by the embodiments of the present invention, by receiving a data read request sent from a data accessor, judging whether there is target data corresponding to the data read request in a cache of the computing node, and if yes, returning the target data to the data accessor; if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor, cache mechanism of data is realized at the block driver layer in kernel state, rounds of switch between kernel state and user state are reduced, the use mode of the distributed block device is equalized with the use convention of the local disk, and use cost is lowered at the same time of effectively utilizing resources and effectively enhancing access efficiency.
[0160] In the method and system of accessing distributed block storage system in kernel state, computer equipment and storage medium provided by the embodiments of the present invention, through such functions as loading a preconfigured kernel module at a device driver layer of the Linux system, providing the kernel module with a cache, enabling the kernel module to be communicable with the distributed block storage cluster through a preset communication protocol, and directly realizing cache at the computing node (corresponding to the client side), the task originally processed by the server side is moved forward to the client side, whereby servicing capability and response speed of the entire cluster are enhanced.
[0161] As should be noted, the various embodiments are progressively described in this Description, identical or similar sections of the embodiments can be cross-referenced from Date recue / Date received 2021-11-03 one another, while the gist of each embodiment lies in its difference from other embodiments.
Particularly, with regard to system or system embodiment, since these are substantially similar to method embodiment, their descriptions are relatively simple, as relevant sections can be cross-referenced from the corresponding sections of the method embodiment. The foregoing descriptions of system or system embodiment are merely schematic, and units explained as separate parts may be or may not be physically separate, while parts shown as units may be or may not be physical units, that is to say, these can be located at a single site, and can also be distributed on a plural of network units. It is possible to select partial or entire modules therefrom as practically required to realize the objectives of the embodiment solutions to the effect that they are understandable and implementable without creative effort from persons ordinarily skilled in the art.
Particularly, with regard to system or system embodiment, since these are substantially similar to method embodiment, their descriptions are relatively simple, as relevant sections can be cross-referenced from the corresponding sections of the method embodiment. The foregoing descriptions of system or system embodiment are merely schematic, and units explained as separate parts may be or may not be physically separate, while parts shown as units may be or may not be physical units, that is to say, these can be located at a single site, and can also be distributed on a plural of network units. It is possible to select partial or entire modules therefrom as practically required to realize the objectives of the embodiment solutions to the effect that they are understandable and implementable without creative effort from persons ordinarily skilled in the art.
[0162] As comprehensible to persons ordinarily skilled in the art, the entire or partial steps in the aforementioned embodiments can be completed via hardware, or via a program instructing relevant hardware, the program can be stored in a computer-readable storage medium, and the storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.
[0163] The foregoing embodiments are merely preferred embodiments of the present invention, and they are not to be construed as restrictive to the present invention. Any amendment, equivalent substitution, and improvement makeable within the spirit and principle of the present invention shall all fall within the protection scope of the present invention.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
Claims (10)
1. A method of accessing distributed block storage system in kernel state, the distributed block storage system containing a computing node and a distributed block storage cluster, characterized in that the method comprises the following steps:
receiving a data read request sent from a data accessor;
judging whether there is target data corresponding to the data read request in a cache of the computing node; and if yes, returning the target data to the data accessor; and if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
receiving a data read request sent from a data accessor;
judging whether there is target data corresponding to the data read request in a cache of the computing node; and if yes, returning the target data to the data accessor; and if not, sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
2. The method of accessing distributed block storage system in kernel state according to Claim 1, characterized in that the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor includes:
generating a corresponding thread in a preconfigured thread pool based on the data read request;
and requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
generating a corresponding thread in a preconfigured thread pool based on the data read request;
and requesting target data corresponding to the data read request from the distributed block storage cluster through execution of the thread and returning the target data to the data accessor.
3. The method of accessing distributed block storage system in kernel state according to Claim 2, characterized in that the step of sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor further includes:
writing the target data into the cache after the target data corresponding to the data read request Date recue / Date received 2021-11-03 has been requested from the distributed block storage cluster through execution of the thread.
writing the target data into the cache after the target data corresponding to the data read request Date recue / Date received 2021-11-03 has been requested from the distributed block storage cluster through execution of the thread.
4. The method of accessing distributed block storage system in kernel state according to anyone of Claims 1 to 3, characterized in further comprising:
receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
receiving a data write request sent from the data accessor, the data write request including to-be-processed data to be written in the distributed block storage cluster;
writing the to-be-processed data in the cache of the computing node, and generating a corresponding thread in the preconfigured thread pool; and writing the to-be-processed data in the distributed block storage cluster through execution of the thread.
5. The method of accessing distributed block storage system in kernel state according to Claim 4, characterized in that, prior to the step of generating a corresponding thread in the preconfigured thread pool, the method further comprises:
generating a corresponding data write task based on the data write request;
and periodically executing the data write task to preprocess the to-be-processed data;
and that the step of writing the to-be-processed data in the distributed block storage cluster through execution of the thread includes:
writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
generating a corresponding data write task based on the data write request;
and periodically executing the data write task to preprocess the to-be-processed data;
and that the step of writing the to-be-processed data in the distributed block storage cluster through execution of the thread includes:
writing the preprocessed to-be-processed data in the distributed block storage cluster through execution of the thread.
6. The method of accessing distributed block storage system in kernel state according to Claim 1 or 2, characterized in that the computing node includes a computer equipment installed with a Linux system, and that the method further comprises a process of realizing the cache, including:
loading a preconfigured kernel module at a device driver layer of the Linux system, wherein the kernel module contains a cache.
Date recue / Date received 2021-11-03
loading a preconfigured kernel module at a device driver layer of the Linux system, wherein the kernel module contains a cache.
Date recue / Date received 2021-11-03
7. The method of accessing distributed block storage system in kernel state according to Claim 6, characterized in that the kernel module is communicable with the distributed block storage cluster through a preset communication protocol, so as to perform read and/or write operation(s).
8. A distributed block storage system, comprising a computing node and a distributed block storage cluster, characterized in that the computing node includes:
a data receiving module, for receiving a data read request sent from a data accessor;
a data judging module, for judging whether there is target data corresponding to the data read request in a cache of the computing node;
a data returning module, for returning the target data to the data accessor;
and a request sending module, for sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
a data receiving module, for receiving a data read request sent from a data accessor;
a data judging module, for judging whether there is target data corresponding to the data read request in a cache of the computing node;
a data returning module, for returning the target data to the data accessor;
and a request sending module, for sending the data read request to the distributed block storage cluster, so as to obtain target data corresponding to the data read request and return the target data to the data accessor.
9. A computer equipment, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to anyone of Claims 1 to 7 are realized when the processor executes the computer program.
10. A computer-readable storage medium, storing thereon a computer program, characterized in that the steps of the method according to anyone of Claims 1 to 7 are realized when the computer program is executed by a processor.
Date recue / Date received 2021-11-03
Date recue / Date received 2021-11-03
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010919808.5A CN112039999A (en) | 2020-09-03 | 2020-09-03 | Method and system for accessing distributed block storage system in kernel mode |
| CN202010919808.5 | 2020-09-03 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA3129982A1 true CA3129982A1 (en) | 2022-03-03 |
Family
ID=73591487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3129982A Pending CA3129982A1 (en) | 2020-09-03 | 2021-09-03 | Method and system for accessing distributed block storage system in kernel mode |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112039999A (en) |
| CA (1) | CA3129982A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052291A (en) * | 2020-09-03 | 2020-12-08 | 苏宁云计算有限公司 | Method and system for accessing distributed block storage system by user mode |
| CN112463662B (en) * | 2020-12-16 | 2024-04-05 | 福州创实讯联信息技术有限公司 | Method and terminal for user mode control of I2C equipment |
| CN116107764B (en) * | 2023-04-03 | 2023-09-26 | 阿里云计算有限公司 | Data processing method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544324B (en) * | 2013-11-11 | 2017-09-08 | 北京搜狐新媒体信息技术有限公司 | A kind of data access method of kernel state, apparatus and system |
| CN106648886A (en) * | 2016-10-20 | 2017-05-10 | 郑州云海信息技术有限公司 | Realization method and apparatus for distributed storage system |
| CN111008233B (en) * | 2019-11-24 | 2023-02-28 | 浪潮电子信息产业股份有限公司 | Method, device and equipment for accessing KV database |
-
2020
- 2020-09-03 CN CN202010919808.5A patent/CN112039999A/en active Pending
-
2021
- 2021-09-03 CA CA3129982A patent/CA3129982A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN112039999A (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3129982A1 (en) | Method and system for accessing distributed block storage system in kernel mode | |
| US10282293B2 (en) | Method, switch, and multiprocessor system using computations and local memory operations | |
| CN108762668B (en) | Method and device for processing write conflict | |
| CA3129984A1 (en) | Method and system for accessing distributed block storage system in user mode | |
| CN109564502B (en) | Processing method and device applied to access request in storage device | |
| CN112732501B (en) | Test method and multiprocessor SOC chip | |
| US7721278B2 (en) | Modular server architecture for multi-environment HTTP request processing | |
| CN109857545B (en) | Data transmission method and device | |
| EP4287023A1 (en) | Service processing method and device | |
| CN113794764A (en) | Request processing method and medium for server cluster and electronic device | |
| CN113032099A (en) | Cloud computing node, file management method and device | |
| US7043603B2 (en) | Storage device control unit and method of controlling the same | |
| JP4322240B2 (en) | Reboot method, system and program | |
| CN109347936B (en) | Redis proxy client implementation method, system, storage medium and electronic device | |
| CN116414735A (en) | Data storage method, system, storage access configuration method and related equipment | |
| CN114706531A (en) | Data processing method, device, chip, equipment and medium | |
| US20100250651A1 (en) | Data access method for making asynchronous request to block device | |
| US20230244417A1 (en) | Storage node, storage device, and network chip | |
| US20230281141A1 (en) | Method for order-preserving execution of write request and network device | |
| CN113805789A (en) | Metadata processing method in storage device and related device | |
| CN112948336B (en) | Data acceleration method, cache unit, electronic device and storage medium | |
| CN113468195B (en) | Server data cache updating method, system and main database server | |
| EP4123470A1 (en) | Data access method and apparatus | |
| CN117093158B (en) | Storage node, system and data processing method and device of distributed storage system | |
| US11677624B2 (en) | Configuration of a server in view of a number of clients connected to the server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |