CN109375988B - Distributed lock implementation method and device - Google Patents
Distributed lock implementation method and device Download PDFInfo
- Publication number
- CN109375988B CN109375988B CN201811291959.XA CN201811291959A CN109375988B CN 109375988 B CN109375988 B CN 109375988B CN 201811291959 A CN201811291959 A CN 201811291959A CN 109375988 B CN109375988 B CN 109375988B
- Authority
- CN
- China
- Prior art keywords
- distributed lock
- rbd
- distributed
- daemon process
- sanlock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000006837 decompression Effects 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000003116 impacting effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a distributed lock implementation method and device. The method relates to a virtualization computing platform technology, and solves the problems that the memory burden is increased and the performance is influenced by the compatibility of an integrated I/O model and an RBD distributed lock. The method comprises the following steps: the node calls the handle and opens a daemon process special for the distributed lock information; and the node uses the daemon process to perform I/O operation on the shared storage of the distributed lock. The technical scheme provided by the invention is suitable for the Sanlock distributed lock, and the efficient RBD-supporting distributed lock system is realized.
Description
Technical Field
The present invention relates to a virtualization computing platform technology, and in particular, to a distributed lock implementation method and apparatus.
Background
With the rapid development of computer technology and network technology, large clusters are widely used in practical production environments. Through virtualization, the server cluster constitutes a virtualization system of the cloud computing platform. In a virtualization system, coordination is often required, different systems or different hosts in the same system share the same resource or a group of resources, and mutual exclusion is often required to prevent mutual interference to ensure consistency, so that a distributed lock is required to ensure the implementation of the above situations.
Distributed locks generally need to support use in large-scale environments, and therefore, a leased distributed lock needs to be used to ensure the uniqueness of a node that acquires a lock resource, and to ensure that the same node acquires the resource uniquely. Since the acquirer of the distributed lock is a member of the host in the cluster, the implementation of the distributed lock is divided into two types: one is a network type, DLM implementation; one is a shared storage type, namely sanlock implementation.
As distributed storage Ceph, the distributed storage Ceph has the characteristics of expansibility and robustness, and is cheaper, so the distributed storage Ceph is popular with cluster users. Their iSCSI services are generally used, but the I/O path for iSCSI services is too long, thus severely impacting the performance of their distributed storage. However, since Ceph provides an RBD storage pool, I/O is performed through RBDs, and communication is directly performed with Ceph nodes, the I/O path is short, and therefore the performance is good. Moreover, when the RBD is used, multipath is carried by the RBD, and third-party multipath software is not needed, so that more users want to use the RBD mode to perform Ceph I/O, but no distributed locking scheme capable of supporting the RBD exists.
In past attempts, an I/O model mode of the sanlock integrated RBD is realized, but the mode of increasing the I/O model directly increases the memory occupied by the sanlock by more than 300M, the cost of the increase of the memory is too high, and the consumed resource amount is too large.
Disclosure of Invention
In order to solve the technical problem, the invention provides a distributed lock implementation method and a distributed lock implementation device. The read-write operation of the distributed lock is carried out through the special daemon process of the third-party agent, the distributed lock system supporting the RBD with high efficiency is realized, and the problems that the memory burden is increased and the performance is influenced when the integrated I/O model is compatible with the RBD distributed lock are solved.
In order to achieve the purpose of the invention, the invention provides a distributed lock implementation method, which comprises the following steps:
the node calls the handle and opens a daemon process special for the distributed lock information;
and the node uses the daemon process to perform I/O operation on the shared storage of the distributed lock.
Preferably, when multiple I/O operations exist, the node completes the multiple I/O operations through different threads in the daemon process.
Preferably, the step of performing the I/O operation on the shared storage of the distributed lock specifically includes:
communicating with the shared storage of the distributed lock via a Local Socket of the node.
Preferably, the information interacted by the node with the shared storage of the distributed lock includes operation information of the I/O operation and volume information targeted by the operation.
Preferably, the I/O operation on the shared storage is realized by performing the I/O operation on a reliable autonomous distributed object storage block device (RBD) in the distributed storage system.
Preferably, after the step of performing the I/O operation on the shared storage of the distributed lock by the node using the daemon, the method further includes:
and checking the states of the reliable autonomous distributed object and the RBD when the I/O operation fails, closing the handle which is opened originally when the states are abnormal, and calling the handle again to open the reliable autonomous distributed object and the RBD.
The invention also provides a distributed lock implementation device, which comprises:
the special process management module is used for calling the handle and opening a special daemon process for the distributed lock information;
and the I/O module is used for performing I/O operation on the shared storage of the distributed lock by using the daemon process.
Preferably, the I/O module is specifically configured to, when multiple I/O operations exist, complete the multiple I/O operations by the node through different threads in the daemon process.
Preferably, the I/O module communicates with the shared storage of the distributed lock via a Local Socket.
Preferably, the dedicated process management module is further configured to close an originally opened handle when an I/O operation fails due to an abnormality, and to open the reliable autonomous distributed object and the RBD by recalling the handle.
The invention provides a method and a device for realizing a distributed lock. Through a special daemon process, the third-party agent I/O operation is realized, the problems that the memory burden is increased and the performance is influenced due to the fact that an integrated I/O model is compatible with the RBD distributed lock are solved, and the efficient RBD supporting distributed lock system is realized.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
Fig. 1 is a schematic flowchart of a distributed lock implementation method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a distributed lock implementation apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
In past attempts, an I/O model mode of the sanlock integrated RBD is realized, but the mode of increasing the I/O model directly increases the memory occupied by the sanlock by more than 300M, the cost of the increase of the memory is too high, and the consumed resource amount is too large.
In order to solve the above problem, embodiments of the present invention provide a distributed lock implementation method and apparatus. The read-write operation of the distributed lock is carried out through the special daemon process of the third-party agent, the distributed lock system supporting the RBD with high efficiency is realized, and the problems that the memory burden is increased and the performance is influenced when the integrated I/O model is compatible with the RBD distributed lock are solved.
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
The embodiment of the invention provides a method for realizing a distributed lock, and the flow for realizing the distributed lock by using the method is shown in figure 1, and comprises the following steps:
and 101, calling a handle by the node, and opening a daemon process special for the distributed lock information.
In this step, when there are multiple I/O operations, the node completes the multiple I/O operations through different threads in the daemon process. The thread number is dynamically adjusted by a dynamic multithreading technology, so that I/O blockage caused by high pressure of I/O is avoided.
The embodiment of the invention adds a third-party I/O proxy mode by calling the handle, so that the sanlock type distributed lock can support an I/O model of the RBD, thereby realizing a set of distributed lock system capable of supporting the RBD. The memory space occupied by the sanlock service is reduced, the I/O model of the sanlock supporting the RBD is realized, and the distributed lock model supporting the RBD storage is increased.
And 102, the node uses the daemon process to perform I/O operation on the shared storage of the distributed lock.
In this step, communication is performed with the shared storage of the distributed lock through the Local Socket of the node. And I/O issuing is carried out through Local Socket, so that the stability and the transmission speed of the message are improved.
I/O operations to the shared storage are performed by performing I/O operations to a reliable autonomous distributed object storage block device (RBD) in a distributed storage system.
The information interacted by the node with the shared storage of the distributed lock comprises operation information of the I/O operation and volume information aimed at by the operation.
And 103, checking the states of a Reliable Autonomous Distributed Object (RADOS) and the RBD when the I/O operation fails, closing the originally opened handle when the states are abnormal, and calling the handle again to open the RADOS and the RBD.
The third-party I/O proxy service process is used as a special daemon process to dynamically detect the opened handles of RADOS and RBD, so that the problem that I/O cannot be issued when the opened handles are in problem is avoided.
An embodiment of the present invention further provides a distributed lock implementation method, which can implement the sanlock supporting the RBD.
In the embodiment of the invention, a daemon process is added, and I/O operation is carried out through the daemon process. Because the I/O model of the integrated RBD causes the memory occupied by sanlock to be larger, and also causes the memory to increase along with the increase of the opened files, the embodiment of the invention uses the agent of a third party to process the I/O of the RBD, simultaneously keeps the long connection of the RBD, and avoids the increase of the memory occupied by the process caused by frequently opening and closing the image of the RBD. The method is realized by a third party agent, so that the memory information occupied by the process is reduced, and meanwhile, the I/O pressure of the RBD is dynamically reduced through a dynamic thread pool.
The method comprises the steps that sanlock communicates through Local Socket when I/O is carried out, pressure sharing when sanlock carries out I/O is achieved through a dynamic multithreading mechanism in a daemon process, states of RBD and RADS are checked when I/O fails, if problems exist, an originally opened handle is closed, and RADS and RBD are opened again. Through the realization of the third-party agent, the checking operation of the handle of the RBD is added, and the problem that the I/O cannot be issued due to the problem of the opened handle is avoided. The Local Socket is used for realizing the data transmission, and simultaneously, the memory occupied by the sanlock process can be reduced.
An embodiment of the present invention provides a method for implementing a distributed lock, which implements a read-write model of sanlock on an RBD volume, that is, sanlock can be used on an RBD, thereby implementing a set of distributed lock systems supporting RBD.
1) Increasing the support of sanlock to RBD, modifying the diskio process of sanlock, dynamically supporting RBD reading and writing, simultaneously realizing Local client, and transmitting information to be subjected to I/O operation and I/O volume information through Local Socket.
2) The method realizes a daemon process supporting the RBD read-write flow, realizes a server end of the RBD I/O, receives the transmission content of a client end called by sanlock, carries out the RBD I/O, and simultaneously realizes a dynamic multithreading technology, thereby avoiding the decompression operation when the pressure of the sanlock to the I/O of the RBD is higher than that, simultaneously realizing the long connection of the RBD read-write, and avoiding the increase of the memory occupied by the process due to frequent opening, reading-write and closing.
An embodiment of the present invention further provides a distributed lock implementation apparatus, the structure of which is shown in fig. 2, including:
the special process management module 201 is used for calling a handle and opening a special daemon process for the distributed lock information;
and the I/O module 202 is used for performing I/O operation on the shared storage of the distributed lock by using the daemon process.
Preferably, the I/O module 202 is specifically configured to, when multiple I/O operations exist, the node completes the multiple I/O operations through different threads in the daemon process.
Preferably, the I/O module 202 communicates with the shared storage of the distributed lock via a Local Socket.
Preferably, the dedicated process management module 201 is further configured to close an originally opened handle when an I/O operation fails due to an abnormality, and to open the reliable autonomous distributed object and the RBD by recalling the handle.
The embodiment of the invention provides a method and a device for realizing a distributed lock. Through a special daemon process, the third-party agent I/O operation is realized, the problems that the memory burden is increased and the performance is influenced due to the fact that an integrated I/O model is compatible with the RBD distributed lock are solved, and the efficient RBD supporting distributed lock system is realized.
Through increasing third party agent, make sanlock support the read-write flow of RBD volume to make sanlock can compatible RBD, realized one set of distributed lock system that can support RBD, more convenient let the cluster use distributed block storage, guarantee the data uniformity of distributed block storage.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (10)
1. A distributed lock implementation method, comprising:
after the sanlock integrates an I/O model of the RBD, if the I/O operation of the RBD needs to be processed, the node calls a handle and starts a daemon process special for distributed lock information of a third-party agent;
the node uses the daemon process to perform I/O operation on shared storage of a distributed lock system of the RBD;
the daemon process of the third-party agent processes the I/O of the RDB and keeps long connection of the RDB, sanlock communicates through Local Socket, and the daemon process shares pressure on sanlock I/O through a dynamic multithreading mechanism.
2. The distributed lock implementation method of claim 1, wherein when multiple I/O operations exist, the node completes the multiple I/O operations through different threads in the daemon process.
3. The distributed lock implementation method according to claim 1, wherein the step of performing the I/O operation on the shared storage of the distributed lock specifically includes:
communicating with the shared storage of the distributed lock via a Local Socket of the node.
4. The distributed lock implementation method of claim 3, wherein the information that the node interacts with the shared storage of the distributed lock comprises operation information of I/O operations and volume information for operations.
5. The distributed lock implementation method of claim 1, wherein the I/O operations on the shared storage are implemented by performing I/O operations on a reliable autonomous distributed object storage block device (RBD) in a distributed storage system.
6. The distributed lock implementation method of claim 1, wherein after the step of the node performing the I/O operation on the shared storage of the distributed lock using the daemon process, the node further comprises:
and checking the states of the reliable autonomous distributed object and the RBD when the I/O operation fails, closing the handle which is opened originally when the states are abnormal, and calling the handle again to open the reliable autonomous distributed object and the RBD.
7. A distributed lock implementation apparatus, comprising:
the special process management module is used for calling a handle and opening a daemon process special for the distributed lock information of a third-party agent if the I/O operation of the RBD is to be processed after the input/output model of the RBD is integrated by the sanlock;
the I/O module is used for performing I/O operation on shared storage of the RBD distributed lock system by using the daemon process;
the daemon process of the third-party agent processes the I/O of the RDB and keeps long connection of the RDB, sanlock communicates through Local Socket, and the daemon process shares pressure on sanlock I/O through a dynamic multithreading mechanism.
8. The distributed lock realization apparatus of claim 7,
the I/O module is specifically configured to complete multiple I/O operations through different threads in the daemon process when the multiple I/O operations exist.
9. The distributed lock implementation apparatus of claim 7, wherein the I/O module communicates with the shared storage of the distributed lock via a Local Socket.
10. The distributed lock realization apparatus of claim 7,
the special process management module is also used for reliably and autonomously opening the distributed objects and the RBDs when I/O operation fails due to abnormity, closing the handles which are opened originally and re-calling the handles to open the reliable and autonomously distributed objects and the RBDs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811291959.XA CN109375988B (en) | 2018-11-01 | 2018-11-01 | Distributed lock implementation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811291959.XA CN109375988B (en) | 2018-11-01 | 2018-11-01 | Distributed lock implementation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109375988A CN109375988A (en) | 2019-02-22 |
| CN109375988B true CN109375988B (en) | 2021-07-30 |
Family
ID=65396841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811291959.XA Active CN109375988B (en) | 2018-11-01 | 2018-11-01 | Distributed lock implementation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109375988B (en) |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5699500A (en) * | 1995-06-01 | 1997-12-16 | Ncr Corporation | Reliable datagram service provider for fast messaging in a clustered environment |
| US7565406B2 (en) * | 2002-07-24 | 2009-07-21 | Sun Microsystems, Inc. | Last thread lock management for multi-threaded process and distributed data systems |
| CN106572130B (en) * | 2015-10-09 | 2020-10-27 | 阿里巴巴集团控股有限公司 | Method and apparatus for implementing distributed lock management |
| CN106201357A (en) * | 2016-07-18 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The construction method of a kind of storage pool and system |
| CN106909384B (en) * | 2017-02-27 | 2020-09-04 | 郑州云海信息技术有限公司 | Sanlock optimization method and device based on cluster file system |
| CN107357664B (en) * | 2017-07-13 | 2020-09-08 | 苏州浪潮智能科技有限公司 | Method and device for realizing distributed lock |
| CN107239238B (en) * | 2017-07-27 | 2020-09-04 | 郑州云海信息技术有限公司 | Storage IO operation method and device based on distributed lock |
| CN107517277B (en) * | 2017-10-09 | 2020-05-19 | 苏州浪潮智能科技有限公司 | Method and device for realizing sanlock |
| CN107888689B (en) * | 2017-11-16 | 2019-04-30 | 无锡地铁集团有限公司 | Locking resource allocation method based on shared storage |
-
2018
- 2018-11-01 CN CN201811291959.XA patent/CN109375988B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109375988A (en) | 2019-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10649953B2 (en) | Blockchain-based data migration method and apparatus | |
| CN109493076B (en) | Kafka message unique consumption method, system, server and storage medium | |
| CN111367659B (en) | Resource management method, equipment and medium for nodes in Kubernetes | |
| KR101871383B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN111767143A (en) | Transaction data processing method, device, equipment and system | |
| CN109246182B (en) | Distributed lock manager and implementation method thereof | |
| US20190324814A1 (en) | Resource release method, resource allocation method, devices, and computer program products | |
| US20160087923A1 (en) | Selective message republishing to subscriber subsets in a publish-subscribe model | |
| CN111064626B (en) | Configuration updating method, device, server and readable storage medium | |
| US10936217B2 (en) | Providing virtual volume flexibility on a storage device cluster | |
| US11811784B2 (en) | Integrity verified paths between entities in a container-orchestration system | |
| CN115913937B (en) | Container multi-network card network configuration method, device, equipment and storage medium | |
| CN107517277B (en) | Method and device for realizing sanlock | |
| US10402252B1 (en) | Alternative event reporting for peripheral devices | |
| CN111309264B (en) | Method, system, device and medium for making directory quota compatible with snapshot | |
| CN109375988B (en) | Distributed lock implementation method and device | |
| CN109445966B (en) | Event processing method, device, medium and computing equipment | |
| CN106790521B (en) | System and method for distributed networking by using node equipment based on FTP | |
| CN109408485B (en) | Distributed lock implementation method and system | |
| US20170364293A1 (en) | Method and apparatus for data processing | |
| CN109376135B (en) | Cluster file system management method and system | |
| US10949114B2 (en) | Data storage management devices using utilization rates and methods for data storage management using utilization rates | |
| CN109144740B (en) | Distributed lock implementation method and device | |
| CN110058790B (en) | Method, apparatus and computer program product for storing data | |
| CN108848156B (en) | Access gateway processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |