CN116521943A - Resource lock management method and device, electronic equipment and storage medium - Google Patents

Resource lock management method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116521943A
CN116521943A CN202310430029.2A CN202310430029A CN116521943A CN 116521943 A CN116521943 A CN 116521943A CN 202310430029 A CN202310430029 A CN 202310430029A CN 116521943 A CN116521943 A CN 116521943A
Authority
CN
China
Prior art keywords
resource
node
lock
locked
hash
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310430029.2A
Other languages
Chinese (zh)
Inventor
程康
杨旭荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sangfor Technologies Co Ltd
Original Assignee
Sangfor Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sangfor Technologies Co Ltd filed Critical Sangfor Technologies Co Ltd
Priority to CN202310430029.2A priority Critical patent/CN116521943A/en
Publication of CN116521943A publication Critical patent/CN116521943A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • G06F16/9014Indexing; Data structures therefor; Storage structures hash tables
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • G06F16/9024Graphs; Linked lists

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention is suitable for the technical field of computers, and provides a resource lock management method, a device, electronic equipment and a storage medium, wherein the resource lock management method comprises the following steps: determining a hash value corresponding to the unique identifier of the resource to be locked; determining a first hash bucket corresponding to a hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked; determining a lock node of the resource to be locked based on a linked list interval corresponding to the first hash bucket; and acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.

Description

Resource lock management method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for managing resource locks, an electronic device, and a storage medium.
Background
The resource lock is a mechanism for coordinating concurrent execution of multiple threads by a computer, and the resource lock in the related art comprises a global lock and a table-level lock in a database, wherein the global lock can lock the whole database instance, and the table-level lock can lock the whole table. The disadvantage of the related art is that the granularity of the locks is too large, the probability of lock collision is large, and the concurrency of the system is low.
Disclosure of Invention
In order to solve the problems, the embodiments of the present invention provide a method, an apparatus, an electronic device, and a storage medium for managing a resource lock, so as to at least solve the problem that the granularity of the lock is too large and the concurrency of the system is low in the related art.
The technical scheme of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for managing a resource lock, where the method includes:
determining a hash value corresponding to the unique identifier of the resource to be locked;
determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked;
determining a lock node of the resource to be locked based on a linked list interval corresponding to the first hash bucket;
and acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
In the above solution, the determining, based on the linked list interval corresponding to the first hash bucket, the lock node of the resource to be locked includes:
Determining whether a linked list interval corresponding to the first hash bucket comprises a lock node of the resource to be locked;
if the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked, creating the lock node of the resource to be locked;
inserting the newly-built lock node of the resource to be locked into the doubly-linked list;
and if the linked list interval corresponding to the first hash bucket comprises the lock node of the resource to be locked, determining the lock node of the resource to be locked from the linked list interval corresponding to the first hash bucket.
In the above solution, after determining the lock node of the resource to be locked from the linked list interval corresponding to the first hash bucket, the method further includes:
acquiring the current reference count of the resource lock of the resource to be locked in the memory;
and if the current reference count is the same as the expected original value, adding 1 to the current reference count of the resource lock of the resource to be locked in the memory.
In the above solution, the inserting the newly-built lock node of the resource to be locked into the doubly linked list includes:
acquiring a newly-built lock node of the resource to be locked;
determining the insertion position of the lock node of the resource to be locked;
Inserting the lock node of the resource to be locked into the insertion position;
updating the index of the hash bucket corresponding to the lock node of the resource to be locked; the index of each hash bucket points to the head node of the corresponding linked list interval; the lock nodes in the linked list interval corresponding to each hash bucket are sorted in an increasing mode according to the size of the hash value.
In the above solution, the determining an insertion position of the lock node of the resource to be locked includes:
determining a second hash bucket corresponding to the hash value of the lock node of the resource to be locked;
traversing the linked list interval from a head node of the linked list interval corresponding to the second hash bucket;
determining whether the hash value of the traversed current node is larger than or equal to the hash value of the lock node of the resource to be locked, and whether the first operation result is equal to the second operation result; the first operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the second operation result represents an operation result of right shift operation on the hash value of the current node;
and if the first operation result is equal to the second operation result and the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, determining that the insertion position is between the previous node of the current node and the current node.
In a second aspect, an embodiment of the present invention provides a resource lock management apparatus, including:
the first determining module is used for determining a hash value corresponding to the unique identifier of the resource to be locked;
the second determining module is used for determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked;
the third determining module is used for determining the lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket;
and the acquisition module is used for acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, where the processor and the memory are connected to each other, where the memory is configured to store a computer program, where the computer program includes program instructions, and where the processor is configured to invoke the program instructions to perform the steps of the resource lock management method provided in the first aspect of the embodiment of the present invention.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium comprising: the computer readable storage medium stores a computer program. The steps of the resource lock management method as provided in the first aspect of the embodiment of the present invention are implemented when the computer program is executed by a processor.
In this embodiment, a hash value corresponding to a unique identifier of a resource to be locked is determined, and a first hash bucket corresponding to the hash value in a preset hash table is determined. And determining a lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket, and acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked. The preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, each hash bucket corresponds to a section of linked list interval of the double-linked list, and each lock node on the double-linked list represents a resource lock of a resource to be locked. When accessing the resource to be locked, the embodiment obtains the corresponding lock node from the doubly linked list according to the hash value corresponding to the unique identifier of the resource to be locked, and obtains the resource lock of the resource to be locked from the lock node. According to the embodiment, the resource locks are managed based on the hash table and the doubly-linked list, each resource lock corresponds to one node in the doubly-linked list, each resource to be locked can have a special lock, the granularity of the resource lock is minimum, the probability of lock conflict under a concurrent scene can be reduced, and the concurrency of the system is improved.
Drawings
FIG. 1 is a schematic diagram of an implementation flow of a resource lock management method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a correspondence between a hash table and a doubly linked list according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a hash table and doubly linked list according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a hash table and doubly linked list creation process provided by an embodiment of the present invention;
FIG. 5A is a schematic flow chart of searching for an insertion position of a locked node according to an embodiment of the present invention;
FIG. 5B is a schematic diagram of a process for inserting a lock node into a designated location according to an embodiment of the present invention;
FIG. 5C is a flowchart of updating a bucket index according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an acquisition flow of a lock node according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a deletion process of a resource lock according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a resource lock management device according to an embodiment of the present invention;
fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
A database is a shared resource used by multiple users, and when multiple users access data concurrently, multiple transactions can occur in the database to access the same data simultaneously. Incorrect data may be read and stored if concurrent operations are not controlled, disrupting database consistency.
Locking is a very important technique for implementing concurrency control of databases, and a transaction makes a request to a system to lock a certain data object before operating on it. The transaction has some control over the data object after locking, and other transactions cannot update the data object until the transaction releases the lock. Concurrent access to shared resources may cause data inconsistency if not locked, and in order to solve the concurrent access problem, the shared resources are usually locked before being accessed, so that only one thread is ensured to access at the same time.
The granularity of a lock refers to the granularity of a blocked resource, such as a data item, record, file, or entire database, with smaller lock granularity providing higher parallelism for transactions. The larger the granularity of a lock in general, the higher the probability of contending for the lock between multiple incoherent transactions, meaning that the more severe the lock conflict, the lower the concurrency and throughput of the overall system.
The related art has the following resource lock schemes:
1. coarse grain lock/large lock.
Typically represented as global locks and table level locks in MySQL databases are the largest 2 types of locks in MySQL that are granular. The global lock may lock the entire database instance, while the table level lock may lock the entire table.
The method has the characteristics of simple realization and less resource consumption; the disadvantage is large granularity of lock, high probability of collision of lock, and low concurrency.
2. Segment locks based on static hash tables.
An array containing N locks is pre-established and used as a hash table lock_hash_table; calculating an integer original hash value hash_int according to a unique identification key of the resource to be locked; then performing modular calculation on N, and locating an index bucket_index=hash_int mod N of a certain hash bucket of the hash table; and finally, the lock_hash_table [ socket_index ] corresponding to the bucket is the fine-grained lock needed by the resource to be locked.
The scheme has the advantages that the granularity of the lock is reduced relative to a large lock, and the lock conflict is reduced, so that the concurrency and the throughput are improved; the disadvantage is that the lock granularity generally decreases with the number N of locks, so a large number of locks need to be created in advance, consuming a large amount of memory resources, and on the other hand, a suitable hash algorithm needs to be selected.
3. Ha Xisuo based on segment locks and HashMap.
For each resource to be locked, assuming its unique identity is key, then a dedicated lock-released_lock is dynamically created for it when locking is required, note here that it is created on demand rather than in advance; after creation, the dedicated_lock is stored in the HashMap, and HashMap [ key ] = dedicated_lock, so that when other threads lock the resource concurrently, the created dedicated_lock can be obtained from the HashMap, and the reference count of the dedicated_lock is increased by 1. When releasing the dedicated_lock, the reference needs to be subtracted by 1, and when the reference count is 0, the lock needs to be deleted from HashMap to release the resource, hashMap. Meanwhile, in order to ensure thread safety when the dedicated_lock is stored and acquired in the HashMap, a segment lock is used to optimize thread synchronization.
The scheme has the advantages that the granularity of the lock is minimum, each resource can have a special lock, and the lock is dynamically created and destroyed, so that the cost of memory resources can be saved; the disadvantage is that to ensure thread security, a segment lock needs to be introduced, and if the segment lock conflicts are large, a large additional time overhead is caused.
It can be seen that the three solutions described above have more or less problems:
1. the granularity of the lock is too large, the lock conflict is serious, and the concurrency and throughput are low;
2. memory resources are occupied greatly, and the hash algorithm is seriously depended on the quality;
3. the extra time cost is too large, and the method is only suitable for application scenes with some locks occupying longer time.
Aiming at the defects of the related technology, the embodiment of the invention provides a resource lock management method which can minimize the granularity of locks and improve the concurrency of a system. In order to illustrate the technical scheme of the invention, the following description is made by specific examples.
Fig. 1 is a schematic implementation flow diagram of a resource lock management method provided by an embodiment of the present invention, where an execution body of the resource lock management method is a thread accessing a resource to be locked, and referring to fig. 1, the resource lock management method includes:
s101, determining a hash value corresponding to the unique identifier of the resource to be locked.
In this embodiment, the resource to be locked may be a data item, a record, a file, a DataBase, or the like, and the unique IDentifier refers to identification information capable of indicating the identity of the resource to be locked, for example, the name of the file may be used as the unique IDentifier of the file, and a DataBase IDentifier (DBID) of the DataBase may be used as the unique IDentifier of the DataBase.
And carrying out hash calculation on the unique identifier of the resource to be locked to obtain a hash value corresponding to the unique identifier of the resource to be locked.
Here, the hash algorithm used for performing the hash calculation in this embodiment may be selected according to the actual situation, and this embodiment is not limited to the hash algorithm.
S102, determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list characterizes a resource lock of a resource to be locked.
In this embodiment, a hash table is preset, where the preset hash table includes a plurality of hash buckets, and the hash buckets correspond to the same doubly linked list.
Hash tables are data structures that are accessed directly from a Key value (Key value) that accesses a record by mapping the Key value to a location in the table to speed up the lookup. This mapping function is called a hash function and the array storing the records is called a hash table.
A doubly linked list is one type of linked list having two pointers in each data node that point to the immediate successor and immediate predecessor, respectively. Therefore, starting from any node in the doubly linked list, its predecessor and successor nodes can be conveniently accessed.
In this embodiment, all hash buckets of the preset hash table correspond to the same doubly-linked list, the doubly-linked list is formed by connecting a plurality of lock nodes, each hash bucket points to one lock node of the doubly-linked list, and each hash bucket corresponds to a section of linked list interval of the doubly-linked list.
The hash table is preset as an index of the resource lock and is used for optimizing access of the resource lock. Essentially, the lock pointer array is an array of lock pointers with a length of N, and N is usually an integer power of 2, so that subsequent bit operation is convenient. Each element of the array, called a bucket, stores the pointer of the first lock on the bucket. Each time a resource lock is accessed, hash calculation is needed to be carried out according to a unique identification key of the resource to be locked to obtain a keyHash, then N is subjected to modulo access, the N is positioned on a certain socket, and the access position of the resource lock can be quickly positioned through the socket.
The bi-directional linked list is used for storing all the created resource locks, and the lock nodes under each socket are tightly connected and respectively point to the former node and the latter node through the prev pointer and the next pointer. And the lock nodes under the socket are sorted in an increasing way according to the keyHash, and the socket points to the node with the smallest keyHash, so that the subsequent searching time can be saved.
Each lock node stores the resource lock information of a resource to be locked, and the lock node can also store the information such as the unique identification, hash value, reference count of the resource to be locked and the like.
As shown in fig. 2, fig. 2 is a schematic diagram of a correspondence relationship between a hash table and a doubly linked list according to an embodiment of the present invention. The hash table is used as an index of the resource lock, and includes N bins (hash buckets), where N is typically an integer power of 2, for example, may be 64. The directional bidirectional linked list stores all resource locks, lock nodes under the same socket are closely connected, and are sorted in an increasing mode according to the keyHash. The bucket points to the lock node with the smallest keyHash under the bucket.
The socket 0 as in fig. 2 points to Lock node Lock0, lock0 storing a unique identification key, keyHash, rwLock of the resource to be locked (resource Lock) and a reference count (count) of the resource Lock.
Wherein Lock0 and Lock1 belong to socket 0, but the keyHash of Lock1 is greater than the keyHash of Lock0, so Lock1 is behind Lock0, and the index of socket 0 points to Lock0.
In practical applications, the types of resource locks are various and classified into row-level locks, table-level locks and page-level locks according to the granularity of the locks. The lock level is divided into a shared lock and an exclusive lock. The division is based on the usage mode, and can be classified into optimistic locks and pessimistic locks. The locking method is divided into automatic locking and explicit locking. The Data Locks (DML, data Locks) and Data Dictionary Locks (DDL) are divided by operation. The embodiment does not limit the type of the resource lock, and can be any resource lock.
According to the embodiment, a first Hash bucket corresponding to the Hash value can be determined according to a consistency Hash algorithm, hash calculation is carried out according to a unique identification key of a resource to be locked to obtain a keyHash, then N is subjected to modulo operation by the keyHash, and the N is positioned on the first Hash bucket.
S103, determining the lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket.
Each hash bucket corresponds to a section of linked list interval of the doubly linked list, for example, the lock nodes under the same hash bucket are progressively ordered according to the hash value, and the index of the hash bucket points to the lock node with the smallest hash value under the hash bucket.
And traversing the chain table interval from the lock node with the smallest hash value under the hash bucket, and finding out the lock node of the resource to be locked.
For example, if the lock node stores a unique identification and hash value of the resource to be locked, the lock node of the resource to be locked may be determined from the hash value or the unique identification. If the hash values or unique identifications are the same, the current node traversed is considered to be the lock node of the resource to be locked.
S104, acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
Each lock node stores resource lock information of a resource to be locked, and the resource lock information can be a section of characters or a data structure. After the thread obtains the resource lock of the resource to be locked from the lock node, the resource lock can be utilized to access the resource to be locked.
When a plurality of threads access the resource to be locked, the threads are placed in a preset queue according to a preset sequence, and only one thread is allowed to use the resource lock to access the resource to be locked at a time, so that the other threads are prevented from preempting the resource.
In this embodiment, a hash value corresponding to a unique identifier of a resource to be locked is determined, and a first hash bucket corresponding to the hash value in a preset hash table is determined. And determining a lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket, and acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked. The preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, each hash bucket corresponds to a section of linked list interval of the double-linked list, and each lock node on the double-linked list represents a resource lock of a resource to be locked. When accessing the resource to be locked, the embodiment obtains the corresponding lock node from the doubly linked list according to the hash value corresponding to the unique identifier of the resource to be locked, and obtains the resource lock of the resource to be locked from the lock node. According to the embodiment, the resource locks are managed based on the hash table and the doubly-linked list, each resource lock corresponds to one node in the doubly-linked list, and each resource to be locked can have a special lock, so that the granularity of the resource lock is minimum, the probability of lock conflict under a concurrent scene can be reduced, and the concurrency of the system is improved.
The embodiment provides the resource lock with the minimum granularity in a low-cost mode (low memory overhead and low additional time overhead), and can be used for realizing high-performance, high-concurrency and high-throughput concurrency programming under various business scenes.
For example, in a user space file system (FUSE, filesystem inUserspace), concurrent access to tens of thousands of files is required, and in this embodiment, a resource lock may be set for each file in the FUSE, and when multiple threads concurrently access the files in the FUSE, since each file has its own dedicated lock, the probability of lock conflict in the concurrent scenario may be effectively reduced, the concurrency of the system may be improved, and high performance file reading and writing may be implemented.
In online Transaction processing (OLTP, online Transaction Processing), OLTP represents a very high transactional system, typically a highly available online system, with small transactions and small queries being the dominant, in such systems, transactions processed per second by a single database often exceed hundreds, or thousands, of Select statements executing in thousands or even tens of thousands per second. Typical OLTP systems are e-commerce systems, banks, securities, etc., and business databases such as eBay are very typical OLTP databases. The DML statement (comprising Insert, update and Delete) in MySQL OLTP scene has a great deal of concurrent operation access lock_sys- > mutex global lock protected key data structure, and the lock conflict is serious, so that the performance is reduced sharply. By using the embodiment, a resource lock can be set for each page in the database, the unique identifier of the page is page_no, and when concurrent inquiry is performed, the special fine-granularity lock of each page is obtained according to the page_no of the page, and as each page has the own special lock, the lock conflict can be greatly reduced, and the concurrency of the system is improved.
The embodiment can be applied to cloud computing platforms, such as kubernetes and Redis. In kubernetes, a ConfigMap instance may be created to store hash tables and doubly linked lists, each lock node in the doubly linked list corresponds to a kubernetes resource, and a dedicated resource lock may be set for each resource in kubernetes. When multiple clients of the tenant of kubernetes access kubernetes resources concurrently, as each resource has own special lock, the probability of lock conflict can be reduced, and kubernetes concurrency is improved.
In an embodiment, the determining, based on the linked list interval corresponding to the first hash bucket, the lock node of the resource to be locked includes:
determining whether a linked list interval corresponding to the first hash bucket comprises a lock node of the resource to be locked;
if the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked, creating the lock node of the resource to be locked;
inserting the newly-built lock node of the resource to be locked into the doubly-linked list;
and if the linked list interval corresponding to the first hash bucket comprises the lock node of the resource to be locked, determining the lock node of the resource to be locked from the linked list interval corresponding to the first hash bucket.
As described above, if the lock node stores a unique identification and a hash value of the resource to be locked, the lock node of the resource to be locked may be determined from the hash value or the unique identification.
If the linked list interval corresponding to the first hash bucket does not have the lock node of the resource to be locked, a new lock node is created, the resource lock of the resource to be locked is stored, and the newly-built lock node is inserted into the double linked list.
In this embodiment, a large number of lock nodes do not need to be built in advance, but lock nodes are newly built according to actual requirements, so that memory consumption is smaller.
Fig. 3 is a schematic diagram of a data structure of a hash table and a doubly linked list according to an embodiment of the present invention, which mainly includes 3 structures:
FGLock (Fine-Grained Lock) is used to manage some global configurations such as keys, index chunks, and ordered doubly linked list of storage locks.
LockList, the ordered double linked list of storage lock, is used for managing the head node head of the linked list and the number count of lock nodes.
Lock, lock node, package of resource Lock, additionally record key (unique identification), keyHash, count (reference count of resource Lock) and other important information. Each lock node includes a successor node pointer (prev) and a successor node pointer (nest).
In practical applications, FGLock structures are first created and initialized.
The initial size of the optional input index during initialization, that is, the initial number of the buckets (hash buckets), defaults to 1024, each bucket (essentially a pointer) occupies 8 bytes of memory under a 64-bit system, and when the size is 1024, only 8×1024=8kb of memory is needed, so that less memory is consumed. The size can be properly adjusted according to actual needs, and in a particularly high concurrency scene, the access performance of the lock can be optimized by increasing the size.
As shown in fig. 4, fig. 4 is a schematic diagram of a hash table and doubly linked list creation process provided in an embodiment of the application of the present invention. The process comprises the following steps:
s401, inputting the size of the index.
size defaults to 1024 and size is the number of buckets.
S402, judging whether the Size is an integer power of 2.
Yes, S404 is executed, and no S403 is executed.
S403, converting the size to an integer power of 2 slightly larger than it.
S404, calculating the keyfields according to the size, and facilitating subsequent hash module transportation.
Keyshifts=sizeof(int)-log2(size)
S405, creating index buckets.
Buckets=malloc(sizeof(Lock*)*size)
S406, creating a doubly linked list linked_list.
linked_list=&LockList{head:&Lock{}}
After the hash table and the doubly-linked list are created, the doubly-linked list is free of lock nodes, and the lock nodes are created according to actual needs, so that the cost of memory resources can be reduced.
In an embodiment, the inserting the newly-built lock node of the resource to be locked into the doubly linked list includes:
acquiring a newly-built lock node of the resource to be locked;
determining the insertion position of the lock node of the resource to be locked;
inserting the lock node of the resource to be locked into the insertion position;
updating the index of the hash bucket corresponding to the lock node of the resource to be locked; the index of each hash bucket points to the head node of the corresponding linked list interval; the lock nodes in the linked list interval corresponding to each hash bucket are sorted in an increasing mode according to the size of the hash value.
For example, according to the hash value of the unique identifier of the resource to be locked, determining a hash bucket corresponding to the lock node of the resource to be locked, and finding the insertion position of the lock node of the resource to be locked from the hash bucket. After the lock node of the resource to be locked is inserted into the insertion position, the pointer of the lock node is updated to point to the index of the hash bucket.
In an embodiment, the determining the insertion position of the lock node of the resource to be locked includes:
determining a second hash bucket corresponding to the hash value of the lock node of the resource to be locked;
traversing the linked list interval from a head node of the linked list interval corresponding to the second hash bucket; the lock nodes in the linked list interval corresponding to each hash bucket are progressively ordered according to the size of the hash value;
Determining whether the hash value of the traversed current node is larger than or equal to the hash value of the lock node of the resource to be locked, and whether the first operation result is equal to the second operation result; the first operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the second operation result represents an operation result of right shift operation on the hash value of the current node;
and if the first operation result is equal to the second operation result and the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, determining that the insertion position is between the previous node of the current node and the current node.
In an embodiment, the determining the second hash bucket corresponding to the hash value of the lock node of the resource to be locked includes:
performing right shift operation on the hash value of the lock node of the resource to be locked to obtain the first operation result;
and determining the second hash bucket according to the first operation result.
For example, a right-shift 16-bit exclusive-or operation is performed on the hash value.
Hash tables typically use a hash modulo approach to locate the bucket, which requires converting the hash value to decimal, and then performing a modulo operation on the hash table length to obtain a remainder, which is relatively inefficient in processing speed. According to the embodiment, the socket is positioned by adopting a right shift operation mode for the hash value, and the bit operation is directly operated on the memory data without being converted into decimal, so that the processing speed is very high, and the searching performance of the data is improved.
In another embodiment, it is further determined whether the second hash bucket is empty, that is, whether there is a lock node in the linked list interval corresponding to the second hash bucket. If the second hash bucket is empty (no lock node), the traversal starts from the hash bucket adjacent to the left of the second hash bucket, and if the hash bucket adjacent to the left of the second hash bucket is still empty, the traversal starts from the head node of the doubly linked list.
In this embodiment, the lock nodes in the linked list intervals are connected in an ascending order according to the hash value, and the linked list intervals are traversed from the head node of the corresponding linked list interval of the second hash bucket, that is, from the lock node with the smallest hash value. The first traversal, the current node is the head node of the linked list interval (the lock node with the smallest hash value).
Determining whether the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, and whether the operation result of performing right shift operation on the hash value of the resource to be locked is equal to the operation result of performing right shift operation on the hash value of the current node (namely, determining whether the hash value corresponds to the same hash bucket), and if the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, determining that the insertion position of the lock node of the resource to be locked is between the previous node of the current node and the current node.
Because the lock nodes of the bidirectional linked list are connected in an increasing order according to the hash value, the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, which means that the current node is positioned behind the lock node of the resource to be locked, the lock node of the resource to be locked is inserted in front of the current node, namely the lock node of the resource to be locked is the previous node of the current node.
In an embodiment, the method further comprises:
if the hash value of the current node is smaller than the hash value of the lock node of the resource to be locked and the first operation result is equal to the second operation result, determining whether the next node of the current node is empty;
and if the next node of the current node is empty, determining the insertion position as the next node of the current node.
The next node of the current node is null, which indicates that the current node is the last node of the doubly linked list, the hash value of the current node is smaller than the hash value of the lock node of the resource to be locked, and the current node is located before the lock node of the resource to be locked, so that the lock node of the resource to be locked is inserted behind the current node, namely the lock node of the resource to be locked is the next node of the current node.
In an embodiment, the method further comprises:
if the next node of the current node is not empty, determining whether the next node of the current node belongs to a later hash bucket;
and if the next node of the current node belongs to a later hash bucket, determining the insertion position as between the next node of the current node and the current node.
The next node of the current node is not null, which indicates that there is a lock node behind the current node, determines whether the next node of the current node belongs to a later hash bucket, and can perform right shift operation on the hash value of the next node of the current node. In this case, the lock node of the resource to be locked is inserted between the next node of the current node and the current node.
In an embodiment, the method further comprises:
if the next node of the current node does not belong to the subsequent hash bucket, continuing to traverse the doubly linked list from the next node of the current node until the insertion position is found.
If the next node of the current node and the lock node of the resource to be locked belong to the same hash bucket, the lock node in the link list interval is described as not being traversed, and the bidirectional link list is continuously traversed until the insertion point of the lock node of the resource to be locked is found.
According to the embodiment, a large number of resident resource locks are not required to be created in advance, and the resource locks are dynamically created according to the requirement, so that occupied memory resources are fewer, and the degree of dependence on the hash algorithm is low.
In an embodiment, after the inserting the lock node of the resource to be locked into the insertion position, the method further comprises:
updating a relay node and a successor node of the lock node of the resource to be locked;
directing a first pointer of the former relay node to a lock node of the resource to be locked;
and under the condition that the subsequent node is not empty, pointing a second pointer of the subsequent node to the lock node of the resource to be locked.
In practical applications, the first pointer is a nest pointer and the second pointer is a prev pointer. Each lock node has a nest pointer and prev pointer.
And after the lock node of the resource to be locked is inserted into the doubly linked list, updating the relay node and the successor node of the lock node of the resource to be locked. And directing a nest pointer of the successor node to a lock node of the resource to be locked, and directing a prev pointer of the successor node to the lock node of the resource to be locked under the condition that the successor node is not empty because the successor node may be empty. The pointer of the updated lock node points backwards, so that the lock node can be conveniently searched subsequently.
In an embodiment, the updating the index of the hash bucket corresponding to the lock node of the resource to be locked includes:
determining whether a third hash bucket corresponding to the lock node of the resource to be locked has other nodes except the lock node of the resource to be locked;
and if the third hash bucket has no other nodes, pointing the index of the third hash bucket to the lock node of the resource to be locked.
After the lock node of the resource to be locked is inserted into the third hash bucket, if the third hash bucket only has the lock node of the resource to be locked, the lock node of the resource to be locked is the head node of the linked list interval of the third hash bucket, and the index of the third hash bucket is pointed to the lock node of the resource to be locked.
In an embodiment, the method further comprises:
if the third hash bucket has other nodes, determining whether the hash value of the lock node of the resource to be locked is smaller than the hash value of the lock node pointed by the index of the third hash bucket;
and if the index of the third hash bucket is smaller than the index of the first hash bucket, pointing the index of the third hash bucket to the lock node of the resource to be locked.
If the third hash bucket has other lock nodes except the lock node of the resource to be locked, further judging whether the hash value of the lock node of the resource to be locked is smaller than the hash value of the lock node pointed by the index of the third hash bucket, if so, indicating that the lock node of the resource to be locked is the head node of the third hash bucket, and pointing the index of the third hash bucket to the lock node of the resource to be locked. And updating the index of the hash bucket, so that the lock node can be conveniently searched subsequently.
In an embodiment, after the lock node of the resource to be locked is determined from the linked list interval corresponding to the first hash bucket, the method further includes:
acquiring the current reference count of the resource lock of the resource to be locked in the memory;
and if the current reference count is the same as the expected original value, adding 1 to the current reference count of the resource lock of the resource to be locked in the memory.
The embodiment adopts a comparison and exchange (CAS) technology to realize lock-free, wherein the CAS is one of atomic operations and can be used for realizing uninterrupted data exchange operation in multi-thread programming, thereby avoiding the problem of inconsistent data caused by uncertainty of an execution sequence and unpredictability of interruption when multiple threads rewrite certain data at the same time. CAS contains three operands: memory location, expected original value, and new value. When the CAS operation is executed, the value of the memory location is compared with the expected original value, and if the memory location is matched with the expected original value, the processor automatically updates the value of the memory location to a new value; if there is no match, the processor does nothing and only one of the multiple threads will succeed in performing the CAS operation at the same time.
Lock-free, simply speaking, does not directly use locks, reducing the overhead occupied by locks in the system. The Lock-free algorithm has significant features compared to Lock-based algorithms: suspending a thread while performing data access does not prevent other threads from continuing to execute. This means that multiple lock-free threads can access the same data at the same time without data contention or corruption.
In order to solve the thread security problem in the concurrent scenario, no additional locks are introduced in this embodiment, because these locks generate a large overhead and become a new bottleneck point for the system. The embodiment adopts CAS atomic operation to realize lock-free, avoids the overheads such as context switching and scheduling, greatly reduces overhead, and reduces the additional time overhead introduced in the lock management process, thereby realizing high concurrency and high throughput of the whole system.
In an embodiment, after the inserting the newly-built lock node of the resource to be locked into the doubly linked list, the method further includes:
and setting the reference count of the resource lock of the newly-built resource to be locked to 1.
For the newly created resource lock, its reference count is set to 1.
Referring to fig. 5A, fig. 5B, and fig. 5C, fig. 5A is a schematic flow chart of searching for an insertion position of a locked node according to an application embodiment of the present invention, fig. 5B is a schematic flow chart of inserting a locked node into a designated position according to an application embodiment of the present invention, and fig. 5C is a schematic flow chart of updating a socket index according to an application embodiment of the present invention. Fig. 5A, 5B and 5C, taken together, are a schematic flow diagram of a lock node insert insertion doubly linked list.
The search for the insertion location includes steps S501-S521, the insertion of the specified location includes steps S522-S528, and the update of the index includes steps S529-S536.
The process comprises the following steps:
s501, inputting a lock to be inserted.
S502, positioning an index socket according to newLock.
Bucket=newLock.keysheh>>keyshifts
S503, attempting to insert newLock into the section corresponding to the socket on the linked_list.
linked_list is a doubly linked list.
S504, searching for the position of the newLock inserted into the linked_list.
S505, judging whether the socket is not empty.
S506 is not executed for null, and S508 is executed for null.
S506, searching for an insertion point from the socket.
currentSearch=bucket
S507, the traversal starts from currentSearch.
Traversing from the head node of the socket.
S508, searching for the first non-empty reftbucket on the left side of the bucket.
S509, judging whether the reftbucket is not empty.
S510 is executed for null, and S511 is not executed for null.
S510, searching from the head node of the bidirectional link table.
currentSearch=linked_list.head
S511, starting searching from the reftBucket.
currentSearch=leftBucket
S512, judging currentsearch.keyhash= newlock.keyhash +.and currentsearch.key= newlock.key.
And judging whether the hash value of the traversed current node is the same as the hash value of the lock to be inserted, and the unique identification is the same.
The same performs S513, and the different performs S515.
S513 illustrates that currentSearch is an existing copy of newLock.
Similarly, newLock already exists.
S514, returning to false, which indicates that there is a copy of newLock, and no repeated insertion is required.
S515, judging whether the current node and the newLock are in the same socket and are successor nodes of the newLock.
currentSearch.keyHash>>keyshifts==newLock.keyHash>>keyshifts&&currentSearch.keyHash>=newLock.keyHash。
The hash value right-shift operation of the current node and the hash value right-shift operation of newLock have the same operation result, and are described in the same socket.
The hash value of the current node is greater than or equal to the hash value of the newLock, indicating that the current node is a successor node of the newLock.
Yes, S516 is performed, and no S517 is performed.
S516, the position inserted into the linked_list is (currentSearch. Prev, nil, currentSearch).
The insertion position is between the previous node of the current node and the current node.
S517, it is determined whether currentsearch. Next is not null.
Whether the next node of the current node is not null is judged, S519 is executed if not null, and S518 is executed if null.
S518, the position inserted into the linked_list is (currentSearch, nil, nil).
The insertion position is the node next to the current node.
S519, judging whether the next node of the current node belongs to the following socket.
currentSearch.next.keyHash>>keyshifts>newLock.keyHash>>keyshifts
And the operation result of right shift operation on the hash value of the next node of the current node is larger than the operation result of right shift operation on the hash value of newLock, and the next node of the current node is considered to belong to the following socket.
Yes, S520 is executed, otherwise S521 is executed.
S520, the position inserted into the linked_list is (currentSearch, nil, currentSearch.
The insertion position is between the next node of the current node and the current node.
S521, the insertion point is not found, and the next node is continuously searched.
currentSearch=currentSearch.next
S522, insert newLock into the found position (left. Nil, right).
S523, updating the successor node and successor node of newLock to left and right, respectively.
newLock.prev=left
newLock.next=right
The reference count is set to 1
newLock.count=1
S524, whether the CAS is used to successfully point the Next pointer of the relay node left to the inserted new node newLock.
CAS(&left.next,right,newLock)
Yes, S526 is executed, otherwise S525 is executed.
S525, the insertion fails, which indicates that other threads are modified concurrently and retry is performed.
S526, whether the subsequent node right is not null is determined.
Yes, S527 is executed, otherwise insertion is completed.
S527, whether the CAS is used to successfully point the Prev pointer of the subsequent node right to the inserted new node newLock.
CAS(&right.prev,left,newLock)
Yes, S528 is executed, otherwise S525 is executed.
S528, the insertion is successful.
S529, an attempt is made to update the index.
S530, judging whether the socket is empty.
And judging whether the socket where the newLock is positioned is empty or not before the newLock is inserted into the linked list.
Yes, S531 is performed, otherwise S533 is performed.
S531, whether to modify the bucket pointing to newLock using CAS.
CAS (& bucket, nil, newLock), points the bucket's index to newLock.
Yes, S535 is performed, otherwise S532 is performed.
S532, the modification fails, which means that other threads are modified concurrently and retry is performed.
S533,bucket.keyHash>newLock.keyHash。
And judging whether the hash value of the lock node pointed by the index of the socket is larger than that of the newLock.
Yes, S534 is executed, otherwise S535 is executed.
S534, whether to use CAS to modify the bucket to point to newLock.
CAS (& bucket, bucket, newLock) determines whether the index of a bucket is successfully modified using CAS to point to newLock.
Yes, S535 is performed, otherwise S532 is performed.
S535, the index is successfully updated, or the index does not need to be updated.
S536, return true, indicate that the insertion was successful.
In an embodiment, the determining whether the linked list interval corresponding to the first hash bucket includes the lock node of the resource to be locked includes:
traversing the linked list interval from a head node of the linked list interval corresponding to the first hash bucket; the lock nodes in the linked list interval corresponding to each hash bucket are progressively ordered according to the size of the hash value;
Determining whether the traversed current node is a lock node of the resource to be locked;
if the current node is not the lock node of the resource to be locked, determining whether the hash value of the resource to be locked is larger than the hash value of the current node, and whether the third operation result is equal to the fourth operation result; the third operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the fourth operation result represents an operation result of right shift operation on the hash value of the current node;
if the hash value of the resource to be locked is larger than the hash value of the current node and the third operation result is equal to the fourth operation result, continuing to traverse the next node of the current node until traversing to the last node of the linked list interval;
and if the hash value of the resource to be locked is smaller than the hash value of the current node or the third operation result is not equal to the fourth operation result, determining that the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked.
In this embodiment, the hash value is determined by a right shift operation to determine whether the hash value belongs to the same hash bucket, and the specific implementation process is detailed in the above embodiment, which is not repeated here.
In an embodiment, the determining whether the current node traversed is the lock node of the resource to be locked includes:
determining whether the hash value of the resource to be locked is equal to the hash value of the current node;
and if so, determining the current node as the lock node of the resource to be locked.
Here, whether the current node is the lock node of the resource to be locked is determined by whether the hash values are the same, if so, the current node is the lock node of the resource to be locked, and if not, the current node is not the lock node of the resource to be locked.
Further, if the lock node stores the unique identifier of the corresponding resource to be locked, it may also be determined whether the current node is the lock node of the resource to be locked by the unique identifier of the resource to be locked. The unique identity is the same as the lock node of the resource to be locked, otherwise it is not.
Referring to fig. 6, fig. 6 is a schematic diagram of a lock node acquisition process according to an embodiment of the present invention, where the process includes:
s601, inputting a unique identification key of the resource to be locked.
S602, sipHash is carried out according to the key value to obtain a hash value keyHash.
S603, positioning an index socket according to the keyHash.
bucket=keyHash>>keyshifts
S604, traversing a lock linked list under the socket.
currentlock=socket, from the head node under the socket.
S605, whether currentLock is not null.
It is determined whether the current node traversed is not null.
Yes, S606 is executed, otherwise S611 is executed.
S606, judging whether the current lock is a lock corresponding to the resource.
currentLock.keyHash==keyHash&&currentLock.key==key。
Yes, S607 is performed, otherwise S609 is performed.
S607, it is determined whether the current lock is being deleted.
currentLock.count==0
Otherwise, S608 is executed, and S611 is executed.
S608, whether the reference count of the lock was successfully incremented by 1.
CAS(&currentLock.count,count,count+1)
Yes, S614 is performed, otherwise S611 is performed.
S609,currentLock.keyHash<keyHash&&current.keyHash>>keyshifts==keyHash>>keyshifts。
And judging whether the hash value of the current lock node is smaller than the keyHash of the resource to be locked, and whether the current lock node and the lock node of the resource to be locked are in the same socket.
Yes, S610 is performed, otherwise S611 is performed.
S610, currentLock points to the next node.
currentLock = currentLock.
S611, it is stated that the found created lock is not present or is being deleted, at which point a new lock needs to be created.
S612, whether newLock is successfully inserted into the doubly linked list linked_list.
Yes, S613 is performed, otherwise S603 is performed.
S613, returning to the newly created newLock.
S614, returning to the currentLock which has been created before.
In an embodiment, after the resource lock of the resource to be locked is used, the method further comprises:
Subtracting 1 from the reference number of the lock node of the resource to be locked;
determining whether the reference number of the lock node of the resource to be locked is 0;
and if the reference number of the lock node of the resource to be locked is 0, deleting the lock node of the resource to be locked from the doubly linked list.
After the use of the resource lock is completed, the user needs to explicitly call an interface for deleting the lock, indicating that the lock is not needed for a while. At this point, the reference count of the resource lock is decremented by 1, and if the reference count of the resource lock is 0, which indicates that no user is currently using the lock, the resource lock is automatically deleted to release the resource.
In another embodiment, in addition to modifying the reference count, the index of the hash bucket needs to be updated, modifying the pointers of the successor node and successor node.
Referring to fig. 7, fig. 7 is a schematic diagram of a process for deleting a resource lock according to an embodiment of the present invention, where the process includes:
s701, inputting a resource lock which needs to be deleted.
S702, reducing the reference count of lock by 1.
S703, it is determined whether the reference count of lock is 0.
The reference count is 0 and S704 is performed.
S704, attempting to delete the resource lock to release the resource.
S705, if the lock is pointed to by the bucket index, then the index needs to be updated to be the successor node of the lock.
S706, positioning the index according to lock.
And positioning the socket according to the hash value of the lock.
Bucket=lock.keyhash>>keyshifts
S707, modifying the subsequent node of the bucket pointing lock by using the CAS.
S708, attempting to delete the lock node from the linked_list.
Attempting to delete the lock node from the doubly linked list.
S709, a relay node and a subsequent node of the lock node are obtained.
Left=lock.prev
Right=lock.nest
S710, whether the subsequent node of left is successfully modified to right by using CAS.
CAS(&left,nest,lock,right)
Yes, S711 is executed, otherwise S714 is executed.
S711, whether the subsequent node right is not null.
Yes, S712 is performed, otherwise S713 is performed.
S712, whether the relay node of right is successfully modified to left by using CAS.
CAS(&right,prev,lock,left)
Yes, S713 is performed, and otherwise S714 is performed.
S713, the lock node is successfully deleted from the doubly linked list.
S714, the modification fails, which means that other threads are modified concurrently and retry is performed.
The embodiment can dynamically create and destroy the resource lock as required, release the memory resource in time and reduce the memory overhead.
In practical application, on a host with 8 logical CPUs and 16GB memory, a total of 10 tens of thousands of small files are written concurrently by using 16 threads, and before writing the files, a corresponding lock is required to be obtained, so as to simulate a scene of ensuring the safety of writing the files in a concurrent scene, and test data of various schemes are shown in the following table 1:
TABLE 1
It can be seen that the total time consumption of this embodiment is the shortest, and it can be seen that concurrency and additional time overhead are best; in addition, memory usage is also lowest in addition to coarse-grain locks/big locks, which is also contemplated, since coarse-grain locks/big locks have only one global big lock, and this embodiment also requires maintaining some indexes and fine-grain locks.
Therefore, the embodiment can realize the minimum granularity of the lock, the highest concurrency of the system and low memory overhead, can dynamically create and destroy the lock as required, and has lower dependence on the hash algorithm. And the additional time cost is small, and the method is suitable for wide application scenes.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The technical schemes described in the embodiments of the present invention may be arbitrarily combined without any collision.
In addition, in the embodiments of the present invention, "first", "second", etc. are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence.
Referring to fig. 8, fig. 8 is a schematic diagram of a resource lock management device according to an embodiment of the present invention, as shown in fig. 8, the device includes:
the first determining module is used for determining a hash value corresponding to the unique identifier of the resource to be locked;
the second determining module is used for determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked;
the third determining module is used for determining the lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket;
and the acquisition module is used for acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
In an embodiment, the third determining module determines, based on a linked list interval corresponding to the first hash bucket, a lock node of the resource to be locked, where the lock node is specifically configured to:
determining whether a linked list interval corresponding to the first hash bucket comprises a lock node of the resource to be locked;
if the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked, creating the lock node of the resource to be locked;
inserting the newly-built lock node of the resource to be locked into the doubly-linked list;
and if the linked list interval corresponding to the first hash bucket comprises the lock node of the resource to be locked, determining the lock node of the resource to be locked from the linked list interval corresponding to the first hash bucket.
In an embodiment, the device further comprises: a reference counting module for: acquiring the current reference count of the resource lock of the resource to be locked in the memory; and if the current reference count is the same as the expected original value, adding 1 to the current reference count of the resource lock of the resource to be locked in the memory.
In an embodiment, the third determining module inserts the newly-built lock node of the resource to be locked into the doubly linked list, specifically for:
Acquiring a newly-built lock node of the resource to be locked;
determining the insertion position of the lock node of the resource to be locked;
inserting the lock node of the resource to be locked into the insertion position;
updating the index of the hash bucket corresponding to the lock node of the resource to be locked; the index of each hash bucket points to the head node of the corresponding linked list interval; the lock nodes in the linked list interval corresponding to each hash bucket are sorted in an increasing mode according to the size of the hash value.
In an embodiment, the third determining module determines an insertion position of the lock node of the resource to be locked, specifically for:
determining a second hash bucket corresponding to the hash value of the lock node of the resource to be locked;
traversing the linked list interval from a head node of the linked list interval corresponding to the second hash bucket;
determining whether the hash value of the traversed current node is larger than or equal to the hash value of the lock node of the resource to be locked, and whether the first operation result is equal to the second operation result; the first operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the second operation result represents an operation result of right shift operation on the hash value of the current node;
And if the first operation result is equal to the second operation result and the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, determining that the insertion position is between the previous node of the current node and the current node.
In an embodiment, the third determining module determines a second hash bucket corresponding to a hash value of the lock node of the resource to be locked, and is specifically configured to:
performing right shift operation on the hash value of the lock node of the resource to be locked to obtain the first operation result;
and determining the second hash bucket according to the first operation result.
In an embodiment, the device further comprises:
the insertion position determining module is configured to determine whether a next node of the current node is empty if the hash value of the current node is smaller than the hash value of the lock node of the resource to be locked, or if the first operation result is not equal to the second operation result; and if the next node of the current node is empty, determining the insertion position as the next node of the current node.
In an embodiment, the insertion position determining module is further configured to:
if the next node of the current node is not empty, determining whether the next node of the current node belongs to a later hash bucket;
And if the next node of the current node belongs to a later hash bucket, determining the insertion position as between the next node of the current node and the current node.
In an embodiment, the device further comprises:
the pointer points to an updating module and is used for updating a successor node and a successor node of the lock node of the resource to be locked; directing a first pointer of the former relay node to a lock node of the resource to be locked; and under the condition that the subsequent node is not empty, pointing a second pointer of the subsequent node to the lock node of the resource to be locked.
In an embodiment, the third determining module updates an index of a hash bucket corresponding to the lock node of the resource to be locked, specifically:
determining whether a third hash bucket corresponding to the lock node of the resource to be locked has other nodes except the lock node of the resource to be locked;
and if the third hash bucket has no other nodes, pointing the index of the third hash bucket to the lock node of the resource to be locked.
In an embodiment, the third determining module is further configured to:
if the third hash bucket has other nodes, determining whether the hash value of the lock node of the resource to be locked is smaller than the hash value of the lock node pointed by the index of the third hash bucket;
And if the index of the third hash bucket is smaller than the index of the first hash bucket, pointing the index of the third hash bucket to the lock node of the resource to be locked.
In an embodiment, the third determining module determines whether the linked list interval corresponding to the first hash bucket includes the lock node of the resource to be locked, and is specifically configured to:
traversing the linked list interval from a head node of the linked list interval corresponding to the first hash bucket; the lock nodes in the linked list interval corresponding to each hash bucket are progressively ordered according to the size of the hash value;
determining whether the traversed current node is a lock node of the resource to be locked;
if the current node is not the lock node of the resource to be locked, determining whether the hash value of the resource to be locked is larger than the hash value of the current node, and whether the third operation result is equal to the fourth operation result; the third operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the fourth operation result represents an operation result of right shift operation on the hash value of the current node;
if the hash value of the resource to be locked is larger than the hash value of the current node and the third operation result is equal to the fourth operation result, continuing to traverse the next node of the current node until traversing to the last node of the linked list interval;
And if the hash value of the resource to be locked is smaller than the hash value of the current node or the third operation result is not equal to the fourth operation result, determining that the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked.
In an embodiment, the device further comprises:
a deleting module, configured to reduce the number of references of the lock node of the resource to be locked by 1; determining whether the reference number of the lock node of the resource to be locked is 0; and if the reference number of the lock node of the resource to be locked is 0, deleting the lock node of the resource to be locked from the doubly linked list.
In practical applications, the first determining module, the third determining module and the acquiring module may be implemented by a processor in the electronic device, such as a central processing unit (CPU, central Processing Unit), a digital signal processor (DSP, digital Signal Processor), a micro control unit (MCU, microcontrollerUnit) or a programmable gate array (FPGA, field-Programmable GateArray), etc.
It should be noted that: in the resource lock management device provided in the above embodiment, only the division of the above modules is used for illustration, and in practical application, the above processing allocation may be performed by different modules according to needs, that is, the internal structure of the device is divided into different modules, so as to complete all or part of the above processing. In addition, the resource lock management device and the resource lock management method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, which are not repeated herein.
The resource lock management device may be in the form of an image file, and after the image file is executed, the image file may be operated in the form of a container or a virtual machine, so as to implement the resource lock management method described in the application. Of course, the method is not limited to the image file form, and any software form capable of implementing the resource lock management method described in the application is within the scope of protection of the application.
Based on the hardware implementation of the program modules, and in order to implement the method of the embodiment of the application, the embodiment of the application also provides an electronic device. Fig. 9 is a schematic diagram of a hardware composition structure of an electronic device according to an embodiment of the present application, as shown in fig. 9, the electronic device includes:
a communication interface capable of information interaction with other devices such as a network device and the like;
and the processor is connected with the communication interface so as to realize information interaction with other equipment and is used for executing the method provided by one or more technical schemes on the electronic equipment side when the computer program is run. And the computer program is stored on the memory.
Of course, in practice, the various components in the electronic device are coupled together by a bus system. It will be appreciated that a bus system is used to enable connected communications between these components. The bus system includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled as bus systems in fig. 9.
The electronic device may be in a cluster form, for example, in a cloud computing platform form, where the cloud computing platform is a service form that uses computing virtualization, network virtualization, and storage virtualization technologies to organize a plurality of independent server physical hardware resources into pooled resources, and is a structure based on software defined resources based on development of virtualization technologies, and may provide resource capabilities in forms of virtual machines, containers, and the like. The method and the system have the characteristics of flexibility, elasticity, distribution, multiple tenants, on demand and the like, and are a novel IT (information technology) and software delivery mode by eliminating the fixed relation between hardware and an operating system, relying on the communication uniform resource scheduling of a network and then providing needed virtual resources and services.
Current cloud computing platforms support several service modes:
SaaS (Software as a Service ): the cloud computing platform user does not need to purchase the software, but rents the software deployed on the cloud computing platform instead, the user does not need to maintain the software, and the software service provider can manage and maintain the software in full right;
PaaS (Platform as a Service ): a cloud computing platform user (typically a software developer at this time) may build new applications on the architecture provided by the cloud computing platform or extend existing applications without having to purchase development, quality control, or production servers;
IaaS (Infrastructure as a Service ): the cloud computing platform provides data centers, infrastructure hardware and software resources through the internet, and the cloud computing platform in the IaaS mode can provide servers, operating systems, disk storage, databases and/or information resources.
The memory in the embodiments of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device.
It will be appreciated that the memory can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), a programmable Read Only Memory (PROM, programmable Read-Only Memory), an erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), an electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), a magnetic random access Memory (FRAM, ferromagnetic random access Memory), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a compact disk Read-Only Memory (CD-ROM, compactDisc Read-Only Memory); the magnetic surface memory may be a disk memory or a tape memory. The volatile Memory may be a random access Memory (RAM, randomAccess Memory) that acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM, static RandomAccess Memory), synchronous static random access memory (SSRAM, synchronous Static RandomAccess Memory), dynamic random access memory (DRAM, dynamic RandomAccess Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic RandomAccess Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double DataRate Synchronous Dynamic RandomAccess Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic RandomAccess Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic RandomAccess Memory), direct memory bus random access memory (DRRAM, direct Rambus RandomAccess Memory). The memory described in the embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.
The method disclosed in the embodiments of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general purpose processor, DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present application. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly embodied in a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium having a memory, and the processor reads the program in the memory and performs the steps of the method in combination with its hardware.
Optionally, when the processor executes the program, a corresponding flow implemented by the electronic device in each method of the embodiments of the present application is implemented, and for brevity, will not be described herein again.
In an exemplary embodiment, the present application further provides a storage medium, i.e. a computer storage medium, in particular a computer readable storage medium, for example comprising a first memory storing a computer program, which is executable by a processor of an electronic device to perform the steps of the aforementioned method. The computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, electronic device, and method may be implemented in other manners. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
Alternatively, the integrated units described above may be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributing to the related art, and the computer software product may be stored in a storage medium, and include several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
The technical solutions described in the embodiments of the present application may be arbitrarily combined without any conflict.
In addition, in the examples of this application, "first," "second," etc. are used to distinguish similar objects and not necessarily to describe a particular order or sequence.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (16)

1. A method for resource lock management, comprising:
determining a hash value corresponding to the unique identifier of the resource to be locked;
determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked;
determining a lock node of the resource to be locked based on a linked list interval corresponding to the first hash bucket;
and acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
2. The method of claim 1, wherein the determining the lock node for the resource to be locked based on the linked list interval corresponding to the first hash bucket comprises:
determining whether a linked list interval corresponding to the first hash bucket comprises a lock node of the resource to be locked;
if the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked, creating the lock node of the resource to be locked;
inserting the newly-built lock node of the resource to be locked into the doubly-linked list;
And if the linked list interval corresponding to the first hash bucket comprises the lock node of the resource to be locked, determining the lock node of the resource to be locked from the linked list interval corresponding to the first hash bucket.
3. The method of claim 2, wherein after the determining the lock node for the resource to be locked from the linked list interval corresponding to the first hash bucket, the method further comprises:
acquiring the current reference count of the resource lock of the resource to be locked in the memory;
and if the current reference count is the same as the expected original value, adding 1 to the current reference count of the resource lock of the resource to be locked in the memory.
4. The method of claim 2, wherein inserting the newly created lock node for the resource to be locked into the doubly linked list comprises:
acquiring a newly-built lock node of the resource to be locked;
determining the insertion position of the lock node of the resource to be locked;
inserting the lock node of the resource to be locked into the insertion position;
updating the index of the hash bucket corresponding to the lock node of the resource to be locked; the index of each hash bucket points to the head node of the corresponding linked list interval; the lock nodes in the linked list interval corresponding to each hash bucket are sorted in an increasing mode according to the size of the hash value.
5. The method of claim 4, wherein the determining the insertion location of the lock node of the resource to be locked comprises:
determining a second hash bucket corresponding to the hash value of the lock node of the resource to be locked;
traversing the linked list interval from a head node of the linked list interval corresponding to the second hash bucket;
determining whether the hash value of the traversed current node is larger than or equal to the hash value of the lock node of the resource to be locked, and whether the first operation result is equal to the second operation result; the first operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the second operation result represents an operation result of right shift operation on the hash value of the current node;
and if the first operation result is equal to the second operation result and the hash value of the current node is greater than or equal to the hash value of the lock node of the resource to be locked, determining that the insertion position is between the previous node of the current node and the current node.
6. The method of claim 5, wherein the determining a second hash bucket corresponding to the hash value of the lock node of the resource to be locked comprises:
Performing right shift operation on the hash value of the lock node of the resource to be locked to obtain the first operation result;
and determining the second hash bucket according to the first operation result.
7. The method of claim 5, wherein the method further comprises:
if the hash value of the current node is smaller than the hash value of the lock node of the resource to be locked, or the first operation result is not equal to the second operation result, determining whether the next node of the current node is empty;
and if the next node of the current node is empty, determining the insertion position as the next node of the current node.
8. The method of claim 7, wherein the method further comprises:
if the next node of the current node is not empty, determining whether the next node of the current node belongs to a later hash bucket;
and if the next node of the current node belongs to a later hash bucket, determining the insertion position as between the next node of the current node and the current node.
9. The method of claim 4, wherein after the inserting the lock node of the resource to be locked into the insertion location, the method further comprises:
Updating a relay node and a successor node of the lock node of the resource to be locked;
directing a first pointer of the former relay node to a lock node of the resource to be locked;
and under the condition that the subsequent node is not empty, pointing a second pointer of the subsequent node to the lock node of the resource to be locked.
10. The method of claim 4, wherein updating the index of the hash bucket corresponding to the lock node of the resource to be locked comprises:
determining whether a third hash bucket corresponding to the lock node of the resource to be locked has other nodes except the lock node of the resource to be locked;
and if the third hash bucket has no other nodes, pointing the index of the third hash bucket to the lock node of the resource to be locked.
11. The method of claim 10, wherein the method further comprises:
if the third hash bucket has other nodes, determining whether the hash value of the lock node of the resource to be locked is smaller than the hash value of the lock node pointed by the index of the third hash bucket;
and if the index of the third hash bucket is smaller than the index of the first hash bucket, pointing the index of the third hash bucket to the lock node of the resource to be locked.
12. The method of claim 2, wherein the determining whether the linked list interval corresponding to the first hash bucket includes the lock node of the resource to be locked comprises:
Traversing the linked list interval from a head node of the linked list interval corresponding to the first hash bucket; the lock nodes in the linked list interval corresponding to each hash bucket are progressively ordered according to the size of the hash value;
determining whether the traversed current node is a lock node of the resource to be locked;
if the current node is not the lock node of the resource to be locked, determining whether the hash value of the resource to be locked is larger than the hash value of the current node, and whether the third operation result is equal to the fourth operation result; the third operation result represents an operation result of right shift operation on the hash value of the resource to be locked; the fourth operation result represents an operation result of right shift operation on the hash value of the current node;
if the hash value of the resource to be locked is larger than the hash value of the current node and the third operation result is equal to the fourth operation result, continuing to traverse the next node of the current node until traversing to the last node of the linked list interval;
and if the hash value of the resource to be locked is smaller than the hash value of the current node or the third operation result is not equal to the fourth operation result, determining that the linked list interval corresponding to the first hash bucket does not comprise the lock node of the resource to be locked.
13. The method of claim 1, wherein after the resource lock for the resource to be locked is used, the method further comprises:
subtracting 1 from the reference number of the lock node of the resource to be locked;
determining whether the reference number of the lock node of the resource to be locked is 0;
and if the reference number of the lock node of the resource to be locked is 0, deleting the lock node of the resource to be locked from the doubly linked list.
14. A resource lock management apparatus, comprising:
the first determining module is used for determining a hash value corresponding to the unique identifier of the resource to be locked;
the second determining module is used for determining a first hash bucket corresponding to the hash value in a preset hash table; the preset hash table comprises a plurality of hash buckets, the hash buckets correspond to the same double-linked list, and each hash bucket corresponds to a section of linked list interval of the double-linked list; each lock node on the doubly linked list represents a resource lock of a resource to be locked;
the third determining module is used for determining the lock node of the resource to be locked based on the linked list interval corresponding to the first hash bucket;
and the acquisition module is used for acquiring the resource lock of the resource to be locked from the lock node of the resource to be locked.
15. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the resource lock management method of any of claims 1 to 13 when the computer program is executed by the processor.
16. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the resource lock management method according to any of claims 1 to 13.
CN202310430029.2A 2023-04-10 2023-04-10 Resource lock management method and device, electronic equipment and storage medium Pending CN116521943A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310430029.2A CN116521943A (en) 2023-04-10 2023-04-10 Resource lock management method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310430029.2A CN116521943A (en) 2023-04-10 2023-04-10 Resource lock management method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116521943A true CN116521943A (en) 2023-08-01

Family

ID=87391415

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310430029.2A Pending CN116521943A (en) 2023-04-10 2023-04-10 Resource lock management method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116521943A (en)

Similar Documents

Publication Publication Date Title
CN110287044B (en) Lock-free shared memory processing method and device, electronic equipment and readable storage medium
US9734607B2 (en) Graph processing using a mutable multilevel graph representation
US9047334B1 (en) Merge-update for efficient atomic memory modification in concurrent computer systems
US7376674B2 (en) Storage of multiple pre-modification short duration copies of database information in short term memory
US9563477B2 (en) Performing concurrent rehashing of a hash table for multithreaded applications
US9208258B2 (en) Locking and traversal methods for ordered tree data structures
US10108653B2 (en) Concurrent reads and inserts into a data structure without latching or waiting by readers
US20110060724A1 (en) Distributed database recovery
US20060020634A1 (en) Method, system and program for recording changes made to a database
US11392567B2 (en) Just-in-time multi-indexed tables in a shared log
CN111459920B (en) Multi-version concurrency control method and system based on virtual global clock synchronization
US6952707B1 (en) Efficient sequence number generation in a multi-system data-sharing environment
CN111316255B (en) Data storage system and method for providing a data storage system
CN110399333B (en) Method, apparatus and computer program product for deleting snapshots
US11221777B2 (en) Storage system indexed using persistent metadata structures
US20060277221A1 (en) Transactional file system with client partitioning
Hu et al. Parallel multi-split extendible hashing for persistent memory
US20110099347A1 (en) Managing allocation and deallocation of storage for data objects
US12019629B2 (en) Hash-based data structure
CN110832473A (en) Log structure management system and method
US20170177615A1 (en) TRANSACTION MANAGEMENT METHOD FOR ENHANCING DATA STABILITY OF NoSQL DATABASE BASED ON DISTRIBUTED FILE SYSTEM
CN116521943A (en) Resource lock management method and device, electronic equipment and storage medium
CN113722021B (en) Object sharing by entities using data structures
CN115048046A (en) Log file system and data management method
CN114691307A (en) Transaction processing method and computer system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination