WO2020107352A1

WO2020107352A1 - Log sequence number generation method and apparatus and readable storage medium

Info

Publication number: WO2020107352A1
Application number: PCT/CN2018/118292
Authority: WO
Inventors: 袁振南; 朱阅岸; 简怀兵
Original assignee: 袁振南; 区链通网络有限公司
Priority date: 2018-11-29
Filing date: 2018-11-29
Publication date: 2020-06-04
Also published as: CN109791541B; CN109791541A

Abstract

A log sequence number generation method and apparatus and a readable storage medium. The method comprises: using a lock manager to receive a locking application for operating an element (S1); determining whether the element is available or not (S2); and if the element is available, using the lock manager to lock the element and generate a log sequence number corresponding to the operation (S3).

Description

Log sequence number generating method, device and readable storage medium

【Technical Field】

The present application relates to the field of databases, and in particular, to a log sequence number generation method, device, and readable storage medium.

【Background technique】

The traditional log manager is a key component of a database management system (DataBase Management System, DBMS), which uses a centralized design approach. All transactions are appended to the log record in a global buffer. The log gives the sequence of all update operations of the database system. Log sequence number (LSN) is the address of the log record on the disk. It can be used as a logical timestamp to indicate the sequence of update actions, and it is also the basis for log record replay when the system is restored.

In a traditional DBMS, when a transaction needs to operate on a database object, it needs to apply to the lock manager to block the database object. After the application is successful, the corresponding operation can be performed. In addition, if the operation is to be recorded in the log, the corresponding LSN needs to be generated, including the following steps: 1) first obtain the corresponding exclusive lock on the log buffer; 2) generate the corresponding LSN for this write operation; 3) release The corresponding exclusive lock. Every time an exclusive lock needs to be added before generating the log record sequence number, other transactions compete for the exclusive lock. In concurrent scenarios, this exclusive lock is often easy to become a system bottleneck.

[Invention content]

The technical problem mainly solved by the present application is to provide a log sequence number generation method, device and readable storage medium, which can solve the problem that the exclusive lock used for LSN generation in the prior art easily becomes a system bottleneck.

In order to solve the above technical problems, the present application provides a method for generating a log sequence number. The method includes: using a lock manager to receive an application for locking an element; judging whether an element is available; if the element is available, using a lock manager Lock the element and generate the log sequence number corresponding to the operation.

In order to solve the above technical problems, the present application provides a log sequence number generating device. The device includes: a receiving module for receiving a lock application for operating an element with a lock manager; and a determining module for determining whether the element is available ; Generate module, used to lock the element when the element is available and generate the log sequence number corresponding to the operation.

In order to solve the above technical problems, the present application provides a log sequence number generating device, which includes a processor and a memory, the processor is coupled to the memory; the processor is used to execute instructions to use the lock manager to receive the lock for operating the element Apply; determine whether the element is available; if the element is available, use the lock manager to lock the element and generate the log sequence number corresponding to the operation.

In order to solve the above technical problem, the present application provides a readable storage medium that stores instructions, and when the instructions are executed, the foregoing method is implemented.

The beneficial effects of this application are: use the lock manager to receive the lock application for operating on the element; determine whether the element is available; if the element is available, use the lock manager to lock the element and generate the log sequence number corresponding to the operation, by the lock The manager is responsible for generating the log sequence number without using the log manager. During the operation of the transaction and the recording of this operation, it is only necessary to compete for locks in the lock manager. There is no need to compete for the exclusive lock to generate the log sequence number. The exclusive lock no longer exists, reducing a competitive operation in the critical section and improving the performance of the database.

【Explanation】

In order to more clearly explain the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings. among them:

FIG. 1 is a schematic flowchart of an embodiment of a method for generating a log sequence number of this application;

2 is a schematic flowchart of a specific embodiment of a method for generating a log sequence number of this application;

3 is a schematic diagram of a hash table in a specific embodiment of a method for generating a log sequence number of this application;

4 is a schematic structural diagram of an embodiment of an apparatus for generating a log sequence number in this application;

5 is a schematic structural diagram of another embodiment of an apparatus for generating a log sequence number of this application;

6 is a schematic structural diagram of an embodiment of a readable storage medium of the present application.

【detailed description】

The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Those in the following embodiments that do not conflict may be combined with each other. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

As shown in FIG. 1, an embodiment of the method for generating a log sequence number of the present application includes:

S1: Use the lock manager to receive the lock application for operating on the element.

In the database system, in order to improve the operating efficiency of the system, multiple transactions are often executed concurrently. Concurrent execution may cause multiple transactions to read/write the same data at the same time, which may lead to the destruction of data consistency, such as the reading of dirty data and the loss of updates. To ensure data consistency, a database management system (DataBase Management System, DBMS) needs to provide a concurrency control protocol to achieve different isolation levels for concurrent transactions. Among them, the lock-based concurrency control protocol (referred to as the lock protocol) is a commonly used concurrency Control protocol.

In the lock protocol, the transaction needs to obtain the control right of the operation object (ie element) before performing the operation. This process can also be called locking. The lock manager in the DBMS is responsible for the application and release of locks. The transaction needs to apply to the lock manager to lock the object, and the operation can only be performed on the object after the application is successful.

In an embodiment of the invention, the lock manager is implemented by a hash table, or the lock manager includes at least one hash table. The key code of the hash table is the database object id. To prevent a single hash table from becoming a hotspot in the system, the lock manager can include multiple hash tables. Multiple hash tables can be obtained by segmenting the overall hash table of all database objects in the management database. Different hash The database objects in the table do not conflict with each other, in which case a single hash table can be regarded as a segment of the overall hash table.

S2: Determine whether the element is available.

In order to prevent conflicts, it is necessary to determine whether the element is available after receiving the lock application, that is, whether the operation of the application can be performed on the element (that is, the operation corresponding to the lock application). Specifically, it can be determined whether there are other transactions that are operating on the element and excluding applications. If no other transaction is operating on the element, or there is another transaction that is operating on the element but the operation does not exclude the requested operation (for example, both are read operations), the element is available, otherwise the element is not available.

If the element is available, jump to S3; if the element is not available, you can choose to add the application to the lock application queue and suspend the transaction.

S3: Use the lock manager to lock the element and generate the log sequence number corresponding to the operation.

The order between locking and LSN generation is not limited. Locking generally includes recording lock information at the element, such as information about the transaction applying for locking, the type of lock (mutually exclusive lock, shared lock, intention lock, etc.), etc. After locking, the transaction can operate on the element. You can use the lock manager to allocate storage space for the operation in the log file buffer to generate the log sequence number (LSN) corresponding to the operation without using the log manager to compete for exclusive locks. The LSN marks the storage location corresponding to the log records of this operation.

Optionally, the operation includes a write operation, and the write operation includes at least one of adding, modifying, and deleting. After performing the write operation, the database objects in the database will change and should be recorded in the log. Although the lock may be applied before the read operation, but it will not change the database object, you can choose not to generate LSN for the read operation. Of course, LSNs can also be generated for read operations.

The number of log files can be one or more. When the number of log files is greater than one, different log files can be operated in parallel, that is, the following conditions are met: 1) transactions that operate on different log files do not conflict with each other and can be executed in parallel, that is, logs can be written in parallel; 2) system During crash recovery, parsing on different log files can be performed simultaneously and played back in parallel. Optionally, each log file corresponds to only one hash table. For example, the log files and the hash table have a one-to-one correspondence, which can achieve parallel LSN generation and further improve the efficiency of concurrent system operation.

Even in the case where only a lock manager including only one hash table and one log file are used, the log file generated by the method provided in this embodiment may not be completely consistent with the log file generated by the traditional centralized log manager, but In the log file generated by the method provided in this embodiment, the operation record sequence for the same database object is consistent with the traditional log and the actual sequence, and does not affect the actual use of the log file.

Through the implementation of this embodiment, the lock manager is responsible for generating the log sequence number without using the log manager. During the operation of the transaction and the recording of this operation, the lock manager only needs to compete with the lock, and no competition is required. Generate an exclusive lock for the log sequence number. The exclusive lock no longer exists, reducing a contention operation in the critical section and improving the performance of the database.

As shown in FIGS. 2 and 3, in a specific embodiment of the present application, the log sequence number generation method includes:

S11: Use the lock manager to receive a lock application for operating on the element.

This embodiment is based on the previous embodiment, using a hash table to implement the lock manager, and the same parts as in the previous embodiment are not repeated.

Hash tables (also called hash tables) are data structures that are accessed directly based on key codes. The key code of the hash table is the database object id. In other words, it accesses records by mapping key codes to a location in the table to speed up the search. This mapping function is called a hash function (hash function).

A hash table can be regarded as an array + linked list data structure. Each element in the array represents the key code of a database object in the database. Each element in the array corresponds to a hash bucket. The hash bucket stores the record Lock information for database objects.

S12: Calculate the hash function value of the element.

S13: Find a hash bucket corresponding to the hash function value in at least one hash table.

S14: Determine whether there are other locks that exclude operations in the hash bucket.

If there is no other lock that excludes the operation in the hash bucket, it means that the element is available, jump to S15.

S15: Use the lock manager to insert a lock into the hash bucket corresponding to the hash function value, and generate a log sequence number corresponding to the operation in the log file buffer corresponding to the hash table where the hash bucket is located.

The generated log sequence number LSN _i , i represents the sequence number of this operation.

As shown in FIG. 4, an embodiment of the log sequence number generating device of the present application includes:

The receiving module 11 is used to receive a lock application for operating an element by using a lock manager.

The judgment module 12 is used to judge whether the element is available.

The generating module 13 is configured to use a lock manager to lock the element when the element is available and generate a log sequence number corresponding to the operation.

As shown in FIG. 5, another embodiment of the log sequence number generating device of the present application includes: a processor 110 and a memory 120.

The processor 110 controls the operation of the log sequence number generating device. The processor 110 may also be called a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip with signal sequence processing capabilities. The processor 110 may also be a general-purpose processor, a digital signal sequence processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 120 stores instructions and data necessary for the processor 110 to work.

The processor 110 is configured to execute instructions to implement the method provided by any embodiment and possible combination of the method for generating a log sequence number of the present application.

As shown in FIG. 6, an embodiment of the readable storage medium of the present application includes a memory 210, and the memory 210 stores instructions, which are executed to implement the method provided by any embodiment of the method for generating a log sequence number of the present application and possible combinations .

The memory 210 may include read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), flash memory (Flash Memory), hard disk, optical disk, and the like.

In the several embodiments provided in this application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the module or unit is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be The combination can either be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application essentially or part of the contribution to the existing technology or all or part of the technical solution can be embodied in the form of a software product, the computer software product is stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .

The above are only the embodiments of the present application, and therefore do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by the description and drawings of this application, or directly or indirectly used in other related technologies In the field, the same reason is included in the scope of patent protection of this application.

Claims

A log sequence number generating method, characterized in that the method includes:

Use the lock manager to receive lock applications for operating on elements;

Determine whether the element is available;

If the element is available, the lock manager is used to lock the element and generate a log sequence number corresponding to the operation.
The method of claim 1, wherein:

The lock manager includes at least one hash table.
The method according to claim 2, characterized in that

The judging whether the element is available includes:

Calculate the hash function value of the element;

Find a hash bucket corresponding to the hash function value in the at least one hash table;

Determine whether there are other locks in the hash bucket that exclude the operation;

If there is no other lock in the hash bucket that excludes the operation, the element is determined to be available, otherwise the element is determined to be unavailable.
The method according to claim 3, characterized in that

The use of the lock manager to lock the element and generate a log sequence number corresponding to the operation includes:

Use the lock manager to insert the lock into a hash bucket corresponding to the hash function value, and generate the operation correspondence in a log file buffer corresponding to the hash table where the hash bucket is located Log sequence number.
The method according to any one of claims 2-4, characterized in that

The number of the hash tables is greater than one, and each of the hash tables corresponds to a log file, and different log files can be operated in parallel.
The method according to any one of claims 1-4, characterized in that

The operation includes a write operation.
A log sequence number generating device, characterized in that it includes:

The receiving module is used to receive a lock application for operating an element by using a lock manager;

The judgment module is used to judge whether the element is available;

The generating module is configured to use the lock manager to lock the element when the element is available and generate a log sequence number corresponding to the operation.
A log sequence number generating device, characterized in that it includes a processor and a memory, and the processor is coupled to the memory;

The processor is used to execute an instruction to use a lock manager to receive an application for locking an element; determine whether the element is available; if the element is available, use the lock manager to lock and generate the element The log sequence number corresponding to the operation.
The device according to claim 8, characterized in that

The lock manager includes at least one hash table.
The device according to claim 9, characterized in that

The processor is used to execute an instruction to calculate a hash function value of the element; find a hash bucket corresponding to the hash function value in the at least one hash table; determine whether there is exclusion in the hash bucket Other locks of the operation; if there is no other lock in the hash bucket that excludes the operation, the lock manager is used to insert the lock into the hash bucket corresponding to the hash function value, and A log sequence number corresponding to the operation is generated in a log file buffer corresponding to the hash table where the hash bucket is located.
The device according to claim 9 or 10, characterized in that

The number of the hash tables is greater than one, and each of the hash tables corresponds to a log file, and different log files can be operated in parallel.
The device according to any one of claims 8-10, characterized in that

The operation includes a write operation.
A readable storage medium storing instructions, characterized in that, when the instructions are executed, the method according to any one of claims 1-6 is implemented.