WO2020238485A1 - 一种数据库处理方法、装置及计算机可读存储介质 - Google Patents

一种数据库处理方法、装置及计算机可读存储介质 Download PDF

Info

Publication number
WO2020238485A1
WO2020238485A1 PCT/CN2020/085848 CN2020085848W WO2020238485A1 WO 2020238485 A1 WO2020238485 A1 WO 2020238485A1 CN 2020085848 W CN2020085848 W CN 2020085848W WO 2020238485 A1 WO2020238485 A1 WO 2020238485A1
Authority
WO
WIPO (PCT)
Prior art keywords
lock
cache
thread
processing method
log collection
Prior art date
Application number
PCT/CN2020/085848
Other languages
English (en)
French (fr)
Inventor
林品
丁岩
卢勤元
戚晨
俞义方
赵培
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to EP20814719.9A priority Critical patent/EP3979097A4/en
Priority to US17/615,581 priority patent/US11928132B2/en
Publication of WO2020238485A1 publication Critical patent/WO2020238485A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2358—Change logging, detection, and notification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing

Definitions

  • the embodiments of the present disclosure relate to, but are not limited to, a database processing method, device, and computer-readable storage medium.
  • the database receives various services sent by the client and converts them into transactions on the database to run concurrently. Because the records in the table may be locked or the entire table may be locked directly during the operation of the transaction, this will cause lock waiting and block business operation. When the operation and maintenance personnel find that the business running time is increasing and the performance of the entire system is degraded, they will think of inquiring whether a lock wait has occurred. However, the database only supports querying information about lock waits currently occurring, and cannot trace historical lock wait information.
  • Parallel database playback often causes playback failures due to lock waiting, and even causes playback threads to hang. It was too late for the operation and maintenance personnel to investigate after the problem was discovered, because the lock waiting information at that time could not be viewed. Therefore, it is necessary to improve this problem.
  • the embodiments of the present disclosure provide a database processing method, device, and computer-readable storage medium to improve database operation and maintenance efficiency.
  • At least one embodiment of the present disclosure provides a database processing method, including: after a lock wait is generated, writing lock wait related information into a lock wait log.
  • At least one embodiment of the present disclosure provides a database processing device, including a memory and a processor, the memory stores a program, and when the program is read and executed by the processor, the program described in any of the embodiments Database processing method.
  • At least one embodiment of the present disclosure provides a computer-readable storage medium that stores one or more programs, and the one or more programs can be executed by one or more processors to Implement the database processing method described in any embodiment.
  • FIG. 1 is a flowchart of a database processing method provided by an embodiment of the disclosure
  • FIG. 2 is a schematic diagram of a database processing device provided by an embodiment of the disclosure.
  • FIG. 3 is a flowchart of execution steps of a database startup module provided by an embodiment of the disclosure.
  • FIG. 4 is a flowchart of the execution steps of a business thread module provided by an embodiment of the disclosure.
  • FIG. 5 is a flowchart of the execution steps of the log collection module provided by an embodiment of the disclosure.
  • FIG. 6 is a flowchart of the execution steps of the database exit module provided by an embodiment of the disclosure.
  • FIG. 7 is a flowchart of a MySQL database processing method provided by an embodiment of the disclosure.
  • FIG. 8 is a flowchart of a MariaDB database processing method provided by an embodiment of the disclosure.
  • FIG. 9 is a flowchart of a Percona database processing method provided by an embodiment of the disclosure.
  • FIG. 10 is a flowchart of a PostgreSQL database processing method provided by an embodiment of the disclosure.
  • FIG. 11 is a sequence diagram of transaction execution provided by an embodiment of the disclosure.
  • FIG. 12 is a block diagram of a database processing device provided by an embodiment of the disclosure.
  • FIG. 13 is a block diagram of a computer-readable storage medium provided by an embodiment of the disclosure.
  • At least one embodiment of the present disclosure proposes a database processing method, which includes: recording lock waiting information generated on the database in a lock waiting log file for subsequent query.
  • the lock waiting information can be recorded in real time, and all lock waiting information can be recorded.
  • a lock wait occurs when a business thread executes a transaction, it needs to find out all other transactions that block the current transaction.
  • the time stamp of the lock wait time, related information about the requested transaction, and the blocked transaction The related information is written to the global cache.
  • the lock waiting log collection thread reads the global lock waiting information cache every preset time (such as one second) or when it is awakened (receiving a wake-up command), and writes the cache content into the lock waiting log.
  • a double cache mode of lock waiting information is designed: the log collection thread obtains the cache one that has written the lock waiting information, and then The second cache is provided to business threads. The log collection threads switch between the two caches to ensure that the log collection thread does not block business threads when writing log files, thereby improving system performance.
  • an embodiment of the present disclosure provides a database processing method, including:
  • Step 101 After the lock wait is generated, write lock wait related information into the lock wait log.
  • the lock waiting information is recorded in the lock waiting log, which facilitates subsequent query of lock waiting information and traces the lock waiting history information, thereby reducing the difficulty and cost of operation and maintenance, and improving the efficiency of database operation and maintenance.
  • step 101 after the lock wait is generated, writing lock wait related information into the lock wait log includes:
  • the log collection thread acquires the lock waiting information cache, and writes the lock waiting related information in the lock waiting information cache to the lock waiting log.
  • the service thread may also write the lock waiting information into the lock waiting log.
  • the lock waiting information cache is a global cache.
  • the log collection thread asynchronously writes the log mechanism in the background to avoid the business thread from generating additional IO operations and avoid affecting the database performance.
  • the lock wait related information includes at least one of the following: the lock wait generation time (for example, recording the time stamp of the lock wait generation time), related information about the requested transaction, and related information about the blocked transaction .
  • the lock wait generation time for example, recording the time stamp of the lock wait generation time
  • related information about the requested transaction for example, related information about the blocked transaction.
  • a requesting transaction refers to a transaction that is blocked due to a lock request
  • a blocking transaction refers to a transaction that blocks the current requesting transaction because a lock has been acquired.
  • the related information of the requested transaction includes at least one of the following: link number, transaction number, lock number, lock type, lock waiting time of the requested transaction. It should be noted that this is only an example, and other information related to the requested transaction may also be included.
  • the related information of the blocking transaction includes at least one of the following: the link number of the blocking transaction, the transaction number, the lock number, and the lock type. It should be noted that this is only an example, and other information related to blocking transactions can also be included.
  • the related information of the requested transaction and the related information of the corresponding blocking transaction can be written in pairs.
  • the method further includes: creating the log collection thread and the lock waiting information cache when the database is started.
  • the step of the log collection thread acquiring the lock waiting information cache includes: the log collection thread acquiring the lock waiting information cache when a preset condition is met.
  • the meeting the preset condition includes: receiving a wake-up instruction, or reaching the wake-up time.
  • the wake-up can be periodic, such as 1s as the cycle, and the wake-up time is reached every 1s.
  • the wake-up command can be sent by the service thread to the log collection thread. For example, when a business thread writes lock waiting related information into the cache, if the cache is found to be full, it sends a wake-up instruction to the log collection thread to take the lock waiting related information in the cache and write it into the lock waiting log.
  • the above preset conditions are only examples, and other conditions can be set as needed.
  • the lock waiting information cache may include at least two caches, for example, two caches.
  • the business thread uses one of the caches.
  • the log collection thread uses the cache currently used by the business thread, the business thread is notified to use another cache. In this way, the business thread and the log collection thread can be prevented from blocking each other, and the concurrency performance can be improved.
  • the step of obtaining the lock waiting information cache includes: obtaining a first cache used by a service thread in the lock waiting information cache;
  • the method further includes: switching the second cache in the lock waiting information cache to the service thread for use.
  • the first cache and the second cache only generally refer to two caches in the lock waiting information cache, and they are not fixed to a certain cache.
  • the double-buffer mechanism prevents the business thread and the background log writing thread from competing for the same buffer (cache) and blocking each other, which improves the concurrency of the business thread and avoids reducing database performance.
  • the method further includes: when the database is closed, closing the log collection thread and releasing the lock waiting information cache.
  • An embodiment of the present disclosure provides a database processing device, as shown in FIG. 2, including:
  • the business thread module 201 is configured to write lock wait related information into the lock wait information cache when a lock wait occurs;
  • the log collection thread module 202 is configured to acquire the lock waiting information cache, and write the lock waiting related information in the lock waiting information cache into the lock waiting log.
  • the log collection thread module 202 obtaining the lock waiting information cache includes: obtaining the first cache used by the service thread module in the lock waiting information cache;
  • the log collection thread module 202 is further configured to, after acquiring the lock waiting information cache, switch the second cache in the lock waiting information cache to the service thread module 201 for use.
  • the first cache and the second cache only generally refer to two caches in the lock waiting information cache, and they are not fixed to a certain cache.
  • the business thread module 201 is also configured to execute transactions.
  • the database processing device further includes: a database startup module 203;
  • the database startup module 203 is configured to create the log collection thread module and the lock waiting information cache when the database is started.
  • the database startup module 203 can also be used to create a lock waiting log file.
  • the database processing device further includes: a database exit module 204;
  • the database exit module 204 is configured to close the log collection thread module and release the lock waiting information cache when the database is closed. Specifically, the service thread module 201 and the log collection thread module 202 are notified to exit.
  • FIG. 3 is a flowchart of the steps performed by the database activation module 203 provided by an embodiment of the disclosure. As shown in Figure 3, the steps performed by the database startup module 203 include:
  • Step 301 Initialize the lock waiting information cache.
  • a lock waiting information buffer is allocated for the lock waiting information, where the lock waiting information buffer includes two buffers.
  • the log collection thread writes the contents of one buffer into the file, the business thread uses another buffer.
  • the dual-buffer mechanism ensures that the business thread and the log collection thread will not conflict when using the buffer, improving concurrency performance.
  • Step 302 Create a lock waiting log file.
  • step 302 may not be executed.
  • Step 303 Create a log collection thread.
  • the log collection thread is responsible for writing the lock wait related information stored in the lock wait cache to the lock wait log.
  • Fig. 4 is a flowchart of the steps executed by the service thread module 201 provided by an embodiment of the disclosure.
  • the function of writing lock waiting transaction information into the cache is added to the original process.
  • the business thread finds that other transactions have already applied for the lock, and stores the transaction information and all other transaction information that has already applied for the lock in the buffer of the lock waiting information cache.
  • the steps performed by the business thread module 201 include:
  • Step 401 the service thread takes out the available buffer from the buffer in the currently used lock waiting information buffer
  • Step 402 the service thread writes the related information of the requested transaction into the lock waiting information cache.
  • Step 403 The service thread writes the related information of the blocked transaction into the lock waiting information cache.
  • step 402 there is no order relationship between step 402 and step 403, and step 403 can be performed first, and then step 402 is performed.
  • the time when the lock wait is generated can also be written into the lock wait information cache. After the service thread saves the lock waiting information, the service thread module 201 enters the lock waiting state.
  • FIG. 5 is a flowchart of the steps performed by the log collection thread module 202 provided by an embodiment of the disclosure. As shown in FIG. 5, the steps performed by the log collection thread module 202 include:
  • step 501 the log collection thread enters a sleep state. For example, it goes to sleep after starting.
  • step 502 the log collection thread is awakened or 1 second expires.
  • the wake-up period is 1 second.
  • Step 503 Obtain the buffer being used by the service thread from the lock waiting information cache, such as buffer1.
  • Step 504 Switch another buffer (such as buffer2) in the lock waiting information cache to the service thread for use.
  • Step 505 Write the lock waiting information stored in buffer 1 into the lock waiting log file.
  • step 506 the log collection thread module 202 judges whether the log collection thread has exited, if it exits, it ends; if it does not exit, it returns to step 501.
  • Fig. 6 is a flowchart of the steps performed by the database exit module 204 provided by an embodiment of the disclosure. As shown in Figure 6, the functions of closing the log collection thread and releasing the lock waiting for the information cache are added to the original process.
  • the steps performed by the database exit module 204 include:
  • Step 601 close the log collection thread
  • Step 602 Release the lock and wait for the information cache.
  • the lock waiting information is written into the buffer through the business thread, and the log collection thread records the lock waiting information in the buffer in the log file, which facilitates the traceability of the lock waiting history information and reduces the difficulty and cost of operation and maintenance.
  • the asynchronous log writing mechanism in the background the business thread avoids additional IO operations.
  • the use of double cache and background asynchronous log writing mechanism makes this solution not reduce database performance.
  • the double buffer mechanism prevents business threads and log collection threads from competing for the same buffer and blocking each other, which improves the concurrency of business threads.
  • This example provides a method to record lock waiting history information based on MySQL database.
  • the method for recording lock waiting history information based on MySQL database is as follows, including:
  • Step 701 Create a lock waiting information cache that stores lock waiting information during MySQL startup.
  • Step 702 Create a log collection thread.
  • step 703 the database is successfully started.
  • step 704 the SQL request from the client is cyclically monitored.
  • step 705 after MySQL is started, it starts to monitor connection requests from the client. After listening to the connection request, it will create a link to receive SQL requests on the link and create a business thread to handle these requests.
  • Step 706 Parse the received SQL statement.
  • Step 707 Apply for a lock for the SQL statement.
  • step 708 if lock waiting is found during the lock application process, the related information of the requested transaction and the related information of all blocked transactions are written into the lock waiting information cache.
  • step 709 after waiting for the execution of the blocking transaction to end, the requesting transaction can apply for a lock, and then execute the SQL statement.
  • Step 710 Reply to the client.
  • This method continues to receive the SQL request, and repeats steps 705 to 710 after receiving the request.
  • the log collection thread is created during the database startup phase and writes lock waiting information to the log file.
  • step 711 the log collection thread sleeps for 1 second.
  • step 712 the log collection thread obtains the buffer currently being used by the business thread under the protection of the mutex, and the business thread will be blocked at this time.
  • step 713 the log collection thread switches another buffer to the service thread to use, releases the mutex obtained in step 712, and the service thread can continue to run.
  • step 714 the log collection thread writes the cache content obtained in step 712 into the lock waiting log file, and returns to step 711.
  • MySQL starts shutting down the database after receiving the shutdown (shutdown) command.
  • Other operations in the database shutdown phase have been omitted, and only the steps related to this solution are described below.
  • step 715 the service thread is notified to exit, and the log collection thread is notified to exit.
  • Step 716 Wait for the service thread and the log collection thread to exit.
  • Step 717 Release the lock and wait for the information cache.
  • This example provides a method to record lock waiting history information based on Mariadb.
  • the method for recording lock waiting history information based on Mariadb database is as follows, including:
  • Step 801 Create a lock waiting information cache that stores lock waiting information during MariaDB startup.
  • Step 802 Create a log collection thread.
  • step 803 the database is successfully started.
  • Step 804 Loop to monitor the SQL request from the client.
  • step 805 after MariaDB is started, it starts to monitor connection requests from the client. After listening to the connection request, it will create a link to receive SQL requests on the link and create a business thread to handle these requests.
  • Step 806 Parse the received SQL statement.
  • Step 807 Apply for a lock for the SQL statement.
  • Step 808 if lock waiting is found during the lock application process, the related information of the requested transaction and all related information of the blocked transaction are written into the lock waiting cache.
  • step 809 after waiting for the execution of the blocking transaction to end, the requesting transaction can apply for a lock, and then execute the SQL statement.
  • step 810 a response is returned to the client.
  • This method continues to receive the SQL request, and repeats steps 805 to 810 after receiving the request.
  • the log collection thread is created during the database startup phase and is responsible for writing lock waiting information to the log file.
  • Step 811 the log collection thread sleeps for 1 second.
  • step 812 the log collection thread obtains the buffer currently being used by the business thread under the protection of the mutex, and the business thread will be blocked at this time.
  • step 813 the log collection thread switches another buffer to the service thread to use, releases the mutex obtained in step 812, and the service thread can continue to run.
  • step 814 the log collection thread writes the cache content obtained in step 812 into the lock waiting log file, and returns to step 811.
  • MariaDB began to shut down the database after receiving the shutdown command. Other operations in the database shutdown phase have been omitted, and only the steps related to this solution are described below.
  • step 815 the service thread is notified to exit, and the log collection thread is notified to exit.
  • Step 816 Wait for the service thread and the log collection thread to exit.
  • Step 817 Release the lock and wait for the information cache.
  • This example provides a method for recording lock waiting history information based on Percona database, as shown in Figure 9, including:
  • Step 901 Create a lock waiting information cache for storing lock waiting information during the startup of Percona.
  • Step 902 Create a log collection thread.
  • step 903 the database is started successfully.
  • Step 904 Loop to monitor the SQL request from the client.
  • Step 905 After Percona is started, it starts to monitor connection requests from the client. After listening to the connection request, it will create a link to receive SQL requests on the link and create a business thread to handle these requests;
  • Step 906 parse the received SQL statement
  • Step 907 apply for a lock for the SQL statement
  • Step 908 if lock waiting is found during the lock application process, the related information of the requested transaction and all related information of the blocked transaction are written into the lock waiting cache;
  • Step 909 after waiting for the blocking transaction to be executed, the requesting transaction can apply for a lock, and then execute the SQL statement;
  • Step 910 a response is returned to the client.
  • This method continues to receive the SQL request, and repeats steps 905 to 910 after receiving the request.
  • the log collection thread is created during the database startup phase and is responsible for writing lock waiting information to the log file.
  • step 911 the log collection thread sleeps for 1 second.
  • step 912 the log collection thread obtains the buffer currently being used by the business thread under the protection of the mutex, and the business thread will be blocked at this time.
  • step 913 the log collection thread switches another buffer to the service thread to use, releases the mutex obtained in step 912, and the service thread can continue to run.
  • Step 914 write the cache content obtained in step 912 into the lock waiting log file, and return to step 911 to execute again.
  • Percona started shutting down the database after receiving the shutdown command. Other operations in the database shutdown phase have been omitted, and only the steps related to this solution are described below.
  • step 915 the service thread is notified to exit, and the log collection thread is notified to exit.
  • Step 916 Wait for the service thread and the log collection thread to exit.
  • Step 917 Release the lock and wait for the information cache.
  • This example provides a method for recording lock waiting history information based on a PostgreSQL database, as shown in Figure 10, including:
  • Step 1001 Create a lock waiting information cache for storing lock waiting information during the startup of PostgreSQL.
  • Step 1002 Create a log collection thread.
  • Step 1003 the database is started successfully.
  • Step 1004 cyclically monitor the SQL request from the client.
  • Step 1005 After PostgreSQL is started up, it starts to monitor connection requests from the client. After listening to the connection request, it will create a link to receive SQL requests on the link and create a business thread to handle these requests.
  • Step 1006 Parse the received SQL statement.
  • Step 1007 Apply for a lock for the SQL statement.
  • Step 1008 If lock waiting is found during the lock application process, the related information of the requested transaction and the related information of all blocked transactions are written into the lock waiting cache.
  • Step 1009 after waiting for the execution of the blocking transaction to end, the requesting transaction can apply for a lock, and then execute the SQL statement.
  • Step 1010 respond back to the client.
  • the log collection thread is created during the database startup phase and is responsible for writing lock waiting information to the log file.
  • Step 1011 the log collection thread sleeps for 1 second.
  • step 1012 the log collection thread obtains the buffer currently being used by the business thread under the protection of the mutex, and the business thread will be blocked at this time.
  • step 1013 the log collection thread switches another buffer to the service thread to use, releases the mutex obtained in step 1012, and the service thread can continue to run.
  • step 1014 the log collection thread writes the cache content obtained in step 1012 into the lock waiting log file, and returns to step 1011 to execute again.
  • PostgreSQL starts to shut down the database after receiving the shutdown command.
  • Other operations in the database shutdown phase have been omitted, and only the steps related to this solution are described below.
  • Step 1015 notify the service thread to exit, and notify the log collection thread to exit.
  • Step 1016 Wait for the service thread and the log collection thread to exit.
  • Step 1017 Release the lock and wait for the information cache.
  • the implementation of technical solutions during transaction operation includes:
  • Step 1101 business thread 1 starts transaction 1;
  • Step 1102 transaction 1 applies for a lock
  • Step 1104 transaction 1 starts to execute
  • Step 1105 transaction 2 applies for a lock
  • Step 1106 Transaction 2 detects lock waiting
  • Step 1107 the business thread 2 writes the related information of the transaction 2 (request transaction) and the transaction 1 (blocking transaction) into the cache 1;
  • Step 1108 the execution of transaction 1 ends
  • Step 1109 transaction 1 releases the lock
  • Step 1110 Transaction 1 responds to the client
  • Step 1111 the log collection thread obtains the cache currently used by the business thread (ie, cache 1);
  • Step 1112 the log collection thread switches the buffer 2 to the business thread for use
  • Step 1113 the log collection thread writes the content of the cache 1 into the lock waiting log file
  • Step 1114 transaction 2 applies for the lock
  • Step 1115 the execution of transaction 2 ends
  • Step 1116 Transaction 2 returns a response to the client.
  • an embodiment of the present disclosure provides a database processing device 120, which includes a memory 1210 and a processor 1220.
  • the memory 1210 stores a program.
  • the program is read and executed by the processor 1220, , To implement the database processing method described in any embodiment.
  • an embodiment of the present disclosure provides a computer-readable storage medium 130.
  • the computer-readable storage medium 130 stores one or more programs 1310, and the one or more programs 1310 can be Or executed by multiple processors to implement the database processing method described in any embodiment.
  • Such software may be distributed on a computer-readable medium, and the computer-readable medium may include a computer storage medium (or a non-transitory medium) and a communication medium (or a transitory medium).
  • the term computer storage medium includes volatile and non-volatile memory implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Sexual, removable and non-removable media.
  • Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage device, or Any other medium used to store desired information and that can be accessed by a computer.
  • communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

提供一种数据库处理方法、装置及计算机可读存储介质。所述数据库处理方法包括：产生锁等待后，将锁等待相关信息写入锁等待日志。

Description

一种数据库处理方法、装置及计算机可读存储介质 技术领域
本公开实施例涉及但不限于一种数据库处理方法、装置及计算机可读存储介质。
背景技术
数据库接收客户端发送过来的各种业务，转换为数据库上的事务并发地运行。由于事务运行过程中可能对表中的记录加锁或直接对整张表加锁，这就会产生锁等待从而阻塞业务运行。当运维人员发现业务运行时间增加、整个系统性能下降时，会想到去查询是否产生了锁等待。但是，数据库只支持查询当前正在发生的锁等待信息，无法追溯历史的锁等待信息。
数据库并行回放时经常因为锁等待导致回放失败，甚至导致回放线程挂起。运维人员在发现问题后再去排查已经晚了，因为当时的锁等待信息已经无法查看。因此，有必要针对此问题进行改进。
发明内容
本公开的实施例提供了一种数据库处理方法、装置及计算机可读存储介质，提高数据库运维效率。
本公开的至少一实施例提供了一种数据库处理方法，包括：产生锁等待后，将锁等待相关信息写入锁等待日志。
本公开的至少一实施例提供了一种数据库处理装置，包括存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现任一实施例所述的数据库处理方法。
本公开的至少一实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现任一实施例所述的数据库处理 方法。
附图说明
附图用来提供对本公开技术方案的进一步理解，并且构成说明书的一部分，与本公开的实施例一起用于解释本公开的技术方案，并不构成对本公开技术方案的限制。
图1为本公开一实施例提供的数据库处理方法的流程图；
图2为本公开一实施例提供的数据库处理装置的示意图；
图3为本公开一实施例提供的数据库启动模块执行步骤的流程图；
图4为本公开一实施例提供的业务线程模块执行步骤的流程图；
图5为本公开一实施例提供的日志搜集模块执行步骤的流程图；
图6为本公开一实施例提供的数据库退出模块执行步骤的流程图；
图7为本公开一实施例提供的MySQL数据库处理方法的流程图；
图8为本公开一实施例提供的MariaDB数据库处理方法的流程图；
图9为本公开一实施例提供的Percona数据库处理方法的流程图；
图10为本公开一实施例提供的PostgreSQL数据库处理方法的流程图；
图11为本公开一实施例提供的事务执行时序图；
图12为本公开一实施例提供的数据库处理装置的框图；以及
图13为本公开一实施例提供的计算机可读存储介质的框图。
具体实施方式
为使本公开的目的、技术方案和优点更加清楚明白，下文中将结合附图对本公开的实施例进行详细说明。需要说明的是，在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互任意组合。
在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且，虽然在流程图中示出了逻辑顺序，但是在 某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
在大多数情况下数据库排查问题时都需要查询历史的锁等待信息。因此，记录历史锁等待信息显地尤为必要，可以帮助运维人员精确地定位问题。本公开的至少一实施例提出了一种数据库处理方法，该方法包括：将数据库上产生过的锁等待信息记录到锁等待日志文件中，供后续查询。在一个示例性实施例中，可以对锁等待信息实时进行记录，并且可以记录全部锁等待信息。
在本公开的至少一实施例中，业务线程执行事务时如果产生了锁等待，则需要找出阻塞当前事务的所有其他事务，将发生锁等待时刻的时间戳、请求事务的相关信息、阻塞事务的相关信息写入全局缓存。锁等待日志搜集线程每隔预设时间(比如一秒)或被唤醒时(接收到唤醒指令)读取一次全局的锁等待信息缓存，并将缓存内容写入到锁等待日志中。为了最大程度地减小因记录锁等待日志带来的性能损耗，在本公开一实施例中，设计了锁等待信息双缓存模式：日志搜集线程获取已经写入锁等待信息的缓存一，然后将缓存二提供给业务线程使用。两个缓存之间通过日志搜集线程互相切换，保证日志搜集线程写日志文件时不会阻塞业务线程，从而提高系统的性能。
如图1所示，本公开一实施例提供一种数据库处理方法，包括：
步骤101，产生锁等待后，将锁等待相关信息写入锁等待日志。
根据本实施例提供的方案，将锁等待信息记录到锁等待日志中，便于后续查询锁等待信息，追溯锁等待历史信息，从而降低了运维难度和成本，提高数据库运维效率。
在一实施例中，所述步骤101中，产生了锁等待后，将锁等待相关信息写入锁等待日志包括：
业务线程在产生了锁等待时，将锁等待相关信息写入锁等待信息缓存；以及
日志搜集线程获取所述锁等待信息缓存，将所述锁等待信息缓存中的锁等待相关信息写入锁等待日志。
需要说明的是，在其他实施例中，也可以是业务线程将锁等待信息写入锁等待日志。
在一个示例性实施例中，锁等待信息缓存为全局缓存。
根据本实施例提供的方案，由日志搜集线程在后台异步写日志机制，避免业务线程产生额外的IO操作，避免影响数据库性能。
在一个示例性实施例中，所述锁等待相关信息包括以下至少之一：所述锁等待产生时刻(比如，记录锁等待产生时刻的时间戳)、请求事务的相关信息、阻塞事务的相关信息。在一个示例性实施例中，请求事务是指因请求锁而被阻塞的事务；阻塞事务是指因已获取锁而阻塞当前请求事务的事务。
在一实施例中，所述请求事务的相关信息包括以下至少之一：所述请求事务所在链路编号、事务编号、锁编号、锁类型、锁等待时间。需要说明的是，此处仅为示例，还可包括其他与请求事务相关的信息。
在一实施例中，所述阻塞事务的相关信息包括以下至少之一：所述阻塞事务所在链路编号、事务编号、锁编号、锁类型。需要说明的是，此处仅为示例，还可包括其他与阻塞事务相关的信息。
在一个示例性实施例中，一个请求事务可能存在多个阻塞事务。可将请求事务的相关信息和其对应的阻塞事务的相关信息成对写入。
在一实施例中，所述方法还包括：在数据库启动时，创建所述日志搜集线程和所述锁等待信息缓存。
在一实施例中，所述日志搜集线程获取所述锁等待信息缓存的步骤包括：日志搜集线程在满足预设条件时获取所述锁等待信息缓存。
在一实施例中，所述满足预设条件包括：接收到唤醒指令，或者，到达唤醒时间。唤醒可以是周期性的，比如以1s为周期，每过1s则到达唤醒时间。唤醒指令可以由业务线程发送给日志搜集线程。比如，当业务线程将锁等待相关信息写入缓存时，如果发现缓存已满，则发送唤醒指令给日志搜集线程，以便将缓存中的锁等待相关信息取走写入锁等待日志。需要说明的是，上述预设条件仅为示例，可以根据需 要设置其他条件。
在一个示例性实施例中，锁等待信息缓存可以包括至少两个缓存，比如，包括两个缓存。业务线程使用其中一个缓存。在日志搜集线程使用业务线程当前使用的缓存时，通知业务线程使用另一个缓存。这样，可以避免业务线程和日志搜集线程互相阻塞，提高并发性能。
在一实施例中，所述获取所述锁等待信息缓存的步骤包括：获取所述锁等待信息缓存中业务线程使用的第一缓存；
所述获取所述锁等待信息缓存后，所述方法还包括：将所述锁等待信息缓存中的第二缓存切换给业务线程使用。需要说明的是，第一缓存和第二缓存仅泛指锁等待信息缓存中的两个缓存，非固定对应某一缓存。
本实施例提供的方案，双缓存机制避免业务线程和后台写日志线程竞争同一个buffer(缓存)而互相阻塞，提高了业务线程的并发性，避免降低数据库性能。
在一实施例中，所述方法还包括：在数据库关闭时，关闭所述日志搜集线程和释放所述锁等待信息缓存。
本公开一实施例提供一种数据库处理装置，如图2所示，包括：
业务线程模块201，被构造成在产生了锁等待时，将锁等待相关信息写入锁等待信息缓存；以及
日志搜集线程模块202，被构造成获取所述锁等待信息缓存，将所述锁等待信息缓存中的锁等待相关信息写入锁等待日志。
在一实施例中，所述日志搜集线程模块202获取所述锁等待信息缓存包括：获取所述锁等待信息缓存中所述业务线程模块使用的第一缓存；
所述日志搜集线程模块202还被构造成，获取所述锁等待信息缓存后，将所述锁等待信息缓存中的第二缓存切换给所述业务线程模块201使用。需要说明的是，第一缓存和第二缓存仅泛指锁等待信息缓存中的两个缓存，非固定对应某一缓存。
在一个示例性实施例中，业务线程模块201还被构造成执行事务。
在一实施例中，所述数据库处理装置还包括：数据库启动模块203；
所述数据库启动模块203被构造成，在数据库启动时，创建所述日志搜集线程模块和所述锁等待信息缓存。所述数据库启动模块203还可用于创建锁等待日志文件。
在一实施例中，所述数据库处理装置还包括：数据库退出模块204；
所述数据库退出模块204被构造成，在数据库关闭时，关闭所述日志搜集线程模块和释放所述锁等待信息缓存。具体的，通知所述业务线程模块201和日志搜集线程模块202退出。
下面通过具体实例对本公开作进一步说明。
图3为本公开一实施例提供的数据库启动模块203执行的步骤的流程图。如图3所示，数据库启动模块203执行的步骤包括：
步骤301，初始化锁等待信息缓存。
在该步骤中，为锁等待信息分配锁等待信息缓存，其中，锁等待信息缓存中包括两个buffer。当日志搜集线程将一个buffer的内容写入文件的过程中，业务线程去使用另外一个buffer。双buffer机制保证业务线程和日志搜集线程使用buffer时不会冲突，提高并发性能。
步骤302，创建锁等待日志文件。
锁等待信息缓存中临时存储的锁等待信息最终持久化到锁等待日志文件中。需要说明的是，如果在数据库启动前已建立锁等待日志文件，也可以不执行步骤302。
步骤303，创建日志搜集线程。
数据库在启动阶段完成初始化锁等待信息缓存后，创建日志搜集线程。该日志搜集线程负责将锁等待缓存中保存的锁等待相关信息写入到锁等待日志中。
需要说明的是，上述步骤301～303无顺序关系，可以调整执行顺序。
图4为本公开一实施例提供的业务线程模块201执行的步骤的流 程图。在原有流程中加入将锁等待事务信息写入缓存的功能。业务线程在申请锁的过程中发现已经有其他事务申请了该锁，将本事务信息和已经申请该锁的所有其他事务信息存储到锁等待信息缓存的buffer中。
如图4所示，业务线程模块201执行的步骤包括：
步骤401，业务线程从当前使用的锁等待信息缓存中的buffer取出可用缓存；
步骤402，业务线程将请求事务的相关信息写入锁等待信息缓存；以及
步骤403，业务线程将阻塞事务的相关信息写入锁等待信息缓存。
需要说明的是，步骤402和步骤403无顺序关系，可以先执行步骤403，再执行步骤402。
需要说明的是，还可将锁等待产生时刻写入锁等待信息缓存。在业务线程保存好锁等待信息后，业务线程模块201进入锁等待状态。
图5为本公开一实施例提供的日志搜集线程模块202执行的步骤的流程图。如图5所示，日志搜集线程模块202执行的步骤包括：
步骤501，日志搜集线程进入睡眠状态。比如，启动后进入睡眠状态。
步骤502，日志搜集线程被唤醒或1秒超时。在本实施例中，唤醒周期为1秒。
步骤503，从锁等待信息缓存中获取业务线程正在使用的buffer，比如buffer1。
步骤504，将锁等待信息缓存中的另一buffer(比如buffer2)切换给业务线程使用。
步骤505，将buffer 1中保存的锁等待信息写入锁等待日志文件中。
步骤506，日志搜集线程模块202判断日志搜集线程是否退出，如果退出，结束，如果未退出，返回步骤501。
图6为本公开一实施例提供的数据库退出模块204执行的步骤的 流程图。如图6所示，在原有流程中加入了关闭日志搜集线程和释放锁等待信息缓存的功能。
如图6所示，数据库退出模块204执行的步骤包括：
步骤601，关闭日志搜集线程；以及
步骤602，释放锁等待信息缓存。
本公开一实施例中，通过业务线程将锁等待信息写入buffer，日志搜集线程将buffer中的锁等待信息记录在日志文件中，便于追溯锁等待历史信息，降低了运维难度和成本。通过后台异步写日志机制，避免业务线程产生额外的IO操作。采用双缓存以及后台异步写日志机制使得本方案不会降低数据库性能。双缓存机制避免业务线程和日志搜集线程竞争同一个buffer而互相阻塞，提高了业务线程的并发性。
本公开的实施例提供的方案适用于所有的关系型数据库。下面举例进行说明。
示例一
本示例提供一种基于MySQL数据库实现记录锁等待历史信息的方法。如图7所示，基于MySQL数据库实现记录锁等待历史信息的方法如下，包括：
(一)启动MySQL
步骤701，MySQL启动过程中创建存储锁等待信息的锁等待信息缓存。
步骤702，创建日志搜集线程。
步骤703，数据库启动成功。
步骤704，循环监听来自客户端的SQL请求。
(二)创建业务线程
步骤705，MySQL启动完成后开始监听来自客户端的连接请求，监听到连接请求后会创建一个链路来接收该链路上的SQL请求并创建一个业务线程来处这些请求。
步骤706，解析接收到的SQL语句。
步骤707，为该SQL语句申请锁。
步骤708，在申请锁过程中如果发现存在锁等待则将请求事务的相关信息和所有阻塞事务的相关信息写入到锁等待信息缓存中。
步骤709，等待阻塞事务执行结束后，请求事务可以申请到锁，然后执行该SQL语句。
步骤710，向客户端回响应。
该方法继续接收SQL请求，接收到请求后重复执行步骤705～710。
(三)创建日志搜集线程
日志搜集线程在数据库启动阶段被创建，将锁等待信息写入日志文件。
步骤711，日志搜集线程睡眠1秒。
步骤712，日志搜集线程在互斥量的保护下获取业务线程当前正在使用的缓存，此时业务线程会被阻塞。
步骤713，日志搜集线程将另一个缓存切换给业务线程使用，释放步骤712获取的互斥量，业务线程可以继续运行。
步骤714，日志搜集线程将步骤712获取的缓存内容写入锁等待日志文件中，返回步骤711。
(四)关闭MySQL
MySQL在收到shutdown(关闭)命令后开始关闭数据库。数据库关闭阶段的其他操作已被省略，下面仅介绍本方案相关的步骤。
步骤715，通知业务线程退出，通知日志搜集线程退出。
步骤716，等待业务线程和日志搜集线程退出完毕。
步骤717，释放锁等待信息缓存。
示例二
本示例提供一种基于Mariadb实现记录锁等待历史信息的方法。如图8所示，基于Mariadb数据库实现记录锁等待历史信息的方法如下， 包括：
(一)启动MariaDB
步骤801，MariaDB启动过程中创建存储锁等待信息的锁等待信息缓存。
步骤802，创建日志搜集线程。
步骤803，数据库启动成功。
步骤804，循环监听来自客户端的SQL请求。
(二)创建业务线程
步骤805，MariaDB启动完成后开始监听来自客户端的连接请求，监听到连接请求后会创建一个链路来接收该链路上的SQL请求并创建一个业务线程来处这些请求。
步骤806，解析接收到的SQL语句。
步骤807，为该SQL语句申请锁。
步骤808，申请锁过程中如果发现存在锁等待，则将请求事务的相关信息和所有阻塞事务的相关信息写入到锁等待缓存中。
步骤809，等待阻塞事务执行结束后，请求事务可以申请到锁，然后执行该SQL语句。
步骤810，向客户端回响应。
该方法继续接收SQL请求，接收到请求后重复执行步骤805～810。
(三)创建日志搜集线程
日志搜集线程在数据库启动阶段被创建，负责将锁等待信息写入日志文件。
步骤811，日志搜集线程睡眠1秒。
步骤812，日志搜集线程在互斥量的保护下获取业务线程当前正在使用的缓存，此时业务线程会被阻塞。
步骤813，日志搜集线程将另一个缓存切换给业务线程使用，释放步骤812获取的互斥量，业务线程可以继续运行。
步骤814，日志搜集线程将步骤812获取的缓存内容写入锁等待 日志文件中，返回步骤811。
(四)关闭MariaDB
MariaDB在收到shutdown命令后开始关闭数据库。数据库关闭阶段的其他操作已被省略，下面仅介绍本方案相关的步骤。
步骤815，通知业务线程退出，通知日志搜集线程退出。
步骤816，等待业务线程和日志搜集线程退出完毕。
步骤817，释放锁等待信息缓存。
示例三
本示例提供一种基于Percona数据库实现记录锁等待历史信息的方法，如图9所示，包括：
(一)启动Percona
步骤901，Percona启动过程中创建存储锁等待信息的锁等待信息缓存。
步骤902，创建日志搜集线程。
步骤903，数据库启动成功。
步骤904，循环监听来自客户端的SQL请求。
(二)创建业务线程
步骤905，Percona启动完成后开始监听来自客户端的连接请求，监听到连接请求后会创建一个链路来接收该链路上的SQL请求并创建一个业务线程来处这些请求；
步骤906，解析接收到的SQL语句；
步骤907，为该SQL语句申请锁；
步骤908，申请锁过程中如果发现存在锁等待则将请求事务的相关信息和所有阻塞事务的相关信息写入到锁等待缓存中；
步骤909，等待阻塞事务执行结束后请求事务可以申请到锁，然后执行该SQL语句；
步骤910，向客户端回响应。
该方法继续接收SQL请求，接收到请求后重复执行步骤905～910。
(三)创建日志搜集线程
日志搜集线程在数据库启动阶段被创建，负责将锁等待信息写入日志文件。
步骤911，日志搜集线程睡眠1秒。
步骤912，日志搜集线程在互斥量的保护下获取业务线程当前正在使用的缓存，此时业务线程会被阻塞。
步骤913，日志搜集线程将另一个缓存切换给业务线程使用，释放步骤912获取的互斥量，业务线程可以继续运行。
步骤914，将步骤912获取的缓存内容写入锁等待日志文件中，返回步骤911重新执行。
(四)关闭Percona
Percona在收到shutdown命令后开始关闭数据库。数据库关闭阶段的其他操作已被省略，下面仅介绍本方案相关的步骤。
步骤915，通知业务线程退出，通知日志搜集线程退出。
步骤916，等待业务线程和日志搜集线程退出完毕。
步骤917，释放锁等待信息缓存。
示例四
本示例提供一种基于PostgreSQL数据库实现记录锁等待历史信息的方法，如图10所示，包括：
(一)启动PostgreSQL
步骤1001，PostgreSQL启动过程中创建存储锁等待信息的锁等待信息缓存。
步骤1002，创建日志搜集线程。
步骤1003，数据库启动成功。
步骤1004，循环监听来自客户端的SQL请求。
(二)创建业务线程
步骤1005，PostgreSQL启动完成后开始监听来自客户端的连接请求，监听到连接请求后会创建一个链路来接收该链路上的SQL请求并创建一个业务线程来处这些请求。
步骤1006，解析接收到的SQL语句。
步骤1007，为该SQL语句申请锁。
步骤1008，申请锁过程中如果发现存在锁等待则将请求事务的相关信息和所有阻塞事务的相关信息写入到锁等待缓存中。
步骤1009，等待阻塞事务执行结束后，请求事务可以申请到锁，然后执行该SQL语句。
步骤1010，向客户端回响应。
继续接收SQL请求，接收到请求后重复执行步骤1005～1010。
(三)创建日志搜集线程
日志搜集线程在数据库启动阶段被创建，负责将锁等待信息写入日志文件。
步骤1011，日志搜集线程睡眠1秒。
步骤1012，日志搜集线程在互斥量的保护下获取业务线程当前正在使用的缓存，此时业务线程会被阻塞。
步骤1013，日志搜集线程将另一个缓存切换给业务线程使用，释放步骤1012获取的互斥量，业务线程可以继续运行。
步骤1014，日志搜集线程将步骤1012获取的缓存内容写入锁等待日志文件中，返回步骤1011重新执行。
(四)关闭PostgreSQL
PostgreSQL在收到shutdown命令后开始关闭数据库。数据库关闭阶段的其他操作已被省略，下面仅介绍本方案相关的步骤。
步骤1015，通知业务线程退出，通知日志搜集线程退出。
步骤1016，等待业务线程和日志搜集线程退出完毕。
步骤1017，释放锁等待信息缓存。
需要说明的是，上述各数据库仅为示例，本公开不限于此，其他 数据库也可应用本公开。
数据库启动后锁等待信息全局缓存以及锁等待日志搜集线程全部创建完毕，下面结合附图对事务运行过程中技术方案的实施作进一步的详细描述。
如图11所示，事务运行过程中技术方案的实施包括：
步骤1101，业务线程1开启事务1；
步骤1102，事务1申请锁；
步骤1103，业务线程2开启事务2；
步骤1104，事务1开始执行；
步骤1105，事务2申请锁；
步骤1106，事务2检测到锁等待；
步骤1107，业务线程2将事务2(请求事务)和事务1(阻塞事务)的相关信息写入缓存1中；
步骤1108，事务1执行结束；
步骤1109，事务1释放锁；
步骤1110，事务1向客户端回响应；
步骤1111，日志搜集线程获取业务线程当前使用的缓存(即缓存1)；
步骤1112，日志搜集线程将缓存2切换给业务线程使用；
步骤1113，日志搜集线程将缓存1的内容写入锁等待日志文件中；
步骤1114，事务2申请到锁；
步骤1115，事务2执行结束；
步骤1116，事务2向客户端返回响应。
如图12所示，本公开一实施例提供了一种数据库处理装置120，包括存储器1210和处理器1220，所述存储器1210存储有程序，所述程序在被所述处理器1220读取执行时，实现任一实施例所述的数据库处理方法。
如图13所示，本公开一实施例提供了一种计算机可读存储介质 130，所述计算机可读存储介质130存储有一个或者多个程序1310，所述一个或者多个程序1310可被一个或者多个处理器执行，以实现任一实施例所述的数据库处理方法。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。

Claims (11)

  1. 一种数据库处理方法，包括：
    产生锁等待后，将锁等待相关信息写入锁等待日志。
  2. 根据权利要求1所述的数据库处理方法，其中，所述产生锁等待后，将锁等待相关信息写入锁等待日志的步骤包括：
    业务线程在产生了锁等待时，将锁等待相关信息写入锁等待信息缓存；以及
    日志搜集线程获取所述锁等待信息缓存，将所述锁等待信息缓存中的锁等待相关信息写入锁等待日志。
  3. 根据权利要求1所述的数据库处理方法，其中，所述锁等待相关信息包括以下至少之一：所述锁等待产生时刻、请求事务的相关信息、阻塞事务的相关信息。
  4. 根据权利要求3所述的数据库处理方法，其中，
    所述请求事务的相关信息包括以下至少之一：所述请求事务所在链路编号、事务编号、锁编号、锁类型、锁等待时间；并且
    所述阻塞事务的相关信息包括以下至少之一：所述请求事务所在链路编号、事务编号、锁编号、锁类型。
  5. 根据权利要求2所述的数据库处理方法，其中，所述日志搜集线程获取所述锁等待信息缓存的步骤包括：日志搜集线程在满足预设条件时获取所述锁等待信息缓存。
  6. 根据权利要求5所述的数据库处理方法，其中，所述满足预设条件包括：接收到唤醒指令，或者，到达唤醒时间。
  7. 根据权利要求2至6任一所述的数据库处理方法，其中，所述日志搜集线程获取所述锁等待信息缓存的步骤包括：所述日志搜集线程获取所述锁等待信息缓存中业务线程使用的第一缓存；并且
    在所述日志搜集线程获取所述锁等待信息缓存后，所述的数据库处理方法还包括：将所述锁等待信息缓存中的第二缓存切换给业务线程使用。
  8. 根据权利要求2至6任一所述的数据库处理方法，还包括：在数据库启动时，创建所述日志搜集线程和所述锁等待信息缓存。
  9. 根据权利要求2至6任一所述的数据库处理方法，还包括：在数据库关闭时，关闭所述日志搜集线程和释放所述锁等待信息缓存。
  10. 一种数据库处理装置，包括存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现如权利要求1至9任一项所述的数据库处理方法。
  11. 一种计算机可读存储介质，其中，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至9任一项所述的数据库处理方法。
PCT/CN2020/085848 2019-05-30 2020-04-21 一种数据库处理方法、装置及计算机可读存储介质 WO2020238485A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20814719.9A EP3979097A4 (en) 2019-05-30 2020-04-21 DATABASE PROCESSING METHOD AND DEVICE AND COMPUTER READABLE RECORDING MEDIA
US17/615,581 US11928132B2 (en) 2019-05-30 2020-04-21 Database processing method and apparatus, and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910464772.3 2019-05-30
CN201910464772.3A CN112015827A (zh) 2019-05-30 2019-05-30 一种数据库处理方法、装置及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2020238485A1 true WO2020238485A1 (zh) 2020-12-03

Family

ID=73501249

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/085848 WO2020238485A1 (zh) 2019-05-30 2020-04-21 一种数据库处理方法、装置及计算机可读存储介质

Country Status (4)

Country Link
US (1) US11928132B2 (zh)
EP (1) EP3979097A4 (zh)
CN (1) CN112015827A (zh)
WO (1) WO2020238485A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1808389A (zh) * 2006-02-20 2006-07-26 南京联创科技股份有限公司 帐务后台内存数据库中共享内存的自治锁方法
CN102831156A (zh) * 2012-06-29 2012-12-19 浙江大学 一种云计算平台上的分布式事务处理方法
CN103885726A (zh) * 2014-03-20 2014-06-25 东蓝数码股份有限公司 一种高效的多线程日志写入方法
US20180144015A1 (en) * 2016-11-18 2018-05-24 Microsoft Technology Licensing, Llc Redoing transaction log records in parallel

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996027157A1 (fr) * 1995-02-28 1996-09-06 Ntt Data Communications Systems Corporation Systeme associatif decentralise et traitements de journaux et de reprise dans celui-ci
US8037476B1 (en) * 2005-09-15 2011-10-11 Oracle America, Inc. Address level log-based synchronization of shared data
US7818743B2 (en) * 2005-09-22 2010-10-19 International Business Machines Corporation Logging lock data
US7571301B2 (en) * 2006-03-31 2009-08-04 Intel Corporation Fast lock-free post-wait synchronization for exploiting parallelism on multi-core processors
US8041691B2 (en) * 2007-07-30 2011-10-18 Oracle International Corporation Acquiring locks in wait mode in a deadlock free manner
US7644106B2 (en) * 2007-07-30 2010-01-05 Oracle International Corporation Avoiding lock contention by using a wait for completion mechanism
US8131983B2 (en) * 2008-04-28 2012-03-06 International Business Machines Corporation Method, apparatus and article of manufacture for timeout waits on locks
US7937378B2 (en) * 2008-08-13 2011-05-03 Oracle America, Inc. Concurrent lock-free skiplist with wait-free contains operator
US20100057965A1 (en) * 2008-08-29 2010-03-04 International Business Machines Corporation Extension of Lock Discipline Violation Detection for Lock Wait Patterns
CN101876932A (zh) * 2009-11-30 2010-11-03 中国移动通信集团浙江有限公司 内存数据库监控的方法、系统及设备
US8607239B2 (en) * 2009-12-31 2013-12-10 International Business Machines Corporation Lock mechanism to reduce waiting of threads to access a shared resource by selectively granting access to a thread before an enqueued highest priority thread
US8607238B2 (en) * 2011-07-08 2013-12-10 International Business Machines Corporation Lock wait time reduction in a distributed processing environment
CN102945278B (zh) * 2012-11-09 2015-12-09 华为技术有限公司 一种数据库记录重做日志的方法和装置
US8954974B1 (en) * 2013-11-10 2015-02-10 International Business Machines Corporation Adaptive lock list searching of waiting threads
US9684614B2 (en) * 2014-01-27 2017-06-20 Signalchip Innovations Private Limited System and method to convert lock-free algorithms to wait-free using a hardware accelerator
US9378069B2 (en) * 2014-03-05 2016-06-28 International Business Machines Corporation Lock spin wait operation for multi-threaded applications in a multi-core computing environment
US9740582B2 (en) * 2015-12-30 2017-08-22 Sybase, Inc. System and method of failover recovery
US11157332B2 (en) * 2016-07-06 2021-10-26 International Business Machines Corporation Determining when to release a lock from a first task holding the lock to grant to a second task waiting for the lock
CN106951488B (zh) * 2017-03-14 2021-03-12 海尔优家智能科技（北京）有限公司 一种日志记录方法和装置
US10719249B1 (en) * 2019-01-31 2020-07-21 EMC IP Holding Company LLC Extent lock resolution in active/active replication

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1808389A (zh) * 2006-02-20 2006-07-26 南京联创科技股份有限公司 帐务后台内存数据库中共享内存的自治锁方法
CN102831156A (zh) * 2012-06-29 2012-12-19 浙江大学 一种云计算平台上的分布式事务处理方法
CN103885726A (zh) * 2014-03-20 2014-06-25 东蓝数码股份有限公司 一种高效的多线程日志写入方法
US20180144015A1 (en) * 2016-11-18 2018-05-24 Microsoft Technology Licensing, Llc Redoing transaction log records in parallel

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3979097A4 *

Also Published As

Publication number Publication date
US11928132B2 (en) 2024-03-12
US20220237209A1 (en) 2022-07-28
EP3979097A1 (en) 2022-04-06
CN112015827A (zh) 2020-12-01
EP3979097A4 (en) 2023-06-14

Similar Documents

Publication Publication Date Title
US8712961B2 (en) Database caching utilizing asynchronous log-based replication
US8190946B2 (en) Fault detecting method and information processing apparatus
CN104156361B (zh) 一种实现数据同步的方法及系统
US11429599B2 (en) Method and apparatus for updating database by using two-phase commit distributed transaction
US20160188611A1 (en) Archival management of database logs
US20130117234A1 (en) Database Log Parallelization
US7890455B2 (en) System and apparatus to ensure a low-latency read of log records from a database management system (“DBMS”)
US9652492B2 (en) Out-of-order execution of strictly-ordered transactional workloads
WO2020025049A1 (zh) 数据同步的方法、装置、数据库主机及存储介质
WO2020253314A1 (zh) 分布式数据库的事务监控方法及装置、系统、存储介质
US11176004B2 (en) Test continuous log replay
CN108460047B (zh) 数据同步方法及设备
JP2013045356A (ja) トランザクション同時実行制御システム、トランザクション同時実行制御方法、およびプログラム
WO2020238485A1 (zh) 一种数据库处理方法、装置及计算机可读存储介质
US10324811B2 (en) Opportunistic failover in a high availability cluster
US9020905B2 (en) Synchronizing database and non-database resources without a commit coordinator
US20140149697A1 (en) Memory Pre-Allocation For Cleanup and Rollback Operations
US20140157048A1 (en) Transactions and failure
US20130018861A1 (en) Dual locking mechanism for a domain
US11301341B2 (en) Replication system takeover with handshake
CN111367625B (zh) 一种线程的唤醒方法及装置，存储介质和电子设备
CN109857523B (zh) 一种用于实现数据库高可用性的方法及装置
CN107239474B (zh) 一种数据记录方法及装置
US20180136847A1 (en) Control device and computer readable recording medium storing control program
JP2000163294A (ja) データベース管理方法及びその装置並びにプログラムを記録した機械読み取り可能な記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20814719

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020814719

Country of ref document: EP

Effective date: 20220103