CN107239474B - Data recording method and device - Google Patents
Data recording method and device Download PDFInfo
- Publication number
- CN107239474B CN107239474B CN201610189066.9A CN201610189066A CN107239474B CN 107239474 B CN107239474 B CN 107239474B CN 201610189066 A CN201610189066 A CN 201610189066A CN 107239474 B CN107239474 B CN 107239474B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- operation request
- operation information
- lock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
Abstract
The embodiment of the application provides a data recording method and a data recording device, when an operation request for data in a database is received, a database management system executes operation of obtaining a lock corresponding to the data according to the operation request, and then judges whether to operate the data in a cache recording mode or not according to a result of obtaining the lock. By the method, the database management system can automatically determine whether to operate the data in a cache recording mode according to the lock acquisition result, so that the problem of thread overstock is solved, and the problems of low database operation efficiency and complicated operation caused by the fact that the cache recording mode needs manual configuration and maintenance are avoided.
Description
Technical Field
The present application relates to the field of information technologies, and in particular, to a data recording method and apparatus.
Background
In the information-oriented society of today, people generally use database technology to store a large amount of data, and when some data needs to be used, the corresponding data is accessed through a database management system.
In the prior database technology, one of the characteristics is that the data sharing performance is high, and because the data stored and managed in the database is oriented to the overall user, the data can be shared and used by a plurality of users or application programs. However, this brings a problem that data consistency is difficult to guarantee, so in the prior art, data currently accessed by a thread is usually locked in a database by using a database lock (i.e., an exclusive lock), so that when other threads access the data, the data cannot be accessed because the data is already locked, and the consistency of the data in the database is guaranteed.
However, for data frequently accessed by a large number of threads (i.e., hot spot data), since the database locking method is adopted, a large number of threads cannot access the locked data, so that a large number of threads are backlogged (i.e., threads that cannot access the data are queued to wait for accessing the data according to the chronological order of accessing the data), and the backlogged threads cannot perform subsequent operations.
For example, assuming that it takes 20s for a thread to access data a and perform the corresponding operation, the number of operations for the data a is at most 3 times in 1 minute, that is, the upper limit of the operation frequency for the data a is 3 times/minute, and the thread backlog is caused as long as the frequency of accessing the data a by the thread exceeds the frequency.
Even if the non-waiting database lock is adopted for the hot spot data, the thread can only be ensured to continue to execute other operations after reading the non-waiting database lock of the data, the thread cannot access the data and execute corresponding operations without waiting for accessing the data, so the thread does not finish accessing and operating the data substantially, and the thread needs to continuously try to access and operate the data subsequently.
Therefore, in the prior art, people configure the hot spot data in a database management system as a cache record mode for the hot spot data, then record the operation information of each thread on the hot spot data in a cache record mode, and periodically write the hot spot record according to each recorded operation information.
Specifically, a worker determines each piece of data frequently accessed by a large number of threads as hot spot data according to manual experience, and configures a cache identifier for each piece of hot spot data in a database management system, so that when the threads access the data, the hot spot data can be determined as a cache recording mode according to the cache identifier (that is, each piece of hot spot data is configured as a cache recording mode). When the thread accesses the hot data and determines that the hot data is in a cache record mode, the thread can record operation information to be executed in a cache record table of the database according to the cache record mode, then subsequent operations can be continuously executed, the database management system drags the operation record in the cache record table according to a fixed period, carries out uniform calculation on the operation information corresponding to the hot data aiming at each hot data in the cache record table, determines a calculation result, and finally modifies the hot data according to the calculation result through the thread. The process of recording each operation information in the cache record table is also referred to as a recording process, and the process of calculating the operation information in the cache record table and modifying the corresponding hot spot data according to the calculation result is also referred to as a complementary writing process.
For example, assuming that the database management system configures data a into a cache record mode, and assuming that 90 threads initiate an access request to the data a within 1 minute, the database management system needs to record the operations that the 90 threads need to perform on the data a in a cache record table of the database. Then, when the database management system fetches the operation records in the cache record table according to a fixed time period, for the data a, the database management system may determine 90 operation records, perform calculation on the 90 operation records, obtain a calculation result, and finally access the data a through a thread and execute a corresponding operation according to the calculation result.
Therefore, multiple operations on the hot spot data can be unified into one operation in a cache recording mode, the problem of thread backlog caused by frequent access of the same hot spot data by a thread is reduced, and meanwhile, the problem of frequent access failure of the hot spot data by the thread is solved as the operation information is recorded in the cache recording table and is equivalent to the operation information which is already operated.
However, the method of using the cache record needs to determine the hot spot data manually, and the worker needs to manually configure the hot spot data in the database management system in advance as the cache record method, so that the method of using the cache record is complex to operate and low in efficiency. Moreover, some hot data may be frequently accessed only in a period of time, and are idle in other periods of time, but the database management system still adopts a cache record mode for the operation request for accessing the hot data as long as the access mode of the hot data is not changed. Furthermore, in the cache record mode in the prior art, actually, operations of the database management system on data processing are added, for each operation requiring cache record, when the operation is recorded in the cache record table, a recording process is added, and when the operation record in the cache record table is fished, a reading process is added, so that the method using the cache record is no longer suitable for the hot data with no busy access request, and therefore, the access mode of the hot data also needs to be manually restored, so that the maintenance work of the database is increased, and the operation efficiency of the database is low.
Therefore, in the prior art, for hot data, both a traditional database locking mode and a cache recording mode have the problems of low database operation efficiency, need of manually adjusting a data access mode and inflexibility in use.
Disclosure of Invention
The embodiment of the application provides a data recording method and a data recording device, which are used for solving the problems of low database operation efficiency, manual adjustment of a database management system and inflexibility in use caused by the conventional data recording method.
The data recording method provided by the embodiment of the application comprises the following steps:
receiving an operation request aiming at data in a database, wherein the operation request carries operation information;
acquiring a lock corresponding to the data according to the operation request;
judging whether the lock is acquired;
if so, locking the data and operating the data according to the operation information;
and if not, operating the data in a cache recording mode.
An embodiment of the present application provides a data access apparatus, including:
the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for receiving an operation request aiming at data in a database, and the operation request carries operation information;
the lock acquisition module is used for acquiring a lock corresponding to the data according to the operation request;
and the judgment execution module is used for judging whether the lock is acquired, if so, locking the data, operating the data according to the operation information, and if not, operating the data in a cache recording mode.
The embodiment of the application provides a data recording method and a data recording device, when an operation request for data in a database is received, a database management system executes operation of obtaining a lock corresponding to the data according to the operation request, and then judges whether to operate the data in a cache recording mode or not according to a result of obtaining the lock. By the method, the database management system can automatically determine whether to operate the data in a cache recording mode according to the lock acquisition result, so that the problem of thread overstock is solved, and the problems of low database operation efficiency and complicated operation caused by the fact that the cache recording mode needs manual configuration and maintenance are avoided.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a data recording process provided by an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data recording apparatus according to an embodiment of the present application.
Detailed Description
In the embodiment of the application, when an operation request for data in a database is received, a lock corresponding to the data is acquired according to the operation instruction, whether the lock is acquired is judged, if yes, the data is locked, the data is operated according to operation information carried by the operation request, and if not, the data is operated in a cache recording mode. According to the method, whether the data needs to be operated in the cache recording mode or not is determined according to the result of obtaining the lock of each data, so that the advantage that the data needs to be operated in the cache recording mode according to the hotspot data is guaranteed, and the problems that in the prior art, the operation efficiency of a database is low and the maintenance cost is high due to the fact that the data which is operated in the cache recording mode needs to be manually configured in advance and the data access mode needs to be manually restored are solved.
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a data recording process provided in an embodiment of the present application, which specifically includes the following steps:
s101: an operation request for data in a database is received.
Since the data recording process may be a process of performing corresponding operation on data according to a received operation request in a database, in this embodiment of the present application, an operation request for the data in the database may be received by a database management system independent from the database, or an operation request for the data in the database may be directly received by the database, and the following description will be given by taking only an example that the database management system receives an operation request for the data in the database. The operation request carries operation information on the data (e.g., operations such as adding, deleting, and changing the data).
In addition, the operation request may be an operation request initiated by a thread according to a currently executed database transaction, that is, the operation request belongs to one database transaction, and since the operation request is for data in a database, the operation request carries an identifier of the data, so that the database operating system may determine data corresponding to the operation request according to the identifier of the data, so that the thread initiating the operation request may access the corresponding data.
The database may be a single device, or may be a system including a plurality of devices.
Further, since a cache record mode needs to be adopted in the method of the present application, a corresponding cache record table should exist in the database. The cache record table may be pre-established, or may be established according to the use requirement when the cache record table needs to be used in the subsequent operation.
For example, suppose that in an account database for recording account funds, the database management system of the database receives a data operation request a of a thread α for an account a, wherein the operation information of the operation request is that 10 elements are transferred to the account a, and the identifier of the data is: account a.
S102: and acquiring a lock corresponding to the data according to the operation request.
In this embodiment, after the database management system receives the operation request, in order to ensure the consistency of the data in the database, the database management system may perform the same lock acquisition operation as in the prior art on the thread.
In addition, in the present application, since data frequently accessed to a thread needs to be operated in a cache record manner, in order to avoid backlogging of the thread when the thread needs to operate the data in the cache record manner, the acquired database lock may be a non-waiting database lock.
Specifically, consistent with the prior art, the database management system determines, according to the received operation request, an identifier of data carried in the operation request, determines, according to the identifier of the data, data that needs to be operated, and performs, through the thread, an operation of obtaining a non-waiting database lock corresponding to the data on the data.
Continuing with the above example, if the database management system determines that the identifier of the data for which the operation request a is directed is account a according to the operation request a, the database management system performs an operation of acquiring a non-waiting database lock of the account a on the account a through a thread α.
S103: and judging whether the lock is acquired, if so, executing step S104, and otherwise, executing step S105.
In the embodiment of the present application, for each data frequently accessed by each thread, that is, for hotspot data, because there are many threads accessing each hotspot data, there is a high probability that the hotspot data is in a locked state by other threads, and further, when one thread accesses one hotspot data, the failure probability of the lock executed by the thread in step S102 to acquire the hotspot data is also high. That is, after step S102 is executed, if it is determined that acquiring the non-waiting database lock corresponding to the data fails, the probability that the data is hot data is high.
As can be seen, for each hot data, when the thread executes the operation of acquiring the non-waiting database lock corresponding to each hot data, the failure probability of acquiring the non-waiting database lock is very high (that is, the non-waiting database lock is not acquired), and thus in the present application, it may be determined whether the data is the hot data according to the result of acquiring the non-waiting database lock corresponding to the data by the thread. However, when the thread acquires the non-waiting database lock, due to reasons such as network timeout and data error, the thread may also fail to acquire the non-waiting database lock corresponding to the data, so when determining whether the data is hot data, it is also necessary to determine that the reason why the thread fails to acquire the non-waiting database lock corresponding to the data is that the data is already locked, and it is determined that the data has a high probability of being hot data, and then the data may be operated in a cache record manner.
In addition, when each thread locks each data in the application, each thread uses a non-waiting database lock, so that after each thread fails to acquire the non-waiting database lock, the hot data can be continuously operated in a cache record mode subsequently, and the situation that subsequent operation cannot be performed due to thread backlog is avoided.
By the method, the database management system can automatically identify the hot spot data without pre-configuring the hot spot data.
Specifically, the database management system determines whether the thread acquires the non-waiting database lock corresponding to the data according to the result of the thread executing the non-waiting database lock corresponding to the data in step S102, if the thread acquires the non-waiting database lock corresponding to the data, step S104 is executed, and if the thread does not acquire the non-waiting database lock corresponding to the data and the reason why the thread does not acquire the non-waiting database lock corresponding to the data is that the data is already locked by other threads, step S105 is executed.
In addition, if the thread does not acquire the non-waiting database lock corresponding to the data and the reason is not that the data is locked by other threads, the database management system can display prompt information of operation failure and display the failure reason, as in the prior art.
Continuing with the above example, assuming that the thread α does not acquire the non-waiting database lock of the account a, the database management system determines, according to the result of the thread α acquiring the non-waiting database lock of the account a, that the thread α has failed to acquire the non-waiting database lock, and the reason for the failure is that the account a has been locked by other threads, then step S105 is executed.
S104: and locking the data and operating the data according to the operation information.
In the embodiment of the present application, as in the prior art, when a thread successfully acquires a non-waiting database lock of the data, the non-waiting database lock may be used to lock the data, and then the data may be operated according to the operation information carried in the operation request.
S105: and operating the data in a cache recording mode.
In this embodiment, as for the reason described in step S103, only when it is determined that the reason that the thread fails to acquire the non-waiting database lock corresponding to the data is that the data is already locked, it can be determined that the data is hot data, that is, only if the reason is satisfied, the data needs to be operated in a cache record manner. That is, when it is determined that the reason that the thread does not acquire the non-waiting database lock corresponding to the data is that the data is already locked, it is determined that the data has a high probability of being hot data, and the data needs to be operated in a cache record manner.
Specifically, in the recording process, the database management system records the operation information in the cache record table through the thread. However, at this time, since the operation information is merely recorded and the data has not been operated according to the operation information, it is also necessary to perform a post-write process.
In the write-back process, firstly, the database management system can salvage the specified amount of operation information in the cache record table according to a preset time period. And secondly, determining each data corresponding to each operation information according to each operation information obtained by fishing. Then, for each determined data, according to the retrieved operation information corresponding to the data, determining the time sequence of the operation information corresponding to the data, and calculating the complementary writing operation information corresponding to the data according to the time sequence of the operation information. And finally, executing the complementary writing process on the data corresponding to each operation information obtained by calculation according to the complementary writing operation information.
In addition, for each determined data that needs to execute the write-once process, the data management system needs to start a thread to execute the process, and the thread may first acquire a non-waiting database lock corresponding to the data and then update the data according to the write-once operation information. Moreover, when the data is updated, the thread only needs to perform one operation on the data according to the write-once operation information, so that the operation efficiency of the database can be greatly improved, but at the same time, only an operation record (i.e. only one operation record) performed according to the write-once operation information is recorded in the log of the data, and the operation record of this time is actually calculated according to each operation information, if no operation record corresponding to each operation information exists in the data log, the operation record in the log of the data is incomplete, that is, there is no detailed operation record corresponding to each operation information, and the operation record corresponding to each operation information is actually important data (for example, for account data, account details are one item of data which is important, that is, the operation record corresponding to each operation information is important), therefore, in the present application, the log information of the data may be updated according to each operation information corresponding to the write-once operation information.
And after the data is updated according to the complementary writing operation information and the log of the data is updated according to each operation information corresponding to the complementary writing operation information, the operation information in the cache record table can be updated, and each operation information corresponding to the complementary writing operation information is deleted to prevent repeated operation on the data and complete the complementary writing process.
Further, it should be noted that, when recording each operation information in the cache record table, each operation information may be recorded in the cache record table according to the time sequence of each operation information, so that when performing the complementary writing process, the complementary writing process may be performed according to the time sequence of each operation information recorded in the cache record table.
Continuing with the above example, assuming that it is determined to operate the account a in a cache record manner, the operation information a will be recorded in a cache record table in the database. It is assumed that, when the database management system fetches a specified amount of operation information from the cache record table according to a preset time period, the fetched operation information is shown in table 1.
| Operation information | Identification of data |
| Operation information a | Account A |
| Operation information b | Account E |
| Operation information c | Account A |
| Operation information d | Account A |
| Operation information e | Account G |
| Operation information f | Account B |
TABLE 1
It can be seen that the data corresponding to the operation information obtained by the database management system at this time is: account a, account B, account E, account G, and according to the correspondence between the operation information and each data shown in table 1, for account a, 3 pieces of operation information may be determined: operation information a, operation information c, and operation information d.
Further assume that the operation information c and the operation information d are: transfer 20 out of account a and transfer 100 into account a. Then the operation information a according to the previous assumption is: transferring 10 yuan to account A, and calculating the complementary writing operation information aiming at the account A as follows: transfer 90 dollars to account a. Then the write-once process can be subsequently performed according to the write-once operation information.
With the data recording method shown in fig. 1, when an operation request for data in a database is received, whether the data needs to be operated by using cache recording is determined by obtaining different results of locks corresponding to the data. According to the method, the hot spot data does not need to be checked manually, the cache recording mode is configured for each hot spot data, and the data needing to be operated in the cache recording mode can be automatically determined. The advantages of the cache recording mode are guaranteed, meanwhile, the problems that data need to be manually configured in advance and the access mode of the data needs to be manually restored are solved, the operation efficiency of the database is improved, and the maintenance cost is reduced.
In addition, when the data management system salvages the specified amount of operation information in the cache record table according to the preset time period, in order to prevent the problem that the operation information is repeatedly salvaged by different threads and repeated operation is caused to the data due to the fact that the time consumption of the process of rewriting each data is long at the last time, in the application, the time required by the rewriting process corresponding to the specified amount of operation information does not exceed the preset time period.
Specifically, first, the database management system may determine the average time required for fishing one operation information per time based on the pre-recorded time taken for fishing each operation information (i.e., the time taken from the beginning of fishing to the end of fishing is determined based on the time stamp in the history of fishing each operation information), determining the time required for calculating the operation information once on average (e.g., calculating two operation information and determining the time of the complementary writing operation information corresponding to the two operation information) based on the time taken for calculating each operation information and determining the corresponding complementary writing operation information recorded in advance (i.e., determining the time taken from the start of calculation to the end of calculation based on the time stamp in the history of calculating each operation information), the time required to perform the one-time complementary writing process on average is determined according to the pre-recorded time consumed by each complementary writing process.
Then, the longest time required for the write-once process corresponding to a specified number of pieces of operation information to be fished at one time can be determined according to the average time required for fishing one piece of operation information each time, the average time required for calculating one piece of operation information, and the average time required for executing one write-once process (for example, when the specified number of pieces of operation information is 100 pieces of operation information, it is determined that 100 pieces of operation information are fished at one time, and the 100 pieces of operation information all correspond to the same data, that is, the time required for calculating 100 times).
And finally, judging whether the maximum time required by the rewriting process exceeds the preset time period according to the preset time period for fishing the operation information in the cache record table, if so, determining that the quantity of the operation information required by the rewriting process needs to be reduced until the maximum time required by the rewriting process does not exceed the preset time period, otherwise, determining that the value of the specified quantity is available, and fishing the operation information according to the specified quantity. Wherein the initial value of the specified number may be preset manually.
And, in order to improve the efficiency of the write-back process, when it is determined that the maximum time required for the write-back process does not exceed the preset time period, the database management system may further gradually increase the value of the specified number until the maximum time required for the write-back process exceeds the preset time period, and then it is determined that the value of the specified number does not exceed the current value of the specified number.
Of course, the actual usage value of the specified number may also be set directly according to manual experience. Furthermore, considering that the number of threads in the database system is limited, it is considered that when the data corresponding to the retrieved operation information are different, each thread may need to perform a plurality of write-back processes for different data within the preset time period. Therefore, when the value of the specified number is determined, the number of times of the complementary writing process which can be executed by all threads in the preset time period can be determined according to the number of the threads in the database, and the number is used as the value for determining the specified number. (e.g., assuming a total of 50 threads in the database, each time the write-back process takes a minimum of 10 seconds, it can be determined that a total of 50 threads in the database can execute within 1 minute
A second complementary write process, so the value of the number of executions may not exceed 300). Of course, the value of the designated number may also be determined by weighting and calculating a plurality of parameters, and how to determine the value of the designated number is not particularly limited in this application.
For example, if a preset time period in a certain database management system is 1 minute, and if the operation information in the cache record table is fished for the mth time, and the fished operation information is more, it takes 1 minute and 10 seconds to complete the write-back process, when the operation information in the cache record table is fished for the M +1 time, because the write-back process after the mth time is not completed, the operation information fished for the M +1 time is the same as the operation information fished for the mth time, resulting in repeated operation on data, and other operation information in the cache record table cannot be fished subsequently. Therefore, it is required to ensure that the time required by the complementary writing process corresponding to the specified amount of operation information does not exceed the preset time period.
Further, since the specified number of values is determined, it cannot be guaranteed that the time required for the overwriting process does not exceed the preset time period in actual use. Therefore, in order to prevent each operation information from being repeatedly operated, in the present application, before each time each operation information in the cache record table is fished, it may be further determined whether the last overwriting process has ended, if not, the current fishing of each operation information in the cache record table is stopped, and if so, each operation information in the cache record table is normally fished. When it is determined that the latest write-once process is not finished, the value of the specified number may be determined to be too large, so that the time required by the write-once process exceeds the preset time period, and then, the database management system may further reduce the value of the specified number at this time to prevent the time required by the write-once process from exceeding the preset time period.
Furthermore, in the present application, since the write-back process is also required to be executed by a thread, when the operation information retrieved by the database management system corresponds to a plurality of data, a thread needs to be started for each data to execute the write-back process, so that the plurality of write-back processes occupy the plurality of threads. However, in the database management system, the number of threads is fixed, so when the database management system operates data in a cache record manner, the corresponding write-back process may occupy all threads, and further, if the number of operation information to be written back is large, no thread may be used to execute other database transactions. Therefore, in order to prevent all threads in the database management system from being used in the write-once process, in the present application, the database management system may suspend the write-once process within a preset sleep time duration, where the sleep time may be set to start when each write-once process ends, that is, after each write-once process ends, the write-once process is suspended within the sleep time duration.
Still further, in step S103, when it is determined that the thread does not acquire the non-waiting database lock corresponding to the data, in addition to determining the probability of the hot data when the data is acquired according to the reason that the thread acquires the non-waiting database lock, in this application, the access frequency of each data accessed by each thread within a certain time range may be determined, then the average frequency of the data accessed by the thread in the database is determined, when the frequency of the data accessed by the thread is higher than the average frequency, it is determined that the data has a high probability of being hot data, the data may be operated in a cache recording manner, and when the frequency of the data accessed by the thread is lower than the average frequency, it is determined that the data has a high probability of not being hot data, and the data does not need to be operated in a cache recording manner.
Based on the data recording process shown in fig. 1, an embodiment of the present application further provides a data recording apparatus, as shown in fig. 2.
Fig. 2 is a schematic structural diagram of a data recording apparatus according to an embodiment of the present application, which specifically includes:
a receiving module 201, configured to receive an operation request for data in a database, where the operation request carries operation information;
an acquire lock module 202, configured to acquire a lock corresponding to the data according to the operation request;
the judgment execution module 203 is configured to judge whether the lock is acquired, if so, lock the data, and operate the data according to the operation information, and if not, operate the data in a cache recording manner.
The judgment execution module 203 is specifically configured to lock the data by using a non-waiting database lock.
The judgment execution module 203 is further configured to determine, before executing the data operation in the cache record manner, that the reason why the lock is not acquired is that the data has been locked.
The judgment execution module 203 is specifically configured to record the operation information in a cache record table, retrieve a specified number of operation information in the cache record table according to a preset time period, and execute a write-back process on data corresponding to each retrieved operation information according to each retrieved operation information.
And the time required by the complementary writing process corresponding to the specified amount of operation information does not exceed the preset time period.
The judgment execution module 203 is further configured to suspend the overwriting procedure within a preset sleep time length according to the sleep time length.
Specifically, the data recording apparatus shown in fig. 2 may be located in a database, or may be located in a database management system independent of the database, where the database may be a single device, or may be a system composed of multiple devices.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (8)
1. A data recording method, characterized in that the method comprises:
receiving an operation request aiming at data in a database, wherein the operation request carries operation information;
acquiring a lock corresponding to the data according to the operation request;
judging whether the lock is acquired;
if so, locking the data in the database aimed at by the operation request by adopting a non-waiting database lock, and operating the data in the database aimed at by the operation request according to the operation information;
if not, and it is determined that the data in the database targeted by the operation request is locked, determining that the data in the database targeted by the operation request is hot data, and operating the data in the database targeted by the operation request in a cache record mode;
the method further comprises the following steps:
if the lock corresponding to the data is not acquired, determining the access frequency of each data accessed by each thread within a preset time range, determining the average frequency of each data accessed by each thread in the database and the probability of the data in the database, which is aimed at by the operation request, accessed by each thread; when the probability that the data in the database aimed at by the operation request is accessed by the thread is higher than the average probability, determining that the data in the database aimed at by the operation request is hot data, and operating the data in the database aimed at by the operation request in a cache record mode.
2. The method of claim 1, wherein prior to manipulating the data in a cache record manner, the method further comprises:
determining that the reason the lock was not acquired is that the data has been locked.
3. The method of claim 1, wherein the data is operated in a cache record mode, and specifically comprises:
recording the operation information in a cache record table;
and according to a preset time period, fishing out the specified amount of operation information in the cache record table, and according to the fished operation information, executing a complementary writing process on the data corresponding to the fished operation information respectively.
4. The method of claim 3, wherein the time required for the complementary writing process corresponding to the specified amount of operation information does not exceed the preset time period.
5. The method of claim 3, wherein the method further comprises:
and according to a preset sleep time length, pausing the complementary writing process in the sleep time length.
6. A data recording apparatus, comprising:
the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for receiving an operation request aiming at data in a database, and the operation request carries operation information;
the lock acquisition module is used for acquiring a lock corresponding to the data according to the operation request;
the judging and executing module is used for judging whether the lock is acquired; if so, locking the data in the database aimed at by the operation request by adopting a non-waiting database lock, and operating the data in the database aimed at by the operation request according to the operation information; if not, and it is determined that the data in the database targeted by the operation request is locked, determining that the data in the database targeted by the operation request is hot data, and operating the data in the database targeted by the operation request in a cache record mode; if the lock corresponding to the data is not acquired, determining the access frequency of each data accessed by each thread within a preset time range, and determining the average frequency of each data accessed by each thread in the database and the probability of the data in the database, which is aimed at by the operation request, being accessed by each thread; when the probability that the data in the database aimed at by the operation request is accessed by the thread is higher than the average probability, determining that the data in the database aimed at by the operation request is hot data, and operating the data in the database aimed at by the operation request in a cache record mode.
7. The apparatus of claim 6, wherein the determination execution module is further configured to determine that the reason the lock was not acquired is that the data has been locked before performing the operation on the data in the cache record manner.
8. The apparatus according to claim 6, wherein the determining and executing module is specifically configured to record the operation information in a cache record table, retrieve a specified number of operation information in the cache record table according to a preset time period, and execute a write-back process on data corresponding to each retrieved operation information according to each retrieved operation information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610189066.9A CN107239474B (en) | 2016-03-29 | 2016-03-29 | Data recording method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610189066.9A CN107239474B (en) | 2016-03-29 | 2016-03-29 | Data recording method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107239474A CN107239474A (en) | 2017-10-10 |
| CN107239474B true CN107239474B (en) | 2021-05-04 |
Family
ID=59983381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610189066.9A Active CN107239474B (en) | 2016-03-29 | 2016-03-29 | Data recording method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107239474B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200387627A1 (en) * | 2019-06-04 | 2020-12-10 | Digital Asset Holdings, LLC | Multi-user database system and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1630269A (en) * | 2003-12-17 | 2005-06-22 | 微软公司 | Mesh networks with end device recognition |
| CN101645837A (en) * | 2009-09-04 | 2010-02-10 | 成都市华为赛门铁克科技有限公司 | Method and device for realizing load balancing |
| CN101788995A (en) * | 2009-12-31 | 2010-07-28 | 成都市华为赛门铁克科技有限公司 | Hotspot data identification method and device |
| CN101867527A (en) * | 2010-07-06 | 2010-10-20 | 重庆大学 | Layering Chord routing method based on physical position |
| CN104252386A (en) * | 2013-06-26 | 2014-12-31 | 阿里巴巴集团控股有限公司 | Data update locking method and equipment |
| CN104462127A (en) * | 2013-09-22 | 2015-03-25 | 阿里巴巴集团控股有限公司 | Recorded data updating method and device |
| CN104932841A (en) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | Saving type duplicated data deleting method in cloud storage system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103095686B (en) * | 2012-12-19 | 2016-06-08 | 华为技术有限公司 | Focus metadata access control method and service device |
| WO2014101108A1 (en) * | 2012-12-28 | 2014-07-03 | 华为技术有限公司 | Caching method for distributed storage system, node and computer readable medium |
| US9519668B2 (en) * | 2013-05-06 | 2016-12-13 | International Business Machines Corporation | Lock-free creation of hash tables in parallel |
| US10229161B2 (en) * | 2013-09-20 | 2019-03-12 | Oracle International Corporation | Automatic caching of scan and random access data in computing systems |
| CN104657492A (en) * | 2015-03-06 | 2015-05-27 | 蔡伟英 | Method and system for searching setting items based on voice recognition |
| CN104834607B (en) * | 2015-05-19 | 2018-02-23 | 华中科技大学 | A kind of hit rate for improving distributed caching and the method for reducing solid state hard disc abrasion |
-
2016
- 2016-03-29 CN CN201610189066.9A patent/CN107239474B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1630269A (en) * | 2003-12-17 | 2005-06-22 | 微软公司 | Mesh networks with end device recognition |
| CN101645837A (en) * | 2009-09-04 | 2010-02-10 | 成都市华为赛门铁克科技有限公司 | Method and device for realizing load balancing |
| CN101788995A (en) * | 2009-12-31 | 2010-07-28 | 成都市华为赛门铁克科技有限公司 | Hotspot data identification method and device |
| CN101867527A (en) * | 2010-07-06 | 2010-10-20 | 重庆大学 | Layering Chord routing method based on physical position |
| CN104252386A (en) * | 2013-06-26 | 2014-12-31 | 阿里巴巴集团控股有限公司 | Data update locking method and equipment |
| CN104462127A (en) * | 2013-09-22 | 2015-03-25 | 阿里巴巴集团控股有限公司 | Recorded data updating method and device |
| CN104932841A (en) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | Saving type duplicated data deleting method in cloud storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107239474A (en) | 2017-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10025844B2 (en) | Query dispatching system and method | |
| US9779127B2 (en) | Integrating database management system and external cache | |
| US11157370B2 (en) | Consistent backup of a distributed database system | |
| WO2020181810A1 (en) | Data processing method and apparatus applied to multi-level caching in cluster | |
| WO2013138770A1 (en) | Systems and methods for supporting inline delegation of middle-tier transaction logs to database | |
| CN107016016B (en) | Data processing method and device | |
| CN107168775B (en) | Method and device for automatically processing suspension affairs | |
| TWI743719B (en) | Method and device for simultaneously executing transactions in block chain, computer readable storage medium and computing equipment | |
| US20170329836A1 (en) | Database transfer of changes | |
| CN109634974A (en) | A kind of data processing method, system and associated component | |
| WO2019109854A1 (en) | Data processing method and device for distributed database, storage medium, and electronic device | |
| US9170837B2 (en) | Transaction concurrent execution control system, method and program for carrying out a control of concurrently executing a transaction, including measuring execution time from starting to ending of transaction execution | |
| US20130024863A1 (en) | System and method for providing dynamic transaction optimizations | |
| CN106874343B (en) | Data deletion method and system for time sequence database | |
| CN103377292B (en) | Database result set caching method and device | |
| CN109522273B (en) | Method and device for realizing data writing | |
| CN107239474B (en) | Data recording method and device | |
| US20110093688A1 (en) | Configuration management apparatus, configuration management program, and configuration management method | |
| US11132351B2 (en) | Executing transactions based on success or failure of the transactions | |
| US7523088B2 (en) | Method for increasing system resource availability in database management systems | |
| US20150081647A1 (en) | Server and method for updating data of server | |
| KR101884726B1 (en) | Method, apparatus, and computer program stored in computer readable medium for reading block in database system | |
| CN106648550B (en) | Method and device for concurrently executing tasks | |
| US8630976B2 (en) | Fast search replication synchronization processes | |
| CN111367625B (en) | Thread awakening method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201014 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20201014 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |