CN116089403A - Flow playback method and system for database - Google Patents
Flow playback method and system for database Download PDFInfo
- Publication number
- CN116089403A CN116089403A CN202310375916.4A CN202310375916A CN116089403A CN 116089403 A CN116089403 A CN 116089403A CN 202310375916 A CN202310375916 A CN 202310375916A CN 116089403 A CN116089403 A CN 116089403A
- Authority
- CN
- China
- Prior art keywords
- transactions
- transaction
- database
- played back
- playback
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000001419 dependent effect Effects 0.000 claims description 29
- 230000000694 effects Effects 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000012545 processing Methods 0.000 description 21
- 238000012360 testing method Methods 0.000 description 17
- 238000012986 modification Methods 0.000 description 12
- 230000004048 modification Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241000579895 Chlorostilbon Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000010976 emerald Substances 0.000 description 1
- 229910052876 emerald Inorganic materials 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000010977 jade Substances 0.000 description 1
- 239000003607 modifier Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- ZLIBICFPKPWGIZ-UHFFFAOYSA-N pyrimethanil Chemical compound CC1=CC(C)=NC(NC=2C=CC=CC=2)=N1 ZLIBICFPKPWGIZ-UHFFFAOYSA-N 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
In the database flow playback technology, it is difficult to improve the playback efficiency while guaranteeing the accuracy of the database flow playback. The embodiment of the specification discloses a method and a system for replaying flow of a database, wherein the method comprises the following steps: acquiring operation information of a plurality of transactions to be played back of a database, wherein the operation information comprises transaction operation content and transaction execution sequence; analyzing and obtaining a dependency relationship among a plurality of transactions to be played back based on the operation information; and playing back a plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprise at least two transactions to be played back, wherein the concurrent playback transactions have no dependency relationship. According to the method, in the database flow playback technology, the concurrent form of database flow playback based on the transaction to be played back, the execution sequence of the transaction and the dependency relationship among the transactions can be realized, and the playback efficiency can be improved while the accurate playback is ensured.
Description
Technical Field
The present disclosure relates to the field of database technologies, and in particular, to a method and a system for playback of a database flow.
Background
Database traffic playback is a traffic playback technique applied to a database, and refers to recording the request traffic of the database (may include operation request related data such as operations performed on the database in an online environment, response data of the operations, etc.) for a period of time by using a traffic recording technique, and playing back the recorded request traffic to the database when the database is migrated to other systems/platforms/devices or in a development/test environment, etc. Through the flow playback of the database, the recording operation can be executed in the database under some scenes, so that whether the database can normally run or not is tested, and the functions of automatic testing and the like of the database are realized. Database traffic playback can be achieved by a variety of methods, but it is difficult to achieve both ensuring accurate playback and improving playback efficiency.
Therefore, there is a need for a method and system for playback of database traffic to more accurately and efficiently implement playback of database traffic.
Disclosure of Invention
An aspect of embodiments of the present disclosure provides a method for playback of a database, the method including: acquiring operation information of a plurality of transactions to be played back of a database, wherein the operation information comprises transaction operation content and transaction execution sequence; analyzing and obtaining the dependency relationship among the plurality of transactions to be played back based on the operation information; and playing back the plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprises at least two transactions to be played back, wherein the dependency relationship does not exist between the concurrent playback transactions.
Another aspect of embodiments of the present description provides a flow playback system of a database, the system comprising: the transaction information acquisition module is used for acquiring operation information of a plurality of transactions to be played back of the database, wherein the operation information comprises transaction operation contents and transaction execution sequences; the analysis module is used for analyzing and obtaining the dependency relationship among the plurality of transactions to be played back based on the operation information; and the playback module is used for playing back the plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprise at least two transactions to be played back, wherein the dependency relationship does not exist between the concurrent playback transactions.
Another aspect of embodiments of the present description provides a flow playback device for a database, comprising at least one storage medium for storing computer instructions and at least one processor; the at least one processor is configured to execute the computer instructions to implement a method for playback of traffic from a database provided by an embodiment of the present specification.
Drawings
The present specification will be further elucidated by way of example embodiments, which will be described in detail by means of the accompanying drawings. The embodiments are not limiting, in which like numerals represent like structures, wherein:
FIG. 1 is an exemplary application scenario diagram of a database's traffic playback system, shown according to some embodiments of the present description;
FIG. 2 is an exemplary flow chart of a method of traffic playback of a database shown in accordance with some embodiments of the present description;
FIG. 3 is an exemplary diagram illustrating playback of a plurality of transactions to be played back to a database according to some embodiments of the present description;
fig. 4 is an exemplary block diagram of a database flow playback system according to some embodiments of the present description.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some examples or embodiments of the present specification, and it is possible for those of ordinary skill in the art to apply the present specification to other similar situations according to the drawings without inventive effort. Unless otherwise apparent from the context of the language or otherwise specified, like reference numerals in the figures refer to like structures or operations.
It will be appreciated that "system," "apparatus," "unit" and/or "module" as used herein is one method for distinguishing between different components, elements, parts, portions or assemblies of different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
As used in this specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
A flowchart is used in this specification to describe the operations performed by the system according to embodiments of the present specification. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.
For database traffic playback, it is necessary to ensure playback accuracy so that the purpose of accurately and effectively testing the database and the like is achieved. Playback accuracy can be understood as: under the condition that the database runs without problems, the operation result data obtained by replaying the recorded request flow and the original operation result data of the recorded request flow (namely, the operation result data obtained when the request flow is originally executed on line) are consistent. In addition, in addition to the need to ensure playback accuracy, it is desirable to be able to increase the efficiency of database traffic playback. In the flow playback technology, in order to ensure playback accuracy, each operation is usually played back sequentially to the database according to the execution timing sequence of each operation in the recorded request flow, so that it is difficult to realize that the playback accuracy is ensured, and meanwhile, the efficiency of the flow playback of the database can be further improved, for example, a plurality of operations in the recorded request flow are played back concurrently, so as to improve the processing efficiency and the resource utilization rate of the processing device.
Based on this, some embodiments of the present disclosure propose a method and a system for playback of a flow of a database, so that multiple operations can be played back concurrently while playback accuracy is ensured, so as to improve flow playback efficiency.
Fig. 1 is an exemplary application scenario diagram of a database's traffic playback system, according to some embodiments of the present description.
The database 130 of the online environment may be various types of databases. The database 130 of the online environment may be deployed on a processing device (e.g., other processing devices than processing device 120).
The database 110 of the test environment may be various types of databases that require traffic playback, such as a corresponding database when the database 130 of the online environment is migrated to other systems/platforms/devices or in a mid-scene (which may be collectively referred to as a test environment) of a development/test environment. The database 110 of test environments may be deployed on the processing device 120 or other processing devices.
A processing device (e.g., processing device 120 or other processing device) may include one or more sub-processing devices. In some embodiments, the processing device may include various types of processors, systems, platforms, etc., or any combination thereof.
In some embodiments, the processing device 120 may obtain the request traffic record from other devices or by the device itself to the database 130 of the online environment, obtain the recorded request traffic 121 for the database 130 of the online environment, and the recorded request traffic 121 may be the request traffic 122 to be played back, and may also play back the request traffic 122 to be played back to the database 110 of the test environment. For example, playback of the to-be-played-back request traffic 122 to the database 110 of the test environment may include: b=b+1 in the to-be-played back request traffic 122; a=a×b; c= 2*c; d=d-1, etc. are performed in the database 110 of the test environment. In some embodiments, in particular, the to-be-played back request traffic 122 may include a plurality of to-be-played back transactions, and the processing device 120 may obtain operation information of the plurality of to-be-played back transactions, where the operation information may include transaction operation contents and a transaction execution order of each to-be-played back transaction. In some embodiments, the processing device 120 may also parse out dependencies between the plurality of transactions to be played back based on the operational information of the plurality of transactions to be played back. In some embodiments, the processing device 120 may further play back the plurality of transactions to be played back to the database 110 of the test environment according to the dependency relationship between the plurality of transactions to be played back, including playing back the plurality of transactions to be played back concurrently without the dependency relationship, so as to implement the flow playback to the database 110 of the test environment. For more details on the method of traffic playback of the database see fig. 2 and its associated description.
The above information transfer relationship between the respective constituent parts of the scene 100 is merely an example, and the present specification is not limited thereto.
It should be noted that the application scenario is provided for illustrative purposes only and is not intended to limit the scope of the present description. Many modifications and variations will be apparent to those of ordinary skill in the art in light of the present description. For example, the scenario 100 may also include a storage device. As another example, the scenario 100 may be implemented on other devices to implement similar or different functionality. However, variations and modifications do not depart from the scope of the present description.
Fig. 2 is an exemplary flow chart of a method of traffic playback of a database, shown in accordance with some embodiments of the present description.
In some embodiments, the process 200 may be implemented by the processing device 120. In some embodiments, the process 200 may be implemented by a database's traffic playback system 400.
In some embodiments, step 202 may be performed by transaction information acquisition module 410.
A database transaction may include one or more database operations that form a logical whole, i.e., a transaction. As an example, for a transaction for online shopping, the transaction may include the following database operations: querying the database for the inventory of purchased goods and updating the inventory, generating a trade order and saving the trade order to the database, querying the database for user information (e.g., the number of purchases) about the trade and updating the user information, etc., which constitute a logical whole, i.e., a trade, i.e., a transaction.
For a transaction, executing a transaction is performing all of the operations included in the transaction. The transactions are independent from each other. If all operations in a database transaction are executed successfully, the transaction is considered to be executed successfully, and the transaction can be submitted, and the data of the operation changes (such as new, deleted, modified, etc.) in the transaction are saved in the database. If an operation in a database transaction fails, the transaction is considered to be an execution failure, and the previously executed operation in the transaction rolls back (i.e., the modification of the previously executed operation is cancelled).
Transactions included in request traffic to be played back (e.g., request traffic recorded by various traffic recording techniques to database 130 of the online environment) in a database (e.g., database 110 of the test environment) are referred to as transactions to be played back.
In some embodiments, the request traffic to be played back may be obtained, and the plurality of transactions to be played back included therein may be obtained in various possible manners based on the request traffic to be played back.
As an example: the plurality of operations included in the request flow to be played back may be obtained based on the request flow to be played back, and then operation information of each operation may be obtained (for example, operation information of each operation is obtained from a database operation log when the request flow to be played back is originally executed on line, for more description of the database operation log, see later), where operation related information of data for each operation, such as a data ID, operation execution content, such as an operation execution statement, a transaction to which the operation belongs, such as a transaction ID, may be included; and, the transaction to which each operation belongs may be obtained based on the operation information of each operation (for example, the transaction ID to which each operation belongs, or other information capable of obtaining the transaction to which each operation belongs), so that a plurality of transactions included in the request flow to be played back, that is, a plurality of transactions to be played back, may be obtained.
The operation information of the transaction to be played back may include various operation information related to the transaction to be played back, such as transaction operation content, transaction execution order, and the like.
The transaction operation content refers to operation content included in a transaction to be played back, and may include operation information of each operation included in the transaction, for example, information of data for each operation, such as a data ID, operation execution content, such as an execution statement of the operation, and operation information of a transaction to which the operation belongs, such as a transaction ID. The transaction operation content can be obtained in various possible manners. For example, in the foregoing example method of obtaining a plurality of transactions to be played back included in the request flow to be played back, operation information of operations included in each transaction to be played back may be obtained.
The transaction execution order of the plurality of transactions to be played back refers to the original execution order of the plurality of transactions to be played back (i.e. the execution order of the plurality of transactions to be played back when the request traffic to be played back is executed in original line). For example, the plurality of transactions to be replayed includes transaction a, transaction B, transaction C, and transaction D, which have a transaction execution order of a first execution of transaction a, a second execution of transaction C, a third execution of transaction B, and a fourth execution of transaction D.
In some embodiments, a database operation log (such as a database increment log of the database 130 in the online environment) of the original online execution of the playback request traffic may be obtained, where the database operation log may include operation execution record information (such as an execution time of each operation, an identifier of an execution sequence number, etc., information about data for the operation, such as a data ID, etc.). In some embodiments, the order of execution of the plurality of transactions to be played back may be derived based on the database oplog.
For example only, for a plurality of transactions to be played back: transaction a, transaction B, transaction C, transaction D, according to the operation execution record information in the database operation log, may be obtained: transaction a and transaction B are for the same piece of data in the database (e.g., data with ID of 001), the execution time of the last operation performed in transaction a is 2022.05.05-09:31:02, the execution time of the first operation performed in transaction B is 2022.05.05-09:32:05, and then transaction a is performed before transaction B. The execution sequence among the rest of the transactions to be played back can be similarly determined, and after the execution sequence among the transactions is determined, the execution sequence of a plurality of the transactions to be played back can be obtained. In some embodiments, the order of execution of the plurality of transactions to be played back may also be obtained in other various possible ways that may occur.
And 204, analyzing and obtaining the dependency relationship among the plurality of transactions to be played back based on the operation information.
In some embodiments, step 204 may be performed by parsing module 420.
For any two transactions to be played back, if transaction dependence exists between the two transactions to be played back, the two transactions to be played back are considered to have a dependence relationship, otherwise, the two transactions to be played back do not have a dependence relationship. That is, inter-transaction dependencies refer to the transaction relationships between transactions for which transaction dependencies exist.
Wherein, the existence of transaction dependence between two transactions to be played back means that: the execution of one transaction to be replayed may have an impact on the execution of another transaction to be replayed. For two transactions to be replayed that have a dependency relationship, the affected transaction to be replayed is referred to as a dependent transaction, and the other transaction to be replayed that has an effect on the dependent transaction is referred to as a dependent transaction.
As an example, there are transaction a, transaction C, transaction B executed in order of ordering, the operation content of transaction a is to insert order data 1 of one commodity a in the order table according to shopping order submitted by the user, the operation content of transaction C is to query the order data 1 of commodity a in the order table, the operation content of operation B is to delete the order data 1 of commodity a in the order table according to cancel shopping order request submitted by the user, transaction B and transaction a exist transaction-dependent and transaction B (being dependent transaction) depend transaction a (being dependent transaction), and transaction C and transaction a exist transaction-dependent and transaction C (being dependent transaction) depend transaction a (being dependent transaction).
As yet another example, as shown in fig. 2, there are transaction B, transaction a, transaction C, transaction D performed in the order of the following ordering, where the operation content of transaction B is to update b=b+1 for field B as calculated below, the operation content of transaction a is to update a=a×b for field a as calculated below, the operation content of transaction C is to update c= 2*c for field C as calculated below, the operation content of transaction D is to update d=d-1 for field D as calculated below, then transaction a and transaction B have transaction dependency and transaction a (are dependent transactions) depend on transaction B (are dependent transactions), and there is no transaction dependency between transaction C and transaction a, transaction B, transaction D.
From the description of the dependency relationship between the transactions, it can be easily understood that according to the operation content such as data aimed by the two transactions and the transaction operation information such as the execution order of the two transactions, whether the two transactions have the dependency relationship or not can be analyzed, and when the dependency relationship exists, which one of the two transactions is the dependent transaction and which one is the dependent transaction can be determined. In some embodiments, the dependency relationship between the plurality of transactions to be played back can be obtained through various existing or future possible parsing methods based on the operation information of each transaction to be played back.
And step 206, playing back the plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprises at least two transactions to be played back, wherein the dependency relationship does not exist between the concurrent playback transactions.
In some embodiments, step 206 may be performed by the playback module 430.
In step 206, playing back a certain transaction to be played back to the database refers to playing back each operation included in the transaction to be played back to the database (which may be understood as executing each operation included in the transaction to be played back to the database).
Concurrent playback of at least two (e.g., two or more) transactions to be played back may refer to simultaneous playback of at least two transactions to be played back. For example, at least two transactions to be replayed are respectively placed in a plurality of threads, and the plurality of threads can be executed together, thereby realizing the concurrent replay of the at least two transactions to be replayed to the database. The simultaneous playback of at least two transactions to be played back may also be achieved in other various possible manners, which are not described here again.
In some embodiments, a part of the transactions to be played back may be selected from the plurality of transactions to be played back without the dependency according to the requirement, or the plurality of transactions to be played back without the dependency may be played back concurrently.
By the embodiment, at least two to-be-played-back transactions without the dependency relationship are played back concurrently, so that the flow playback efficiency can be improved, and the playback accuracy is not influenced.
In some embodiments, for a transaction to be played back for which there is a dependency, playback may be performed in the following order: the dependent transaction plays back after the playback of the dependent transaction associated therewith is completed. For example, for two transactions to be replayed where there is a dependency, the dependent transaction and the dependent transaction are placed in the same thread, and the thread is configured such that the dependent transaction is re-executed after the dependent transaction execution is completed. Playback of the transaction to be played back in the aforementioned order, in which the dependency relationship exists, may also be implemented in other various possible manners, which will not be described herein.
As an example, as shown in fig. 3, where transaction a and transaction B have transaction dependencies and transaction a (is a dependent transaction) depends on transaction B (is a dependent transaction), there is no dependency between transaction C and transaction a, transaction B, transaction D, then the 4 transactions to be played back may be played back according to the following: and enabling the transaction B, the transaction C and the transaction D to be replayed concurrently, and enabling the transaction A to be replayed after the transaction B is replayed completely. The specific implementation method of this mode may be, for example: transaction a and transaction B are placed in thread 1 and configured such that transaction a is re-executed after transaction B execution is completed, and transaction C and transaction D are placed in thread 2 and thread 3, respectively, with thread 1, thread 2, and thread 3 executing together.
By the embodiment, the transaction to be played back with the dependency relationship is played back according to the sequence of the dependent transaction playback after the dependent transaction, so that the playback accuracy can be effectively ensured.
It should be noted that the above descriptions of the respective flows are merely for illustration and description, and do not limit the application scope of the present specification. Various modifications and changes to the flow may be made by those skilled in the art under the guidance of this specification. However, such modifications and variations are still within the scope of the present description. For example, changes to the flow steps associated with the present description, such as adding pre-processing steps, storing steps, other data processing steps, etc.
Fig. 4 is an exemplary block diagram of a database flow playback system according to some embodiments of the present description.
The database's traffic playback system 400 may be implemented on the processing device 120. As shown in fig. 4, the database's flow playback system 400 may include a transaction information acquisition module 410, a parsing module 420, and a playback module 430.
In some embodiments, the transaction information acquisition module 410 may be configured to acquire operation information of a plurality of transactions to be played back of the database, where the operation information includes transaction operation content and a transaction execution order. The transaction information acquisition module 410 may be further configured to acquire database operation logs related to the plurality of transactions to be played back; and the transaction execution order may be derived based on the database oplog. In some embodiments, when the dependency relationship exists between two to-be-played-back transactions, the method includes: the execution of one of the transactions to be played back has an effect on the execution of the other transaction to be played back.
In some embodiments, the transaction information acquisition module 410 may be further configured to acquire a request traffic to be played back of the database, where the request traffic to be played back includes a plurality of operations; determining transactions to which the plurality of operations respectively belong; and obtaining the plurality of transactions to be played back according to the plurality of operations and the transactions to which the plurality of operations respectively belong.
In some embodiments, the parsing module 420 may be configured to parse a dependency relationship between a plurality of transactions to be played back based on the operation information. In some embodiments, the parsing module 420 may be further configured to determine, based on the database oplog, operational performance record information related to the plurality of transactions to be played back, the operational performance record information including one or more of: the execution time of the operation included in the transaction, the operation identification of the operation included in the transaction, and the information of the data aimed at by the operation included in the transaction; and obtaining the transaction execution order based on the operation execution record information.
In some embodiments, the playback module 430 may be configured to play back a plurality of transactions to be played back to the database according to the dependency relationship, including at least two transactions to be played back for which there is no dependency relationship between concurrent playback transactions. In some embodiments, the playback module 430 may be further configured to play back at least two of the to-be-played back transactions for which the dependency relationship exists between the transactions in the following order: the dependent transaction plays back after the playback of the dependent transaction associated therewith is completed.
For a detailed description of the modules of the system shown above, reference may be made to the flow chart portion of the present description, such as fig. 2 and its associated description.
It should be understood that the system shown in fig. 4 and its modules may be implemented in a variety of ways. For example, in some embodiments, the system and its modules may be implemented in hardware, software, or a combination of software and hardware. Wherein the hardware portion may be implemented using dedicated logic; the software portions may then be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those skilled in the art will appreciate that the methods and systems described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system of the present specification and its modules may be implemented not only with hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but also with software executed by various types of processors, for example, and with a combination of the above hardware circuits and software (e.g., firmware).
It should be noted that the above description of the flow playback system of the database and its modules is for convenience of description only and is not intended to limit the present description to the scope of the illustrated embodiments. It will be appreciated by those skilled in the art that, given the principles of the system, various modules may be combined arbitrarily or a subsystem may be constructed in connection with other modules without departing from such principles. For example, in some embodiments, the transaction information acquisition module 410 and the parsing module 420 may be different modules in a system, or may be one module to implement the functionality of two or more modules described above. For example, each module may share one memory module, or each module may have a respective memory module. Such variations are within the scope of the present description.
The embodiments of the present specification also provide a flow playback device of a database, including at least one storage medium and at least one processor, the at least one storage medium storing computer instructions; the at least one processor is configured to execute the computer instructions to implement a method of traffic playback for a database. The method comprises the following steps: acquiring operation information of a plurality of transactions to be played back of a database, wherein the operation information comprises transaction operation content and transaction execution sequence; analyzing and obtaining a dependency relationship among a plurality of transactions to be played back based on the operation information; and playing back a plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprise at least two transactions to be played back, wherein the concurrent playback transactions have no dependency relationship.
Possible benefits of embodiments of the present description include, but are not limited to: the method comprises the steps of obtaining operation information such as transaction operation content, transaction execution sequence and the like of a plurality of transactions to be replayed of a request flow to be replayed, analyzing the operation information to obtain the dependency relationship among the plurality of transactions to be replayed, and replaying the plurality of transactions to be replayed to a database according to the dependency relationship among the transactions, wherein the plurality of transactions to be replayed without the dependency relationship are replayed concurrently, so that the flow replay efficiency can be improved by concurrently replaying a plurality of operations while the replay accuracy is ensured. It should be noted that, the advantages that may be generated by different embodiments may be different, and in different embodiments, the advantages that may be generated may be any one or a combination of several of the above, or any other possible advantages that may be obtained.
While the basic concepts have been described above, it will be apparent to those skilled in the art that the foregoing detailed disclosure is by way of example only and is not intended to be limiting. Although not explicitly described herein, various modifications, improvements, and adaptations to the present disclosure may occur to one skilled in the art. Such modifications, improvements, and modifications are intended to be suggested within this specification, and therefore, such modifications, improvements, and modifications are intended to be included within the spirit and scope of the exemplary embodiments of the present invention.
Meanwhile, the specification uses specific words to describe the embodiments of the specification. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the present description. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the present description may be combined as suitable.
Furthermore, those skilled in the art will appreciate that the various aspects of the specification can be illustrated and described in terms of several patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the present description may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the specification may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
The computer storage medium may contain a propagated data signal with the computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take on a variety of forms, including electro-magnetic, optical, etc., or any suitable combination thereof. A computer storage medium may be any computer readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code located on a computer storage medium may be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or the like, or a combination of any of the foregoing.
The computer program code necessary for operation of portions of the present description may be written in any one or more programming languages, including an object oriented programming language such as Java, scala, smalltalk, eiffel, JADE, emerald, C ++, c#, vb net, python and the like, a conventional programming language such as C language, visual Basic, fortran 2003, perl, COBOL 2002, PHP, ABAP, a dynamic programming language such as Python, ruby and Groovy, or other programming languages and the like. The program code may execute entirely on the user's computer or as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any form of network, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet), or the use of services such as software as a service (SaaS) in a cloud computing environment.
Furthermore, the order in which the elements and sequences are processed, the use of numerical letters, or other designations in the description are not intended to limit the order in which the processes and methods of the description are performed unless explicitly recited in the claims. While certain presently useful inventive embodiments have been discussed in the foregoing disclosure, by way of various examples, it is to be understood that such details are merely illustrative and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements included within the spirit and scope of the embodiments of the present disclosure. For example, while the system components described above may be implemented by hardware devices, they may also be implemented solely by software solutions, such as installing the described system on an existing server or mobile device.
Likewise, it should be noted that in order to simplify the presentation disclosed in this specification and thereby aid in understanding one or more inventive embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof. This method of disclosure, however, is not intended to imply that more features than are presented in the claims are required for the present description. Indeed, less than all of the features of a single embodiment disclosed above.
In some embodiments, numbers describing the components, number of attributes are used, it being understood that such numbers being used in the description of embodiments are modified in some examples by the modifier "about," approximately, "or" substantially. Unless otherwise indicated, "about," "approximately," or "substantially" indicate that the number allows for a 20% variation. Accordingly, in some embodiments, numerical parameters set forth in the specification and claims are approximations that may vary depending upon the desired properties sought to be obtained by the individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and employ a method for preserving the general number of digits. Although the numerical ranges and parameters set forth herein are approximations that may be employed in some embodiments to confirm the breadth of the range, in particular embodiments, the setting of such numerical values is as precise as possible.
Each patent, patent application publication, and other material, such as articles, books, specifications, publications, documents, etc., referred to in this specification is incorporated herein by reference in its entirety. Except for application history documents that are inconsistent or conflicting with the content of this specification, documents that are currently or later attached to this specification in which the broadest scope of the claims to this specification is limited are also. It is noted that, if the description, definition, and/or use of a term in an attached material in this specification does not conform to or conflict with what is described in this specification, the description, definition, and/or use of the term in this specification controls.
Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments of this specification. Other variations are possible within the scope of this description. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present specification may be considered as consistent with the teachings of the present specification. Accordingly, the embodiments of the present specification are not limited to only the embodiments explicitly described and depicted in the present specification.
Claims (10)
1. A method of traffic playback of a database, the method comprising:
acquiring operation information of a plurality of transactions to be played back of a database, wherein the operation information comprises transaction operation content and transaction execution sequence;
analyzing and obtaining the dependency relationship among the plurality of transactions to be played back based on the operation information;
and playing back the plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprises at least two transactions to be played back, wherein the dependency relationship does not exist between the concurrent playback transactions.
2. The method of claim 1, wherein the playing back the plurality of transactions to be played back to the database further comprises:
and playing back at least two transactions to be played back, wherein the transactions have the dependency relationship, according to the following sequence: the dependent transaction plays back after the playback of the dependent transaction associated therewith is completed.
3. The method of claim 1, wherein the presence of the dependency relationship between two of the transactions to be played back represents: the execution of one of the transactions to be played back has an effect on the execution of the other transaction to be played back.
4. The method as recited in claim 1, further comprising:
acquiring database operation logs related to the plurality of transactions to be played back;
and obtaining the execution order of the transaction based on the database operation log.
5. The method of claim 4, wherein the deriving the transaction execution order based on the database oplog comprises:
determining, based on the database oplog, operational performance record information related to the plurality of transactions to be played back, the operational performance record information including one or more of: the execution time of the operation included in the transaction, the operation identification of the operation included in the transaction, and the information of the data aimed at by the operation included in the transaction;
and obtaining the transaction execution order based on the operation execution record information.
6. The method as recited in claim 1, further comprising:
obtaining a request flow to be replayed of the database, wherein the request flow to be replayed comprises a plurality of operations;
determining transactions to which the plurality of operations respectively belong;
and obtaining the plurality of transactions to be played back according to the plurality of operations and the transactions to which the plurality of operations respectively belong.
7. A database traffic playback system, the system comprising:
the transaction information acquisition module is used for acquiring operation information of a plurality of transactions to be played back of the database, wherein the operation information comprises transaction operation contents and transaction execution sequences;
the analysis module is used for analyzing and obtaining the dependency relationship among the plurality of transactions to be played back based on the operation information;
and the playback module is used for playing back the plurality of transactions to be played back to the database according to the dependency relationship, wherein the plurality of transactions to be played back comprise at least two transactions to be played back, wherein the dependency relationship does not exist between the concurrent playback transactions.
8. The system of claim 7, wherein the playback module is further configured to:
and playing back at least two transactions to be played back, wherein the transactions have the dependency relationship, according to the following sequence: the dependent transaction plays back after the playback of the dependent transaction associated therewith is completed.
9. The system of claim 7, wherein the transaction information acquisition module is further configured to:
acquiring database operation logs related to the plurality of transactions to be played back;
and obtaining the execution order of the transaction based on the database operation log.
10. A flow playback device for a database, comprising at least one storage medium and at least one processor, the at least one storage medium for storing computer instructions; the at least one processor is configured to execute the computer instructions to implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310375916.4A CN116089403A (en) | 2023-04-11 | 2023-04-11 | Flow playback method and system for database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310375916.4A CN116089403A (en) | 2023-04-11 | 2023-04-11 | Flow playback method and system for database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116089403A true CN116089403A (en) | 2023-05-09 |
Family
ID=86208676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310375916.4A Pending CN116089403A (en) | 2023-04-11 | 2023-04-11 | Flow playback method and system for database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116089403A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150309768A1 (en) * | 2014-04-28 | 2015-10-29 | Sonos, Inc. | Preference Conversion |
| CN110287006A (en) * | 2019-06-10 | 2019-09-27 | 广州思创科技发展有限公司 | A kind of data dispatching method and system |
| CN110806923A (en) * | 2019-10-29 | 2020-02-18 | 百度在线网络技术（北京）有限公司 | Parallel processing method and device for block chain tasks, electronic equipment and medium |
| CN115167989A (en) * | 2022-05-05 | 2022-10-11 | 山东浪潮科学研究院有限公司 | HLC-based distributed transaction parallel playback method and system |
| CN115421880A (en) * | 2022-09-22 | 2022-12-02 | 金篆信科有限责任公司 | Transaction processing method, device, equipment and medium |
| CN115509694A (en) * | 2022-10-08 | 2022-12-23 | 北京火山引擎科技有限公司 | Transaction processing method and device, electronic equipment and storage medium |
-
2023
- 2023-04-11 CN CN202310375916.4A patent/CN116089403A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150309768A1 (en) * | 2014-04-28 | 2015-10-29 | Sonos, Inc. | Preference Conversion |
| CN110287006A (en) * | 2019-06-10 | 2019-09-27 | 广州思创科技发展有限公司 | A kind of data dispatching method and system |
| CN110806923A (en) * | 2019-10-29 | 2020-02-18 | 百度在线网络技术（北京）有限公司 | Parallel processing method and device for block chain tasks, electronic equipment and medium |
| CN115167989A (en) * | 2022-05-05 | 2022-10-11 | 山东浪潮科学研究院有限公司 | HLC-based distributed transaction parallel playback method and system |
| CN115421880A (en) * | 2022-09-22 | 2022-12-02 | 金篆信科有限责任公司 | Transaction processing method, device, equipment and medium |
| CN115509694A (en) * | 2022-10-08 | 2022-12-23 | 北京火山引擎科技有限公司 | Transaction processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10552301B2 (en) | Completing functional testing | |
| US9274782B2 (en) | Automated computer application update analysis | |
| US20080086660A1 (en) | Test data management | |
| US20200249922A1 (en) | Method and apparatus for data compilation, electronic device and computer readable storage medium | |
| US7185235B2 (en) | Test and verification framework | |
| US20200089594A1 (en) | Artificial intelligence for load testing | |
| US20140081902A1 (en) | Integrating Data Transform Test with Data Transform Tool | |
| CN112765017A (en) | Data query performance test method and device based on MySQL database | |
| US10241888B2 (en) | Method to verify correctness of computer system software and hardware components and corresponding test environment | |
| US20120054724A1 (en) | Incremental static analysis | |
| KR102271857B1 (en) | Test automation system | |
| CN110928786A (en) | Testing method and device for financial program | |
| CN115080433A (en) | Testing method and device based on flow playback | |
| CN112905449B (en) | Target test method, device, equipment and storage medium | |
| CN117370767A (en) | User information evaluation method and system based on big data | |
| US8739091B1 (en) | Techniques for segmenting of hardware trace and verification of individual trace segments | |
| CN116089403A (en) | Flow playback method and system for database | |
| CN115373688B (en) | Optimization method and system of software development thread and cloud platform | |
| US8863095B2 (en) | Recording and playback of system interactions in different system environments | |
| CN113360365A (en) | Flow testing method and flow testing system | |
| CN110990267B (en) | Data processing method and device | |
| CN115495017B (en) | Data storage method and system based on big data | |
| US8321844B2 (en) | Providing registration of a communication | |
| CN114863585B (en) | Intelligent vehicle testing and monitoring system and method and cloud platform | |
| CN118261143B (en) | Json data analysis method, device, equipment and medium in Unity environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230509 |