Disclosure of Invention
In view of the above, the embodiment of the invention discloses a flow playback method and a flow playback device, which improve the playback success rate and solve the problem of low test efficiency caused by low playback success rate in the prior art.
The embodiment of the invention discloses a flow playback method, which comprises the following steps:
receiving a plurality of request data sent by a network layer, and storing the request data into a message queue;
sequentially acquiring request data from the message queue until the request data does not exist in the message queue;
storing the request data with the pre-dependent data and incomplete pre-dependent data into a message queue;
the playback operation will be performed with no pre-dependency data or with request data for which the pre-dependency data is complete.
Optionally, the method further comprises:
determining a pre-dependency relationship between the plurality of request data;
and configuring pre-dependency data for each request data of the plurality of request data based on the pre-dependency relationship among the plurality of request data.
Optionally, the method further comprises:
and desensitizing the request data.
Optionally, if the playback operation of the requested data fails, the method further includes:
acquiring a reason for failure of playback of the request data;
if the playback of the request data fails, the request data is subjected to desensitization processing, and the desensitization processing operation process of the request data is called;
and executing reverse compensation operation of the desensitization processing operation based on the desensitization processing operation process of the request data to obtain a compensation result.
Optionally, the method further comprises:
and carrying out playback operation on the request data again based on the compensation result.
Optionally, the method further comprises:
obtaining result data generated in the playback operation process;
judging whether the association relation between result data generated in the playback operation process meets a preset rule or not;
if the association relationship between the result data generated in the playback operation process meets the preset rule, the result data generated in the playback operation process is correct data.
Optionally, the storing the plurality of request data in a message queue includes:
determining an execution order of the plurality of request data;
and storing the received plurality of request data according to the execution sequence of the plurality of request data.
The embodiment of the invention discloses a flow playback device, which comprises:
a receiving unit, configured to receive a plurality of request data sent by a network layer, and store the plurality of request data in a message queue;
the acquisition unit is used for sequentially acquiring the request data from the message queue until the request data do not exist in the message queue;
the pre-dependency identification unit is used for storing the request data with pre-dependency data and incomplete pre-dependency data into the message queue;
and the playback unit is used for performing playback operation on the request data without the pre-dependent data or with the pre-dependent data complete.
Optionally, the method further comprises:
and desensitizing the request data.
Optionally, if the playback operation of the requested data fails, the method further includes:
a failure cause acquisition unit configured to acquire a cause of playback failure of the request data;
a first retrieving unit, configured to retrieve a desensitization processing operation procedure of the request data if the reason for the playback failure of the request data is that the request data is desensitized;
and the reverse compensation unit is used for executing reverse compensation operation of the desensitization processing operation based on the desensitization processing operation process of the request data to obtain a compensation result.
Optionally, the method further comprises:
a result data acquisition unit configured to acquire result data generated during a playback operation;
a judging unit for judging whether the association relationship between the result data generated in the playback operation process meets a preset rule;
and the determining unit is used for determining that the result data generated in the playback operation process is correct data if the association relation between the result data generated in the playback operation process meets the preset rule.
The embodiment of the invention discloses a flow playback method and a device, comprising the following steps: receiving a plurality of request data sent by a network layer, and storing the request data into a message queue; sequentially acquiring request data from the message queue until the request data does not exist in the message queue; storing the request data with the pre-dependent data and incomplete pre-dependent data into a message queue; the playback operation will be performed with no pre-dependency data or with request data for which the pre-dependency data is complete. As can be seen from this, in the present embodiment, when the flow playback is performed, the priority dependency of the request data is determined in consideration of the sequence of the request data, and when the request data has the priority dependency and the priority data is incomplete, the playback is not immediately ended, but the request data is stored again in the message queue, and the request data is waited for being extracted again. Therefore, the situation that playback fails due to the fact that the sequency among network traffic is ignored is avoided, the success rate of traffic playback is greatly improved, and therefore testing efficiency is improved.
Furthermore, since the desensitization processing is performed on the request data, data loss or data error is likely to be caused, in order to avoid playback failure caused by such a situation, in this embodiment, the desensitization operation is reversely compensated, so that the problem of data loss or data error is solved, the success rate of flow playback is improved, and the efficiency of testing is further improved.
In addition, in the embodiment, by detecting the association relationship between the result data generated in the playback process, verification of the correctness of the playback result is realized, and the accuracy of flow playback is further ensured.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a flow chart of a flow playback method provided by an embodiment of the present invention is shown, and in this embodiment, the method includes:
s101: receiving a plurality of request data sent by a network layer, and storing the request data into a message queue;
in this embodiment, the request data may be represented as request information for triggering a certain operation in the service, where the request that the playback operation may be performed may include: query requests, write requests, etc.
In general, a service is executed on a line, and network traffic data including request data is generated during the execution. For example, the payment service includes the following service requests in the execution process: ordering, withholding, confirming receipt, refund application, refund confirmation, etc. Wherein, before executing the playback operation, the playback system needs to acquire the request data generated in the process of executing the business process on line.
In the prior art, the network layer may copy the request data generated during the service execution process and forward the request data to the playback system. However, since the service is performed sequentially, the network layer copies the request data in the order of the request data and forwards the request data to the playback system in the copied order. However, due to problems such as network delay or non-uniformity in network execution times of different machines, the order of the request data received by the playback system is disordered. In this case, when the playback system performs the playback operation based on the acquired request data, a disorder may also occur, for example, for some request data that needs to have a pre-dependency, if before the request data is performed, the pre-dependency data corresponding to the request data does not perform the playback operation, the request data cannot be played back, so that the whole playback process ends with failure. For example, the payment request includes request data such as order, deduction, confirmation of receipt, refund application, refund confirmation, etc., and some request data require the execution result of the previous step, but if the order of the request data is disordered, playback will fail, for example, when the order operation is not executed yet when the deduction is executed, the order information corresponding to the deduction request cannot be queried, and then the deduction operation cannot be executed.
In order to solve the above-described problem, before performing a playback operation, request data transmitted by a network layer is stored in a message queue, and a playback system pulls data from the message queue.
In this embodiment, when storing request data in a message queue, the following two methods are included:
in the first embodiment, the request data is stored in the order of execution of the request data, and the request data is output in the order of storage, that is, the order of execution is first output. Wherein storing the plurality of request data into the message queue comprises:
determining an execution order of the plurality of request data;
and storing the received plurality of request data according to the execution sequence of the plurality of request data.
The sequence tag may be set in advance for each request data, and the request data may be sequentially stored in the message queue according to the preset sequence tag.
In this way, the playback system can ensure the sequentiality of the request data when executing the request data. For the storage mode of mode one, an ordered message queue may be used, for example, a Kafka ordered queue.
And storing the received request data in the order of the received request data.
Based on the second storage mode, the data in the message queue follows the first-in first-out principle, and when the data is pulled from the message queue, the data is also pulled based on the storage sequence of the request data, namely, the previously stored request data is pulled first.
In the storage manner of the first embodiment, the playback of the request data in order can be guaranteed, but in order to achieve the purpose of ordering the request data stored in the message column, the service in the playback system needs to be modified, so that the system is more invasive.
In the case of the above-described mode two, in order to secure the sequence of playback and avoid intrusion into the system, the playback system may execute the following flow (S102 to S104):
even if the storage method of the first embodiment is adopted, the request data may be processed in the following manner S102 to S104.
S102: sequentially acquiring request data from the message queue until the request data does not exist in the message queue;
in this embodiment, the end of the playback of the traffic is marked as being indicated by the fact that the message queue no longer contains the requested data.
S103: storing the request data with the pre-dependent data and incomplete pre-dependent data into a message queue;
in this embodiment, the plurality of request data to be played back are sequential, or it may be understood that the request data with the subsequent execution order needs the execution result of the request data with the preceding execution order, that is, the execution result of the request data with the preceding execution order needs to be detected if the request data with the subsequent execution order is played back.
In this embodiment, the preceding request data on which the request data depends is executed as the preceding dependent data of the request data whose execution order is subsequent.
In this embodiment, before executing S103, the pre-dependency of each request data is set, and specifically further includes:
determining a pre-dependency relationship between the plurality of request data;
and configuring pre-dependency data for each request data of the plurality of request data based on the pre-dependency relationship among the plurality of request data.
In this embodiment, some of the plurality of request data for which playback is to be performed have a pre-dependency, and some do not require a pre-dependency, for example, for a request for an order and a request for deduction in a payment request: the order request is one that does not require pre-dependency, and the paid pre-dependency data is the data generated by the order request.
In this embodiment, the determination of the pre-dependency relationship between the plurality of request data may be performed in two ways:
firstly, presetting a prepositive dependency relationship among a plurality of request data;
determining a dependency relationship among a plurality of request data based on a preset rule;
the preset rule may be a preset financial transaction rule, assuming that the request data is in the financial domain.
In this embodiment, by configuring the pre-dependency data of each request data, a dependency dictionary may be generated, which may contain the pre-dependency relationship of each request data.
In this embodiment, after the playback system pulls the request data from the message queue, playback failure due to the disordered sequence of the request data can be avoided by performing the following operations:
judging whether the request data has pre-dependent data or not;
if the request data has the pre-dependent data, judging whether the pre-dependent data corresponding to the request data is complete or not;
and if the pre-dependent data corresponding to the request data is incomplete, the request data is restored to the message queue.
Illustrating: in the process of data playback, if the request data is a payment request, inquiring whether a payment order corresponding to the payment request exists through a storage service of playback, and if the payment order does not exist, (wherein the payment order is pre-dependent data of the payment request), indicating that the request of the order is not played back, then the payment request needs to be stored in a message queue again.
In this embodiment, the request data is restored to the message queue, and typically, the request data may be stored to the tail end of the message queue and waiting for the playback system to pull the request data again. Wherein, before the playback system pulls the request data from the message queue again, the playback system pulls other request data from the message queue for playback operation, and when the request data is pulled again, the playback system may already play back the pre-dependent data of the request data, thus avoiding the problem of playback failure caused by lack of the pre-dependent data.
As is clear from the above operation S103, in this embodiment, in the case where the pre-dependent data of the request data is incomplete, the playback operation is not stopped (i.e., the playback operation is not directly declared to be failed), but the request data is restored in the message queue, and after other request data is executed, the request data is pulled again.
S104: the playback operation will be performed with no pre-dependency data or with request data for which the pre-dependency data is complete.
In this embodiment, some of the request data is free of the pre-dependent data, and the playback operation can be directly performed for the request data having such a feature.
Illustrating: if the request data is an order request, since the order request does not have pre-dependent data, the playback operation of the order request can be directly performed.
Or alternatively, the process may be performed,
in the case where the request data has the pre-dependent data and the pre-dependent data is complete, the playback operation may also be performed on the request data.
Illustrating: if the request data is a payment request, if a payment order corresponding to the payment request exists (i.e. the corresponding order operation is played back), the request data is provided with the complete pre-dependent data of the payment request, and the playback operation can be performed on the request data.
In this embodiment, when the flow playback is performed, the priority dependency of the request data is determined in consideration of the sequence of the request data, and when the request data has the priority dependency and the priority data is incomplete, the playback is not immediately ended, but the request data is stored again in the message queue, and the request data is waited for being pulled again. Therefore, the problem of playback failure caused by neglecting the sequency among network flows is avoided, and the success rate of flow playback is greatly improved, so that the efficiency of testing is improved.
In this embodiment, the data generated during the online execution of the service may include some sensitive information, such as account numbers and passwords of clients, transaction amounts, and the like. In order to ensure the security of the client information, the received request data needs to be desensitized before the traffic playback.
However, after the desensitization processing is performed on the request data, data loss or data errors are easily caused, for example, when the account number and the password of the client are hidden, errors are generated in the account number and the password of the user, and after the transaction amount is hidden, the data loss is caused, and in the playback process, playback failure is caused due to the data errors or the data loss.
In order to solve the problem of playback failure caused by the desensitization processing, in this embodiment, the desensitization processing is compensated reversely, and specifically, referring to fig. 2, there is shown a further flow diagram of a playback method provided by an embodiment of the present invention, where the method includes:
s201: if the playback operation of the request data fails, acquiring the reason of the playback failure of the request data;
in this example, a plurality of reasons for playback failure are preset, and if playback failure occurs in the playback system during the playback operation, the reasons for playback failure can be determined based on the operation process.
S202: if the playback of the request data fails, the request data is subjected to desensitization processing, and the desensitization processing operation process of the request data is called;
s203: and executing reverse compensation operation of the desensitization processing operation based on the desensitization processing operation of the request data to obtain a compensation result.
In this embodiment, the basis of the compensation operation is desensitization, what desensitization is performed, and some operations need to be performed reversely to compensate, for example, if the desensitization is to eliminate data, reverse compensation is to complement missing data, and if the data is modified, reverse compensation may be to adjust erroneous data. In this way, the playback operation can be normally performed by the reverse compensation operation.
Illustrating: in online operation, the user pays with a balance and pays successfully, but desensitizes the data prior to playback, for example: erasing the account information and balance information of the user, the process of the reverse compensation operation corresponding to the desensitization processing operation may include: an account is created and a balance is set. Thus, account information and balance information exist, and playback failure caused by data loss can be avoided when playback operation is performed.
S204: and carrying out playback operation on the request data again based on the compensation result.
In this embodiment, after the compensation operation is performed, since the problem of data missing or the like due to the desensitization processing is complemented, the playback operation of the requested data can be performed again based on the compensated result, and in this case, playback failure due to the desensitization processing is avoided.
In this embodiment, since the desensitization processing is performed on the request data, there is a high probability of data missing or data error, and in order to avoid playback failure caused by such a situation, in this embodiment, the desensitization operation is reversely compensated, so as to make up for the problem of data missing or data error, and improve the success rate of flow playback, thereby further improving the efficiency of testing.
In this embodiment, in order to further ensure the correctness of the result obtained by the playback operation, the result data obtained by the playback operation may be further checked, referring to fig. 3, another flow chart of flow playback provided by the embodiment of the present invention is shown, and in this embodiment, the method includes:
s301: obtaining result data generated in the playback operation process;
s302: judging whether the association relation between result data generated in the playback operation process meets a preset rule or not;
s303: if the association relationship between the result data generated in the playback operation process meets the preset rule, the result data generated in the playback operation process is correct data.
If the association relationship between the result data generated during the playback operation does not meet the preset rule, the result data is indicated to be in error, and in this case, the cause of the error may be determined, and the error may be corrected based on the cause of the error.
In this embodiment, for playback operations in different fields, the association relationship between the obtained result data may follow different rules, for example, if the result data belongs to the financial field, the association relationship between the obtained result data may follow accounting rules.
Illustrating: assuming that under normal conditions, a payment will generate an order and include at least two accounting flows and two updates of the user balance, after the playback operation of the payment is performed, the verification can be performed by the following rules: 1. whether the order contains two running water or not, and the amount is consistent; 2. whether corresponding orders are generated in each running water or not, and the amounts are consistent; 3. and whether the running water of the accounts is consistent with the change of the balance or not within a certain time.
In the embodiment, the verification of the correctness of the playback result is realized by detecting the association relationship between the result data generated in the playback process, so that the accuracy of flow playback is further ensured.
Referring to fig. 4, an interactive schematic diagram of a flow playback provided by an embodiment of the present invention is shown, in this embodiment, the method includes:
s401: forwarding request data generated by a service system to a message queue based on a network layer;
s402: acquiring request data from a message queue, and performing desensitization processing on the request data;
s403: verifying the front-end dependency relationship of the request data;
s404: if the request data has pre-dependency and the pre-dependency data is incomplete, storing the request data into a message queue again;
s405: if the request data does not have the pre-dependency and the pre-dependency data is complete, performing playback operation on the request data;
s406: if the playback operation of the request data fails, and the reason for the playback failure of the request data is that the request data is desensitized, the reverse compensation operation is performed;
the specific reverse compensation operation process comprises the following steps:
invoking a desensitization processing operation of the request data;
based on the desensitization processing operation process of the request data, performing reverse compensation operation of the desensitization processing operation to obtain a compensation result;
s407: performing playback operation again on the request data based on the compensation result;
s408: and checking the correctness of the result data generated in the playback process.
The checking method can comprise the following steps:
obtaining result data generated in the playback operation process;
judging whether the association relation between result data generated in the playback operation process meets a preset rule or not;
if the association relationship between the result data generated in the playback operation process meets the preset rule, the result data generated in the playback operation process is correct data.
In this embodiment, when the flow playback is performed, the priority dependency of the request data is determined in consideration of the sequential nature of the request data, and when the request data has the priority dependency and the priority data is incomplete, the playback is not immediately ended, but the request data is stored in the message queue again, and the request data is waiting to be extracted again. Therefore, the situation that playback fails due to the fact that the sequency among network traffic is ignored is avoided, the success rate of traffic playback is greatly improved, and therefore testing efficiency is improved.
Furthermore, since the desensitization processing is performed on the request data, data loss or data error is likely to be caused, in order to avoid playback failure caused by such a situation, in this embodiment, the desensitization operation is reversely compensated, so that the problem of data loss or data error is solved, the success rate of flow playback is improved, and the efficiency of testing is further improved.
In addition, in the embodiment, by detecting the association relationship between the result data generated in the playback process, verification of the correctness of the playback result is realized, and the accuracy of flow playback is further ensured.
Referring to fig. 5, a schematic structural diagram of a flow playback device according to an embodiment of the present invention is shown, where in this embodiment, the device includes:
a receiving unit 501, configured to receive a plurality of request data sent by a network layer, and store the plurality of request data in a message queue;
an obtaining unit 502, configured to sequentially obtain request data from the message queue until no request data exists in the message queue;
a pre-dependency identification unit 503, configured to store request data in which pre-dependency data exists and the pre-dependency data is incomplete, into a message queue;
a first playback unit 504, configured to perform a playback operation on the request data where the pre-dependency data does not exist or where the pre-dependency data is complete.
Optionally, the method further comprises:
a pre-dependency determination unit configured to determine a pre-dependency between the plurality of request data;
and a configuration unit configured to configure the pre-dependency data for each request data of the plurality of request data based on the pre-dependency relationship between the plurality of request data.
Optionally, the method further comprises:
and the desensitization unit is used for carrying out desensitization processing on the request data.
Optionally, if the playback operation of the requested data fails, the method further includes:
a failure cause acquisition unit configured to acquire a cause of playback failure of the request data;
a first retrieving unit, configured to retrieve a desensitization processing operation procedure of the request data if the reason for the playback failure of the request data is that the request data is desensitized;
and the reverse compensation unit is used for executing reverse compensation operation of the desensitization processing operation based on the desensitization processing operation process of the request data to obtain a compensation result.
Optionally, the method further comprises:
and the second playback unit is used for carrying out playback operation on the request data again based on the compensation result.
Optionally, the method further comprises:
a result data acquisition unit configured to acquire result data generated during a playback operation;
a judging unit for judging whether the association relationship between the result data generated in the playback operation process meets a preset rule;
and the determining unit is used for determining that the result data generated in the playback operation process is correct data if the association relation between the result data generated in the playback operation process meets the preset rule.
Optionally, the receiving unit includes:
an execution order determining subunit configured to determine an execution order of the plurality of request data;
and the storage subunit is used for storing the received plurality of request data according to the execution sequence of the plurality of request data.
By the device of the embodiment, when the flow playback is performed, the prepositive dependency of the request data is judged by considering the sequence of the request data, and when the request data has the prepositive dependency and the prepositive data is incomplete, the playback is not immediately finished, but the request data is stored in the message queue again and is waited to be extracted again. Therefore, the situation that playback fails due to the fact that the sequency among network traffic is ignored is avoided, the success rate of traffic playback is greatly improved, and therefore testing efficiency is improved.
Furthermore, since the desensitization processing is performed on the request data, data loss or data error is likely to be caused, in order to avoid playback failure caused by such a situation, in this embodiment, the desensitization operation is reversely compensated, so that the problem of data loss or data error is solved, the success rate of flow playback is improved, and the efficiency of testing is further improved.
In addition, in the embodiment, by detecting the association relationship between the result data generated in the playback process, verification of the correctness of the playback result is realized, and the accuracy of flow playback is further ensured.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.