CN111858506B - Test data processing method and device, electronic equipment and storage medium - Google Patents
Test data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111858506B CN111858506B CN202010617683.0A CN202010617683A CN111858506B CN 111858506 B CN111858506 B CN 111858506B CN 202010617683 A CN202010617683 A CN 202010617683A CN 111858506 B CN111858506 B CN 111858506B
- Authority
- CN
- China
- Prior art keywords
- log
- target
- test data
- service scene
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 224
- 238000003672 processing method Methods 0.000 title abstract description 15
- 239000012634 fragment Substances 0.000 claims abstract description 56
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000011144 upstream manufacturing Methods 0.000 claims abstract description 44
- 238000012545 processing Methods 0.000 claims description 40
- 230000015654 memory Effects 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 239000002699 waste material Substances 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 2
- 230000011218 segmentation Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000004904 shortening Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a test data processing method, a device, electronic equipment and a storage medium, and relates to the technical field of testing in big data technology. The specific implementation scheme is as follows: acquiring a full log fragment corresponding to test data required by a target service scene to be tested from a full log on line of a server system; intercepting a target log fragment belonging to the target service scene and a service scene upstream of the target service scene in a service flow from the total log fragment; and generating the test data on line based on the target log segment. When the test data generated by the method is used for testing the target service scene, the upstream party does not need to communicate with the target service scene, the test data is passively waited for being provided by the upstream service scene, the test of the target service scene can be directly finished by using the test data, the test time is effectively shortened, the test progress is accelerated, and the test efficiency of the target service scene can be effectively improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to the field of testing technologies in big data technologies, and in particular, to a method and apparatus for processing test data, an electronic device, and a storage medium.
Background
In the prior art, a complex server system usually splits a plurality of parts of services to complete research, development and test of projects. In the test stage of each service scene of the research and development life cycle, the testers are strongly dependent on the upstream service party for acquiring test data, and the upstream testers are required to cooperate for providing.
If the upstream service party does not provide the test data in time, the test progress of the downstream service scenario is delayed, so that the acquisition of the test data of the existing downstream service scenario is very passive, and the test efficiency is greatly reduced.
Disclosure of Invention
In order to solve the technical problems, the application provides a test data processing method, a device, electronic equipment and a storage medium.
According to an aspect of the present application, there is provided a method for processing test data, wherein the method includes:
acquiring a full log fragment corresponding to test data required by a target service scene to be tested from a full log on line of a server system;
intercepting a target log fragment belonging to the target service scene and a service scene upstream of the target service scene in a service flow from the total log fragment;
and generating the test data on line based on the target log segment.
According to another aspect of the present application, there is provided a test data processing apparatus, wherein the apparatus includes:
the acquisition module is used for acquiring a full log fragment corresponding to test data required by a target service scene to be tested from a full log on line of the server system;
the intercepting module is used for intercepting the target log fragments belonging to the target service scene and the service scene upstream of the target service scene in the service flow from the total log fragments;
and the generation module is used for generating the test data on line based on the target log segment.
According to still another aspect of the present application, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to yet another aspect of the present application, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method as described above.
According to yet another aspect of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
According to the technology of the application, the generated test data not only comprises the data of the target service scene, but also comprises the data of the upstream service scene, so that when the generated test data of the embodiment of the application is adopted to test the target service scene, the user does not need to communicate with an upstream party, passively waits for the upstream service scene to provide the test data, and can directly utilize the test data to complete the test of the target service scene, thereby effectively shortening the test time, accelerating the test progress and further effectively improving the test efficiency of the target service scene.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present application;
FIG. 2 is a schematic diagram according to a second embodiment of the present application;
FIG. 3 is a schematic diagram according to a third embodiment of the present application;
FIG. 4 is a schematic diagram according to a fourth embodiment of the present application;
FIG. 5 is a schematic diagram according to a fifth embodiment of the present application;
FIG. 6 is a schematic diagram according to a sixth embodiment of the present application;
fig. 7 is a block diagram of an electronic device for implementing a method of processing test data according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
FIG. 1 is a schematic diagram according to a first embodiment of the present application; as shown in fig. 1, the present embodiment provides a method for processing test data, which specifically includes the following steps:
s101, acquiring a full log segment corresponding to test data required by a target service scene to be tested from a full log on line of a server system;
s102, intercepting a target log fragment belonging to a target service scene and a service scene upstream of the target service scene in a service flow from the total log fragments;
s103, generating test data on line based on the target log segment.
The execution main body of the test data processing method of the embodiment is a test data processing device, and the test data processing device may be an electronic entity or may also be a software integrated application, and when in use, the test data processing device runs on a computer device and can generate test data on line based on data on line.
The embodiment is mainly used for generating the test data required by the test target service scene. The service end system of the embodiment comprises a plurality of service scenes in the service flow, and the target service scene is only one service scene in the service end system. For example, the service end system of the embodiment may be a service end system of an e-commerce, and for example, in a business process, a plurality of business scenarios including ordering, payment, distribution, after-sales, and the like may be included. Or the server system of the embodiment may be a system that includes a plurality of service scenarios in other service flows, which are not described herein in detail.
Because the test data to be generated is used for testing the target service scene of the server system on line, the acquired total log segments are required to be the total log segments corresponding to the test data required by the target service scene to be tested. If the full log segment includes the information of the target service scene, the full log segment must include the test data of the target service scene, so the full log segment may be the full log segment corresponding to the test data required by the target service scene. Specifically, the full log segment includes instruction information of the target service scene. In addition, since the full log segment is obtained from the full log on the line of the server system, the full log segment necessarily includes instruction information in other service scenarios.
The full log on the line of the server system in this embodiment may be historical data of a full binlog corresponding to an on-line database of the server system, where the full log may include instruction information in each service scenario in a service flow corresponding to each piece of service data. However, because the user is in operation with the service, the service is often interrupted, for example, some users cancel the order after the order is placed, or pay for the cancellation order and apply for refunds. At this time, the corresponding business data and the corresponding full log do not include the information of the distributed business scenario. Therefore, in this embodiment, a full log segment corresponding to test data required by a target service scenario to be tested needs to be obtained based on the target service scenario to be tested, that is, the full log segment must include instruction information corresponding to the target service scenario.
The full log segment acquired in this embodiment may include instruction information on the full business flow of the business data. However, the test data to be acquired in the embodiment is only used for testing the target service scenario, so that instruction information corresponding to the service scenario located behind the target service scenario in the service flow in the full log fragment does not have any contribution to generating the test data and can not be reserved. Specifically, the target log segments belonging to the target service scene and the service scene upstream of the target service scene in the service flow may be intercepted from the full log segments.
Because the acquired target log segment includes the target service scene and the instruction information of the service scene at the upstream of the target service scene in the service flow, the test data of the target service scene can be generated on line based on the target log segment, so that the current data of the target service scene can be included, and the data of the upstream scene of the target scene can be included.
According to the test data processing method, a full log segment corresponding to test data required by a target service scene to be tested is obtained from a full log on line of a server system; intercepting a target log fragment belonging to a target service scene and a service scene upstream of the target service scene in a service flow from the total log fragments; based on the target log segment, test data is generated online. Because the target log segment of the embodiment not only includes the target service scene and the information of the service scene upstream of the target service scene, the correspondingly generated test data not only includes the data of the target service scene, but also includes the data of the service scene upstream, when the generated test data of the embodiment is adopted to test the target service scene, the user does not need to communicate with the upstream party, passively waits for the upstream service scene to provide the test data, and can directly utilize the test data to complete the test of the target service scene, thereby effectively shortening the test time, accelerating the test progress, and further effectively improving the test efficiency of the target service scene.
FIG. 2 is a schematic diagram according to a second embodiment of the present application; as shown in fig. 2, the method for processing test data according to the present embodiment further describes the technical solution of the present application in more detail on the basis of the technical solution of the embodiment shown in fig. 1. As shown in fig. 2, the method for processing test data in this embodiment may specifically include the following steps:
s201, according to a target service scene to be tested, a unique identifier of test data required in the target service scene is fished out on line;
in this embodiment, the unique identifier of the test data must be selected with reference to the target service scenario, or else, the unique identifier of the test data may not have the data under the corresponding target service scenario, and thus may not be the test data.
For example, if the service end system is an e-commerce service end system, when the target service scenario is delivery, the order number of one piece of service data with the delivery scenario may be taken from online as the unique identifier of the test data to be generated. Because the order number is unique and unchanged in different business scenarios of the business process, the order number can be used as a unique identifier of the test data. Similarly, in other server systems, the unique identifier of the test data can also be based on the identifier which is unchanged in the whole process of the service data in the service flow.
S202, acquiring a full log fragment from a full log on a line according to a unique identifier of test data;
similarly, the on-line full log in this embodiment is a binlog full log in the history data of the on-line database. Based on the unique identification of the test data, full log segments may be searched and obtained from the full log. By the method, the full log fragments are acquired, and the accuracy of the subsequently generated test data can be effectively ensured.
S203, dividing the total log segments into target log segments comprising the target service scene and the service scene upstream of the target service scene and waste log segments comprising the service scene downstream of the target service scene according to a plurality of service scenes included in the service flow of the server system;
because the total log segments comprise instruction information of each service scene in the service flow, when test data in a target service scene is generated, the target service scene and the log segments of the service scene upstream of the target service scene only contribute to the generation of the test data, while the log segments of the service scene downstream of the target service scene do not contribute to the generation of the test data, based on the instruction information, the total log segments are divided into target log segments comprising the service scene belonging to the target service scene and the service scene upstream of the target service scene and waste log segments comprising the service scene downstream of the target service scene according to a plurality of service scenes in the service flow, and the waste log segments can not be reserved.
This step is a specific implementation manner of step S102 in the embodiment shown in fig. 1.
In addition, optionally, step S102 in the embodiment shown in fig. 1 may further include the following steps when specifically implemented:
(a) Dividing the total log segments according to a plurality of service scenes included in the service flow of the server system to obtain log sub-segments of each service scene;
(b) And merging the log sub-fragments belonging to the target service scene and the service scene upstream of the target service scene in the service flow according to the sequence in the service flow to generate the target log fragment.
The method is that after the log sub-segments are segmented according to a plurality of service scenes, the needed log sub-segments are combined. The target log fragment generated in this way is the same as the target log fragment generated in step S203 described above.
S204, synchronizing the target log segments on the line to be off-line;
s205, executing the synchronized target log fragments by adopting an off-line database to generate test data.
Specifically, the target log segments may be copied from online, enabling the synchronization of the target log segments on-line to off-line. Because the target log segments include one instruction information, and corresponding test data is required to be generated, the synchronized target log segments are required to be executed by adopting an off-line database, specifically, the operations of the instruction information can be sequentially executed according to the sequence of the instruction information in the target log segments from front to back, and the test data can be correspondingly generated in the off-line database.
By adopting the mode of the embodiment, the test data of the target service scene can be generated one by one, the test data of the required number is finally obtained, and the test requirement of the target service scene is met.
According to the test data processing method, the test data can be generated on line based on the on-line information, the generated test data not only comprises the data of the target service scene, but also comprises the data of the upstream service scene, so that the test of the target service scene can be effectively guaranteed without passively waiting for the upstream service scene to provide the test data, the test of the target service scene can be directly completed by using the test data, the test efficiency of the target service scene can be effectively improved, and the test quality can be effectively guaranteed. In addition, the technical scheme of the embodiment can also effectively ensure the accuracy of the generated test data.
FIG. 3 is a schematic diagram according to a third embodiment of the present application; as shown in fig. 3, the method for processing test data in this embodiment may specifically include the following steps:
s301, after testing is carried out by adopting test data, acquiring an update log segment of an off-line database;
s302, acquiring an incremental log fragment based on the updated log fragment and a target log fragment of test data;
s303, restoring test data based on the increment log segments.
Similarly, the execution body of the test data processing method of the present embodiment is a test data processing apparatus.
The test data of this embodiment may be the test data generated in the embodiment shown in fig. 1 or fig. 2 described above. Specifically, after the generated test data is adopted to test the target service scene of the server system, the test data can change, and the offline database can update the log segment, so that the updated log segment can be obtained.
In addition, the target log segment is also synchronized off-line as the test data is generated. At this time, the update log segment and the target log segment may be compared to obtain an incremental log segment of the update log segment relative to the target log segment. Because all instruction information of the increment is recorded in the increment log segment, the changed test data can be restored based on the increment log segment, and the test data can be restored.
It should be noted that, the processing method of the test data in this embodiment may be configured as a single embodiment, or may be combined with the embodiment shown in fig. 1 or fig. 2 to implement further processing of the test data.
According to the test data processing method, after test data are adopted to execute a test, an update log segment of an off-line database is obtained; based on the updated log segments and the target log segments of the test data, incremental log segments are obtained; based on the increment log segments, the test data is restored, after the test data is changed, the test data can be restored, the repeated usability of the test number can be improved, the same test data can be repeatedly used for a plurality of times, and the utilization rate of the test data is improved.
Further optionally, step S303 in the embodiment shown in fig. 3, restoring test data based on the incremental log segment may include, in a specific implementation: and according to the reverse sequence of a plurality of operations sequentially included in the log fragments, performing the reverse operation of the plurality of operations sequentially by adopting the offline database so as to rollback and update the log fragments to the target log fragments, thereby realizing the restoration of the test data.
Alternatively, in the present embodiment, the inverse operation of each operation may be specifically implemented according to the following rule:
(1)insert→delete；
(2)delete→insert；
(3)update(old，new)→update(new，old)。
the above-described mode (1) indicates that the inverse operation of the insert operation corresponds to the delete operation; the above-described mode (2) indicates that the inverse operation of the delete operation corresponds to an insert operation; the above-mentioned mode (3) indicates that if the old update operation is to update old to new, the corresponding inverse operation may be to update new to old.
According to the method, according to the reverse sequence of the operations of the incremental log segments, the offline database is adopted to perform the reverse operation on each operation, so that the test data can be restored, the repeated usability of the test number can be improved, the same test data can be repeatedly used for a plurality of times, and the utilization rate of the test data is improved.
FIG. 4 is a schematic diagram according to a fourth embodiment of the present application; as shown in fig. 4, the present embodiment provides a test data processing apparatus 400, including:
the acquiring module 401 is configured to acquire, from a full log on a line of the server system, a full log segment corresponding to test data required by a target service scenario to be tested;
an intercepting module 402, configured to intercept, from the total log segments, a target log segment of a service scenario belonging to a target service scenario and an upstream service scenario of the target service scenario in a service flow;
a generating module 403, configured to generate test data online based on the target log segment.
The implementation principle and the technical effect of the processing of the test data by using the above modules in the processing device 400 of the test data in this embodiment are the same as those of the implementation of the above related method embodiments, and detailed description of the above related method embodiments may be referred to and will not be repeated here.
FIG. 5 is a schematic diagram according to a fifth embodiment of the present application; as shown in fig. 5, the processing device 400 for test data according to the present embodiment further describes the technical solution of the present application in more detail on the basis of the technical solution of the embodiment shown in fig. 4.
As shown in fig. 5, in the test data processing apparatus 400 of the present embodiment, an acquisition module 401 includes:
the fishing unit 4011 is used for fishing out the unique identifier of the required test data in the target service scene from the online according to the target service scene to be tested;
the acquiring unit 4012 is configured to acquire a full log segment from the full log on the line according to the unique identifier of the test data.
Further alternatively, as shown in fig. 5, in the test data processing apparatus 400 of the present embodiment, an intercepting module 402 includes:
the dividing unit 4021 is configured to divide the total log segments according to a plurality of service scenarios included in a service flow of the server system, to obtain log sub-segments of each service scenario;
the splicing unit 4022 is configured to combine the log sub-segments belonging to the target service scenario and the service scenario upstream of the target service scenario in the service flow according to the sequence in the service flow, and generate a target log segment.
Or alternatively, in the test data processing apparatus 400 of the present embodiment, the intercepting module 402 is configured to:
according to a plurality of service scenes included in the service flow of the server-side system, the total log fragments are divided into target log fragments including the service scenes belonging to the target service scene and the upstream service scene and waste log fragments including the service scenes belonging to the downstream service scene.
Further alternatively, as shown in fig. 5, in the test data processing apparatus 400 of the present embodiment, the generating module 403 includes:
a synchronizing unit 4031 for synchronizing the target log segments on the line to off-line;
the generating unit 4032 is configured to execute the synchronized target log segment by using the offline database, and generate test data.
The implementation principle and the technical effect of the processing of the test data by using the above modules in the processing device 400 of the test data in this embodiment are the same as those of the implementation of the above related method embodiments, and detailed description of the above related method embodiments may be referred to and will not be repeated here.
FIG. 6 is a schematic diagram according to a sixth embodiment of the present application; as shown in fig. 6, the present embodiment provides a test data processing apparatus 600, including:
an update segment obtaining module 601, configured to obtain an update log segment of the offline database after performing a test with the test data;
the incremental segment acquisition module 602 is configured to acquire an incremental log segment based on the updated log segment and a target log segment of the test data;
the restoration processing module 603 is configured to restore the test data based on the incremental log segment.
Further alternatively, the reduction processing module 603 of this embodiment is configured to:
and according to the reverse sequence of a plurality of operations sequentially included in the log fragments, performing the reverse operation of the plurality of operations sequentially by adopting the offline database so as to rollback and update the log fragments to the target log fragments, thereby realizing the restoration of the test data.
The test data processing apparatus 600 of this embodiment may exist alone, or may be combined with the test data processing apparatus 600 of the embodiment shown in fig. 4 or fig. 5, respectively, to form an alternative apparatus.
The implementation principle and the technical effect of the processing of the test data by using the above modules in the processing device 600 of the test data in this embodiment are the same as those of the implementation of the above related method embodiments, and detailed description of the above related method embodiments may be referred to and will not be repeated here.
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 7, a block diagram of an electronic device implementing a method for processing test data according to an embodiment of the present application is shown. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 7, the electronic device includes: one or more processors 701, memory 702, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 701 is illustrated in fig. 7.
Memory 702 is a non-transitory computer-readable storage medium provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of processing test data provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the method of processing test data provided by the present application.
The memory 702 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., related modules shown in fig. 4 and 5, or related modules shown in fig. 6) corresponding to the test data processing method in the embodiment of the present application. The processor 701 executes various functional applications of the server and data processing, i.e., implements the processing method of test data in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 702.
Memory 702 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device implementing the processing method of the test data, and the like. In addition, the memory 702 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 702 optionally includes memory remotely located with respect to processor 701, which may be connected via a network to an electronic device implementing the method of processing test data. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device implementing the test data processing method may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or otherwise, in fig. 7 by way of example.
The input device 703 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device implementing the processing method of the test data, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. input devices. The output device 704 may include a display apparatus, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the full log segments corresponding to the test data required by the target service scene to be tested are obtained from the full log on line of the server system; intercepting a target log fragment belonging to a target service scene and a service scene upstream of the target service scene in a service flow from the total log fragments; based on the target log segment, test data is generated online. Because the target log fragment in the embodiment of the application not only comprises the target service scene and the information of the service scene at the upstream of the target service scene, the correspondingly generated test data not only comprises the data of the target service scene, but also comprises the data of the service scene at the upstream, when the generated test data in the embodiment of the application is adopted for testing the target service scene, the communication with the upstream party is not needed, the test data is passively waited for being provided by the upstream service scene, the test of the target service scene can be completed by directly utilizing the test data, the test time is effectively shortened, the test progress is accelerated, and the test efficiency of the target service scene can be effectively improved.
According to the technical scheme of the embodiment of the application, the test data can be generated on line based on the on-line information, and the generated test data not only comprises the data of the target service scene, but also comprises the data of the upstream service scene, so that when the target service scene is tested, the test data can be directly finished by utilizing the test data without passively waiting for the upstream service scene to provide the test data, the test efficiency of the target service scene can be effectively improved, and the test quality is effectively ensured. Moreover, the technical scheme of the embodiment of the application can also effectively ensure the accuracy of the generated test data.
According to the technical scheme of the embodiment of the application, after the test is executed by adopting the test data, the updated log segment of the off-line database is obtained; based on the updated log segments and the target log segments of the test data, incremental log segments are obtained; based on the increment log segments, the test data is restored, after the test data is changed, the test data can be restored, the repeated usability of the test number can be improved, the same test data can be repeatedly used for a plurality of times, and the utilization rate of the test data is improved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (14)
1. A method of processing test data, wherein the method comprises:
acquiring a full log fragment corresponding to test data required by a target service scene to be tested from a full log on line of a server system;
intercepting a target log fragment belonging to the target service scene and a service scene upstream of the target service scene in a service flow from the total log fragment;
generating the test data on line based on the target log segment;
intercepting, from the total log segments, a target log segment of a service scenario belonging to the target service scenario and upstream of the target service scenario in a service flow, including:
dividing the total log segments according to a plurality of service scenes included in the service flow of the server system to obtain log sub-segments of each service scene;
and merging the log sub-fragments belonging to the target service scene and the service scene upstream of the target service scene in the service flow according to the sequence in the service flow to generate the target log fragment.
2. The method of claim 1, wherein obtaining, from a full log on-line of a server system, a full log segment corresponding to test data required for a target service scenario to be tested, comprises:
according to the target service scene to be tested, the unique identification of the test data required under the target service scene is fished out on line;
and acquiring the full log fragments from the full log on the line according to the unique identification of the test data.
3. The method of claim 1, wherein intercepting, from the full log segments, a target log segment of a business scenario that belongs to the target business scenario and that is upstream of the target business scenario on a business process, comprises:
and dividing the total log segments into the target log segments comprising the target service scene and the service scene upstream of the target service scene and the waste log segments comprising the service scene downstream of the target service scene according to a plurality of service scenes included in the service flow of the server system.
4. The method of claim 1, wherein generating the test data online based on the target log segment comprises:
synchronizing the target log segment on-line to off-line;
and executing the synchronized target log fragments by adopting an off-line database to generate the test data.
5. The method of any of claims 1-4, wherein, based on the target log segment, after generating the test data online, the method further comprises:
after testing is carried out by adopting the test data, obtaining an updated log segment of the off-line database;
based on the updated log segments and the target log segments of the test data, incremental log segments are obtained;
and restoring the test data based on the increment log segment.
6. The method of claim 5, wherein restoring the test data based on the incremental log segments comprises:
and sequentially executing the reverse operations of the operations by adopting the offline database according to the reverse sequence of the operations sequentially included in the increment log fragments so as to roll back the update log fragments to the target log fragments, thereby realizing the restoration of the test data.
7. A test data processing apparatus, wherein the apparatus comprises:
the acquisition module is used for acquiring a full log fragment corresponding to test data required by a target service scene to be tested from a full log on line of the server system;
the intercepting module is used for intercepting the target log fragments belonging to the target service scene and the service scene upstream of the target service scene in the service flow from the total log fragments;
the generation module is used for generating the test data on line based on the target log segment;
the intercepting module comprises:
the segmentation unit is used for segmenting the total log segments according to a plurality of service scenes included in the service flow of the server-side system to obtain log sub-segments of each service scene;
and the splicing unit is used for merging the log sub-fragments belonging to the target service scene and the service scene upstream of the target service scene in the service process according to the sequence in the service process to generate the target log fragment.
8. The apparatus of claim 7, wherein the acquisition module comprises:
the system comprises a drag-and-drop unit, a drag-and-drop unit and a drag-and-drop unit, wherein the drag-and-drop unit is used for dragging a unique identifier of test data required in the target service scene on line according to the target service scene to be tested;
and the acquisition unit is used for acquiring the full log fragments from the full log on the line according to the unique identification of the test data.
9. The apparatus of claim 7, wherein the intercept module is to:
and dividing the total log segments into the target log segments comprising the target service scene and the service scene upstream of the target service scene and the waste log segments comprising the service scene downstream of the target service scene according to a plurality of service scenes included in the service flow of the server system.
10. The apparatus of claim 7, wherein the generating module comprises:
a synchronization unit for synchronizing the target log segments on line to off-line;
and the generating unit is used for executing the synchronized target log fragments by adopting an off-line database to generate the test data.
11. The apparatus according to any one of claims 7-10, wherein the apparatus further comprises:
the updating segment acquisition module is used for acquiring updating log segments of the off-line database after the testing data are adopted to execute the testing;
the incremental segment acquisition module is used for acquiring an incremental log segment based on the updated log segment and the target log segment of the test data;
and the restoration processing module is used for restoring the test data based on the increment log segment.
12. The apparatus of claim 11, wherein the reduction processing module is configured to:
and sequentially executing the reverse operations of the operations by adopting the offline database according to the reverse sequence of the operations sequentially included in the increment log fragments so as to roll back the update log fragments to the target log fragments, thereby realizing the restoration of the test data.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010617683.0A CN111858506B (en) | 2020-06-30 | 2020-06-30 | Test data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010617683.0A CN111858506B (en) | 2020-06-30 | 2020-06-30 | Test data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111858506A CN111858506A (en) | 2020-10-30 |
| CN111858506B true CN111858506B (en) | 2024-03-22 |
Family
ID=72989192
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010617683.0A Active CN111858506B (en) | 2020-06-30 | 2020-06-30 | Test data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858506B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112433950A (en) * | 2020-11-30 | 2021-03-02 | 天津五八到家货运服务有限公司 | Method for automatically building test environment, electronic equipment and storage medium |
| CN112612664B (en) * | 2020-12-24 | 2024-04-02 | 百度在线网络技术（北京）有限公司 | Electronic equipment testing method and device, electronic equipment and storage medium |
| CN112631920A (en) * | 2020-12-28 | 2021-04-09 | 广州品唯软件有限公司 | Test method, test device, electronic equipment and readable storage medium |
| CN113254308A (en) * | 2021-05-19 | 2021-08-13 | 中国联合网络通信集团有限公司 | Log processing method and device |
| CN114780584B (en) * | 2022-06-22 | 2022-09-02 | 云账户技术(天津)有限公司 | Multi-scene streaming data processing method, system, network equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007264810A (en) * | 2006-03-27 | 2007-10-11 | Fujitsu Ltd | Load test program |
| US8346929B1 (en) * | 2003-08-18 | 2013-01-01 | Oracle America, Inc. | System and method for generating secure Web service architectures using a Web Services security assessment methodology |
| JP2013246512A (en) * | 2012-05-23 | 2013-12-09 | Fujitsu Ltd | Test data generation program, test data generation method and test data generation apparatus |
| CN109101423A (en) * | 2018-08-03 | 2018-12-28 | 挖财网络技术有限公司 | A kind of automated testing method based on server-side log playback |
| CN110362473A (en) * | 2019-05-31 | 2019-10-22 | 口口相传(北京)网络技术有限公司 | Test optimization method and device, storage medium, the terminal of environment |
| CN110554958A (en) * | 2019-08-01 | 2019-12-10 | 北京百度网讯科技有限公司 | Graph database testing method, system, device and storage medium |
| CN111177222A (en) * | 2019-12-26 | 2020-05-19 | 腾讯云计算（北京）有限责任公司 | Model testing method and device, computing equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9348739B2 (en) * | 2014-07-10 | 2016-05-24 | International Business Machines Corporation | Extraction of problem diagnostic knowledge from test cases |
-
2020
- 2020-06-30 CN CN202010617683.0A patent/CN111858506B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8346929B1 (en) * | 2003-08-18 | 2013-01-01 | Oracle America, Inc. | System and method for generating secure Web service architectures using a Web Services security assessment methodology |
| JP2007264810A (en) * | 2006-03-27 | 2007-10-11 | Fujitsu Ltd | Load test program |
| JP2013246512A (en) * | 2012-05-23 | 2013-12-09 | Fujitsu Ltd | Test data generation program, test data generation method and test data generation apparatus |
| CN109101423A (en) * | 2018-08-03 | 2018-12-28 | 挖财网络技术有限公司 | A kind of automated testing method based on server-side log playback |
| CN110362473A (en) * | 2019-05-31 | 2019-10-22 | 口口相传(北京)网络技术有限公司 | Test optimization method and device, storage medium, the terminal of environment |
| CN110554958A (en) * | 2019-08-01 | 2019-12-10 | 北京百度网讯科技有限公司 | Graph database testing method, system, device and storage medium |
| CN111177222A (en) * | 2019-12-26 | 2020-05-19 | 腾讯云计算（北京）有限责任公司 | Model testing method and device, computing equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111858506A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111858506B (en) | Test data processing method and device, electronic equipment and storage medium | |
| CN111913998B (en) | Data processing method, device, equipment and storage medium | |
| CN110619002A (en) | Data processing method, device and storage medium | |
| CN111984476A (en) | Test method and device | |
| CN111858628A (en) | Database-based management method, database-based management platform, electronic device and storage medium | |
| CN112015468A (en) | Interface document processing method and device, electronic equipment and storage medium | |
| CN111158666A (en) | Entity normalization processing method, device, equipment and storage medium | |
| CN110750419B (en) | Offline task processing method and device, electronic equipment and storage medium | |
| CN112435313A (en) | Method and device for playing frame animation, electronic equipment and readable storage medium | |
| CN111339187A (en) | Data processing method, device, equipment and storage medium based on intelligent contract | |
| CN111767149B (en) | Scheduling method, device, equipment and storage equipment | |
| CN111782357B (en) | Label control method and device, electronic equipment and readable storage medium | |
| WO2023169193A1 (en) | Method and device for generating smart contract | |
| CN112084263B (en) | Data synchronization method, device, equipment and storage medium | |
| US20210209075A1 (en) | Data Thinning Method, Data Thinning Apparatus, Electronic Device, and Storage Medium | |
| CN111782542A (en) | Test method, device, equipment, system and computer storage medium | |
| CN113542802A (en) | Video transition method and device | |
| JP2021118004A (en) | Graph calculation processing method, device, electronic equipment and storage medium | |
| CN112583897A (en) | Data destination creating method and device of rule engine, electronic equipment and storage medium | |
| CN111818046A (en) | Method, device, equipment and storage medium for interacting information | |
| CN111767989A (en) | Neural network training method and device | |
| CN111177558A (en) | Channel service construction method and device | |
| CN111639032B (en) | Method and apparatus for testing applications | |
| CN113132126B (en) | Voting weight determination method and related equipment | |
| CN112437152B (en) | Crash processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |