CN117093494A

CN117093494A - Test processing method, device, equipment and storage medium

Info

Publication number: CN117093494A
Application number: CN202311114165.7A
Authority: CN
Inventors: 童璐
Original assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Current assignee: China Construction Bank Corp; CCB Finetech Co Ltd
Priority date: 2023-08-31
Filing date: 2023-08-31
Publication date: 2023-11-21

Abstract

The disclosure provides a test processing method, a device, equipment and a storage medium, which can be applied to the field of computer technology or financial science and technology. The method comprises the following steps: responding to a test request of a target object, and calling a data acquisition interface associated with the test dimension according to the test dimension carried in the test request; dynamically acquiring production data associated with the test dimension from a production system through a data acquisition interface to obtain a dynamic production data set; processing the production data in the dynamic production data set to obtain a dynamic threshold value associated with the test dimension; generating a test case associated with the test dimension based on the dynamic threshold and a case template associated with the test dimension; and evaluating between the dynamic threshold value and the test value obtained by executing the test case to obtain an evaluation result.

Description

Test processing method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technology or financial technology, and in particular, to a test processing method, apparatus, device, storage medium, and program product.

Background

With the development of computer technology, more and more system software is utilized in the computer technology, and the structure of the system software is also more and more complex. To enable high availability of system software, it is generally necessary to test the system software and adjust the system software according to the test result.

In the process of implementing the inventive concept of the present disclosure, the inventor found that the following problems generally exist in the related art: in the process of testing the system software, although automatic testing tools can be used for testing the system software, the testing tools are generally used as auxiliary tools for completing the testing of the system software, and the main system software testing operation also needs to rely on manual participation. Therefore, the existing method for testing the system software has the problems of long time consumption, low accuracy and low intelligent degree.

Disclosure of Invention

In view of the foregoing, the present disclosure provides test processing methods, apparatuses, devices, storage media, and program products.

One aspect of the present disclosure provides a test processing method, including: responding to a test request of a target object, and calling a data acquisition interface associated with the test dimension according to the test dimension carried in the test request; dynamically acquiring production data associated with the test dimension from a production system through the data acquisition interface to obtain a dynamic production data set; processing the production data in the dynamic production data set to obtain a dynamic threshold value associated with the test dimension; generating a test case associated with the test dimension based on the dynamic threshold and a case template associated with the test dimension; and evaluating between the dynamic threshold value and a test value obtained by executing the test case to obtain an evaluation result.

According to an embodiment of the present disclosure, the test dimension is configured with a dimension identifier; processing the production data in the dynamic production data set to obtain a dynamic threshold associated with the test dimension, including: preprocessing the production data in the dynamic production data set to obtain target production data; calling a preset threshold value generation strategy from a database based on the dimension identification of the test dimension; and processing the target production data by using the threshold generation strategy to obtain a dynamic threshold associated with the test dimension.

According to an embodiment of the present disclosure, the threshold generation policy includes a preset float percentage; processing the target production data using the threshold generation strategy to obtain a dynamic threshold associated with the test dimension, including: generating a floating value of the target production data based on the target production data and the preset floating percentage; a dynamic threshold associated with the test dimension is generated based on the target production data and the float value of the target production data.

According to an embodiment of the present disclosure, the preprocessing the production data in the dynamic production data set to obtain target production data includes: determining a data type associated with the test dimension, wherein the data type comprises a mean type and a maximum type; carrying out averaging processing on the production data in the dynamic production data set to obtain the target production data under the condition that the data type associated with the test dimension is the average value type, wherein the target production data is used for representing the average value of all production data in the dynamic production data set; under the condition that the data type associated with the test dimension is the highest value type, sorting the production data in the dynamic production data set according to a preset sorting strategy to obtain a sorting result; and generating the target production data according to the production data arranged at the preset position in the sequencing result, wherein the target production data is used for representing the maximum value of all the production data or the minimum value of the production data in the dynamic generation data set.

According to an embodiment of the present disclosure, the above method further includes: obtaining a test value obtained by executing the test case; and checking the test value to obtain a checking result.

According to an embodiment of the present disclosure, the verifying the test value to obtain a verification result includes: constructing a verification condition according to the test dimension and the dynamic threshold; checking the test value by using the checking condition; generating a verification result passing verification under the condition that the test value is matched with the verification condition; and under the condition that the test value does not match the verification condition, generating and sending alarm information that the verification is not passed.

According to an embodiment of the present disclosure, the evaluating between the dynamic threshold and a test value obtained by executing the test case to obtain an evaluation result includes: outputting a difference value between the dynamic threshold value and the test value by using a calculation engine; and generating the evaluation result according to the difference value.

According to an embodiment of the present disclosure, the above method further includes: after the evaluation result is generated according to the difference value, a visualization component is called; and displaying the evaluation result by using the visualization component.

Another aspect of the present disclosure also provides a test processing apparatus, including: the first calling module is used for responding to a test request of a target object and calling a data acquisition interface associated with the test dimension according to the test dimension carried in the test request; the acquisition module is used for dynamically acquiring production data associated with the test dimension from the production system through the data acquisition interface to obtain a dynamic production data set; the processing module is used for processing the production data in the dynamic production data set to obtain a dynamic threshold value associated with the test dimension; the generation module is used for generating a test case associated with the test dimension based on the dynamic threshold value and a case template associated with the test dimension; and the evaluation module is used for evaluating between the dynamic threshold value and the test value obtained by executing the test case to obtain an evaluation result.

Another aspect of the present disclosure also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the test processing method.

Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the test processing method described above.

Another aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the test processing method described above.

According to the test processing method, the device, the equipment, the storage medium and the program product provided by the embodiment of the disclosure, a data acquisition interface is called according to the test dimension carried in the test request by responding to the test request of the target object; dynamically acquiring production data from a production system through the data acquisition interface to obtain a dynamic production data set; processing the dynamic production data set to obtain a threshold value; generating a test case based on the threshold and the case template; and dynamically evaluating between the dynamic threshold value and the obtained test value of the execution test case to obtain an evaluation result. Because the whole process can be automatically executed in the process of testing and evaluating the target object, the time consumption can be reduced and the intelligent degree can be improved; in addition, the embodiment of the disclosure collects data from the production system, and the data of the production system is closer to an actual application scene, and the data is changed in real time according to actual conditions, so that the collected data is dynamic, the obtained threshold value is changed dynamically, and then the evaluation result between the test value and the threshold value is also dynamic, and further the test accuracy of the target object can be improved. The test processing method provided by the embodiment of the disclosure at least partially solves the problems of long time consumption, low accuracy and low intelligent degree of the related technology, thereby achieving the technical effects of reducing time consumption, improving test accuracy and improving intelligent degree.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of a test processing method, apparatus, device, storage medium and program product according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a test processing method according to an embodiment of the disclosure;

FIG. 3 schematically illustrates a flow chart of a test processing method according to another embodiment of the present disclosure;

FIG. 4 schematically illustrates a block diagram of a test handler according to an embodiment of the present disclosure; and

fig. 5 schematically illustrates a block diagram of an electronic device adapted to implement a test processing method according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the technical scheme of the invention, the related user information (including but not limited to user personal information, user image information, user equipment information, such as position information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) are information and data authorized by a user or fully authorized by all parties, and the processing of the related data such as collection, storage, use, processing, transmission, provision, disclosure, application and the like are all conducted according to the related laws and regulations and standards of related countries and regions, necessary security measures are adopted, no prejudice to the public welfare is provided, and corresponding operation inlets are provided for the user to select authorization or rejection.

At present, the process of testing the system software generally needs to be completed manually, which results in long time consumption, low efficiency of testing and difficulty in meeting the current rapid iteration requirement. In addition, the requirement assessment, scheme design and the like involved in the test process require personnel with higher technical work, and further result in higher personnel thresholds required for completing the test. Meanwhile, the error rate of manually completing the test is also higher, and the accuracy of the test is reduced.

In view of the foregoing, embodiments of the present disclosure provide a test processing method, apparatus, device, storage medium, and program product for reducing time consumption, improving test accuracy, and improving degree of intelligence. Specifically, the method comprises the following steps: responding to a test request of a target object, and calling a data acquisition interface associated with the test dimension according to the test dimension carried in the test request; dynamically acquiring production data associated with the test dimension from a production system through a data acquisition interface to obtain a dynamic production data set; processing the production data in the dynamic production data set to obtain a dynamic threshold value associated with the test dimension; generating a test case associated with the test dimension based on the dynamic threshold and a case template associated with the test dimension; and evaluating between the dynamic threshold value and the test value obtained by executing the test case to obtain an evaluation result.

It should be noted that, the test processing method and apparatus determined in the embodiments of the present disclosure may be used in the field of computer technology or the field of financial technology, and may also be used in any field other than the field of computer technology or the field of financial technology, where the application field of the determined test processing method and apparatus is not limited in the embodiments of the present disclosure.

Fig. 1 schematically illustrates an application scenario diagram of a test processing method, apparatus, device, storage medium and program product according to an embodiment of the present disclosure.

As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 10l, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 through the network 104 using at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103 to receive the evaluation result or to send a test request for the target object, or the like. Various communication client applications, such as a test class application, a financial class application, a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, the third terminal device 103.

The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (merely an example) providing support for test requests for a target object transmitted by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the request, and feed back the processing result (for example, the evaluation result obtained according to the user request, the acquired or generated web page, information, or data) to the terminal device.

It should be noted that the test processing method provided in the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the test processing device provided by the embodiments of the present disclosure may be generally disposed in the server 105. The test processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the test processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

The test processing method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 3 based on the scenario described in fig. 1.

Fig. 2 schematically illustrates a flow chart of a test processing method according to an embodiment of the disclosure.

As shown in fig. 2, the test process of this embodiment includes operations S210 to S250.

In operation S210, in response to the test request for the target object, a data collection interface associated with the test dimension is invoked according to the test dimension carried in the test request.

In operation S220, production data associated with the test dimension is dynamically collected from the production system via the data collection interface, resulting in a dynamic production dataset.

In operation S230, the production data in the dynamic production data set is processed to obtain a dynamic threshold associated with the test dimension.

In operation S240, a test case associated with the test dimension is generated based on the dynamic threshold and the case template associated with the test dimension.

In operation S250, an evaluation is performed between the dynamic threshold value and the test value obtained by executing the test case, to obtain an evaluation result.

According to embodiments of the present disclosure, the target object may include system software to be commissioned or commissioned. The test request may be for the target object to conduct a non-functional test. Nonfunctional testing is a type of software testing that is used to check nonfunctional aspects (e.g., performance, availability, reliability, etc.) of a software application. The test types of the non-functional test may include at least one of: performance testing (e.g., at least one of capacity testing, response time testing, throughput testing, concurrent number testing, etc.), reliability testing, security testing, ease of use testing, etc. The flow of the nonfunctional test can be nonfunctional requirement assessment, nonfunctional scheme generation, nonfunctional test scene and case generation, nonfunctional script writing, nonfunctional test case execution, nonfunctional result data collection and nonfunctional result assessment.

According to embodiments of the present disclosure, the test dimension may be related to a test type, such as a performance dimension (which may be subdivided into a response time dimension, a throughput dimension, a capacity dimension, a concurrency dimension, an extensibility dimension, etc.), a reliability dimension, a security dimension, an ease of use dimension, etc. In general, the test dimension may characterize the purpose of the current test request as well as the test type.

According to an embodiment of the present disclosure, the data acquisition interface is used for acquiring relevant data. Each test dimension may correspond to a plurality of data acquisition interfaces, and one data acquisition interface may also correspond to a plurality of test dimensions. Specifically, the test dimension and the data acquisition interface can be in a pre-binding relationship, binding identifiers are configured for the test dimension and the data acquisition interface, and when the data acquisition interface is called, the data acquisition interface can be called according to the dimension identifier of the test dimension and the binding identifiers.

According to embodiments of the present disclosure, a production system may refer to an information system that supports the daily business processing of an organization, which may include production data, production data processing systems, production networks, and the like. Generally, the data on the production system is obtained by processing actual business and is closer to the actual application scene.

In accordance with embodiments of the present disclosure, with respect to the process of dynamic acquisition, it can be understood that since there is a variation in the actual business, there is a variation in the production data obtained by processing the actual business, and thus the data in the dynamic production data set obtained by each acquisition is different, so it is called dynamic acquisition.

According to embodiments of the present disclosure, the dynamic threshold may be used as a reference value upon which to base the evaluation. The process of processing production data in the dynamic production data set to obtain the dynamic threshold value may be implemented by means of a function.

According to an embodiment of the present disclosure, a case template is used to generate test cases. The test cases may be obtained by content stitching using dynamic thresholds and case templates, or by extracting relevant information from dynamic production datasets or/and test requests using template fields on case templates, and content stitching. For example, the template fields may include dynamic thresholds, interfaces to be tested, number of concurrent loads to be required, etc., through which at least one of dynamic thresholds, production data sets, test requests may be content extracted and spliced to obtain test cases. It will be appreciated that the case templates, template fields, and files used to extract information (e.g., dynamic thresholds, production data sets, test requests, etc.) may all be adapted as desired.

Illustratively, one test case is a single transaction benchmark that tests an interface: single concurrence continues the pressure measurement for 15 minutes, and test results are obtained. Another test case is to test a single transaction load of an interface: and simultaneously concurrence with a loads, wherein a is a positive integer, and continuously performing pressure measurement for 15 minutes, and obtaining a test result. There may also be mixed transaction test cases, batch test cases, etc. for multiple interfaces.

According to embodiments of the present disclosure, an evaluation is performed between a dynamic threshold and a test value obtained by executing a test case, and the obtained evaluation result may include whether the test value satisfies a condition constructed based on the dynamic threshold, whether the test value characterizes a test passing, a difference between the test value and the dynamic threshold, and the like.

According to embodiments of the present disclosure, the data acquisition interface may be provided on an NFR (Non Functional Requirements, non-functional requirements) model. The NFR model is a framework for abstracting and categorizing non-functional requirements for assisting in analyzing and managing non-functional test requirements. In constructing the NFR model, it is possible to construct according to the NFR framework, the constituent elements of which mainly include: NFR coarse dimension category, NFR fine dimension category, NFR index, NFR service level, NFR demand relationship, and NFR change process.

The NFR coarse dimension class may be to classify non-functional test requirements according to their attributes, such as performance dimension class, reliability dimension class, security dimension class, usability dimension class, portability dimension class, etc. This may be a process of high-level classification and abstraction of non-functional requirements.

The NFR fine-dimension categories may be further subdivided for each dimension category. For example, performance dimension classes may include response time NFR, throughput NFR, extensibility NFR, and the like. This may be a process that refines the non-functional requirements.

Determining NFR metrics may be further formulating quantitative or compliant metrics for each subdivided NFR for measuring the degree of implementation of the corresponding test requirements. For example, response time < 1 second, throughput > 5000tps, etc. The metrics may be customized for different systems.

The NFR service level may be partitioned based on a range of index values to meet different levels of test demand levels for clarifying the priority and importance of test demands during design and development. For example, a response time of level 1 < 0.5 seconds, a response time of level 2 < 1 second, a response time of level 3 < 3 seconds, etc.

NFR demand relationships, which identify the influence or dependency of different NFRs, directly or indirectly, for maintaining overall consistency and prioritization in demand analysis and implementation. For example, security requirements can affect the performance of the system.

NFR change procedures for managing the process of continuously changing or optimizing non-functional test requirements in system evolution. This requires attention to the implementation of the original NFR, verifying the dependency of new test requirements on existing test requirements, and formulating a transition scheme.

By performing the above-described processes of determining the coarse dimension category of NFR, determining the fine dimension category of NFR, determining the NFR index, determining the level of NFR service, determining the relationship of NFR requirements, determining the NFR change process, etc. for non-functional test requirements, an NFR model can be constructed, and production data associated with the non-functional test requirements is collected by utilizing a data collection interface configured on the NFR model. Each non-functional test requirement may correspond to at least one NFR model, and one NFR model may correspond to at least one non-functional test requirement.

The NFR model is constructed to facilitate in-depth knowledge of the logic of non-functional attributes from an overall perspective. The NFR model may guide demand analysis, architecture design, and test case formulation. And an inherent relation between different test requirements is established through the model, so that contradiction and conflict between the test requirements are avoided. The NFR model also helps to track changes in demand, providing basis and compliance for continued evolution of the system. In general, the NFR model is utilized to participate in non-functional testing, which is beneficial to realizing a high-quality and sustainable evolution software system.

According to embodiments of the present disclosure, the process of collecting production data using a data collection interface on NFR may be collected by buried site. The process of collecting data may include at least one of the following.

API (Application Programming Interface, application program interface) log collection: the application system detects index data about monitoring service of the pre-buried point by calling the API, the acquisition system acquires API log information, analyzes the monitoring data and acquires production data from the monitoring data. The acquisition mode is relatively transparent, decoupling can be realized, and the resource utilization rate is high.

And (3) collecting a service database: and directly accessing a service database to inquire service data, and obtaining production data. The acquisition mode is simple and convenient to expand, and the acquired production data is accurate.

And (3) middleware collection: the application system issues business event or data change information to the information queue, and the acquisition system subscribes to the information queue of the queue and extracts relevant business production data. The acquisition mode has high utilization rate of resources and high comprehensive performance.

And (3) archiving and collecting a database: the configuration business database files the uncommitted transactions regularly, and the data with changes are collected from the archive table at fixed points to obtain production data. The integrity of the production data acquired by the acquisition mode is high.

Database trigger acquisition: and configuring an INSERT/UPDATE/DELETE trigger in the service table, publishing a trigger program to a message queue, and subscribing the message by the acquisition system for acquisition to obtain production data. The data acquisition mode has high data production efficiency and better timeliness.

Middleware transaction tracking: the application system publishes the transaction information to the information queue, and the acquisition system subscribes the information to analyze the transaction context for acquisition, so as to obtain production data. The acquisition mode can ensure the integrity and timeliness of production data.

Full flow replication and acquisition: and (3) copying a binary log (Binlog) of the database to a message queue in real time by using a flow copying tool (such as Canal), and acquiring real-time data change events by subscribing a message analysis log by an acquisition system to obtain production data. The production data obtained by the acquisition mode has high accuracy and high integrity.

According to an embodiment of the present disclosure, after the production data is collected, the production data may be processed to obtain a dynamic threshold, and specifically, the process of obtaining the dynamic threshold may include the following operations: preprocessing production data in the dynamic production data set to obtain target production data; calling a preset threshold value generation strategy from a database based on the dimension identification of the test dimension; and processing the target production data by using a threshold generation strategy to obtain a dynamic threshold associated with the test dimension. Wherein the dimension identification is configured for the test dimension.

According to an embodiment of the present disclosure, preprocessing production data in a dynamic production dataset to obtain target production data may include the following operations: determining a data type associated with the test dimension, wherein the data type comprises a mean type and a maximum type; under the condition that the data type associated with the test dimension is the average value type, carrying out averaging treatment on the production data in the dynamic production data set to obtain target production data, wherein the target production data is used for representing the average value of all production data in the dynamic production data set; under the condition that the data type associated with the test dimension is the most value type, sorting the production data in the dynamic production data set according to a preset sorting strategy to obtain a sorting result; and generating target production data according to the production data arranged at the preset position in the sorting result, wherein the target production data is used for representing the maximum value or the minimum value of all the production data in the dynamic generation data set.

According to embodiments of the present disclosure, the data types associated with the test dimension may be determined according to test requirements, test attributes, and business requirements. For example, the test requirement of an embodiment is to test the average response time of an interface, in which case the data type is the mean type. For another example, the test requirements of an embodiment are to test the maximum response time, the maximum concurrency number, the minimum response time, etc. of an interface, in which case the data type is the most significant type.

According to an embodiment of the present disclosure, the averaging process is used to obtain an average value of the production data, and specifically, the calculation engine may be used to perform average calculation on the production data to obtain the target production data.

According to the embodiment of the disclosure, the preset ordering strategy can be adaptively adjusted according to actual needs. For example, in the case where the maximum response time is required to be obtained, the production data may be sorted in order of the response time from large to small or from small to large to obtain a sorting result, and the data arranged at a preset position (for example, arranged at the top or bottom) is selected from the sorting result as the target production data.

According to an embodiment of the present disclosure, the threshold generation policy includes a preset float percentage; processing the target production data using a threshold generation strategy, deriving a dynamic threshold associated with the test dimension may include the operations of: generating a floating value of the target production data based on the target production data and a preset floating percentage; a dynamic threshold associated with the test dimension is generated based on the target production data and the float value of the target production data.

According to embodiments of the present disclosure, the threshold generation policies may be stored in a database, and each test dimension may be associated with at least one threshold generation policy. The threshold generation policy may be stored using a dimension identification of the test dimension at the time of storage. The threshold generation strategy may be as shown in equation (1).

Dynamic threshold=target production data× (1+x) =target production data+target production data×x (1)

Wherein x can be a preset floating percentage, and the floating percentage can be adaptively adjusted according to actual needs. The target production data x may represent a floating value of the target production data.

For example, when the test dimension is a performance dimension, the dynamic threshold of an interface with respect to the maximum response time under the test performance dimension may be [ the maximum response time of the interface in the production environment x (1+x) ]; for another example, in the test performance dimension, the dynamic threshold for the average response time for an interface may be [ the average response time of the interface in the production environment x (1+x) ].

According to the embodiment of the disclosure, the dynamic threshold is generated by using the threshold generation strategy, and because the threshold is obtained according to the production data under different conditions, different thresholds can be obtained under different production environments, and further, when the test value obtained by using the threshold to check the execution test case is used, the obtained check result is also dynamically changed, so that the expected result of the test value is dynamically changed under different production environments, and the test result is more close to the actual production environment, and the test accuracy is improved.

According to an embodiment of the present disclosure, the test processing method may further include the operations of: obtaining a test value obtained by executing a test case; and checking the test value to obtain a checking result.

According to an embodiment of the present disclosure, the process of verifying the test value to obtain the verification result may include the following operations: constructing a verification condition according to the test dimension and the dynamic threshold value; checking the test value by using a checking condition; under the condition that the test value is matched with the verification condition, generating a verification result passing verification; and under the condition that the test value does not match the verification condition, generating and sending alarm information that the verification is not passed.

According to an embodiment of the present disclosure, a verification condition is used to verify a test value. The verification condition constructed according to the test dimension and the dynamic threshold value may be greater than or equal to the dynamic threshold value, or less than or equal to the dynamic threshold value, or a dynamic range determined based on the dynamic threshold value. The verification conditions can be specifically adjusted adaptively according to actual needs.

For example, if the service requirement needs to limit the maximum response time, if the test dimension is the response time of an interface and the dynamic threshold is the maximum response time obtained by the formula (1), the verification condition may be that if the maximum response time obtained as a test value after the interface is pressed for a certain time is smaller than the dynamic threshold, the test is passed; and if the dynamic threshold value is greater than or equal to the dynamic threshold value, the test is not passed. For another example, if the service requirement needs to limit the average response time, if the test dimension is the average response time of an interface and the dynamic threshold is the average response time obtained by the formula (1), the verification condition may be that if the average response time obtained as the test value after the interface is pressed for a certain time is less than or equal to the dynamic threshold, the test is passed; greater than the dynamic threshold, the test fails.

According to an embodiment of the present disclosure, operation S250 may include the following operations: outputting a difference value between the dynamic threshold value and the test value by using the calculation engine; and generating an evaluation result according to the difference value.

According to an embodiment of the present disclosure, after obtaining the evaluation result, the following operations may be included: invoking a visualization component; and displaying the evaluation result by using a visualization component.

According to embodiments of the present disclosure, the evaluation result may be a curve or a generated table file drawn according to a difference value between the dynamic threshold value and the test value. The visualization component can include conventional chart presentation class tools, curve presentation class tools, and the like. By displaying the evaluation result by using the visualization component, the method can help related personnel to further analyze the problems existing in the test result and make decisions according to the current test result.

Fig. 3 schematically illustrates a flow chart of a test processing method according to another embodiment of the present disclosure.

As shown in fig. 3, the test processing method of another embodiment may include operations S310 to S350.

In operation S310, an NFR model is constructed according to the NFR framework.

According to the NFR framework, the process of constructing the NFR model may include determining a dimension category, a dimension attribute, an attribute relationship, and the like of the non-functional requirement, and specifically, the processes of determining a coarse dimension category of the NFR, determining a fine dimension category of the NFR, determining an NFR index, determining an NFR service level, determining an NFR requirement relationship, determining an NFR change process, and the like, which are described above, may be performed on the non-functional test requirement, and will not be repeated herein.

In operation S320, production data for different test dimensions is collected according to the NFR model.

Such as interface response time, throughput, transaction volume, demand characteristics, importance, etc. of the performance dimension. The information collecting process may refer to the process of collecting information at the information embedding point described above, and will not be described herein.

In operation S330, dynamic thresholds associated with the test dimensions are generated using a threshold generation policy.

For example, different threshold generation strategies may be generated from different test dimensions, different dimension attributes and in combination with service requirements, and specific reference may be made to the related content of the above-described formula (1), which is not described herein.

In operation S340, a dynamic test case is automatically generated according to the dynamic threshold and the case template.

For example, according to the business requirement, the conventional non-functional case template adopts the collected generated data as source data, obtains a dynamic threshold value based on the formula (1), and modifies part of the content (such as modifying an interface, the dynamic threshold value, a test result and the like) on the non-functional case template to obtain a test case. Or extracting relevant information from the dynamic production data set or/and the test request according to the template field, and performing content splicing to obtain a test case.

Dynamic generation of cases means that the data captured from production each time is dynamically changed, so that the thresholds derived from the threshold production strategy are dynamically changed, i.e., the expected outcome for the test case is dynamically changed.

In operation S350, the test case is executed to obtain a test value, and an evaluation is performed between the test value and the dynamic threshold.

For example, based on the dynamic threshold and the test value obtained by executing the test case, the tool is used to automatically judge whether the test passes or fails, the calculation engine is used to calculate the difference between the test value and the dynamic threshold, a curve is drawn or a table file is generated according to the difference, and the curve or the table file obtained according to the difference is displayed by using the visualization tool.

The test processing method provided by the embodiment of the disclosure can capture related data from production according to the test model, automatically generate a non-functional test scene and case, automatically execute the non-functional test, and automatically judge the non-functional test result according to the rule by the system. The whole process can complete tasks through automatic script writing except for building a test model, so that time is saved and personnel thresholds required for completing nonfunctional tests are reduced.

According to the test processing method provided by the embodiment of the disclosure, data can be grabbed from the production system, the nonfunctional test can be accurately completed according to the production hot spot data and the production real data, the nonfunctional test scene is highly covered, and the accuracy and the efficiency of the nonfunctional test can be improved.

It should be noted that, unless there is an execution sequence between different operations or an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may be different, and multiple operations may also be executed simultaneously in the embodiment of the disclosure.

Based on the test processing method, the disclosure also provides a test processing device. The device will be described in detail below in connection with fig. 4.

Fig. 4 schematically shows a block diagram of a test processing device according to an embodiment of the present disclosure.

As shown in fig. 4, the test processing apparatus 400 of this embodiment includes a first calling module 410, an acquisition module 420, a processing module 430, a generation module 440, and an evaluation module 450.

The first invoking module 410 is configured to invoke a data acquisition interface associated with a test dimension according to the test dimension carried in the test request in response to the test request for the target object.

The collection module 420 is configured to dynamically collect production data associated with the test dimension from the production system through the data collection interface, so as to obtain a dynamic production data set.

The processing module 430 is configured to process the production data in the dynamic production data set to obtain a dynamic threshold associated with the test dimension.

A generation module 440 for generating test cases associated with the test dimension based on the dynamic threshold and the case templates associated with the test dimension.

And the evaluation module 450 is used for evaluating between the dynamic threshold value and the test value obtained by executing the test case to obtain an evaluation result.

According to an embodiment of the present disclosure, the processing module may include a first processing unit, a calling unit, and a second processing unit.

And the first processing unit is used for preprocessing the production data in the dynamic production data set to obtain target production data.

And the calling unit is used for calling a preset threshold value generation strategy from the database based on the dimension identification of the test dimension.

And the second processing unit is used for processing the target production data by utilizing a threshold generation strategy to obtain a dynamic threshold associated with the test dimension.

According to an embodiment of the present disclosure, the second processing unit may include a first generating subunit, and a second generating subunit.

The first generation subunit is used for generating a floating value of the target production data based on the target production data and a preset floating percentage.

A second generation subunit for generating a dynamic threshold associated with the test dimension based on the target production data and the floating value of the target production data.

According to an embodiment of the present disclosure, the first processing unit may include a determining subunit, a processing subunit, a sorting subunit, and a third generating subunit.

And a determining subunit configured to determine a data type associated with the test dimension, wherein the data type includes a mean type and a maximum type.

And the processing subunit is used for carrying out averaging processing on the production data in the dynamic production data set under the condition that the data type associated with the test dimension is the mean value type to obtain target production data, wherein the target production data is used for representing the mean value of all the production data in the dynamic production data set.

And the sorting subunit is used for sorting the production data in the dynamic production data set according to a preset sorting strategy under the condition that the data type associated with the test dimension is the most value type, so as to obtain a sorting result.

And the third generation subunit is used for generating target production data according to the production data arranged at the preset position in the sequencing result, wherein the target production data is used for representing the maximum value or the minimum value of all the production data in the dynamic generation data set.

According to an embodiment of the disclosure, the test processing device may further include an acquisition module, and a verification module.

And the acquisition module is used for acquiring the test value obtained by executing the test case.

And the verification module is used for verifying the test value to obtain a verification result.

According to an embodiment of the present disclosure, the verification module may include a construction unit, a verification unit, a first generation unit, and a second generation unit.

And the construction unit is used for constructing the verification condition according to the test dimension and the dynamic threshold value.

And the verification unit is used for verifying the test value by utilizing the verification condition.

The first generation unit is used for generating a verification result passing the verification under the condition that the test value is matched with the verification condition.

And the second generation unit is used for generating and sending alarm information which does not pass the verification under the condition that the test value does not match the verification condition.

According to an embodiment of the present disclosure, the evaluation module may include an output unit, and a third generation unit.

And the output unit is used for outputting a difference value between the dynamic threshold value and the test value by using the calculation engine.

And the third generation unit is used for generating an evaluation result according to the difference value.

According to an embodiment of the disclosure, the test processing device may further include a second calling module, and a presentation module.

And the second calling module is used for calling the visualization component after the evaluation result is generated according to the difference value.

And the display module is used for displaying the evaluation result by utilizing the visualization component.

According to embodiments of the present disclosure, any of the plurality of modules of the first invoking module 410, the collecting module 420, the processing module 430, the generating module 440, and the evaluating module 450 may be combined in one module to be implemented, or any of the plurality of modules may be split into a plurality of modules. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the first invocation module 410, the acquisition module 420, the processing module 430, the generation module 440, and the evaluation module 450 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the first invocation module 410, the acquisition module 420, the processing module 430, the generation module 440, and the evaluation module 450 may be at least partially implemented as a computer program module, which, when executed, may perform the respective functions.

It should be noted that, in the embodiment of the present disclosure, the test processing device portion corresponds to the test processing method portion in the embodiment of the present disclosure, and the description of the test processing device portion specifically refers to the test processing method portion and is not described herein.

As shown in fig. 5, an electronic device 500 according to an embodiment of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 501 may also include on-board memory for caching purposes. The processor 501 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.

In the RAM 503, various programs and data required for the operation of the electronic apparatus 500 are stored. The processor 501, ROM 502, and RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 502 and/or the RAM 503. Note that the program may be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 500 may also include an input/output (I/O) interface 505, the input/output (I/O) interface 505 also being connected to the bus 504. The electronic device 500 may also include one or more of the following components connected to an input/output (I/O) interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to an input/output (I/O) interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 502 and/or RAM 503 and/or one or more memories other than ROM 502 and RAM 503 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the test processing methods provided by embodiments of the present disclosure.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 501. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, and/or installed from a removable medium 511 via the communication portion 509. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 501. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A test processing method, comprising:

responding to a test request of a target object, and calling a data acquisition interface associated with a test dimension according to the test dimension carried in the test request;

Dynamically acquiring production data associated with the test dimension from a production system through the data acquisition interface to obtain a dynamic production data set;

processing the production data in the dynamic production data set to obtain a dynamic threshold associated with the test dimension;

generating a test case associated with the test dimension based on the dynamic threshold and a case template associated with the test dimension;

and evaluating between the dynamic threshold value and a test value obtained by executing the test case to obtain an evaluation result.

2. The method of claim 1, wherein the test dimension is configured with a dimension identification;

the processing the production data in the dynamic production data set to obtain a dynamic threshold associated with the test dimension includes:

preprocessing the production data in the dynamic production data set to obtain target production data;

calling a preset threshold value generation strategy from a database based on the dimension identification of the test dimension;

and processing the target production data by utilizing the threshold generation strategy to obtain a dynamic threshold associated with the test dimension.

3. The method of claim 2, wherein the threshold generation strategy comprises a preset percentage of float-up;

the processing the target production data by using the threshold generation strategy to obtain a dynamic threshold associated with the test dimension comprises the following steps:

generating a floating value of the target production data based on the target production data and the preset floating percentage;

a dynamic threshold associated with the test dimension is generated based on the target production data and a float value of the target production data.

4. The method of claim 2, wherein the preprocessing the production data in the dynamic production dataset to obtain target production data comprises:

determining a data type associated with the test dimension, wherein the data type comprises a mean type and a maximum type;

carrying out averaging treatment on the production data in the dynamic production data set under the condition that the data type associated with the test dimension is the mean value type to obtain the target production data, wherein the target production data is used for representing the mean value of all production data in the dynamic production data set;

Under the condition that the data type associated with the test dimension is the most value type, sorting the production data in the dynamic production data set according to a preset sorting strategy to obtain a sorting result;

and generating the target production data according to the production data arranged at the preset position in the sorting result, wherein the target production data is used for representing the maximum value of all the production data or the minimum value of the production data in the dynamic generation data set.

5. The method of claim 1, further comprising:

obtaining a test value obtained by executing the test case;

and checking the test value to obtain a checking result.

6. The method of claim 5, wherein the verifying the test value to obtain a verification result comprises:

constructing a verification condition according to the test dimension and the dynamic threshold;

checking the test value by using the checking condition;

generating a verification result passing verification under the condition that the test value is matched with the verification condition;

and generating and sending alarm information which is not checked under the condition that the test value is not matched with the check condition.

7. The method of claim 1, wherein the evaluating between the dynamic threshold and the test value resulting from performing the test case results in an evaluation result comprising:

outputting a difference value between the dynamic threshold value and the test value using a calculation engine;

and generating the evaluation result according to the difference value.

8. The method of claim 7, further comprising:

after the evaluation result is generated according to the difference value, a visualization component is called;

and displaying the evaluation result by using the visualization component.

9. A test handler comprising:

the first calling module is used for responding to a test request of a target object and calling a data acquisition interface associated with a test dimension according to the test dimension carried in the test request;

the acquisition module is used for dynamically acquiring production data associated with the test dimension from a production system through the data acquisition interface to obtain a dynamic production data set;

the processing module is used for processing the production data in the dynamic production data set to obtain a dynamic threshold value associated with the test dimension;

a generation module for generating a test case associated with the test dimension based on the dynamic threshold and a case template associated with the test dimension;

And the evaluation module is used for evaluating between the dynamic threshold value and the test value obtained by executing the test case to obtain an evaluation result.

10. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-8.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.