CN113778842A - Fault-tolerant test method and device, electronic equipment and storage medium - Google Patents

Fault-tolerant test method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN113778842A
CN113778842A CN202010865406.1A CN202010865406A CN113778842A CN 113778842 A CN113778842 A CN 113778842A CN 202010865406 A CN202010865406 A CN 202010865406A CN 113778842 A CN113778842 A CN 113778842A
Authority
CN
China
Prior art keywords
test
application
online
result
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010865406.1A
Other languages
Chinese (zh)
Inventor
王前勇
邱文静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202010865406.1A priority Critical patent/CN113778842A/en
Publication of CN113778842A publication Critical patent/CN113778842A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a fault tolerance test method, a fault tolerance test device, electronic equipment and a storage medium, and relates to the technical field of computers. One embodiment of the method comprises: judging whether the application to be online is a backbone application or not based on the attribute information of the application to be online; if the application to be online is the backbone application, screening a test case template corresponding to the type according to the type of the application to be online; determining a test parameter corresponding to the application to be online based on the test function of the application to be online; and executing fault-tolerant test on the application to be online based on the test case template and the test parameters, and obtaining a test result. The implementation method can solve the problems that when the number of clusters is large and the number of service applications is large, the workload of a test task is large, the test efficiency is low, and the online time of newly developed applications is prolonged.

Description

Fault-tolerant test method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a fault tolerance testing method and apparatus, an electronic device, and a storage medium.
Background
For various applications, they are often developed based on new requirements in order to optimize the application. However, newly developed applications need to be tested, and the applications can be on-line after the tests are completed. The current test mode is generally as follows: and completing the developed application based on the requirement analysis, wherein a worker needs to formulate a test scheme corresponding to the application, and then compile the test scheme into a test case and test the application.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
because each newly developed application needs to make a test scheme and compile a corresponding test case, when the number of clusters is large and the number of service applications is large, the workload of a test task is large, the test efficiency is low, and the online time of the newly developed application is prolonged.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a system, and a storage medium for fault tolerance testing, which can solve the problems that when the number of clusters is large and the number of service applications is large, the workload of a test task is large, the test efficiency is low, and the online time of newly developed applications is prolonged.
To achieve the above object, according to an aspect of an embodiment of the present invention, a method of fault tolerance testing is provided.
The fault tolerance test method of the embodiment of the invention comprises the following steps: judging whether the application to be online is a backbone application or not based on the attribute information of the application to be online; if the application to be online is a backbone application, screening a test case template corresponding to the type according to the type of the application to be online; determining a test parameter corresponding to the application to be on-line based on the test function of the application to be on-line; and executing fault-tolerant test on the application to be on-line based on the test case template and the test parameters, and obtaining a test result.
In one embodiment, the test parameters include input data and a test algorithm corresponding to the application to be online;
the executing fault-tolerant test on the application to be online based on the test case template and the test parameters and obtaining a test result comprises the following steps:
determining a test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and acquiring an expected result corresponding to the application to be online;
executing fault-tolerant test on the application to be online through the test case to obtain an execution result;
and determining the test result of the application to be on-line according to the execution result and the expected result.
In another embodiment, the determining the test result of the application to be online according to the execution result and the expected result includes:
judging whether the similarity between the execution result and the expected result reaches a preset threshold or not;
if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
In another embodiment, after the executing the fault-tolerant test on the application to be online based on the test case template and the test parameters and obtaining the test result, the method further includes:
if the test result is that the test is passed, the application to be online is issued to an online cluster, and the cluster test is executed on the application to be online through the online cluster to obtain a cluster test result;
and if the test result is that the test is not passed, sending a notification message, wherein the notification message comprises the test result.
In another embodiment, after the executing a cluster test on the application to be online by the online cluster to obtain a cluster test result, the method further includes:
judging whether the test data in the cluster test comprises a buried point mark or not;
if not, storing the test data; and if so, not storing the test data.
In yet another embodiment, further comprising:
if the application to be online is not a backbone application, releasing the application to be online to an online cluster;
and executing cluster test on the application to be online through the online cluster to obtain a cluster test result.
In another embodiment, the attribute information includes an identifier, a priority, and whether the to-be-online application is a backbone function.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided an apparatus for fault tolerant testing.
The fault tolerance test device of the embodiment of the invention comprises: the system comprises a receiving unit, a judging unit and a judging unit, wherein the receiving unit is used for judging whether an application to be online is a backbone application or not based on attribute information of the application to be online; the screening unit is used for screening a test case template corresponding to the type of the application to be online according to the type of the application to be online if the application to be online is a backbone application; the determining unit is used for determining a test parameter corresponding to the application to be on-line based on the test function of the application to be on-line; and the test unit is used for executing fault-tolerant test on the application to be on-line based on the test case template and the test parameters and obtaining a test result.
In one embodiment, the test parameters include input data and a test algorithm corresponding to the application to be online;
the test unit is specifically configured to:
determining a test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and acquiring an expected result corresponding to the application to be online;
executing fault-tolerant test on the application to be online through the test case to obtain an execution result;
and determining the test result of the application to be on-line according to the execution result and the expected result.
In another embodiment, the test unit is specifically configured to:
judging whether the similarity between the execution result and the expected result reaches a preset threshold or not;
if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
In yet another embodiment, the test unit is further configured to:
if the test result is that the test is passed, the application to be online is issued to an online cluster, and the cluster test is executed on the application to be online through the online cluster to obtain a cluster test result;
the device further comprises:
and the sending unit is used for sending a notification message if the test result is that the test fails, wherein the notification message comprises the test result.
In yet another embodiment, the apparatus further comprises:
the judging unit is used for judging whether the test data in the cluster test comprises a buried point mark or not;
the storage unit is used for storing the test data if the test data is not stored; and if so, not storing the test data.
In yet another embodiment, the apparatus further comprises:
the issuing unit is used for issuing the application to be online to an online cluster if the application to be online is not a backbone application;
the test unit is further configured to execute a cluster test on the application to be online through the online cluster to obtain a cluster test result.
In another embodiment, the attribute information includes an identifier, a priority, and whether the to-be-online application is a backbone function.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided an electronic apparatus.
An electronic device of an embodiment of the present invention includes: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors realize the fault tolerance testing method provided by the embodiment of the invention.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided a computer-readable medium.
A computer-readable medium of an embodiment of the present invention stores thereon a computer program, which, when executed by a processor, implements the method of fault tolerance testing provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: in the embodiment of the invention, whether the application to be online is the backbone application can be judged according to the attribute information of the application to be online, if the application to be online is the backbone application, the test case template corresponding to the type of the application to be online is screened, the test parameter corresponding to the test case template is determined, and the fault-tolerant test is carried out on the application to be online. In the embodiment of the invention, the fault-tolerant test is carried out only when the application to be online is a main application, the number of applications to be tested can be reduced, and the application to be tested can be screened for the test case template corresponding to the type of the application without rewriting the test case for the application to be tested, so that the time for testing the application can be saved, the workload of a test task can be reduced even when the number of clusters is large and the number of service applications is large, and the test efficiency is improved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of one principal flow of a method of fault tolerance testing in accordance with an embodiment of the present invention;
FIG. 2 is a diagram of an attribute information table for an online application according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a test information table according to an embodiment of the invention;
FIG. 4 is a schematic diagram of a main flow of a method of cluster testing according to an embodiment of the invention;
FIG. 5 is a schematic diagram of yet another major flow of a method of fault tolerance testing in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram of the main elements of an apparatus for fault tolerance testing in accordance with an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
FIG. 8 is a schematic block diagram of a computer system suitable for use in implementing embodiments of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The embodiment of the invention provides a fault tolerance test method which can be used in a test scene of developed applications, for example, a test scene of the applications in a micro service system after development. In the embodiment of the present invention, the specific content of the application to be online may be determined according to a scenario, and may be a minimum unit that can be developed, for example, a part of a functional module of a certain system or a part of a certain application. In the embodiment of the present invention, an example of testing an application developed in a microservice system is described.
An embodiment of the present invention provides a method for fault tolerance testing, which may be performed by a terminal or a server, as shown in fig. 1, and includes the following steps.
S101: and judging whether the application to be on-line is a backbone application or not based on the attribute information of the application to be on-line.
The attribute information of the application to be online is predetermined. In the embodiment of the present invention, before this step, attribute information of an application to be online may be predetermined, and the attribute information of the application to be online may include an identifier, a priority, whether a backbone function is provided, and the like of the application to be online.
The backbone application means that the function realized by the application is the backbone function of the system. In the embodiment of the invention, based on whether the function realized by the application is a backbone function, the application can be divided into backbone application and non-backbone application. Therefore, in the embodiment of the invention, some developed applications can be split and then respectively tested, for example, the applications are split into a minimum unit which can be developed, namely, the applications to be online, so as to realize the accurate division of whether the applications to be online are the backbone applications.
It should be noted that, in the micro service system, the application iterative development speed of each version is faster and the number is larger. For more convenient application management, each application to be online may be digitally managed, for example, in a form mode, attribute information of each application to be online is stored in the form, and the attribute information of each application to be online is stored in the form in a manner of dividing by a row unit, that is, a row of data is the attribute information of one application to be online. The priority, whether the backbone function is used, and the like in the attribute information of the application to be online can also be determined by developers or testers. In this step, the attribute information of the application to be online can be obtained from the stored table.
Specifically, a table for storing attribute information of an application to be online may be as shown in fig. 2. As shown in fig. 2, the attribute information may include a function module to which the application to be tested belongs, a name of the application to be on-line, an overview of the application to be on-line, a priority of the application to be on-line, a technical research description of the application to be on-line, a requirement analysis and prototype design description of the application to be on-line, a version number of the application to be on-line, an owner (developer) corresponding to the application to be on-line, whether the application to be on-line is implemented as a backbone function, and an expected on-line time of the application to be on-line.
In this step, after determining the attribute information of the application to be on-line, it may be determined whether the application to be on-line is a backbone application according to whether the attribute information is a backbone function. If the application to be online realizes the backbone function, the application is the backbone application; if the application to be online is not a backbone function, the application is not a backbone application and is a non-backbone application.
S102: and if the application to be online is the backbone application, screening a test case template corresponding to the type according to the type of the application to be online.
In the embodiment of the invention, a plurality of test case templates are configured in advance, and after the application to be tested is determined to be the backbone application in step S101, the test case template corresponding to the application to be tested can be screened out.
Because the optimization or new function development of the backbone application in the system usually does not change much, the test case template can be configured based on the functions included in the backbone application in the embodiment of the invention, namely, the backbone application is divided into different types of functions, and then a corresponding test case template is configured for each type. In the step, the corresponding test case template can be screened out based on the type of the application to be online. For example, if a system backbone function includes ordering and customer service, corresponding test case templates are configured for both ordering type and customer service type applications to be online. The type of the application to be tested can be determined according to the function module belonging to the attribute information of the application to be tested.
It should be noted that, in the embodiment of the present invention, for each type, a test case template corresponding to each type may be obtained through analysis and abstraction according to the implemented function of each type. When an application with a certain function is tested for the first time, a corresponding test case template needs to be written for the application. In the embodiment of the invention, one type of test case template can be a basic test case written for the type of function, and the basic test case can realize the test of the basic function of the function, so the test case template can be used as the test case template.
S103: and determining test parameters corresponding to the application to be online based on the test function of the application to be online.
In the method, the functions of different applications to be online are not completely the same, so that the personalized data of the applications to be online are required to be combined during testing, and the testing parameters corresponding to the applications to be online are determined based on the testing functions of the applications to be online in the step. The test parameters may specifically include input data and a test algorithm corresponding to the application to be online.
Because different input data and test algorithms can obtain different test results, and the test results are predicted before testing, the input data and the test algorithms and expected results corresponding to the input data and the test algorithms can be determined when the to-be-online application is tested. The input data may include the form, content, etc. of the data input at the time of the test.
Taking the function realized by the application to be online as the customer service response as an example, different results may be obtained by inputting different question ways and adopting different deformation algorithms during the test, so that the question ways and the deformation algorithms adopted by the test and the corresponding expected results need to be determined firstly during the test. The question mode is input data corresponding to the online application during testing, and the deformation algorithm is a testing algorithm used during testing. Specifically, the deformation algorithm may include a machine learning method CNN, a decision tree analysis method, naive bayes, logistic regression, and the like.
In the embodiment of the invention, the input data and the test algorithm used by each type of application to be online during testing can be pre-configured, and then the test parameters corresponding to the application to be online are determined based on the functions of the application to be online. In addition, when the input data and the test algorithm configured for each type of application to be online include multiple types, the specific input data and the test algorithm can be determined based on the input instruction of a tester or a developer in the embodiment of the invention.
S104: and executing fault-tolerant test on the application to be online based on the test case template and the test parameters, and obtaining a test result.
After the test case template and the test parameters corresponding to the application to be online are determined, the test case corresponding to the application to be online can be determined, and thus the fault-tolerant test can be performed on the application to be tested.
Specifically, the test parameters may include input data and a test algorithm corresponding to the application to be online. And bringing the input data and the test algorithm into the test case template, namely combining the test case template, the input data and the test algorithm to determine the test case corresponding to the application to be tested. Meanwhile, in the embodiment of the invention, an expected result corresponding to the application to be online needs to be obtained, so that the execution result can be judged subsequently.
By running the test case, the fault-tolerant test of the application to be on-line can be realized, an execution result is obtained after the test case is run, and the test result of the application to be on-line can be determined according to the execution result and an expected result.
Specifically, according to the execution result and the expected result, it can be determined that the test result of the application to be online can be specifically executed as: judging whether the similarity between the execution result and the expected result reaches a preset threshold or not; if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
In the embodiment of the invention, the fault-tolerant test is executed on the application to be on-line, so that the execution result of the test case after being executed may not be completely the same as the expected result, and a similar threshold value between the execution result and the expected result can be set in the embodiment of the invention. After the execution result is obtained, the degree of similarity between the execution result and the expected result may be determined, and then the magnitude relationship between the degree of similarity and the similarity threshold may be compared. If the similarity degree of the execution result and the expected result is greater than the similarity threshold, determining that the test result of the to-be-on-line application is a test pass; and if the similarity degree of the execution result and the expected result is not larger than the threshold value, determining that the test result of the application to be on-line is a test failure. The threshold may specifically be set to 90%.
In the embodiment of the invention, the similarity between the execution result and the expected result can be calculated by a similarity algorithm, or can be determined by inputting a similarity instruction by a developer or a tester.
It should be noted that, since there may be a plurality of input data and test algorithms determined by one application to be online, after the combination with the test case template, a plurality of test cases may be determined, and then the application to be online needs to test each test case, and then determine the test result according to the average value of the test results of each test case. For example, if the proportion of passing the test in the test result of each test case reaches a preset value, the test result can be determined as passing the test; and if the proportion of passed test in the test result of each test case does not reach the preset value, determining that the test result is failed.
In the embodiment of the invention, the test related information of the application to be online can be stored in a table mode. The test-related information may include test parameters, expected results, similarity thresholds, and the like. Specifically, taking the function realized by the application to be online as the customer service response as an example, the test related information is shown in fig. 3. In fig. 3, the application ID represents the identity of the application to be brought online; the method comprises the steps that a question mark represents input data of an application to be online; the deformation mode represents a test algorithm to be applied online; the similarity threshold represents a threshold of similarity between the execution result and the expected result; the state represents the state of the test result of the application to be online, if the test result of the application to be online is that the test passes, the state is valid, and if the test result of the application to be online is that the test does not pass, the state is invalid; the owner represents the owner (developer) of the application to be online. The test related information of different applications to be online can be separated by an empty row, for example, a row numbered 3 in fig. 3 is empty, and is used for distinguishing different applications to be tested.
In the embodiment of the invention, the application to be tested can set the test frequency during the test, so that the application to be tested can be subjected to multiple fault-tolerant tests to further obtain a relatively stable overall test result. The application to be online generally needs to be tested in three environments, which are respectively: DEV, TEST, and UAT (pre-release) environments. Among them, the DEV environment and the TEST environment are usually used by many users, and are frequently published with a positive version, and the environment is unstable, so that a lower TEST frequency may be set, for example, performed once a day, whereas the UAT environment is more stable in version, and a higher TEST frequency may be set, for example, performed once every two hours. When the application to be online executes multiple fault-tolerant tests, the total test result is determined according to the proportion of the test passing times in each test result to the total test times. For example, if the proportion of the test pass times to the total test pass times in the test results of the test cases reaches a preset value, the total test result can be determined as the test pass; if the proportion of the test passing times in the test result of each test case to the total times does not reach the preset value, the total test result can be determined as the test failing.
In the embodiment of the invention, if the test result is that the test is passed, the application to be online can be issued to the online cluster, then the cluster test is executed in the online cluster, and the cluster test result is obtained; if the test result is that the test is not passed, a notification message can be sent to notify the developer or the tester that the test is not passed, so that the test can be redeveloped.
In the embodiment of the invention, the applications to be online are divided into backbone applications and non-backbone applications, and the functions realized by the non-backbone applications do not affect the main functions of the system, so that the fault-tolerant test can be avoided, and the applications to be online are directly issued to the online cluster for cluster test, namely, the processes of the steps S102, S103 and S104 in the embodiment of the invention are only executed on the backbone applications, so that the number of the applications to be tested can be reduced, and the applications to be online can be screened for test case templates corresponding to the types of the applications to be online without rewriting the test cases for the applications, so that the time for application test can be saved, and the workload of test tasks can be reduced and the test efficiency can be improved even when the number of clusters is large and the number of service applications is large.
It should be noted that after the application to be online is released to the online cluster, the test in the cluster may be performed. The test in the cluster can be executed regularly by setting the test frequency, and then whether the test in the cluster passes or not is determined according to the test result.
In the following, a test process in a cluster after an application to be tested is issued to an online cluster according to an embodiment of the present invention is described with reference to the embodiment shown in fig. 1, and as shown in fig. 4, the method includes the following steps.
S401: and executing cluster test on the application to be on-line in the on-line cluster.
The applications to be online issued to the online cluster include trunk applications and non-trunk applications, and the trunk applications and the non-trunk applications in the online cluster are required to be subjected to cluster testing.
In the embodiment of the invention, the cluster test frequency can be set for the trunk application and the non-trunk application, the cluster test is carried out according to the set cluster test frequency, and meanwhile, when the application to be on-line is the trunk application, the heartbeat test can be set, wherein the heartbeat test refers to the test with higher test frequency.
Specifically, when a high frequency heartbeat test can be used for the backbone application, the test frequency can be set to be performed every two minutes. The cluster test frequency may be set according to requirements, for example, may be set to be executed every two hours. In the embodiment of the invention, the trunk application can be subjected to cluster test according to the cluster test frequency, and the heartbeat test is simultaneously carried out according to the frequency of the heartbeat test, so that the test on the trunk application is enhanced, and the heartbeat test can enable the trunk application to still execute the test process when the cluster test has problems, so as to ensure that the corresponding function can be stably realized after the trunk application is on line.
In the embodiment of the present invention, the test cases used in the cluster test may be selected from a test case set, and the test case set may be pre-established and may be a test case that has been used, for example, a test case used in the fault-tolerant test in step S104.
S402: an overall cluster test result is determined.
The cluster test result comprises a test result of the cluster test and a test result of the heartbeat test. In the embodiment of the invention, when the test result of the cluster test and the test result of the heartbeat test both pass, the total cluster test result can be determined as the test pass.
Since the functions of the non-backbone application affect the backbone functions, the cluster test may be a fault-tolerant test, that is, in the embodiment of the present invention, a threshold of the cluster test may be set, and for all the cluster tests executed at a fixed time, a ratio of the number of times that the test passes to the total number of times of the test reaches the threshold, and then a test result of the cluster test may be determined, and the threshold may be set according to a requirement, for example, set to 90%. The heartbeat test is a test applied to the backbone, and the requirement can be met only when all the test results are passed, that is, the heartbeat test executed regularly needs to pass the test results every time, and the test result of the heartbeat test can be determined to pass the test. The test result in the cluster test is a test pass.
In the embodiment of the invention, when the total cluster test result indicates that the application in the cluster can be on line when the test passes, the passing result can be notified to relevant personnel, such as developers, and the on-line process is executed. When the test result of the total cluster is that the test fails, it is indicated that the application in the cluster needs to be developed again, and the failing result can be notified to the developer, so that the developer needs to develop again.
It should be noted that after the application to be online is issued to the online cluster, in order not to affect the use of the user, the shunting processing needs to be performed, for example, a pin or a vender id field in chatinfo in a protocol in the protocol controls the user to enter the application to be online, so as to ensure that the application to be online is not used by the user, and then the application to be online is used by the user through setting after passing the test.
S403: and storing the test data.
In the embodiment of the present invention, the tested test data is stored, and the test data may include the overall cluster test result and the related data for each test execution. Because the number of tests is large, in order to reduce the generation of test junk data, test cases and test account numbers with fixed numbers can be used in the test process. For the test result, a report can be generated to inform relevant personnel, in order to reduce garbage data and reduce the influence of test data on report generation, embedded point marking can be performed on each test case in the embodiment of the invention, the test data corresponding to the test case with the embedded point marking can not be stored, so that when the test data is stored, the test data used for storage can be distinguished according to the embedded point marking. Specifically, whether the test data comprises the embedded point mark or not can be judged during test data storage, and if yes, the test data is not stored; if not, the test data is stored. The test data corresponding to the test case with the embedded point marks can not be stored, so the test data can not be used for generating the report and can not influence the generation of the report.
It should be noted that the punctuation marking may be performed by setting a white list.
After each test execution, a test report may be generated, where the test report may include test case related information, execution time, test result, and the like of the application to be on-line. The test report can be sent to the relevant personnel by mail or the like.
In the embodiment of the invention, the test in the cluster is executed on the application to be online in the online cluster, so that the function can be stably realized after the application to be online is online.
The tests to be applied online typically include tests performed by developers and tests performed by testers. The developer can automatically execute the test process after developing the application to be online, and then the test is carried out by the tester. The test scene applied in the embodiment of the invention can be tested by a tester, namely, the test process of self test of the developer is executed after the trunk application and the non-trunk application are developed, so as to ensure that the application to be on-line can run. Then, a tester for the backbone application performs the test by the method of the embodiment shown in fig. 1, and issues the test to the on-line cluster after the test is passed, instead of directly issuing the backbone application to the on-line cluster. For the backbone application and the non-backbone application, the test executed by the developer may be a fault-tolerant test, and the specific test mode may be the mode of determining a test case and executing the fault-tolerant test as in the embodiment shown in fig. 1. The overall test execution process of the specific application to be online can be as shown in fig. 5.
As shown in fig. 5, in the embodiment of the present invention, an application to be online is divided into a backbone application and a non-backbone application, and after analyzing and developing the application, the application is determined to be the backbone application or the non-backbone application, and then both the backbone application and the non-backbone application are issued to a pre-distribution cluster after analyzing and developing the application, where the pre-distribution cluster includes the backbone application and the non-backbone application. For the backbone application, after the backbone application is developed, a test case also needs to be determined, wherein if the corresponding function of the backbone application is to execute the test for the first time, a test scheme and the test case need to be written; if the corresponding function of the backbone application is not to execute the test for the first time, the test case can be determined according to the manner of step S102 and step S103. And after the trunk application determines the test case, the timing fault-tolerant test can be executed, and the timing fault-tolerant test is added into a fault-tolerant monitoring queue so as to monitor the fault-tolerant test of the trunk application. The trunk application and the non-trunk application can also execute the process of self test of developers after the development is finished and the non-trunk application is released to the pre-sending cluster, and the fault-tolerant monitoring queue can comprise the test tasks of the trunk application and the non-trunk application. The testing of the trunk application and the testing of the non-trunk application can be to carry out the fault-tolerant test at regular time, and send the relevant information of the test to the pre-sending cluster by request, the relevant information of the test includes the return result of the fault-tolerant test, namely the test result, the test result can be notified to relevant personnel in a reporting mode, and when the trunk application test fails in the test result, the application to be online is returned to carry out the trunk application development again, when the non-trunk application test fails in the test result, the application to be online is returned to carry out the non-trunk application development again, and when the test result is that the test passes, the application to be online is issued to the cluster of the online system, namely to the online cluster. In the online cluster, the cluster test is executed no matter whether the application to be online is a trunk application or a non-trunk application, and meanwhile, the cluster test also comprises heartbeat monitoring of the trunk application, and the cluster test can be a fault-tolerant test executed at regular time. The test result of the cluster test can be returned to the on-line cluster, and the on-line cluster can inform the relevant personnel of the returned test result in a reporting mode.
In order to solve the problems in the prior art, an embodiment of the present invention provides an apparatus 600 for fault tolerance testing, as shown in fig. 6, the apparatus 600 includes:
a receiving unit 601, configured to determine whether an application to be online is a backbone application based on attribute information of the application to be online;
a screening unit 602, configured to screen, if the application to be online is a backbone application, a test case template corresponding to the type according to the type of the application to be online;
a determining unit 603, configured to determine, based on a test function of the application to be online, a test parameter corresponding to the application to be online;
and the test unit 604 is configured to execute a fault-tolerant test on the application to be online based on the test case template and the test parameters, and obtain a test result.
It should be understood that the manner of implementing the embodiment of the present invention is the same as the manner of implementing the embodiment shown in fig. 1, and the description thereof is omitted.
In an implementation manner of the embodiment of the present invention, the test parameters include input data and a test algorithm corresponding to the to-be-online application; the test unit 604 is specifically configured to:
determining a test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and acquiring an expected result corresponding to the application to be online;
executing fault-tolerant test on the application to be online through the test case to obtain an execution result;
and determining the test result of the application to be on-line according to the execution result and the expected result.
In another implementation manner of the embodiment of the present invention, the test unit 604 is further configured to:
judging whether the similarity between the execution result and the expected result reaches a preset threshold or not;
if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
In another implementation manner of the embodiment of the present invention, the testing unit 604 is further configured to issue the to-be-online application to an online cluster if the testing result is that the testing passes, and execute a cluster test on the to-be-online application through the online cluster to obtain a cluster testing result;
the apparatus 600 further comprises:
and the sending unit is used for sending a notification message if the test result is that the test fails, wherein the notification message comprises the test result.
In another implementation manner of the embodiment of the present invention, the apparatus 600 further includes:
the judging unit is used for judging whether the test data result in the cluster test comprises a buried point mark or not;
the storage unit is used for storing the test data if the test data is not stored; if yes, the test data is not stored; .
In another implementation manner of the embodiment of the present invention, the apparatus 600 further includes:
the issuing unit is used for issuing the application to be online to an online cluster if the application to be online is not a backbone application;
the test unit is further configured to execute a cluster test on the application to be online through the online cluster to obtain a cluster test result.
In another implementation manner of the embodiment of the present invention, the attribute information includes an identifier, a priority, and whether the to-be-online application is a backbone function.
It should be understood that the embodiment of the present invention is implemented in the same manner as the embodiment shown in fig. 2 or fig. 4, and is not repeated herein.
In the embodiment of the invention, the applications to be online are divided into the backbone applications and the non-backbone applications, and the functions realized by the non-backbone applications do not influence the main functions of the system, so that the fault-tolerant test can be directly issued to the online cluster without carrying out fault-tolerant test, namely, the fault-tolerant test is only carried out on the backbone applications, thereby reducing the number of the applications to be tested, screening the test case templates corresponding to the types of the applications to be tested, and not rewriting the test cases for the applications to be tested, thereby saving the time of the application test, reducing the workload of the test tasks and improving the test efficiency even when the number of the clusters is large and the number of the service applications is large.
The invention also provides an electronic device and a readable storage medium according to the embodiment of the invention.
The electronic device of the present invention includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the processor, the instructions being executable by the at least one processor to cause the at least one processor to perform a method of fault tolerance testing as provided by embodiments of the present invention.
Fig. 7 shows an exemplary system architecture 700 to which the method of fault tolerance testing or the apparatus of fault tolerance testing of embodiments of the present invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. Various client applications may be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server that provides various services, and may analyze and process data such as a received product information query request, and feed back a processing result (for example, product information — just an example) to the terminal device.
It should be noted that the method for fault tolerance testing provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the device for fault tolerance testing is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, a block diagram of a computer system 800 suitable for use in implementing embodiments of the present invention is shown. The computer system illustrated in FIG. 8 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a unit, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a receiving unit, a screening unit, a determining unit, and a testing unit. Where the names of these elements do not in some cases constitute a limitation of the element itself, for example, a receiving element may also be described as "an element of the function of the receiving element".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to perform the fault tolerance testing methods of the present invention.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. A method of fault tolerance testing, comprising:
judging whether the application to be online is a backbone application or not based on the attribute information of the application to be online;
if the application to be online is a backbone application, screening a test case template corresponding to the type according to the type of the application to be online;
determining a test parameter corresponding to the application to be on-line based on the test function of the application to be on-line;
and executing fault-tolerant test on the application to be on-line based on the test case template and the test parameters, and obtaining a test result.
2. The method of claim 1, wherein the test parameters include input data and a test algorithm corresponding to the application to be online;
the executing fault-tolerant test on the application to be online based on the test case template and the test parameters and obtaining a test result comprises the following steps:
determining a test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and acquiring an expected result corresponding to the application to be online;
executing fault-tolerant test on the application to be online through the test case to obtain an execution result;
and determining the test result of the application to be on-line according to the execution result and the expected result.
3. The method of claim 1, wherein determining the test result of the application to be brought online according to the execution result and the expected result comprises:
judging whether the similarity between the execution result and the expected result reaches a preset threshold or not;
if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
4. The method according to claim 1, wherein after performing fault-tolerant testing on the application to be online based on the test case template and the test parameters and obtaining a test result, the method further comprises:
if the test result is that the test is passed, the application to be online is issued to an online cluster, and the cluster test is executed on the application to be online through the online cluster to obtain a cluster test result;
and if the test result is that the test is not passed, sending a notification message, wherein the notification message comprises the test result.
5. The method of claim 4, after performing a cluster test on the application to be online by the online cluster to obtain a cluster test result, further comprising:
judging whether the test data in the cluster test comprises a buried point mark or not;
if not, storing the test data; and if so, not storing the test data.
6. The method of claim 1, further comprising:
if the application to be online is not a backbone application, releasing the application to be online to an online cluster;
and executing cluster test on the application to be online through the online cluster to obtain a cluster test result.
7. An apparatus for fault tolerant testing, comprising:
the system comprises a receiving unit, a judging unit and a judging unit, wherein the receiving unit is used for judging whether an application to be online is a backbone application or not based on attribute information of the application to be online;
the screening unit is used for screening a test case template corresponding to the type of the application to be online according to the type of the application to be online if the application to be online is a backbone application;
the determining unit is used for determining a test parameter corresponding to the application to be on-line based on the test function of the application to be on-line;
and the test unit is used for executing fault-tolerant test on the application to be on-line based on the test case template and the test parameters and obtaining a test result.
8. The apparatus of claim 7, wherein the test parameters comprise input data and a test algorithm corresponding to the application to be online;
the test unit is specifically configured to:
determining a test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and acquiring an expected result corresponding to the application to be online;
executing fault-tolerant test on the application to be online through the test case to obtain an execution result;
and determining the test result of the application to be on-line according to the execution result and the expected result.
9. The device according to claim 8, characterized in that the test unit is specifically configured to:
judging whether the similarity between the execution result and the expected result reaches a preset threshold or not;
if so, determining that the test result is that the test is passed; if not, determining that the test result is that the test is not passed.
10. The apparatus of claim 7, further comprising:
the issuing unit is used for issuing the application to be online to an online cluster if the application to be online is not a backbone application;
the test unit is further configured to execute a cluster test on the application to be online through the online cluster to obtain a cluster test result.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.
12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.
CN202010865406.1A 2020-08-25 2020-08-25 Fault-tolerant test method and device, electronic equipment and storage medium Pending CN113778842A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010865406.1A CN113778842A (en) 2020-08-25 2020-08-25 Fault-tolerant test method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010865406.1A CN113778842A (en) 2020-08-25 2020-08-25 Fault-tolerant test method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113778842A true CN113778842A (en) 2021-12-10

Family

ID=78835195

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010865406.1A Pending CN113778842A (en) 2020-08-25 2020-08-25 Fault-tolerant test method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113778842A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060265691A1 (en) * 2005-05-20 2006-11-23 Business Machines Corporation System and method for generating test cases
CN101976220A (en) * 2010-09-27 2011-02-16 北京神州泰岳软件股份有限公司 Unit test system
CN103186457A (en) * 2011-12-29 2013-07-03 阿里巴巴集团控股有限公司 Method and device for automatically generating test case
CN104298589A (en) * 2013-07-16 2015-01-21 阿里巴巴集团控股有限公司 Performance test method and performance test equipment
CN105302720A (en) * 2015-10-27 2016-02-03 浪潮电子信息产业股份有限公司 Software regression test case screening method
CN105912461A (en) * 2016-04-05 2016-08-31 浪潮电子信息产业股份有限公司 Test method and system for software agility
WO2017023299A1 (en) * 2015-08-04 2017-02-09 Hewlett Packard Enterprise Development Lp Composing future tests
CN108920139A (en) * 2018-06-29 2018-11-30 北京金山数字娱乐科技有限公司 A kind of program creating method, device, system, electronic equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060265691A1 (en) * 2005-05-20 2006-11-23 Business Machines Corporation System and method for generating test cases
CN101976220A (en) * 2010-09-27 2011-02-16 北京神州泰岳软件股份有限公司 Unit test system
CN103186457A (en) * 2011-12-29 2013-07-03 阿里巴巴集团控股有限公司 Method and device for automatically generating test case
CN104298589A (en) * 2013-07-16 2015-01-21 阿里巴巴集团控股有限公司 Performance test method and performance test equipment
WO2017023299A1 (en) * 2015-08-04 2017-02-09 Hewlett Packard Enterprise Development Lp Composing future tests
CN105302720A (en) * 2015-10-27 2016-02-03 浪潮电子信息产业股份有限公司 Software regression test case screening method
CN105912461A (en) * 2016-04-05 2016-08-31 浪潮电子信息产业股份有限公司 Test method and system for software agility
CN108920139A (en) * 2018-06-29 2018-11-30 北京金山数字娱乐科技有限公司 A kind of program creating method, device, system, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US10623235B2 (en) Correlating computing network events
US11138058B2 (en) Hierarchical fault determination in an application performance management system
CN107608901B (en) Jmeter-based testing method and device, storage medium and electronic equipment
CN110532322B (en) Operation and maintenance interaction method, system, computer readable storage medium and equipment
US10942801B2 (en) Application performance management system with collective learning
CN110851471A (en) Distributed log data processing method, device and system
CN112631919A (en) Comparison test method and device, computer equipment and storage medium
CN117608825A (en) Resource management method based on multi-cloud management platform and related equipment
CN109299124B (en) Method and apparatus for updating a model
US10671470B2 (en) Application performance management system with dynamic discovery and extension
CN113760522A (en) Task processing method and device
CN115309558A (en) Resource scheduling management system, method, computer equipment and storage medium
CN113778842A (en) Fault-tolerant test method and device, electronic equipment and storage medium
CN113918525A (en) Data exchange scheduling method, system, electronic device, medium, and program product
CN114070889A (en) Configuration method, traffic forwarding method, device, storage medium, and program product
CN113434382A (en) Database performance monitoring method and device, electronic equipment and computer readable medium
CN113626869A (en) Data processing method, system, electronic device and storage medium
CN109144788B (en) Method, device and system for reconstructing OSD
CN112948195A (en) Interface testing method and device, electronic equipment and storage medium
CN112579402A (en) Method and device for positioning faults of application system
CN113132447A (en) Reverse proxy method and system
US11330079B2 (en) Environment for developing of distributed multicloud applications
US10235262B2 (en) Recognition of operational elements by fingerprint in an application performance management system
CN117130945B (en) Test method and device
CN113760723B (en) Business process debugging method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination