WO2021248754A1 - 一种系统测试方法、装置、存储介质及电子设备 - Google Patents

一种系统测试方法、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2021248754A1
WO2021248754A1 PCT/CN2020/119616 CN2020119616W WO2021248754A1 WO 2021248754 A1 WO2021248754 A1 WO 2021248754A1 CN 2020119616 W CN2020119616 W CN 2020119616W WO 2021248754 A1 WO2021248754 A1 WO 2021248754A1
Authority
WO
WIPO (PCT)
Prior art keywords
service
tested
test
server
under test
Prior art date
Application number
PCT/CN2020/119616
Other languages
English (en)
French (fr)
Inventor
祁晓光
李标
Original Assignee
北京旷视科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京旷视科技有限公司 filed Critical 北京旷视科技有限公司
Publication of WO2021248754A1 publication Critical patent/WO2021248754A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26—Functional testing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3692—Test management for test results analysis

Definitions

  • This application relates to the technical field of software testing, and specifically to a system testing method, device, storage medium, and electronic equipment.
  • the business system will be tested before it is officially launched.
  • the business system is basically tested when the server environment is relatively stable, and the scope of the test is relatively limited, causing the business system to fail after it goes online. More.
  • One of the objectives of the embodiments of the present application is to provide a system testing method, device, storage medium, and electronic equipment to improve the above technical problems.
  • an embodiment of the present application provides a system testing method, which includes: injecting server exceptions into a server running a system under test; monitoring the operating indicators of the service under test included in the system under test, and testing the system under test The interface function of the service.
  • the above method firstly, by proactively injecting server exceptions into the server running the system under test, the real abnormal condition of the server is simulated, and then the operation indicators of the service under test are monitored and the interface functions of the service under test are tested, and then the subsequent operations are based on the obtained Statistical analysis of the operating indicators and test results of the system can determine the working conditions of the system under test (for example, whether it can work normally, problems in the working process, etc.) when the server is abnormal.
  • the above-mentioned method provides a new solution for testing the working conditions of the business system when the server is abnormal, expands the conventional test range of the business system, and fills the gap in the existing technology.
  • the above-mentioned method is also conducive to exposing and solving problems in the test process as soon as possible, and reducing the probability of failure of the system under test after it is actually online.
  • the method further includes: injecting service exceptions into the service to be tested; monitoring the operation index of the service to be tested, and testing the interface function of the service to be tested.
  • the real abnormal condition of the service to be tested is simulated, so that the working condition of the service to be tested under the abnormal condition can be tested.
  • the method further includes: determining a related service of the service under test, and the related service includes the service under test.
  • the upstream service and/or downstream service of the service monitor the operation index of the related service, and test the interface function of the related service.
  • the services in the system under test may have a dependency relationship.
  • the service dependent on the current service is called the upstream service of the current service, and the service that depends on the current service is called the downstream service of the current service. If the current service is abnormal, it may cause it Abnormal downstream services may also be caused by abnormal upstream services. Therefore, after injecting an exception into the service to be tested, in addition to testing the service to be tested, the working conditions of the upstream and/or downstream services of the service to be tested can also be tested to fully evaluate the impact of the exception in the service invocation chain.
  • the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated test system are also deployed on the DevOps platform; the injecting into the server running the system to be tested
  • the server exception step includes: calling the first exception acquisition interface provided by the DevOps platform to acquire the server exception list, and calling the first exception injection interface provided by the DevOps platform to inject the server in the server exception list to the server Abnormal;
  • the monitoring the operating indicators of the service under test included in the system under test, and testing the interface function of the service under test includes: calling the service instance acquisition interface provided by the DevOps platform to obtain deployment on the DevOps platform And determine the service to be tested included in the system to be tested in the service list; use the operation and maintenance monitoring system to monitor the operating indicators of the service to be tested, and use the automation
  • the test system tests the interface function of the service to be tested.
  • the step of monitoring the operation index of the related service and testing the interface function of the related service includes:
  • the operation and maintenance monitoring system is used to monitor the operation indicators of the related services, and the automated test system is used to test the interface functions of the related services.
  • the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated test system are also deployed on the DevOps platform; the injecting service exceptions into the service to be tested
  • the steps include: invoking the service instance acquisition interface provided by the DevOps platform to obtain a service list formed by the services deployed on the DevOps platform, and determining the service under test included in the system under test in the service list Call the second exception acquisition interface provided by the DevOps platform to obtain the service exception list, and call the second exception injection interface provided by the DevOps platform to inject the service exceptions in the service exception list into the service under test;
  • Monitoring the operation index of the service to be tested and testing the interface function of the service to be tested includes: using the operation and maintenance monitoring system to monitor the operation index of the service to be tested, and using the automated test system to test the The interface function of the service to be tested.
  • the DevOps platform can be understood as a tool used to install and deploy applications. DevOps emphasizes how to efficiently organize the team to complete the software life cycle management through automated tool collaboration and communication, so as to deliver faster, more frequent and more stable Software.
  • the system to be tested, the operation and maintenance monitoring system, and the automated test system involved in the solution of this application are deployed on the DevOps platform for unified management.
  • the DevOps platform also encapsulates service instance acquisition, exception acquisition, and exception injection Such interfaces are available for external calls, making it easy to inject exceptions and test services, and users do not need to write complicated instructions by themselves.
  • the method further includes:
  • the working condition of the service to be tested it is determined whether the service to be tested has a service abnormality.
  • the method further includes:
  • the server abnormality includes at least one of the following: a disk abnormality, an IO abnormality, a memory abnormality, a CPU abnormality, a power failure, and a network failure.
  • the service exception includes at least one of the following: database access timeout, array out of bounds, request exception, and response exception.
  • an embodiment of the present application provides a test platform configured to test the system under test using the above-mentioned system test method.
  • the test platform includes an abnormal encapsulation layer, a platform interface layer, a function implementation layer, and an application layer;
  • the abnormal encapsulation layer includes service abnormalities and server abnormalities
  • the platform interface layer includes a first exception acquisition interface, a second exception acquisition interface, a first exception injection interface, a second exception injection interface, and a service instance acquisition interface, and the platform interface layer is configured to acquire the Service abnormalities and/or server abnormalities;
  • the function realization layer is configured to inject server exceptions into the server running the system under test, monitor the operation indicators of the service under test included in the system under test, and test the interface function of the service under test;
  • the application layer is configured to receive test operations.
  • an embodiment of the present application provides a system testing device, including: an injection module configured to inject server exceptions into a server running a system under test; a testing module configured to monitor the service under test included in the system under test And test the interface function of the service to be tested.
  • the injection module is configured to inject service exceptions into the service to be tested
  • the test module is configured to monitor the operation index of the service to be tested and test the interface function of the service to be tested.
  • an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored on the computer-readable storage medium, and when the computer program instructions are read and run by a processor, the first aspect or The method provided by any one of the possible implementations of the first aspect.
  • an embodiment of the present application provides an electronic device, including: a memory and a processor.
  • the memory stores computer program instructions.
  • the computer program instructions When the computer program instructions are read and run by the processor, the first Aspect or any one of the possible implementations of the first aspect.
  • FIG. 1 shows a schematic diagram of a system under test provided by an embodiment of the present application
  • Figure 2 shows a flowchart of a system testing method provided by an embodiment of the present application
  • FIG. 3 shows a schematic diagram of a test platform provided by an embodiment of the present application
  • Figure 4 shows a functional module diagram of a system test device provided by an embodiment of the present application
  • Fig. 5 shows a structural diagram of an electronic device provided by an embodiment of the present application.
  • the object of the system testing method provided by the embodiment of the application is part or all of the services in a certain business system.
  • the tested business system detects the system under test, and the service under test in the system under test is referred to as the service under test.
  • the service to be tested can be either a traditional service or a microservice, which is not limited in this application.
  • Fig. 1 shows a schematic diagram of a system under test provided by an embodiment of the present application. Please refer to Figure 1.
  • the system under test 100 shown in Figure 1 includes front-end services 110, business-side microservices 120, core-side microservices 130, and middleware 140.
  • the system under test 100 can be deployed on a server or On the server cluster.
  • the system under test 10 may be used in different business scenarios. The following takes it as an example to introduce a video structured scenario, that is, the system under test 100 is a video structured system in this scenario.
  • the so-called video structuring can refer to the intelligent analysis of the original video stream and video file, extract the key information from it, and perform the semantic description of the text.
  • a possible workflow of the video structured system is as follows:
  • the middleware 140 has a tool nature and can implement services with specific functions, for example, Redis (a high-performance database), RabbitMQ (a message queue), Nginx (a web server), and so on.
  • the middleware 140 can be divided into business-side middleware and core-side middleware, which are used by the business-side microservice 120 and the core-side microservice 130, respectively.
  • the deployment sequence of each component in the video structured system is: core-side middleware, core-side microservice 130, business-side middleware, business-side microservice 120, and front-end service 110.
  • the services to be tested in the video structured system may include business-side microservices 120, core-side microservices 130, and middleware.
  • the inventor found the following defects when studying the test work of the video structured system:
  • Defect 1 The test of the service to be tested is based on the premise that the server's CPU, IO, network environment and other indicators are relatively stable. If the server is abnormal, whether the video structured system and the service to be tested included in the system are normal The work lacks the corresponding test plan.
  • Defect 2 The services involved in the video structured system have a certain dependency relationship. If a service is abnormal, the service that relies on it may also be abnormal. For example, according to the previous explanation, the business-side middleware should be deployed before the business-side microservices are started when the video structured system is deployed. Treated as an exception in Redis deployment), it will cause business-side microservices that rely on Redis to fail to work properly. At present, there is still a lack of solutions to effectively test the upstream service and/or downstream service of an abnormal service to determine whether it can work normally.
  • Fig. 1 is only an example, and the system under test involved in the solution of this application does not necessarily adopt the architecture shown in Fig. 1. Therefore, Fig. 1 should not be regarded as a limitation on the scope of protection of this application. However, the two defects mentioned above have universality.
  • FIG. 2 shows a flowchart of a system testing method provided by an embodiment of the present application.
  • the method can be executed by an electronic device.
  • FIG. 5 shows a possible structure of the electronic device.
  • the method includes:
  • Step S200 Inject a server exception into the server running the system to be tested.
  • Step S210 Monitor the operating indicators of the service to be tested included in the system to be tested, and test the interface function of the service to be tested.
  • Server abnormalities can include, but are not limited to, server disk abnormalities (such as bad sectors on the disk, too low remaining disk space, etc.), IO abnormalities (such as IO device errors, etc.), memory abnormalities (such as high memory usage, etc.), CPU abnormalities ( Such as high CPU usage), power failure, and network disconnection, etc., you can sort out possible abnormalities in the server in advance, encapsulate and save these abnormalities, and provide external interfaces so that these abnormalities can be obtained when abnormal injection is required.
  • Injecting server exceptions in step S200 refers to actively creating server exception scenarios to simulate the real abnormal conditions of the server. For example, injecting CPU exceptions into the server through an interface, the internal code of the interface will try to increase the CPU occupancy rate.
  • the server CPU has an abnormal condition, so that the operating condition of the system under test can be tested under such an abnormal condition.
  • the operation index of the service to be tested in step S210 may include the index of a single service to be tested or the overall index of all the services to be tested (or the system to be tested).
  • These indicators include but are not limited to CPU occupancy, IO rate, memory occupancy, virtual memory (such as SWAP under Linux) occupancy, disk occupancy, etc. These indicators are mainly used to describe the running status of the service to be tested from the external environment level. For example, if a service to be tested occupies close to 100% of the CPU for a long time, it can be reasonably inferred that its operation is abnormal.
  • the operating indicators of the service to be tested can be monitored through the operation and maintenance monitoring system, for example, the Prometheus system, the InfluxDB system, etc.
  • Interface testing can use automated testing systems, such as interface testing tools such as Postman and RESTClient.
  • step S210 After performing step S210, perform statistical analysis based on the operating indicators obtained through monitoring and/or the test results obtained through the interface test, for example, perform statistical analysis through the operating indicators obtained through the monitoring; another example, perform statistical analysis on the test results obtained through the interface test.
  • Statistical analysis for another example, statistical analysis of the running indicators obtained by monitoring and the test results obtained by the interface test can determine the working conditions of the service under test, including whether the service under test can work normally, and what abnormalities exist if it fails to work normally. Then, a solution corresponding to the abnormality can be determined.
  • the solution can be understood as a solution for how to make the service under test maintain normal operation even when the server is abnormal.
  • the above statistical analysis process and the process of resolving anomalies can either be manually checked, or can be automatically analyzed and automatically determined based on anomalies, which is not limited in this application. Since the system under test is composed of several services, the working condition of the service under test is determined, that is, the working condition of the entire system under test is determined.
  • the server may have a variety of abnormal conditions, it may be necessary to inject different types of abnormalities into the server multiple times and test separately.
  • only one type of server exception is injected each time, and the solution corresponding to the exception is determined immediately after the injection is completed, and the next type of server exception is injected after the solution is successfully determined.
  • this method provides a new solution for testing the working conditions of the business system when the server is abnormal, which significantly expands the testing scope of the business system in the conventional method and fills the existing technology. blank.
  • the abnormal injection in the above method is also helpful for exposing and solving possible problems of the system under test when facing server abnormalities, and reducing the probability of failure of the system under test after it is actually online.
  • Server anomalies are mainly anomalies in the external environment of the service to be tested, and are biased towards the device level.
  • Service exceptions are mainly internal exceptions generated by the service to be tested, which are biased towards the code logic or data level of the service to be tested.
  • the server in addition to injecting exceptions into the server, it is also possible to inject service exceptions into the service under test in order to simulate the real abnormal condition of the service under test.
  • the operating indicators of the service under test are monitored and the service under test is monitored.
  • the interface function of the service is tested.
  • statistical analysis based on operating indicators and/or test results can determine the working conditions of the service to be tested. For example, statistical analysis is performed through the operating indicators obtained through monitoring; another example is statistical analysis of the test results obtained through interface testing; another example is statistical analysis performed through the operating indicators obtained through monitoring and the test results obtained through interface testing.
  • the operation index can be obtained through monitoring, the test result can be obtained through interface testing, and the solution corresponding to the service anomaly can be further determined, that is, how to eliminate the service anomaly, or how to make the service to be tested when the service is abnormal
  • the service is still able to maintain the normal working plan. Since the system under test is composed of several services, the working condition of the service under test is determined, that is, the working condition of the entire system under test is determined. Injecting service exceptions for the service to be tested is also helpful for exposing and solving possible problems of the system under test when faced with service exceptions, and reducing the probability of failure of the system under test after it is actually online.
  • test for service exceptions and the test for server exceptions are performed independently of each other, that is, they are allowed to select services for testing respectively, and they are also allowed to select independent time periods for testing. For example, you can select the same service to be tested, first test the server abnormality, and then test the service abnormality. When the test service is abnormal, the server can be restored to a normal state.
  • Service exceptions can include, but are not limited to, database access timeout, array out-of-bounds, request exceptions (such as invalid request parameters, etc.) and response exceptions (such as response data errors, etc.). You can sort out the possible exceptions in the service in advance. Exceptions are encapsulated and saved, and interfaces are provided to the outside so that these exceptions can be obtained when exception injection is required.
  • injecting service exceptions into the service to be tested refers to actively creating service exception scenarios to simulate the actual abnormal conditions of the service. For example, by injecting wrong request parameters into the service to be tested through an interface, it can simulate the actual application of the customer. The abnormal condition caused by the illegal request sent by the terminal, so that the running condition of the service to be tested can be tested under such abnormal condition.
  • step S210 As for monitoring the operating indicators of the service to be tested and testing the interface function of the service to be tested after the abnormality is injected, the method is similar to step S210, and the description will not be repeated.
  • the services in the system under test may have a dependency relationship.
  • the service dependent on the current service is called the upstream service of the current service
  • the service that depends on the current service is called the downstream service of the current service.
  • the service and/or downstream service may be referred to as the related service of the current service for short.
  • the upstream service may be referred to as the related service of the current service for short;
  • the downstream service may be referred to as the related service of the current service for short; for another example, both the upstream service and the downstream service are referred to as the related service of the current service for short.
  • the current service is abnormal, it may cause an abnormality in its downstream service, or it may be caused by an abnormality in its upstream service. Therefore, after injecting an exception into the service to be tested, in addition to testing the service to be tested, the working conditions of the related services of the service to be tested can also be tested to fully evaluate the impact of the exception in the service invocation chain.
  • the specific approach is as follows:
  • DevOps combined term of Deployment Development and Operation and Maintenance Operations
  • the DevOps platform can be understood as a tool used to install and deploy applications.
  • DevOps emphasizes how to efficiently organize the team to complete the software life cycle management through automated tool collaboration and communication, so as to deliver faster, more frequent and more stable Software.
  • the system to be tested, the operation and maintenance monitoring system (used to detect the operation indicators of the service, see the preceding text for details) and the automated test system (for the interface function of the test service, see the previous text for details) involved in the solution of this application They are all deployed on the DevOps platform for unified management.
  • the DevOps platform also encapsulates the service instance acquisition, exception acquisition, exception injection and other interfaces for external calls, making the injection of exceptions and the testing of the service under test easier, and users don’t need to do so by themselves. Writing complex instructions is conducive to improving test efficiency.
  • step S200 can be implemented as: calling the first exception acquisition interface provided by the DevOps platform to obtain the server exception list, and calling the first exception injection interface provided by the DevOps platform to inject the server
  • the server in the server exception list is abnormal.
  • the server exception list contains all the server exceptions sorted out and encapsulated in advance.
  • the server exception to be injected it can be part or all of the exceptions in the list, depending on the test requirements. If there are multiple server exceptions that need to be injected, you can inject them sequentially, solve one exception and then inject the next exception, as described above.
  • the first exception injection interface can be a unified interface, and different server exceptions are injected by calling this interface: for example, in the server exception list, each server exception is represented by an exception code, and the first exception injection interface A parameter is provided to specify the exception code during design, so that after the first exception injection interface is called, the internal logic of the interface can determine the type of the current server exception based on the value passed in the parameter, and execute the corresponding exception injection behavior .
  • the first exception injection interfaces are provided for exception injection.
  • Step S210 can be implemented as follows: First, call the service instance acquisition interface provided by the DevOps platform to obtain the service list formed by the services deployed on the DevOps platform.
  • the service list contains all the services deployed on the DevOps platform.
  • the services are also deployed on the DevOps platform, so the service to be tested belonging to the system to be tested can be determined from the service list; then, the operation and maintenance monitoring system can be used to monitor the operating indicators of the service to be tested, and the automated test system can be used to test The interface function of the service to be tested.
  • the service exception list contains all service exceptions sorted out and encapsulated in advance.
  • the service exception to be injected it can be some or all of the exceptions in the list.
  • the second exception acquisition interface and the first exception acquisition interface may be implemented as the same interface or as different interfaces.
  • the second exception injection interface provided by the DevOps platform is called to inject the service exceptions in the service exception list into the service to be tested.
  • the second exception injection interface and the first exception injection interface can be implemented as the same interface or as different interfaces.
  • the second exception injection interface can be a unified interface. Different server exceptions are injected by calling this interface. It can also provide different second exception injection interfaces for exception injection for different types of service exceptions.
  • the implementation can refer to the implementation of the first exception injection interface above, and the explanation will not be repeated.
  • FIG. 3 shows a schematic diagram of the platform.
  • the bottom layer of the platform is the encapsulated abnormality
  • the left side is the server abnormality, for example, the disk abnormality, IO abnormality, power failure, network disconnection, etc. shown in Figure 3.
  • service exceptions such as database access timeout, array out-of-bounds, request exception, response exception, etc. shown in Figure 3.
  • the second layer is the integrated DevOps platform interface, including the first exception acquisition interface, the second exception acquisition interface, the first exception injection interface, the second exception injection interface, and the service instance acquisition interface mentioned above.
  • the third layer is the realization of the platform's possible functions.
  • the left side is the automated test system for interface testing
  • the right side is the operation and maintenance monitoring system for operation index monitoring
  • the middle is the three main methods of the methods provided in the embodiments of this application.
  • the steps are implemented by calling the interface provided by the DevOps platform in the second layer.
  • the left side of the top layer is a Web page for users to directly perform test operations
  • the right side is an API (Application Programming Interface), which is used to interface with third-party applications to integrate test functions into third-party applications.
  • API Application Programming Interface
  • FIG. 4 shows a functional module diagram of a system testing device 300 provided by an embodiment of the present application. 4, the system testing device 300 includes:
  • the injection module 310 is configured to inject server exceptions to the server running the system under test;
  • the test module 320 is configured to monitor the operating indicators of the service to be tested included in the system to be tested, and to test the interface function of the service to be tested.
  • the injection module 310 is further configured to: inject service exceptions into the service to be tested; the testing module 320 is also configured to monitor the operating indicators of the service to be tested, and test the The interface function of the service to be tested.
  • the testing module 320 is further configured to: after injecting service exceptions into the service to be tested, determine the related service of the service to be tested, and the related service includes the service to be tested.
  • the upstream service and/or downstream service of the service monitor the operation index of the related service, and test the interface function of the related service.
  • the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated test system are also deployed on the DevOps platform;
  • the server injecting server exceptions includes: calling the first exception acquisition interface provided by the DevOps platform to obtain a server exception list, and calling the first exception injection interface provided by the DevOps platform to inject the server in the server exception list to the server Abnormal;
  • the test module 320 monitors the operating indicators of the service to be tested included in the system to be tested, and tests the interface functions of the service to be tested, including: calling the service instance acquisition interface provided by the DevOps platform to obtain deployment in the DevOps
  • the service list formed by the services on the platform, and determine the service to be tested included in the system under test in the service list; use the operation and maintenance monitoring system to monitor the operating indicators of the service to be tested, and use the The automated test system tests the interface function of the service to be tested.
  • the server abnormality includes at least one of the following: a disk abnormality, an IO abnormality, a memory abnormality, a CPU abnormality, a power outage, and a network outage.
  • the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated test system are also deployed on the DevOps platform;
  • the injection module 310 injects into the service to be tested Service exceptions include: calling the service instance acquisition interface provided by the DevOps platform to obtain the service list formed by the services deployed on the DevOps platform, and determining the service under test included in the system under test in the service list Call the second exception acquisition interface provided by the DevOps platform to obtain the service exception list, and call the second exception injection interface provided by the DevOps platform to inject the service exceptions in the service exception list into the service under test;
  • test module 320 Monitoring the operation index of the service to be tested and testing the interface function of the service to be tested includes: using the operation and maintenance monitoring system to monitor the operation index of the service to be tested, and using the automated test system to test the service Describe the interface functions of the service to be tested.
  • the service abnormality includes at least one of the following: database access timeout, array out of bounds, request abnormality, and response abnormality.
  • FIG. 5 shows a possible structure of an electronic device 400 provided by an embodiment of the present application.
  • the electronic device 400 can implement the steps of the system test method shown in FIG. 2.
  • the electronic device 400 includes a processor 410, a memory 420, and a communication interface 430. These components are interconnected and communicate with each other through a communication bus 440 and/or other forms of connection mechanisms (not shown).
  • the memory 420 includes one or more (only one is shown in the figure), which can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), and programmable memory only Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory, EEPROM )Wait.
  • RAM Random Access Memory
  • ROM read only memory
  • PROM programmable memory only Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • the processor 410 includes one or more (only one is shown in the figure), which may be an integrated circuit chip with signal processing capability.
  • the aforementioned processor 410 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a micro controller unit (Micro Controller Unit, MCU), a network processor (Network Processor, NP), or other conventional processors; or Dedicated processors, including digital signal processors (Digital Signal Processor, DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
  • CPU Central Processing Unit
  • MCU Micro Controller Unit
  • NP Network Processor
  • NP Network Processor
  • Dedicated processors including digital signal processors (Digital Signal Processor, DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete Gate or transistor logic devices,
  • the communication interface 430 includes one or more (only one is shown in the figure), which can be used to directly or indirectly communicate with other devices to exchange data.
  • the communication interface 430 may include an interface for wired and/or wireless communication.
  • One or more computer program instructions may be stored in the memory 420, and the processor 410 may read and run these computer program instructions to implement the system test method provided by the embodiments of the present application and other desired functions.
  • the structure shown in FIG. 5 is only for illustration, and the electronic device 400 may also include more or fewer components than those shown in FIG. 5, or have a different configuration from that shown in FIG.
  • the components shown in FIG. 5 can be implemented by hardware, software, or a combination thereof.
  • the electronic device 400 may be a physical device, such as a PC, a notebook computer, a tablet computer, a server, an embedded device, etc., or a virtual device, such as a virtual machine, a virtualized container, etc.
  • the electronic device 400 is not limited to a single device, and may also be a combination of multiple devices or one or more clusters formed by a large number of devices.
  • the embodiment of the present application also provides a computer-readable storage medium having computer program instructions stored on the computer-readable storage medium.
  • the computer program instructions When the computer program instructions are read and run by a processor of the computer, the System test method.
  • the computer-readable storage medium may be implemented as the memory 420 in the electronic device 400 in FIG. 5.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation.
  • multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections between devices or units through some communication interfaces, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.
  • the system testing method, device, storage medium, and electronic equipment provided by the embodiments of the application first actively inject server exceptions into the server to simulate the real abnormal conditions of the server, and then monitor the operating indicators of the service to be tested and perform the interface function of the service to be tested Testing, and then performing statistical analysis based on the obtained operating indicators and test results, can determine the working conditions of the system under test when the server is abnormal.
  • this method provides a new solution for testing the working conditions of the business system when the server is abnormal, which significantly expands the testing scope of the business system in the conventional method and fills the existing technology. blank.
  • the abnormal injection in the above method is also beneficial to expose and solve the possible problems of the system under test when facing server abnormalities, and reduce the probability of failure of the system under test after it is actually online.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本申请涉及软件测试技术领域，提供一种系统测试方法、装置、存储介质及电子设备。其中，系统测试方法包括：向运行有待测系统的服务器注入服务器异常；监测待测系统包含的待测服务的运行指标，并测试待测服务的接口功能。该方法通过主动向运行有待测系统的服务器注入服务器异常，模拟服务器真实的异常状况，然后对待测服务的运行指标进行监测以及对待测服务的接口功能进行测试，进而后续基于得到的运行指标和测试结果进行统计分析，就可以确定在服务器出现异常的情况下待测服务、待测系统的工作情况，从而拓展了业务系统的常规测试范围，还有利于尽早暴露并解决在测试过程中出现的问题。

Description

一种系统测试方法、装置、存储介质及电子设备
相关申请的交叉引用
本申请要求于2020年06月09日提交中国专利局的申请号为2020105208162、名称为“一种系统测试方法、装置、存储介质及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及软件测试技术领域，具体而言，涉及一种系统测试方法、装置、存储介质及电子设备。
背景技术
在业务系统正式上线前都会对其进行测试，但在现有技术中，对业务系统进行测试基本上都是建立在服务器环境比较稳定的情况下，测试的范围比较局限，导致业务系统上线后故障较多。
发明内容
本申请实施例的目的之一在于提供一种系统测试方法、装置、存储介质及电子设备，以改善上述技术问题。
为实现上述目的，本申请提供如下技术方案：
第一方面，本申请实施例提供一种系统测试方法，包括：向运行有待测系统的服务器注入服务器异常；监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
在上述方法中，首先通过主动向运行有待测系统的服务器注入服务器异常，模拟服务器真实的异常状况，然后对待测服务的运行指标进行监测以及对待测服务的接口功能进行测试，进而后续基于得到的运行指标和测试结果进行统计分析，就可以确定在服务器出现异常的情况下待测系统的工作情况(例如，是否能够正常工作、工作过程中存在的问题等)。或者说上述方法提供了一种测试业务系统在服务器出现异常的情况下的工作情况的新方案，拓展了业务系统的常规测试范围，填补了现有技术中的空白。
进一步的，上述方法还有利于尽早暴露并解决在测试过程中出现的问题，降低待测系统在实际上线后出现故障的概率。
在第一方面的一种实现方式中，所述方法还包括：向所述待测服务注入服务异常；监测所述待测服务的运行指标，并测试所述待测服务的接口功能。
在上述实现方式中，通过主动向待测服务注入服务异常，模拟待测服务真实的异常状况，从而可以测试待测服务在异常状况下的工作情况。
在第一方面的一种实现方式中，在所述向所述待测服务注入服务异常之后，所述方法还包括：确定所述待测服务的相关服务，所述相关服务包括所述待测服务的上游服务和/或下游服务；监测所述相关服务的运行指标，并测试所述相关服务的接口功能。
待测系统中的服务之间可能具有依赖关系，被当前服务依赖的服务称为当前服务的上游服务，依赖于当前服务的服务称为当前服务的下游服务，若当前服务发生异常，可能导致它的下游服务发生异常，也有可能是它的上游服务异常所导致。因此，在向待测服务注入异常后，除了测试待测服务本身，还可以对待测服务的上游和/或下游服务的工作情况进行测试，全面评估异常在服务调用链中产生的影响。
在第一方面的一种实现方式中，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；所述向运行有待测系统的服务器注入服务器异常的步骤，包括：调用所述DevOps平台提供的第一异常获取接口获取服务器异常列表，并调用所述DevOps平台提供的第一异常注入接口向所述服务器注入所述服务器异常列表中的服务器异常；所述监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能，包括：调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
在第一方面的一种实现方式中，所述监测所述相关服务的运行指标，并测试所述相关服务的接口功能的步骤包括：
调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述相关服务；
利用所述运维监控系统监测所述相关服务的运行指标，并利用所述自动化测试系统测试所述相关服务的接口功能。
在第一方面的一种实现方式中，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；所述向所述待测服务注入服务异常的步骤，包括：调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；调用所述DevOps平台提供的第二异常获取接口获取服务异常列表，并调用所述DevOps平台提供的第二异常注入接口向所述待测服务注入所述服务异常列表中的服务异常；所述监测所述待测服务的运行指标，并测试所述待测服务的接口功能，包括：利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
DevOps平台可以理解为用来安装、部署应用程序的工具，DevOps强调的是高效组织 团队之间如何通过自动化的工具协作和沟通来完成软件的生命周期管理，从而更快、更频繁地交付更稳定的软件。在以上两种实现方式中，本申请方案中涉及的待测系统、运维监控系统以及自动化测试系统均部署在DevOps平台上进行统一管理，同时DevOps平台还封装服务实例获取、异常获取、异常注入等接口供外部调用，使得对于异常的注入以及对服务的测试等操作变得简单，用户无需自行编写复杂的指令。
在第一方面的一种实现方式中，所述方法还包括：
基于通过监测得到的运行指标进行统计分析，确定所述待测服务的工作情况；
根据所述待测服务的工作情况，确定所述待测服务是否出现服务异常。
在第一方面的一种实现方式中，所述方法还包括：
若确定所述待测服务出现服务异常，则获取与所述待测服务的工作情况对应的解决方案。
在第一方面的一种实现方式中，所述服务器异常包括以下至少一种：磁盘异常、IO异常、内存异常、CPU异常、断电以及断网。
在第一方面的一种实现方式中，所述服务异常包括以下至少一种：数据库访问超时、数组越界、请求异常以及响应异常。
第二方面，本申请实施例提供了一种测试平台，所述测试平台配置成利用上述的系统测试方法测试待测系统。
在第二方面的一种实现方式中，所述测试平台包括异常封装层、平台接口层、功能实现层及应用层；
所述异常封装层包括服务异常及服务器异常；
所述平台接口层包括第一异常获取接口、第二异常获取接口、第一异常注入接口、第二异常注入接口及服务实例获取接口，所述平台接口层配置成获取所述异常封装层包括的服务异常和/或服务器异常；
所述功能实现层配置成向运行有待测系统的服务器注入服务器异常，监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能；
所述应用层配置成接收测试操作。
第三方面，本申请实施例提供一种系统测试装置，包括：注入模块，配置成向运行有待测系统的服务器注入服务器异常；测试模块，配置成监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
在第三方面的一种实现方式中，所述注入模块，配置成向所述待测服务注入服务异常；
所述测试模块，配置成监测所述待测服务的运行指标，并测试所述待测服务的接口功能。
第四方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序指令，所述计算机程序指令被处理器读取并运行时，执行第一方面或第一方面的任意一种可能的实现方式提供的方法。
第五方面，本申请实施例提供一种电子设备，包括：存储器以及处理器，所述存储器中存储有计算机程序指令，所述计算机程序指令被所述处理器读取并运行时，执行第一方面或第一方面的任意一种可能的实现方式提供的方法。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本申请的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本申请实施例提供的一种待测系统的示意图；
图2示出了本申请实施例提供的一种系统测试方法的流程图；
图3示出了本申请实施例提供的一种测试平台的示意图；
图4示出了本申请实施例提供的一种系统测试装置的功能模块图；
图5示出了本申请实施例提供的一种电子设备的结构图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
本申请实施例提供的系统测试方法测试的对象为某个业务系统中的部分或全部的服务，被测试的业务系统检测待测系统，待测系统中被测试的服务简称待测服务。其中，待测服务既可以是传统服务也可以是微服务，本申请对此不限定。
图1示出了本申请实施例提供的一种待测系统的示意图。请参照图1，图1中示出的待测系统100包括前端服务110、业务侧微服务120、核心侧微服务130以及中间件140几个组成部分，该待测系统100可以部署在服务器或服务器集群上。待测系统10可能被用于不 同的业务场景，下面以其被用于视频结构化场景为例进行介绍，即在该场景下待测系统100为一个视频结构化系统。所谓视频结构化，可以指通过原始的视频流、视频文件进行智能分析，提取出其中关键信息，并进行文本的语义描述。视频结构化系统的一种可能的工作流程如下：
在用户希望进行视屏结构化时，可以从前端服务110提供的Web页面上勾选摄像头进行视频流采集或者上传视频文件，Web页面上控件的操作逻辑可以由业务侧微服务120提供支持，业务侧微服务120在获取到视频数据后，将其传递给核心侧微服务130，核心侧微服务130会调用相应的算法对视频数据进行结构化分析，并将分析结果返回给业务侧微服务120进行存储。中间件140具有工具性质，可以实现具有特定功能的服务，例如，Redis(一种高性能数据库)、RabbitMQ(一种消息队列)、Nginx(一种Web服务器)等。中间件140可以分为业务侧中间件和核心侧中间件，分别供业务侧微服务120和核心侧微服务130使用。视频结构化系统中各组件的部署顺序为：核心侧中间件、核心侧微服务130、业务侧中间件、业务侧微服务120、前端服务110。
视频结构化系统中的待测服务可以包括业务侧微服务120、核心侧微服务130以及中间件。发明人在研究视频结构化系统的测试工作时发现其中存在如下缺陷：
缺陷一：对于待测服务的测试都是建立在服务器的CPU、IO、网络环境等指标比较稳定的前提下，对服务器出现异常的情况，视频结构化系统以及系统包含的待测服务能否正常工作则缺少相应的测试方案。
缺陷二：视频结构化系统所涉及的服务之间具有一定的依赖关系，若一个服务出现异常，则依赖它的服务也可能会出现异常。例如，根据前文的阐述，视频结构化系统部署时业务侧中间件应该在业务侧微服务启动之前部署，若在某次部署中，业务侧中间件Redis在业务侧微服务启动之后才执行(可视为Redis在部署上的异常)，就会造成依赖于Redis的业务侧微服务无法正常工作。目前还缺乏对某个异常服务的上游服务和/或下游服务进行有效测试、确定其能否正常工作的方案。
以上缺陷，是发明人长期研究现有测试方案后发现的，因此，其发现过程以及下文中本申请实施例针对上述缺陷所提出的解决方案，都应该是发明人在对本申请做出的贡献。
可以理解的，图1仅为示例，本申请方案中涉及的待测系统未必采用图1所示的架构，因此图1不应视为对本申请保护范围的限制，但上面提出的两个缺陷具有普遍性。
图2示出了本申请实施例提供的一种系统测试方法的流程图。该方法可由一电子设备执行，图5示出了该电子设备可能的结构，该电子设备的结构可以参照后文关于图5的相关阐述。参照图2，该方法包括：
步骤S200：向运行有待测系统的服务器注入服务器异常。
步骤S210：监测待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
服务器异常可以包括，但不限于服务器的磁盘异常(如磁盘出现坏道，磁盘剩余空间过低等)、IO异常(如IO设备错误等)、内存异常(如内存占用过高等)、CPU异常(如CPU占用过高等)、断电以及断网等，可以事先梳理好服务器可能存在的异常，对这些异常进行封装保存，并对外提供接口以便在需要进行异常注入时可以获取到这些异常。步骤S200中的注入服务器异常，是指主动制造服务器异常场景用于模拟服务器真实的异常状况，比如，通过某个接口向服务器注入CPU异常，该接口的内部代码会设法提高CPU的占用率，制造服务器CPU出现异常的状况，从而可以在这样的异常状况下测试待测系统的运行状况。
步骤S210中的待测服务的运行指标，既可以包括单个待测服务的指标，也可以包括所有待测服务(或者说待测系统)总体的指标。这些指标包括但不限于CPU占用情况、IO速率、内存占用情况、虚拟内存(如Linux下的SWAP)占用情况、磁盘占用情况等。这些指标主要用于从外部环境的层面描述待测服务的运行状况，例如，若某个待测服务长时间占用CPU接近100％，可以合理推断其运行出现了异常。对待测服务的运行指标可以通过运维监控系统进行监测，例如，Prometheus系统、InfluxDB系统等。
仅仅基于上述运行指标并不能获知待测服务的功能是否正常，例如，某个待测服务虽然运行指标正常，但内部数据已经出现错误，调用该待测服务提供的接口总是向客户端返回错误的数据，则仍然属于一种异常状况。因此，还需要对待测服务的接口进行功能测试。接口测试可以采用自动化测试系统，如Postman、RESTClient等接口测试工具。
执行完步骤S210后，结合通过监测得到的运行指标和/或通过接口测试得到的测试结果进行统计分析，例如，通过监测得到的运行指标进行统计分析；又例如，通过接口测试得到的测试结果进行统计分析；再例如，通过监测得到的运行指标和接口测试得到的测试结果进行统计分析，就可以确定待测服务的工作情况，包括待测服务能否正常工作，若不能正常工作存在怎样的异常等，进而可以确定对应于该异常的解决方案，该解决方案可以理解为如何使得待测服务在服务器出现异常的状况下也可以维持正常运行的方案。上述统计分析过程以及解决异常的过程，既可以采用人工排查的方式，也可以根据异常现象自动分析、自动确定解决方案，本申请对此不作限定。由于待测系统是由若干服务构成的，所以确定了待测服务的工作情况，也就是确定了整个待测系统的工作情况。
由于服务器可能存在多种异常状况，因此可能需要多次向服务器注入不同种类的异常，分别进行测试。在一种实现方式中，为避免干扰，每次只注入一种类型的服务器异常，注入完成立即着手确定对应于该异常的解决方案，成功确定解决方案后再注入下一种类型的服务器异常。
综上所述，在图2示出的方法中，首先主动向服务器注入服务器异常以便模拟服务器真实的异常状况，然后对待测服务的运行指标进行监测以及对待测服务的接口功能进行测试，进而基于得到的运行指标和测试结果进行统计分析，就可以确定在服务器出现异常的情况下待测系统的工作情况。换句话说，该方法提供了一种测试业务系统在服务器出现异常的情况下的工作情况的新方案，显著地拓展了常规方法中对业务系统进行测试的测试范围，填补了现有技术中的空白。另一方面，通过上述方法中的异常注入还有利于尽早暴露并解决在面临服务器异常时待测系统可能出现的问题，降低待测系统在实际上线后出现故障的概率。
服务器异常主要是待测服务所处的外部环境中的异常，偏向于设备层面。除了服务器异常外，还有一类异常是服务异常，服务异常主要是待测服务的内部产生的异常，偏向于待测服务的代码逻辑或数据层面。
在一些实现方式中，除了向服务器注入异常外，还可以向待测服务注入服务异常以便模拟待测服务真实的异常状况，并在异常注入后，对待测服务的运行指标进行监测，以及对待测服务的接口功能进行测试。进而基于运行指标和/或测试结果进行统计分析，就可以确定待测服务的工作情况。例如，通过监测得到的运行指标进行统计分析；又例如，通过接口测试得到的测试结果进行统计分析；再例如，通过监测得到的运行指标和接口测试得到的测试结果进行统计分析。其中，该运行指标可以通过监测得到，该测试结果可以通过接口测试得到，并且还可以进一步确定对应于服务异常的解决方案，即如何消除服务异常，或者如何在服务出现异常的状况下使得待测服务仍然能够维持正常工作的方案。由于待测系统是由若干服务构成的，所以确定了待测服务的工作情况，也就是确定了整个待测系统的工作情况。对待测服务进行服务异常注入还有利于尽早暴露并解决在面临服务异常时待测系统可能出现的问题，降低待测系统在实际上线后出现故障的概率。
需要指出，对服务异常的测试和对服务器异常的测试是相互独立进行的，即允许二者分别选择服务进行测试，也允许二者分别选择独立的时间段进行测试。例如，可以选择相同的待测服务，先测试服务器异常，再测试服务异常，在测试服务异常时，服务器可以恢复成正常状态。
服务异常可以包括，但不限于数据库访问超时、数组越界、请求异常以(如请求参数不合法等)及响应异常(如响应数据错误等)等，可以事先梳理好服务可能存在的异常，对这些异常进行封装保存，并对外提供接口以便在需要进行异常注入时可以获取到这些异常。所谓的向待测服务注入服务异常，是指主动制造服务异常场景用于模拟服务真实的异常状况，比如，通过某个接口向待测服务注入错误的请求参数，就可以模拟实际应用中因客户端发送非法请求所导致的异常状况，从而可以在这样的异常状况下测试待测服务的运 行状况。
至于在异常注入后，监测待测服务的运行指标以及测试待测服务的接口功能，其方法和步骤S210类似，不再重复阐述。
由于服务异常可能有很多类型，同时待测服务也可能有多个，因此可能需要多次向不同的待测服务注入不同种类的异常，分别进行测试。在一种实现方式中，为避免干扰，每次只向一个待测服务注入一种类型的服务异常，注入完成立即着手确定对应于该异常的解决方案，成功确定解决方案后再向该待测服务注入下一种类型的服务异常，若某个待测服务已经注入并解决了所有类型的服务异常，则可以开始下一个待测服务的异常注入过程，直至所有的待测服务都完成异常注入并测试完毕。或者，对于一些相互之间没有依赖关系的待测服务，也可以进行并行测试。
进一步的，之前已经提到，待测系统中的服务之间可能具有依赖关系，被当前服务依赖的服务称为当前服务的上游服务，依赖于当前服务的服务称为当前服务的下游服务，上游服务和/或下游服务可以简称为当前服务的相关服务。例如，上游服务可以简称为当前服务的相关服务；又例如，下游服务可以简称为当前服务的相关服务；再例如，上游服务和下游服务均被简称为当前服务的相关服务。若当前服务发生异常，可能导致它的下游服务发生异常，也有可能是它的上游服务异常所导致。因此，在向待测服务注入异常后，除了测试待测服务本身，还可以对待测服务的相关服务的工作情况进行测试，全面评估异常在服务调用链中产生的影响。具体做法如下：
在向待测服务注入服务异常之后，首先根据服务之间的依赖关系确定待测服务的相关服务(当然如果事先已经记录每个服务的相关服务，直接获取就可以了)，然后监测相关服务的运行指标，并测试相关服务的接口功能，其做法和监测待测服务的运行指标以及测试待测服务的接口功能类似，不再重复。
下面再结合DevOps(部署Development和运维Operations的组合词)平台介绍本申请方案的一种实施方式。DevOps平台可以理解为用来安装、部署应用程序的工具，DevOps强调的是高效组织团队之间如何通过自动化的工具协作和沟通来完成软件的生命周期管理，从而更快、更频繁地交付更稳定的软件。
在一些实现方式中，本申请方案中涉及的待测系统、运维监控系统(用于检测服务的运行指标，具体见前文)以及自动化测试系统(用于测试服务的接口功能，具体见前文)均部署在DevOps平台上进行统一管理，同时DevOps平台还封装服务实例获取、异常获取、异常注入等接口供外部调用，使得对于异常的注入以及对待测服务的测试等操作变得简单，用户无需自行编写复杂的指令，有利于提高测试效率。
作为一种可选的实施方式，在采用DevOps平台时，步骤S200可以实现为：调用DevOps 平台提供的第一异常获取接口获取服务器异常列表，并调用DevOps平台提供的第一异常注入接口向服务器注入服务器异常列表中的服务器异常。其中，服务器异常列表中包含事先梳理并封装好的全部服务器异常，至于要注入的服务器异常可以是该列表中的部分或全部的异常，取决于测试需求。如果有多个服务器异常需要注入，可以采取依次注入的方式，解决一个异常再注入下一个异常，前文已述。
第一异常注入接口可以是一个统一的接口，不同的服务器异常在进行注入时都通过调用该接口完成：例如，在服务器异常列表中，每个服务器异常用一个异常代码表示，第一异常注入接口在设计时提供一个参数用于指定异常代码，从而在第一异常注入接口被调用后，接口的内部逻辑根据该参数传入的值就可以确定当前服务器异常的类型，并执行相应的异常注入行为。当然，也不排除在一些实现方式中，针对不同类型的服务器异常，提供不同的第一异常注入接口用于异常注入。
步骤S210则可以实现为：首先，调用DevOps平台提供的服务实例获取接口获取部署在DevOps平台上的服务形成的服务列表，该服务列表中包含了部署在DevOps平台上的全部服务，由于待测系统的服务也都部署在DevOps平台上，因此可以从该服务列表中确定出属于待测系统的待测服务；然后，可以利用运维监控系统监测待测服务的运行指标，并利用自动化测试系统测试待测服务的接口功能。
进一步的，在采用DevOps平台时，对于测试服务异常的情况，其具体实现如下：
首先，调用DevOps平台提供的服务实例获取接口获取部署在DevOps平台上的服务形成的服务列表，并确定服务列表中待测系统包含的待测服务。
然后，调用DevOps平台提供的第二异常获取接口获取服务异常列表，服务异常列表中包含事先梳理并封装好的全部服务异常，至于要注入的服务异常可以是该列表中的部分或全部的异常，取决于测试需求，第二异常获取接口和第一异常获取接口可以实现为同一接口也可以实现为不同的接口。之后，调用DevOps平台提供的第二异常注入接口向待测服务注入服务异常列表中的服务异常，第二异常注入接口和第一异常注入接口可以实现为同一接口也可以实现为不同的接口。如果有多个服务异常需要注入，可以采取依次注入的方式，解决一个异常再注入下一个异常，前文已述。第二异常注入接口可以是一个统一的接口，不同的服务器异常在进行注入时都通过调用该接口完成，也可以针对不同类型的服务异常，提供不同的第二异常注入接口用于异常注入，其实现可以参考上文对第一异常注入接口的实现，不再重复阐述。
最后，利用运维监控系统监测待测服务的运行指标，并利用自动化测试系统测试待测服务的接口功能，如有需要，还可以利用运维监控系统监测待测服务的相关服务的运行指标，并利用自动化测试系统测试待测服务的相关服务的接口功能。
本申请实施例还提供一种测试平台，该平台用于利用本申请实施例提供的系统测试方法测试待测系统，图3示出了该平台的示意图。参照图3，从下往上看，该平台最下层为封装好的异常，左侧为服务器异常，例如，图3中示出的磁盘异常、IO异常、断电、断网等。右侧为服务异常，例如，图3中示出的数据库访问超时、数组越界、请求异常、响应异常等。
第二层为集成的DevOps平台接口，包括上面提到的第一异常获取接口、第二异常获取接口、第一异常注入接口、第二异常注入接口、服务实例获取接口等。第三层为平台可能的功能实现，左侧为用于接口测试的自动化测试系统，右侧为用于运行指标监测的运维监控系统，中间为本申请实施例所提供的方法中三个主要的步骤，通过调用第二层中DevOps平台提供的接口实现。最上层左侧为Web页面，用于用户直接执行测试操作，右侧为API(Application Programming Interface，应用程序接口)，用于对接第三方应用程序，以便将测试功能集成到第三方应用程序中。
图4示出了本申请实施例提供的系统测试装置300的功能模块图。参照图4，系统测试装置300包括：
注入模块310，配置成向运行有待测系统的服务器注入服务器异常；
测试模块320，配置成监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
在系统测试装置300的一种实现方式中，注入模块310还配置成：向所述待测服务注入服务异常；测试模块320还配置成：监测所述待测服务的运行指标，并测试所述待测服务的接口功能。
在系统测试装置300的一种实现方式中，测试模块320还配置成：在向所述待测服务注入服务异常之后，确定所述待测服务的相关服务，所述相关服务包括所述待测服务的上游服务和/或下游服务；监测所述相关服务的运行指标，并测试所述相关服务的接口功能。
在系统测试装置300的一种实现方式中，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；注入模块310向运行有待测系统的服务器注入服务器异常，包括：调用所述DevOps平台提供的第一异常获取接口获取服务器异常列表，并调用所述DevOps平台提供的第一异常注入接口向所述服务器注入所述服务器异常列表中的服务器异常；测试模块320监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能，包括：调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
在系统测试装置300的一种实现方式中，所述服务器异常包括以下至少一种：磁盘异常、IO异常、内存异常、CPU异常、断电以及断网。
在系统测试装置300的一种实现方式中，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；注入模块310向所述待测服务注入服务异常，包括：调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；调用所述DevOps平台提供的第二异常获取接口获取服务异常列表，并调用所述DevOps平台提供的第二异常注入接口向所述待测服务注入所述服务异常列表中的服务异常；测试模块320监测所述待测服务的运行指标，并测试所述待测服务的接口功能，包括：利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
在系统测试装置300的一种实现方式中，所述服务异常包括以下至少一种：数据库访问超时、数组越界、请求异常以及响应异常。
图5示出了本申请实施例提供的电子设备400的一种可能的结构。该电子设备400可实现图2中所示出的系统测试方法的步骤。请参照图5，电子设备400包括：处理器410、存储器420以及通信接口430，这些组件通过通信总线440和/或其他形式的连接机构(未示出)互连并相互通讯。
其中，存储器420包括一个或多个(图中仅示出一个)，其可以是，但不限于，随机存取存储器(RandomAccessMemory，RAM)，只读存储器(Read Only Memory，ROM)，可编程只读存储器(Programmable Read-Only Memory，PROM)，可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)，电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)等。处理器410以及其他可能的组件可对存储器420进行访问，读和/或写其中的数据。
处理器410包括一个或多个(图中仅示出一个)，其可以是一种集成电路芯片，具有信号的处理能力。上述的处理器410可以是通用处理器，包括中央处理器(CentralProcessingUnit，CPU)、微控制单元(Micro Controller Unit，MCU)、网络处理器(Network Processor，NP)或者其他常规处理器；还可以是专用处理器，包括数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuits，ASIC)、现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
通信接口430包括一个或多个(图中仅示出一个)，可以用于和其他设备进行直接或间接地通信，以便进行数据的交互。通信接口430可以包括进行有线和/或无线通信的接口。
在存储器420中可以存储一个或多个计算机程序指令，处理器410可以读取并运行这些计算机程序指令，以实现本申请实施例提供的系统测试方法以及其他期望的功能。
可以理解，图5所示的结构仅为示意，电子设备400还可以包括比图5中所示更多或者更少的组件，或者具有与图5所示不同的配置。图5中所示的各组件可以采用硬件、软件或其组合实现。电子设备400可能是实体设备，例如PC机、笔记本电脑、平板电脑、服务器、嵌入式设备等，也可能是虚拟设备，例如虚拟机、虚拟化容器等。并且，电子设备400也不限于单台设备，也可以是多台设备的组合或者大量设备构成的一个或多个集群。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序指令，所述计算机程序指令被计算机的处理器读取并运行时，执行本申请实施例提供的系统测试方法。例如，计算机可读存储介质可以实现为图5中电子设备400中的存储器420。
在本申请所提供的实施例中，应该理解到，所揭露装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
再者，在本申请各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。
以上所述仅为本申请的实施例而已，并不用于限制本申请的保护范围，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。
工业实用性
本申请实施例提供的系统测试方法、装置、存储介质及电子设备，首先主动向服务器注入服务器异常以便模拟服务器真实的异常状况，然后对待测服务的运行指标进行监测以及对待测服务的接口功能进行测试，进而基于得到的运行指标和测试结果进行统计分析，就可以确定在服务器出现异常的情况下待测系统的工作情况。换句话说，该方法提供了一种测试业务系统在服务器出现异常的情况下的工作情况的新方案，显著地拓展了常规方法中对业务系统进行测试的测试范围，填补了现有技术中的空白。另一方面，通过上述方法 中的异常注入还有利于尽早暴露并解决在面临服务器异常时待测系统可能出现的问题，降低待测系统在实际上线后出现故障的概率。

Claims (16)

  1. 一种系统测试方法，其特征在于，包括：
    向运行有待测系统的服务器注入服务器异常；
    监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
  2. 根据权利要求1所述的系统测试方法，其特征在于，所述方法还包括：
    向所述待测服务注入服务异常；
    监测所述待测服务的运行指标，并测试所述待测服务的接口功能。
  3. 根据权利要求2所述的系统测试方法，其特征在于，在所述向所述待测服务注入服务异常之后，所述方法还包括：
    确定所述待测服务的相关服务，所述相关服务包括所述待测服务的上游服务和/或下游服务；
    监测所述相关服务的运行指标，并测试所述相关服务的接口功能。
  4. 根据权利要求3所述的系统测试方法，其特征在于，所述监测所述相关服务的运行指标，并测试所述相关服务的接口功能的步骤包括：
    调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述相关服务；
    利用所述运维监控系统监测所述相关服务的运行指标，并利用所述自动化测试系统测试所述相关服务的接口功能。
  5. 根据权利要求1所述的系统测试方法，其特征在于，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；
    所述向运行有待测系统的服务器注入服务器异常的步骤，包括：
    调用所述DevOps平台提供的第一异常获取接口获取服务器异常列表，并调用所述DevOps平台提供的第一异常注入接口向所述服务器注入所述服务器异常列表中的服务器异常；
    所述监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能的步骤，包括：
    调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；
    利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
  6. 根据权利要求1-5任一项所述的系统测试方法，其特征在于，所述方法还包括：
    基于通过监测得到的运行指标进行统计分析，确定所述待测服务的工作情况；
    根据所述待测服务的工作情况，确定所述待测服务是否出现服务异常。
  7. 根据权利要求6所述的系统测试方法，其特征在于，所述方法还包括：
    若确定所述待测服务出现服务异常，则获取与所述待测服务的工作情况对应的解决方案。
  8. 根据权利要求1-7中任一项所述的系统测试方法，其特征在于，所述服务器异常包括以下至少一种：磁盘异常、IO异常、内存异常、CPU异常、断电以及断网。
  9. 根据权利要求2所述的系统测试方法，其特征在于，所述待测系统部署在DevOps平台上，所述DevOps平台上还部署有运维监控系统以及自动化测试系统；
    所述向所述待测服务注入服务异常的步骤，包括：
    调用所述DevOps平台提供的服务实例获取接口获取部署在所述DevOps平台上的服务形成的服务列表，并确定所述服务列表中所述待测系统包含的所述待测服务；
    调用所述DevOps平台提供的第二异常获取接口获取服务异常列表，并调用所述DevOps平台提供的第二异常注入接口向所述待测服务注入所述服务异常列表中的服务异常；
    所述监测所述待测服务的运行指标，并测试所述待测服务的接口功能的步骤，包括：
    利用所述运维监控系统监测所述待测服务的运行指标，并利用所述自动化测试系统测试所述待测服务的接口功能。
  10. 根据权利要求2-9中任一项所述的系统测试方法，其特征在于，所述服务异常包括以下至少一种：数据库访问超时、数组越界、请求异常以及响应异常。
  11. 一种测试平台，其特征在于，所述测试平台配置成利用权利要求1-10任一项所述的系统测试方法测试待测系统。
  12. 根据权利要求11所述的测试平台，其特征在于，所述测试平台包括异常封装层、平台接口层、功能实现层及应用层；
    所述异常封装层包括服务异常及服务器异常；
    所述平台接口层包括第一异常获取接口、第二异常获取接口、第一异常注入接口、第二异常注入接口及服务实例获取接口，所述平台接口层配置成获取所述异常封装层包括的服务异常和/或服务器异常；
    所述功能实现层配置成向运行有待测系统的服务器注入服务器异常，监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能；
    所述应用层配置成接收测试操作。
  13. 一种系统测试装置，其特征在于，包括：
    注入模块，配置成向运行有待测系统的服务器注入服务器异常；
    测试模块，配置成监测所述待测系统包含的待测服务的运行指标，并测试所述待测服务的接口功能。
  14. 根据权利要求12所述的系统测试装置，其特征在于，所述注入模块，配置成向所述待测服务注入服务异常；
    所述测试模块，配置成监测所述待测服务的运行指标，并测试所述待测服务的接口功能。
  15. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序指令，所述计算机程序指令被处理器读取并运行时，执行如权利要求1-10中任一项所述的方法。
  16. 一种电子设备，其特征在于，包括：存储器以及处理器，所述存储器中存储有计算机程序指令，所述计算机程序指令被所述处理器读取并运行时，执行如权利要求1-10中任一项所述的方法。
PCT/CN2020/119616 2020-06-09 2020-09-30 一种系统测试方法、装置、存储介质及电子设备 WO2021248754A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010520816.2 2020-06-09
CN202010520816.2A CN111881014B (zh) 2020-06-09 2020-06-09 一种系统测试方法、装置、存储介质及电子设备

Publications (1)

Publication Number Publication Date
WO2021248754A1 true WO2021248754A1 (zh) 2021-12-16

Family

ID=73156842

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119616 WO2021248754A1 (zh) 2020-06-09 2020-09-30 一种系统测试方法、装置、存储介质及电子设备

Country Status (2)

Country Link
CN (1) CN111881014B (zh)
WO (1) WO2021248754A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114328210A (zh) * 2021-12-24 2022-04-12 中国联合网络通信集团有限公司 一种测试方法、装置及计算机可读存储介质
CN114706733A (zh) * 2022-05-30 2022-07-05 支付宝(杭州)信息技术有限公司 切面程序异常的监控方法和装置
CN115037653A (zh) * 2022-06-28 2022-09-09 北京奇艺世纪科技有限公司 业务流量监控方法、装置、电子设备和存储介质
CN115080436A (zh) * 2022-06-28 2022-09-20 中电金信软件有限公司 测试指标确定方法、装置、电子设备及存储介质
CN116541312A (zh) * 2023-07-06 2023-08-04 广汽埃安新能源汽车股份有限公司 一种汽车软件持续集成测试方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106708700A (zh) * 2016-12-13 2017-05-24 广东金赋科技股份有限公司 一种应用于服务端的运维监控方法和装置
WO2017172559A1 (en) * 2016-03-30 2017-10-05 Microsoft Technology Licensing, Llc Local chat service simulator for bot development
CN108268365A (zh) * 2016-12-30 2018-07-10 腾讯科技（深圳）有限公司 异常任务注入方法、装置和系统
CN109766370A (zh) * 2018-12-27 2019-05-17 口碑(上海)信息技术有限公司 数据处理方法、数据服务系统及设备
CN110262972A (zh) * 2019-06-17 2019-09-20 中国科学院软件研究所 一种面向微服务应用的失效测试工具及方法
CN111104336A (zh) * 2019-12-30 2020-05-05 武汉烽火信息集成技术有限公司 一种基于容器和vnc的服务接口在线测试方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9652365B2 (en) * 2010-08-24 2017-05-16 Red Hat, Inc. Fault configuration using a registered list of controllers
CN105808420B (zh) * 2014-12-31 2018-12-28 阿里巴巴集团控股有限公司 健壮性测试过程的实现方法和装置
US9747153B2 (en) * 2015-06-22 2017-08-29 Microsoft Technology Licensing, Llc Resilience as a service
CN110673993B (zh) * 2019-09-19 2023-05-05 聚好看科技股份有限公司 一种故障注入方法、平台及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017172559A1 (en) * 2016-03-30 2017-10-05 Microsoft Technology Licensing, Llc Local chat service simulator for bot development
CN106708700A (zh) * 2016-12-13 2017-05-24 广东金赋科技股份有限公司 一种应用于服务端的运维监控方法和装置
CN108268365A (zh) * 2016-12-30 2018-07-10 腾讯科技（深圳）有限公司 异常任务注入方法、装置和系统
CN109766370A (zh) * 2018-12-27 2019-05-17 口碑(上海)信息技术有限公司 数据处理方法、数据服务系统及设备
CN110262972A (zh) * 2019-06-17 2019-09-20 中国科学院软件研究所 一种面向微服务应用的失效测试工具及方法
CN111104336A (zh) * 2019-12-30 2020-05-05 武汉烽火信息集成技术有限公司 一种基于容器和vnc的服务接口在线测试方法及装置

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114328210A (zh) * 2021-12-24 2022-04-12 中国联合网络通信集团有限公司 一种测试方法、装置及计算机可读存储介质
CN114706733A (zh) * 2022-05-30 2022-07-05 支付宝(杭州)信息技术有限公司 切面程序异常的监控方法和装置
CN114706733B (zh) * 2022-05-30 2022-09-20 支付宝(杭州)信息技术有限公司 切面程序异常的监控方法和装置
CN115037653A (zh) * 2022-06-28 2022-09-09 北京奇艺世纪科技有限公司 业务流量监控方法、装置、电子设备和存储介质
CN115080436A (zh) * 2022-06-28 2022-09-20 中电金信软件有限公司 测试指标确定方法、装置、电子设备及存储介质
CN115080436B (zh) * 2022-06-28 2023-09-22 中电金信软件有限公司 测试指标确定方法、装置、电子设备及存储介质
CN115037653B (zh) * 2022-06-28 2023-10-13 北京奇艺世纪科技有限公司 业务流量监控方法、装置、电子设备和存储介质
CN116541312A (zh) * 2023-07-06 2023-08-04 广汽埃安新能源汽车股份有限公司 一种汽车软件持续集成测试方法及系统
CN116541312B (zh) * 2023-07-06 2023-09-22 广汽埃安新能源汽车股份有限公司 一种汽车软件持续集成测试方法及系统

Also Published As

Publication number Publication date
CN111881014A (zh) 2020-11-03
CN111881014B (zh) 2022-03-29

Similar Documents

Publication Publication Date Title
WO2021248754A1 (zh) 一种系统测试方法、装置、存储介质及电子设备
CN105389243B (zh) 一种容器监控方法和装置
US9069903B2 (en) Multi-platform test automation enhancement
WO2021169270A1 (zh) 服务器故障预警方法、装置、计算机设备及存储介质
CN112667362B (zh) Kubernetes上部署Kubernetes虚拟机集群的方法与系统
CN110768872A (zh) 巡检方法、系统、装置、计算机设备和存储介质
CN111768097B (zh) 任务执行状态监控方法、装置、系统及存储介质
CN102457578B (zh) 一种基于事件机制的分布式网络监控方法
CN112650676A (zh) 软件测试方法、装置、设备及存储介质
CN114153783B (zh) 多核通信机制的实现方法、系统、计算机设备及存储介质
WO2020253045A1 (zh) 配置化的数据转发异常补处理方法、装置及可读存储介质
US11449407B2 (en) System and method for monitoring computing platform parameters and dynamically generating and deploying monitoring packages
CN115952227A (zh) 数据采集系统及方法、电子设备和存储介质
CN111966599B (zh) 一种虚拟化平台可靠性测试方法、系统、终端及存储介质
CN113031969B (zh) 设备部署巡检方法、装置、计算机设备及存储介质
CN115499493A (zh) 异步事务处理方法、装置、存储介质及计算机设备
CN115202946A (zh) 自动化测试方法、装置、设备、存储介质及程序产品
CN114003416A (zh) 内存错误动态处理方法、系统、终端及存储介质
CN113360389A (zh) 一种性能测试方法、装置、设备及存储介质
CN112035295A (zh) 一种虚拟机崩溃事件处理方法、系统、终端及存储介质
Chen et al. Big data system testing method based on chaos engineering
CN115827287A (zh) Ssd长稳测试结果的统计方法、装置、计算机设备及存储介质
CN117349137A (zh) 分布式系统的自动化测试方法、装置、设备及存储介质
CN116859214A (zh) 远程测试方法、装置、工控机及集成电路测试系统
CN113778842A (zh) 容错测试的方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20939615

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24.03.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20939615

Country of ref document: EP

Kind code of ref document: A1