WO2023230883A1 - 一种测试方法、系统及装置 - Google Patents

一种测试方法、系统及装置 Download PDF

Info

Publication number
WO2023230883A1
WO2023230883A1 PCT/CN2022/096367 CN2022096367W WO2023230883A1 WO 2023230883 A1 WO2023230883 A1 WO 2023230883A1 CN 2022096367 W CN2022096367 W CN 2022096367W WO 2023230883 A1 WO2023230883 A1 WO 2023230883A1
Authority
WO
WIPO (PCT)
Prior art keywords
fault
hardware
module
computer
fault injection
Prior art date
Application number
PCT/CN2022/096367
Other languages
English (en)
French (fr)
Inventor
柴守刚
李汐
蔡建永
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202280007333.XA priority Critical patent/CN117597669A/zh
Priority to PCT/CN2022/096367 priority patent/WO2023230883A1/zh
Publication of WO2023230883A1 publication Critical patent/WO2023230883A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software

Definitions

  • the present application relates to the field of testing technology, and in particular, to a testing method, system and device.
  • fault injection is to use a certain strategy to artificially introduce faults of this fault type into the target system for a specified fault type, so as to observe and analyze the operating behavior of the target system under the condition of injected faults.
  • Fault injection is an important means to achieve safe and realistic simulation of various fault scenarios in the system. Fault injection plays a very important role in the integration and testing process of embedded software and hardware systems such as automotive electronics. Especially in safety-critical systems, comprehensive and sufficient fault testing is an important guarantee for the reliable operation of the system.
  • Safety-critical systems are generally composed of software and hardware, so fault injection testing is also performed from both software and hardware aspects.
  • Commonly used fault injection techniques are divided into two categories, namely software-based fault injection and hardware-based fault injection.
  • software-based fault injection has the problems of high cost and low efficiency, while software-based fault injection cannot cover hardware faults. How to flexibly and comprehensively implement fault injection at the software level and hardware level has become an urgent problem to be solved.
  • a testing method, system and device are proposed that can flexibly and comprehensively implement fault injection at the software level and hardware level.
  • embodiments of the present application provide a testing method, which method is applied to a virtualization software and hardware platform.
  • the virtualization software and hardware platform includes a virtualization hardware module and an embedded software module, wherein the virtualization software and hardware platform includes a virtualization hardware module and an embedded software module.
  • the hardware module is used to represent a virtualized embedded hardware system that can run on a general-purpose computer.
  • the method includes: obtaining fault injection configuration information, the fault injection configuration information is used to indicate the fault injection point and fault type, the The fault injection point is located in the virtualized hardware module or the embedded software module; fault injection is performed to the fault injection point according to the fault type.
  • testing methods provided by the embodiments of this application can be applied to the testing and verification of any electronic control system that combines software and hardware, including automotive electronic control systems, robot electronic control systems, etc.
  • a virtualized embedded hardware system that can run on a general-purpose computer is used to replace the actual hardware device, thereby reducing the testing cost; fault injection into the virtualized embedded hardware system is just like software fault injection. Injection is equally flexible, thereby improving hardware fault test coverage.
  • the method further includes: dynamically translating binary execution instructions of the embedded hardware system into binary execution instructions that can be run on a general-purpose computer, and obtain the Describe the virtualization hardware module.
  • virtualization of the embedded hardware system is achieved through dynamic translation of binary execution instructions, thereby reducing test costs and improving flexibility and coverage.
  • the binary execution instructions of the embedded hardware system are dynamically translated into binary execution instructions that can be run on a general-purpose computer.
  • Instructing to obtain the virtualized hardware module includes: according to test requirements, determining a target binary execution instruction from binary execution instructions of the embedded hardware system and performing the dynamic translation to obtain the virtualization hardware module.
  • the method further includes: after performing the fault injection, generating a control signal;
  • the control signal is sent to the controlled object simulation platform, so that the controlled object simulation platform performs performance simulation on the controlled object according to the control signal, and obtains fault test feedback results.
  • the method further includes: receiving the fault test feedback result; sending the fault feedback result to the fault Inject the console so that the fault injection console performs fault analysis based on the fault test feedback results and the preconfigured expected fault processing results.
  • performing fault injection into the fault injection point according to the fault type includes: : According to the fault type, perform hardware fault injection to the fault injection point located in the virtualized hardware module by modifying the calibration protocol or the universal measurement and calibration protocol.
  • hardware fault simulation can be realized by modifying the calibration protocol or the general measurement and calibration protocol.
  • the method according to the Injecting a fault into the fault injection point according to the fault type includes: injecting a software fault into the fault injection point located in the embedded software module through code injection or state injection according to the fault type.
  • software fault simulation can be achieved through code injection or state injection.
  • test system includes: a fault injection console and a virtualization software and hardware platform.
  • the virtualization software and hardware platform includes a virtualization hardware module and an embedded software module. , wherein the virtualization hardware module is used to represent a virtualized embedded hardware system that can run on a general-purpose computer;
  • the fault injection console is used to collect fault injection configuration information and send the fault injection configuration information to the virtualized software and hardware platform.
  • the fault injection configuration information is used to indicate fault injection points and fault types, so The fault injection point is located in the virtualized hardware module or the embedded software module;
  • the virtualization software and hardware platform is configured to receive the fault injection configuration information and perform fault injection to the fault injection point according to the fault type.
  • the test system further includes: a controlled object simulation platform;
  • the virtualized software and hardware platform is also used to generate a control signal after fault injection, and send the control signal to the controlled object simulation platform;
  • the controlled object simulation platform is used to perform performance simulation on the controlled object according to the control signal to obtain fault test feedback results.
  • the controlled object simulation platform is also used to send the fault test feedback results to the virtualization software and hardware platform;
  • the virtualization software and hardware platform is also used to send the fault test feedback results to the fault injection console;
  • the fault injection console is also used to perform fault analysis based on the fault test feedback results and preconfigured expected fault processing results.
  • the fault injection console includes a fault configuration module and a fault recovery module;
  • the fault configuration module is used to provide a human-computer interaction interface and collect the fault injection configuration information through the human-computer interaction interface;
  • the fault recovery module is configured to receive the fault test feedback result, and perform fault analysis based on the fault test feedback result and the preconfigured expected fault processing result.
  • the virtualized software and hardware platform further includes a hardware fault injection module and a software fault injection module.
  • the hardware fault injection module is configured to inject hardware faults into the fault injection point located in the virtualized hardware module according to the fault type;
  • the software fault injection module is configured to inject software faults into the fault injection point located in the embedded software module according to the fault type.
  • the virtualization software and hardware platform and The controlled object simulation platform is deployed on a cloud server.
  • the virtualization software and hardware platform and the controlled object simulation platform are provided with a unified The simulation clock or hardware clock is configured, and the virtualization software and hardware platform and the controlled object simulation platform perform step synchronization settings through code injection or state injection.
  • inventions of the present application provide a testing device, which is applied to a virtualization software and hardware platform.
  • the virtualization software and hardware platform includes a virtualization hardware module and an embedded software module, wherein the virtualization software and hardware platform includes a virtualization hardware module and an embedded software module.
  • the virtualized hardware module is used to represent a virtualized embedded hardware system that can run on a general-purpose computer.
  • the device includes:
  • Obtaining module used to obtain fault injection configuration information, the fault injection configuration information is used to indicate the fault injection point and fault type, the fault injection point is located in the virtualized hardware module or the embedded software module;
  • An injection module is configured to inject faults into the fault injection points acquired by the acquisition module according to the fault types acquired by the acquisition module.
  • the device further includes:
  • a translation module is used to dynamically translate the binary execution instructions of the embedded hardware system into binary execution instructions that can be run on a general-purpose computer to obtain the virtualization hardware module.
  • the translation module is also used to:
  • a target binary execution instruction is determined from the binary execution instructions of the embedded hardware system and the dynamic translation is performed to obtain the virtualized hardware module.
  • the device further includes:
  • a generation module configured to generate a control signal after performing the fault injection
  • the first sending module is used to send the control signal to the controlled object simulation platform, so that the controlled object simulation platform performs performance simulation on the controlled object according to the control signal and obtains fault test feedback results.
  • the device further includes:
  • a receiving module configured to receive the fault test feedback results
  • the second sending module is configured to send the fault feedback result to the fault injection console, so that the fault injection console performs fault analysis based on the fault test feedback result and the preconfigured expected fault processing result.
  • the injection module is further configured to: according to the fault type, modify the calibration protocol Alternatively, a hardware fault is injected into the fault injection point located in the virtualized hardware module using a universal measurement and calibration protocol.
  • the injection module further Used for: according to the fault type, perform software fault injection into the fault injection point located in the embedded software module through code injection or state injection.
  • embodiments of the present application provide a testing device that can perform one or more of the testing methods of the first aspect or multiple possible implementations of the first aspect.
  • embodiments of the present application provide a computer program product, including a computer readable code, or a non-volatile computer readable storage medium carrying the computer readable code, when the computer readable code is stored electronically
  • the processor in the electronic device executes one or more of the testing methods of the first aspect or multiple possible implementations of the first aspect.
  • embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and the computer program instructions are used by a processor to execute the above-mentioned first aspect or multiple aspects of the first aspect.
  • Figure 1 shows a schematic structural diagram of a test system provided by an embodiment of the present application
  • Figure 2 shows an exemplary structural schematic diagram of the virtualization software and hardware platform provided by the embodiment of the present application
  • Figure 3 shows an exemplary schematic diagram of the hardware fault injection process provided by the embodiment of the present application
  • Figure 4 shows an interactive flow chart of the testing method provided by the embodiment of the present application
  • Figure 5 shows an exemplary structural schematic diagram of a test system provided by an embodiment of the present application
  • Figure 6 shows a schematic structural diagram of a testing device provided by an embodiment of the present application.
  • FIG. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • exemplary means "serving as an example, example, or illustrative.” Any embodiment described herein as “exemplary” is not necessarily to be construed as superior or superior to other embodiments.
  • Software-based fault injection is to inject faults at the software level to generate errors, thereby creating hardware or system-level faults, and then analyze the system feedback data after the fault is injected to conduct system reliability testing.
  • the fault injection point is in the software system, such as in the application programming interface (Application Programming Interface, API) of the application software, or in the API of the operating system (Operating System, OS), etc.
  • fault injection methods can include directly modifying the statements of the executing program, modifying or adding or deleting data in the software system, modifying the running status of the software system, or modifying memory data through software debugging tools. After flexibly selecting fault injection points and fault injection methods, software modules such as application software, operating systems, and underlying driver software can be tested.
  • the above-mentioned software fault injection method has the characteristics of low cost and easy control. It does not require additional hardware equipment. Flexible fault injection points and multiple fault injection methods can be selected at the entire software level. It is widely used in software system testing. The above software-based fault injection has the problem that it can only test software faults but cannot test hardware faults.
  • Hardware-based fault injection is to incorporate the hardware system into the test system, and at the same time select the fault injection point for the hardware and select the corresponding fault injection method.
  • Embedded systems generally include hardware components such as computing, storage, communication, input and output (Input Output, IO). These hardware components themselves or their interfaces can be used as fault injection points.
  • the fault injection method can be that the hardware component itself fails or The hardware interface pin has failed.
  • System reliability testing is performed by introducing faults on hardware components and then analyzing the system feedback data after the fault is injected.
  • the above hardware-based fault injection method requires the construction of a hardware system for each fault test system. It can be seen that if a large number of hardware fault injection tests are performed, a large number of hardware devices need to be configured, which results in high testing costs and low testing efficiency. In addition, in chip packaging, system-on-chip and other scenarios, many hardware components are packaged together, and the hardware fault injection location is inaccessible, resulting in the problem of incomplete fault injection testing, that is, the problem of low hardware fault test coverage.
  • the embodiments of this application provide a testing method, system and device, which can be applied to the testing and verification of any electronic control system that combines software and hardware, including automotive electronic control systems, robot electronic control systems, etc.
  • a virtualized embedded hardware system that can run on a general-purpose computer is used to replace the actual hardware device, thereby reducing the testing cost; fault injection into the virtualized embedded hardware system is just like software fault injection. Injection is equally flexible, thereby improving hardware fault test coverage.
  • the virtualized embedded hardware system and the controlled object simulation platform are all deployed in the cloud, thereby improving the testing efficiency and the utilization rate of the testing system.
  • Figure 1 shows a schematic structural diagram of a test system provided by an embodiment of the present application.
  • the test system provided by the embodiment of the present application includes a fault injection console, a virtualization software and hardware platform, and a controlled object simulation platform.
  • the fault injection console is the entrance to the test system for testers to operate. After testers log in to the fault injection console, they can initiate fault injection tests.
  • the virtualized software and hardware platform can perform software and hardware fault injection after the fault injection test initiated by the tester.
  • the controlled object simulation platform can virtualize the software and hardware platform and initiate simulation test verification after completing fault injection.
  • testers can log in to the fault injection console through the terminal device.
  • the virtualization hardware platform and the controlled object simulation platform can be deployed on the cloud server, and they can be deployed on the same cloud server or on different cloud servers. This is not limited by the embodiments of this application.
  • the terminal device and the cloud server can communicate. The embodiments of this application do not limit the communication method between the terminal device and the cloud server. The specific information of each part that constitutes the test system is described in detail below.
  • the fault injection console is the entrance to the test system for testers to operate.
  • the form of fault injection console includes but is not limited to applications, applets, or web pages.
  • the front-end of the fault injection console provides a human-computer interaction interface for interacting with testers, and the back-end provides functions for interacting with virtualized software and hardware platforms.
  • the fault injection configuration information selected by the tester is transmitted to the virtualization software and hardware platform; on the other hand, the fault test feedback results are received from the virtualization software and hardware platform. Therefore, the terminal device used by testers to log in to the fault injection console only needs to be able to provide a human-computer interaction interface and back-end communication functions.
  • the terminal device involved in the embodiments of this application may be a smartphone, a netbook, a tablet, a notebook, a wearable electronic device (such as a smart bracelet, a smart watch, etc.), a TV, a virtual reality device, etc.
  • the embodiments of this application are not limiting.
  • the fault injection console can be used to collect fault injection configuration information and send the fault injection configuration information to the virtualization hardware platform.
  • the fault injection configuration information can be used to indicate the fault injection point and fault type.
  • the fault injection point can be used to indicate the location of the injected fault.
  • the fault injection point may be located in the virtualized hardware module or the embedded software module.
  • the virtualized hardware module and the embedded software module will be introduced later and will not be detailed here. .
  • When the fault injection point is located in the virtualized hardware module it indicates that the fault to be injected is a hardware fault; when the fault injection point is located in the embedded software module, it indicates that the fault to be injected is a software fault.
  • the fault type can be used to indicate the type of injected fault. Different fault types require different injection methods for fault injection. Fault injection points, fault types and fault injection methods will be introduced later and will not be described in detail here.
  • the fault injection console includes a fault configuration module and a fault recovery module.
  • the fault configuration module can be used to provide a human-computer interaction interface, and collect fault injection configuration information through the human-computer interaction interface.
  • the fault recovery module may be configured to receive the fault test feedback result, and perform fault analysis based on the fault test feedback result and the preconfigured expected fault processing result.
  • the fault test feedback results come from the virtualization software and hardware platform. For fault analysis methods, you can refer to related technologies and will not be repeated here.
  • the tester can select the fault injection point and fault type through the fault injection console; at the same time, the expected fault processing results need to be configured to match the actual fault test results (that is, fault tests from the virtualized software and hardware platform feedback results) for comparison.
  • the embodiment of this application provides a fault mode library based on existing experience (the fault mode library stores multiple fault injection points and multiple fault types) for testers to choose. Of course, testers can also manually Configure fault injection points and fault types.
  • the virtualization software and hardware platform may be configured to receive the fault injection configuration information and perform fault injection to the fault injection point according to the fault type.
  • FIG. 2 shows an exemplary structural diagram of a virtualization software and hardware platform provided by an embodiment of the present application.
  • the virtualization software and hardware platform includes virtualization hardware modules and embedded software modules.
  • the virtualization hardware module may be used to represent a virtualized embedded hardware system that can run on a general-purpose computer.
  • virtualized hardware modules include but are not limited to virtualized central processing unit (CPU), virtualized memory, virtualized bus, virtualized onboard communication module, virtualized IO module, etc.
  • hardware virtualization can be divided into different types such as full virtualization, partial virtualization, and operating system virtualization according to different virtualization levels. The specific type can be flexibly determined according to the testing requirements of the actual system. selection, thereby improving the efficiency of virtualized hardware module runtime.
  • the binary execution instructions of the embedded hardware system can be dynamically translated into binary execution instructions that can be run on a general-purpose computer to obtain the virtualized hardware module.
  • the entire embedded hardware system can be virtualized, and then the embedded hardware system can be simulated and run on a general-purpose computer.
  • the virtualization of the embedded hardware system can be refined to the granularity of the instruction set. This provides a guarantee for the simulation accuracy of the hardware.
  • the dedicated instruction set of the embedded hardware system can be dynamically translated into a general x86 instruction set, and a virtual hardware embedded system can be simulated on a general-purpose computer.
  • dynamically translating the binary execution instructions of the embedded hardware system into binary execution instructions that can be run on a general-purpose computer, and obtaining the virtualized hardware module may include: according to the test requirements, from the embedded The target binary execution instruction is determined among the binary execution instructions of the conventional hardware system and the dynamic translation is performed to obtain the virtualization hardware module.
  • the target binary execution instruction represents any part of the binary execution instructions in the binary execution instructions of the embedded hardware system. It can be seen that in the embodiment of the present application, part of the embedded hardware system can be virtualized, and then part of the embedded hardware system can be simulated and run on a general-purpose computer.
  • the operating system of the embedded hardware system can also be virtualized, and then the operating system of the embedded hardware system can be simulated on a general-purpose computer.
  • the virtualized Electronic Control Unit (ECU) resource cloud is used as the virtualized software and hardware platform, which requires the microcontroller unit (Micro Controller Unit, MCU) and the main processing unit (Master Process Unit). , MPU), memory, IO and communication modules are virtualized to obtain the above virtualization software and hardware platform.
  • MCU Micro Controller Unit
  • Master Process Unit Master Process Unit
  • a fully virtualized MCU is a complete independent operating system that can run the same operating system as the actual hardware MCU, so the actual generated code can run directly on the fully virtualized MCU.
  • open source software such as Quick EMUlator (QEMU) can be used to achieve full virtualization of the MCU.
  • system design based on the C++ language
  • QEMU computer language for system design
  • microarchitecture simulator microarchitecture simulator, macsim
  • EMUlator EMU
  • MPU For MPU systems, because the hardware of the MPU is similar to that of a general-purpose computer, and the main frequency of the MPU is higher, the computing pressure of a fully virtualized MPU is greater for a general-purpose computer. Therefore, partial virtualization can be used to virtualize the MPU or Only the operating system of the MPU is virtualized.
  • technologies such as OpenVZ, Linux Virtual Server (Linux VServer), Linux Containers (LXC), etc. may be used to implement MPU virtualization.
  • Embedded software modules may be used to represent actual software capable of running on virtualized embedded hardware systems.
  • embedded software modules include but are not limited to software modules such as basic software, operating systems, and application software. These software modules are real software codes used for actual generation.
  • the real embedded software module and the high-precision virtualized hardware module provide the basis for constructing a complete high-precision fault injection test.
  • the virtualized software and hardware platform also includes a hardware fault injection module and a software fault injection module.
  • the hardware fault injection module can be used to inject hardware faults into the fault injection point located in the virtualized hardware module according to the fault type
  • the software fault injection module can be used to inject hardware faults into the fault injection point located in the embedded software according to the fault type.
  • the fault injection point of the module performs software fault injection.
  • software faults can be divided into: functional faults, algorithm faults, sequence faults, inspection faults and assignment faults; according to function and operating environment, software faults can be divided into: external interface Types of faults, interaction faults between internal modules, configuration faults, data faults, resource faults, operating system faults, and driver faults. It can be seen that software faults involve many levels and types, and the fault coverage rate is high.
  • software fault injection can be performed through code injection.
  • Code injection is to directly change a certain part of the original code to produce a syntactically correct but semantically incorrect fault point.
  • By applying a patch to a specific location in the target code the behavior of the code under test is changed, thereby achieving the goal of software fault injection. Purpose.
  • software fault injection can be performed through state injection.
  • State injection is to inject faults by changing the running status or behavior of the software system, and by simulating the errors caused by a fault, test the final response of the entire system. For example, to test the system's fault tolerance for data inconsistencies, you can deliberately modify the data type, and then observe the difference between the subsequent output results of the system and the output results of normal operation.
  • software fault injection can be achieved through the following technical means.
  • the first is software fault injection based on piling: piling is performed in the software module to force the software to run and test the fault mode set by the user instead of the normal mode, thereby achieving fault injection.
  • the second type is process-based soft fault injection: In the software fault injection module, a high-priority process dedicated to fault injection is initiated. This process can modify the calculation status to achieve fault injection.
  • the third type is debugger-based soft fault injection: using the tools provided by the debugger to inject errors into a running process by setting breakpoints.
  • the tools provided by the debugger can be debugger (debugger, dbx) or gun debugger (gun debugger, gdb), etc.
  • message-based software fault injection For software modules that communicate through messages, message-based tools can be used to deliberately destroy the content and sequence of messages to generate wrong messages, thereby achieving software fault injection.
  • the fifth type is storage-based software fault injection: using memory operation tools to directly modify the data and status values of software modules in the memory, inject faults into the storage system, and introduce faults through the storage system.
  • the sixth type is command-based software fault injection: using commands from the tester interface or remote maintenance port to change the status of the system such as running, operation, management, operation and maintenance, etc., so that it works in the fault mode, that is, to inject faults into the test software in the system.
  • MCU failure causes the code to not run normally
  • memory failure causes data loss or inability to read and write
  • IO failure causes the inability to connect to external devices.
  • the communication link fails, resulting in the inability to send and receive data normally, etc.
  • this can be achieved by modifying the calibration protocol (CAN Calibration Protocol, CCP) or the Universal Measurement and Calibration Protocol (Universal Measurement and Calibration Protocol, XCP).
  • CCP is a matching calibration protocol based on the Controller Area Network bus (Controller Area Network, CAN) bus, which can detect or modify the variable values inside the ECU online.
  • CANape is a CCP-based ECU calibration and testing tool.
  • the communication between CANape and ECU is realized through a description file, which records the detailed information of each parameter in the ECU, such as the storage address, storage structure, data type and conversion formula of variables in the ECU, etc. Through this file, you can directly access and modify the variables in the ECU.
  • the specific modification process is realized through the mapping mechanism specified by the CCP protocol, which ensures that the content in the description file and the actual variables inside the ECU are synchronized.
  • the XCP protocol is an extension of the CCP protocol. It can adapt to a variety of underlying network protocols and bus types, such as CAN, Ethernet, FlexRay, Serial Communication Interface (SCI), Serial Peripheral Interface ( Serial Peripheral Interface (SPI), Universal Serial Bus (Universal Serial Bus, USB), etc
  • FIG 3 shows an exemplary schematic diagram of the hardware fault injection process provided by the embodiment of the present application.
  • the hardware fault injection module and the virtualization hardware module are connected through a virtual bus.
  • the hardware fault injection module adds fault injection points to the description file of the CCP protocol or XCP protocol, and maps these fault injection points to the registers of various hardware modules such as CPU, memory, IO or communication through the CCP protocol or XCP protocol.
  • the hardware fault injection module modifies the variables in the description file. By modifying these variables and injecting the required various fault status and data (i.e., fault types), these variables will be automatically mapped to the virtualized hardware module, thus achieving Hardware fault injection.
  • the original function of the CCP protocol and the XCP protocol is to calibrate and match the internal variables of the ECU.
  • the mapping relationship between the description files in the CCP protocol and the XCP protocol to the ECU memory address space is used to inject faults.
  • the software fault injection module is similar to the software-based fault injection method in related technologies, while the flexibility of the hardware fault injection module is greatly increased, because after hardware virtualization, fault injection into hardware is like software Fault injection is equally flexible, thereby greatly improving the coverage and efficiency of hardware fault injection testing.
  • virtualized hardware modules can be replicated on a large scale, which greatly reduces costs compared with large-scale deployment of actual embedded hardware systems.
  • the fault injection technology of virtualized hardware is cleverly designed by using the mapping relationship between the description file in the CCP protocol or XCP protocol and the ECU memory address space.
  • the software can directly use the actual production code, which is more accurate than pure software in the loop (Software In Loop, SIL) simulation verification.
  • the virtualized software and hardware platform can also be used to generate control signals after fault injection, and send the control signals to the controlled object simulation platform.
  • the function of the controlled object simulation platform is to replace the actual controlled object with high-precision simulation method.
  • the vehicle simulation tool cloud can be used to replace the actual vehicle.
  • the controlled object simulation platform can be used to: receive control information from the virtualized software and hardware platform; perform performance simulation of the accused crime according to the control signal to obtain fault test feedback results; and send fault test feedback results to the virtualization Software and hardware platforms. Afterwards, after the virtualization software and hardware platform receives the fault test feedback results, it can send the fault test feedback results to the fault recovery module of the fault injection console for analysis.
  • vehicle simulation tools mainly include dynamics simulation tools and economic simulation tools. Therefore, the performance simulation performed in the controlled object simulation platform mainly includes dynamics simulation and economic simulation. Among them, the dynamics simulation mainly simulates the vehicle's acceleration ability, climbing ability, etc., and the economic simulation mainly simulates the vehicle's endurance ability, etc.
  • existing commercial or open source vehicle simulation tools can be directly deployed in the cloud, and then directly connected to the virtualized ECU resource cloud (corresponding to the controlled object simulation platform shown in Figure 1) Virtualization software and hardware platform) can be connected.
  • the virtualized software and hardware platform and the controlled object simulation platform can achieve synchronization by setting a unified simulation clock or configuring a hardware clock, and achieve step synchronization setting through code injection or state injection. Therefore, after deploying the vehicle simulation tool and the virtualized ECU resource cloud in the cloud, you can synchronize the vehicle simulation tool and the virtualized ECU resource cloud by setting a unified simulation clock or configuring a hardware clock.
  • the method of configuring the hardware clock can refer to the method of hardware fault injection, which will not be described again here.
  • the setting method of synchronization step size can refer to the software fault injection method, which will not be described again here.
  • fault injection testing is to observe the deviation between the response of the system after fault injection and the response of the normally working system, thereby evaluating the reliability of the system. Therefore, the accuracy and timeliness of fault feedback play an important role in the test system.
  • fault feedback is the output data or status of the software system, which is generally recorded in the form of logs and then analyzed offline.
  • the hardware-based test system either the log method is used to record the fault in the hardware system and then exported for offline analysis; or the hardware system is connected to the actual controlled system and the feedback data of the controlled system is collected to analyze the fault. analyze.
  • the fault feedback recording method described above shows that by recording logs and analyzing data offline, it is equivalent to the entire fault testing system being an open-loop system.
  • the tested software and hardware system is not connected to the actual controlled object, and lacks Real feedback of software and hardware failures from actual controlled objects.
  • the simulation system of the controlled object is used to replace the actual controlled object.
  • a complete closed-loop system of fault injection, control, simulation, and test result feedback is realized, so that the actual controlled system can be obtained.
  • Real feedback in the event of software and hardware failures improves the accuracy of fault test feedback results; on the other hand, it further reduces testing costs.
  • Figure 4 shows an interactive flow chart of the testing method provided by the embodiment of the present application. This method can be applied to the test system shown in Figure 1. As shown in Figure 4, the method includes:
  • Step S400 The tester configures the fault injection configuration information on the fault injection console.
  • the fault injection configuration information can be used to indicate the fault injection point and fault type.
  • the fault injection point can be used to indicate the location of the injected fault.
  • the fault injection point may be located in a virtualized hardware module or an embedded software module.
  • a virtualized hardware module may be used to represent a virtualized embedded hardware system capable of running on a general-purpose computer.
  • virtualized hardware modules include but are not limited to virtualized central processing unit (CPU), virtualized memory, virtualized bus, virtualized onboard communication module, virtualized IO module, etc.
  • Embedded software modules may be used to represent actual software capable of running on virtualized embedded hardware systems.
  • embedded software modules include but are not limited to software modules such as basic software, operating systems, and application software. These software modules are real software codes used for actual generation.
  • a fault mode library (which stores multiple fault injection points and multiple fault types) is provided based on existing experience for testers to choose. Of course, testers can also manually configure fault injection points and faults. type.
  • Step S401 The fault injection console collects fault injection configuration information.
  • Step S402 The fault injection console sends the fault injection configuration information to the virtualization software and hardware platform.
  • Step S403 The virtualization software and hardware platform performs fault injection to the fault injection point indicated by the fault injection configuration information according to the fault type indicated by the fault injection configuration information.
  • the virtualized software and hardware platform can inject hardware faults to the fault injection point located in the virtualized hardware module through a hardware fault injection module; and inject software faults into the fault injection point located in the embedded software module through a software fault injection module. .
  • the injection methods of software faults and hardware faults are as mentioned above and will not be described again here.
  • Step S404 After performing fault injection, the virtualization software and hardware platform generates a control signal.
  • the control signal can be used to control the controlled object simulation platform to run the controlled object.
  • the control signal may be vehicle simulation control information, and the vehicle simulation control information may be used to control the operation of the vehicle simulated in the vehicle simulation tool cloud.
  • Step S405 The virtualization software and hardware platform sends the control signal to the controlled object simulation platform.
  • Step S406 The controlled object simulation platform performs performance simulation on the controlled object according to the control signal, and obtains fault test feedback results.
  • Step S407 The controlled object simulation platform sends the fault test feedback results to the virtualization software and hardware platform.
  • Step S408 The virtualization software and hardware platform sends the controlled object simulation platform to the fault injection console.
  • Step S409 The fault injection console performs fault analysis based on the fault test feedback results and the preconfigured expected fault processing results.
  • a virtualized embedded hardware system that can run on a general-purpose computer is used to replace the actual hardware device, thereby reducing the testing cost; fault injection into the virtualized embedded hardware system is just like software fault injection. Injection is equally flexible, thereby improving hardware fault test coverage.
  • the virtualized embedded hardware system and the controlled object simulation platform are all deployed in the cloud, thereby improving the testing efficiency and the utilization rate of the testing system.
  • Figure 5 shows an exemplary structural schematic diagram of a test system provided by an embodiment of the present application.
  • the system includes a fault injection console, a virtualized ECU resource cloud and a vehicle simulation tool cloud.
  • the fault injection console can refer to the fault injection console shown in Figure 1
  • the virtualized ECU resource cloud can refer to the virtualized software and hardware platform shown in Figure 1
  • the vehicle simulation tool cloud can refer to the controlled object shown in Figure 1
  • the simulation platform will not be described in detail here.
  • the implementation method is as follows:
  • the tester selects the injection point located in the virtual hardware module through the fault injection console.
  • the fault injection point is the variable torque located on the random access memory (Random Access Memory, RAM) in ECU1.
  • the tester can select ECU1, RAM, variable and torque in sequence on the human-computer interaction interface, and then modify the value of the variable torque to the fault injection value that needs to be tested. It is assumed here that it is modified to 0.
  • the virtualized ECU resource cloud performs fault input.
  • the above fault injection can be implemented by using the ECU memory read and write function of the CCP protocol. It should be noted that what is modified here is not the memory data of the physical ECU, but the memory data of the virtual ECU. The specific process is as follows:
  • Step 1 The fault injection console writes the fault injection configuration information configured by the tester into the CRO (command receiving object) message of the CCP protocol.
  • the first byte of the CRO message is the Command Prompt (CMD), which is used to describe the purpose of the message; the second byte of the CRO message is the command counter (CTR), which is used to track and record communications; CRO
  • CMD Command Prompt
  • CTR command counter
  • CRO third to eighth bytes of the message are used to store specific modified data and its address in memory.
  • the fault injection console needs to write the address and value of the variable torque in the ECU memory into the CRO message.
  • the specific writing method please refer to the CCP protocol, which will not be described here.
  • Step 2 The fault injection console sends the CRO message to the CCP driver module of the virtualized ECU resource cloud through the local CCP driver module. This step is achieved through the virtual CAN bus function provided by the CCP protocol.
  • Step 3 After receiving the CRO message, the CCP module of the virtualized ECU resource cloud parses the commands and parameters in it, and then executes the memory modification instruction, that is, it finds the specified memory address and modifies the content to 0. After completing the modification, the virtualized ECU resource cloud feeds back the modification results to the fault injection console through DTO (data transfer object) messages.
  • DTO data transfer object
  • the vehicle simulation tool cloud performs fault testing.
  • testers can initiate simulation test verification.
  • the virtual ECU resource cloud begins to execute the test cases issued by the tester.
  • the torque variable in the virtual ECU resource cloud memory was modified to 0 by the fault injection function.
  • the test case initiates a steering command in the driver model at this time, because the torque variable in the memory is injected with 0, which is equivalent to not detecting the driver's input torque, and the steering control algorithm will not output the steering assist control signal. Therefore, the corresponding steering action will not occur as expected in the vehicle simulation tool cloud, resulting in a malfunction.
  • the vehicle simulation tool cloud can generate fault test feedback results and send the fault test feedback results to the virtualized ECU resource cloud.
  • the virtualized ECU resource cloud sends the received fault test feedback results to the fault injection console.
  • the fault injection console can perform fault analysis based on fault test feedback results and pre-configured expected fault processing results.
  • test system after the entire test system is deployed in the cloud, it is equivalent to forming a set of virtualized test benches on the cloud. Because this system is a pure software system and does not require any hardware equipment, it can be replicated on a large scale on the deployed cloud platform, so that it can test multiple test cases at the same time, or provide it to many tests in a multi-user manner. Personnel use at the same time.
  • test cases there are 10,000 test cases in the test verification of the vehicle control system. If all these test cases have to be completed through a hardware test bench, the bench cost and time cost will be very high. In fact, most test cases (80% or more) do not have to be completed on the hardware test bench, and can be completed on the virtualization test bench proposed in the embodiment of this application. At this time, only the remaining test cases can be completed. 2,000 test cases with mandatory requirements must pass the hardware test bench and be tested on the hardware test bench. In this way, on the one hand, the pressure on the hardware test bench is reduced, and on the other hand, large-scale parallel testing of 8,000 test cases is performed on the virtualized test bench. The test can be completed in a short time, improving efficiency. .
  • Figure 6 shows a schematic structural diagram of a testing device provided by an embodiment of the present application.
  • the device 600 can be applied to the virtualization software and hardware platform shown in Figure 1. As shown in Figure 6, the device 600 may include:
  • Obtaining module 601 is used to obtain fault injection configuration information.
  • the fault injection configuration information is used to indicate a fault injection point and a fault type.
  • the fault injection point is located in the virtualized hardware module or the embedded software module;
  • the injection module 602 is configured to inject faults into the fault injection points acquired by the acquisition module 601 according to the fault types acquired by the acquisition module 601 .
  • the device further includes:
  • a translation module is used to dynamically translate the binary execution instructions of the embedded hardware system into binary execution instructions that can be run on a general-purpose computer to obtain the virtualization hardware module.
  • the translation module is also used to:
  • a target binary execution instruction is determined from the binary execution instructions of the embedded hardware system and the dynamic translation is performed to obtain the virtualized hardware module.
  • the device further includes:
  • a generation module configured to generate a control signal after performing the fault injection
  • the first sending module is used to send the control signal to the controlled object simulation platform, so that the controlled object simulation platform performs performance simulation on the controlled object according to the control signal and obtains fault test feedback results.
  • the device further includes:
  • a receiving module configured to receive the fault test feedback results
  • the second sending module is configured to send the fault feedback result to the fault injection console, so that the fault injection console performs fault analysis based on the fault test feedback result and the preconfigured expected fault processing result.
  • the injection module is also configured to: according to the fault type, perform hardware fault injection to the fault injection point located in the virtualized hardware module by modifying the calibration protocol or the universal measurement and calibration protocol. injection.
  • the injection module is further configured to inject software faults into the fault injection point located in the embedded software module by means of code injection or state injection according to the fault type.
  • a virtualized embedded hardware system that can run on a general-purpose computer is used to replace the actual hardware device, thereby reducing the testing cost; fault injection into the virtualized embedded hardware system is just like software fault injection. Injection is equally flexible, thereby improving hardware fault test coverage.
  • the virtualized embedded hardware system and the controlled object simulation platform are all deployed in the cloud, thereby improving the testing efficiency and the utilization rate of the testing system.
  • FIG. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device can be used as a terminal device or a cloud server, and the electronic device can be used to deploy the fault injection console, virtualization software and hardware platform or controlled object simulation platform shown in Figure 1.
  • the computing device may include at least one processor 301 , a memory 302 , an input-output device 303 and a bus 304 .
  • processor 301 the computing device may include at least one processor 301 , a memory 302 , an input-output device 303 and a bus 304 .
  • memory 302 the computing device may include at least one processor 301 , a memory 302 , an input-output device 303 and a bus 304 .
  • the processor 301 is the control center of the computing device, and may be a processor or a collective name for multiple processing elements.
  • the processor 301 is a CPU, or it can be an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more microprocessors.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • FPGA Field Programmable Gate Array
  • the processor 301 can perform various functions of the computing device by running or executing software programs stored in the memory 302 and calling data stored in the memory 302.
  • the processor 301 may include one or more CPUs, such as CPU 0 and CPU 1 shown in the figure.
  • the computing device may include multiple processors, such as the processor 301 and the processor 305 shown in FIG. 7 .
  • processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU).
  • a processor here may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).
  • the memory 302 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (Random Access Memory, RAM) or other types that can store information and instructions.
  • ROM read-only memory
  • RAM Random Access Memory
  • dynamic storage device which may also be an electrically erasable programmable read-only memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Test And Diagnosis Of Digital Computers (AREA)

Abstract

本申请实施例提供了一种测试方法、系统及装置，所述方法可应用于虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统，所述方法包括：获取故障注入配置信息，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；按照所述故障类型向所述故障注入点进行故障注入。在本申请实施例中，能够灵活、全面的实现软件层面以及硬件层面的故障注入。本申请提供的实施例可以用于智能汽车或新能源汽车测试。

Description

一种测试方法、系统及装置 技术领域
本申请涉及测试技术领域，尤其涉及一种测试方法、系统及装置。
背景技术
故障注入的本质是针对指定的故障类型，采用某种策略人为地将该故障类型的故障引入目标系统中，从而观察和分析目标系统在注入故障情况下的运行行为。故障注入是实现安全、逼真的模拟系统各类故障场景的重要手段。故障注入在汽车电子等嵌入式软硬件系统的集成与测试过程中占据着非常重要的地位。尤其在安全关键系统中，全面而充分的故障测试，是系统可靠运行的重要保证。
安全关键系统一般都是由软硬件两部分组合而成，因此故障注入测试，也从软件和硬件两方面进行。常用的故障注入技术分为两类，即基于软件的故障注入和基于硬件的故障注入。其中，基于硬件的故障注入存在成本高、效率低的问题，而基于软件的故障注入无法覆盖硬件故障。如何灵活、全面的实现软件层面以及硬件层面的故障注入成为亟待解决的问题。
发明内容
有鉴于此，提出了一种测试方法、系统及装置，能够灵活、全面的实现软件层面以及硬件层面的故障注入。
第一方面，本申请的实施例提供了一种测试方法，所述方法应用于虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统，所述方法包括：获取故障注入配置信息，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；根据所述故障类型向所述故障注入点进行故障注入。
本申请实施例提供的测试方法可以应用于任何软硬件结合的电子控制系统的测试与验证中，包括汽车电子控制系统、机器人电子控制系统等。在本申请实施例中，使用能够在通用计算机上运行的虚拟化的嵌入式硬件系统取代实际的硬件设备，从而降低了测试成本；对虚拟化的嵌入式硬件系统的故障注入就像对软件故障注入一样灵活，从而提升了硬件故障测试的覆盖率。
根据第一方面，在所述方法的第一种可能的实现方式中，所述方法还包括：将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
在本申请实施例中，通过对二进制执行指令的动态翻译，实现了嵌入式硬件系统的虚拟化，从而降低了测试成本，提高了灵活性和覆盖率。
根据第一方面的第一种可能的实现方式，在所述方法的第二种可能的实现方式中， 所述将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块，包括：根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
在本申请实施例中，通过对二进制执行指令中的目标二进制执行指令进行动态翻译，实现了嵌入式硬件系统的部分虚拟化，细化了嵌入式硬件系统的虚拟化力度，提高了灵活性和运行效率。
根据第一方面或者以上第一方面的任意一种可能的实现方式，在所述方法的第三种可能的实现方式中，所述方法还包括：在进行所述故障注入后，生成控制信号；将所述控制信号发送至被控对象仿真平台，以使所述被控对象仿真平台按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
在本申请实施例中，通过引入被控对象仿真平台，一方面实现了闭环控制，提升了故障测试反馈精度，另一方面使用被控对象仿真平台替代实际的被控对象，降低了测试成本。
根据第一方面的第三种可能的实现方式，在所述方法的第四种可能的实现方式中，所述方法还包括：接收所述故障测试反馈结果；将所述故障反馈结果发送至故障注入控制台，以使所述故障注入控制台根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
根据第一方面或者以上第一方面的任意一种可能的实现方式，在所述方法的第五种可能的实现方式中，所述根据所述故障类型向所述故障注入点进行故障注入，包括：根据所述故障类型，通过改造标定协议或者通用测量与标定协议的方式向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入。
在本申请实施例中，通过改造标定协议或者通用测量与标定协议的方式可以实现硬件故障的模拟。
根据第一方面或者以上第一方面的第一种可能的实现方式至第四种可能的实现方式中的任意一种，在所述方法的第六种可能的实现方式中，所述根据所述故障类型向所述故障注入点进行故障注入，包括：根据所述故障类型，通过代码注入或者状态注入的方式向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
在本申请实施例中，通过代码注入或者状态注入可以实现软件故障的模拟。
第二方面，本申请的实施例提供了一种测试系统，所述测试系统包括：故障注入控制台和虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统；
所述故障注入控制台，用于采集故障注入配置信息，并将所述故障注入配置信息发送至所述虚拟化软硬件平台，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
所述虚拟化软硬件平台，用于接收所述故障注入配置信息，并按照所述故障类型向所述故障注入点进行故障注入。
根据第二方面，在所述系统的第一种可能的实现方式中，所述测试系统还包括： 被控对象仿真平台；
所述虚拟化软硬件平台，还用于在进行故障注入后，生成控制信号，并将所述控制信号发送至所述被控对象仿真平台；
所述被控对象仿真平台，用于按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
根据第二方面的第一种可能的实现方式，在所述系统的第二种可能的实现方式中，
所述被控对象仿真平台，还用于将所述故障测试反馈结果发送至所述虚拟化软硬件平台；
所述虚拟化软硬件平台，还用于将所述故障测试反馈结果发送至所述故障注入控制台；
所述故障注入控制台，还用于根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
根据第二方面的第一种可能的实现方式或者第二种可能的实现方式，在所述系统的第三种可能的实现方式中，所述故障注入控制台包括故障配置模块和故障回收模块；
所述故障配置模块，用于提供人机交互界面，并通过所述人机交互界面采集所述故障注入配置信息；
所述故障回收模块，用于接收所述故障测试反馈结果，并根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
根据第二方面或者以上第二方面的任意一种可能的实现方式，在所述系统的第四种可能的实现方式中，所述虚拟化软硬件平台还包括硬件故障注入模块和软件故障注入模块；
所述硬件故障注入模块，用于按照所述故障类型向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入；
所述软件故障注入模块，用于按照所述故障类型向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
根据所述第二方面的第一种可能的实现方式至第三种可能的实现方式中的任意一种，在所述系统的第五种可能的实现方式中，所述虚拟化软硬件平台和所述被控对象仿真平台部署在云服务器上。
根据第二方面或者以上第二方面的任意一种可能的实现方式，在所述系统的第六种可能的实现方式中，所述虚拟化软硬件平台和所述被控对象仿真平台设置有统一的仿真时钟或者配置有硬件时钟，且所述虚拟化软硬件平台和所述被控对象仿真平台通过代码注入或者状态注入的方式进行了步长同步设置。
这样，可以实现虚拟化软硬件平台和被控对象仿真平台之间的同步，便于进行仿真以及收集故障测试反馈结果，从而提高故障分析的准确性。
第三方面，本申请的实施例提供了一种测试装置，所述装置应用于虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统，所述装置包括：
获取模块，用于获取故障注入配置信息，所述故障注入配置信息用于指示故障注 入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
注入模块，用于根据所述获取模块获取的故障类型向所述获取模块获取的故障注入点进行故障注入。
根据第三方面，在所述装置的第一种可能的实现方式中，所述装置还包括：
翻译模块，用于将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
根据第三方面的第一种可能的实现方式，在所述装置的第二种可能的实现方式中，所述翻译模块还用于：
根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
根据第三方面或者以上第三方面的任意一种可能的实现方式，在所述装置第三种可能的实现方式中，所述装置还包括：
生成模块，用于在进行所述故障注入后，生成控制信号；
第一发送模块，用于将所述控制信号发送至被控对象仿真平台，以使所述被控对象仿真平台按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
根据第三方面的第三种可能的实现方式，在所述装置的第四种可能的实现方式中，所述装置还包括：
接收模块，用于接收所述故障测试反馈结果；
第二发送模块，用于将所述故障反馈结果发送至故障注入控制台，以使所述故障注入控制台根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
根据第三方面或者以上第三方面的任意一种可能的实现方式，在所述装置的第五种可能的实现方式中，所述注入模块还用于：根据所述故障类型，通过改造标定协议或者通用测量与标定协议的方式向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入。
根据第三方面或者以上第三方面的第一种可能的实现方式至第五种可能的实现方式中的任意一种，在所述装置的第六种可能的实现方式中，所述注入模块还用于：根据所述故障类型，通过代码注入或者状态注入的方式向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
第四方面，本申请的实施例提供了一种测试装置，该测试装置可以执行上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的测试方法。
第五方面，本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的测试方法。
第六方面，本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的测试方法。
本申请的这些和其他方面在以下(多个)实施例的描述中会更加简明易懂。
附图说明
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本申请的示例性实施例、特征和方面，并且用于解释本申请的原理。
图1示出本申请实施例提供的测试系统的结构示意图；
图2示出了本申请实施例提供的虚拟化软硬件平台的示例性结构示意图；
图3示出本申请实施例提供的硬件故障注入过程的示例性示意图；
图4示出本申请实施例提供的测试方法的交互流程图；
图5示出了本申请实施例提供测试系统的示例性结构示意图；
图6示出本申请实施例提供的测试装置的结构示意图；
图7示出本申请实施例提供的电子设备的结构示意图。
具体实施方式
以下将参考附图详细说明本申请的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本申请，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。
基于软件的故障注入是通过在软件层面注入故障，生成错误，从而制造出硬件或者系统级的故障，然后对注入故障后的系统反馈数据进行分析，来进行系统可靠性测试。在软件故障注入中，故障注入点在软件系统中，比如：在应用软件的应用编程接口(Application Programming Interface，API)中，或者在操作系统(Operating System，OS)的API中等。在软件故障注入中，故障注入的方式可以选择直接修改执行程序的语句、修改或增删软件系统中的数据、修改软件系统的运行状态或者通过软件调试工具修改内存数据等。在灵活选择故障注入点和故障注入方式后，可以对应用软件、操作系统、底层驱动软件等软件模块进行测试。
上述软件故障注入方式具有成本低、容易控制的特点，不需要额外的硬件设备，在整个软件层面都可以选择灵活的故障注入点和多种故障注入方式。在软件系统测试中被广泛使用。上述基于软件的故障注入存在只能测试软件故障而无法测试硬件故障的问题。
基于硬件的故障注入是在测试系统中纳入硬件系统，同时针对硬件选择故障注入点并选择相应的故障注入方式。嵌入式系统中一般包括计算、存储、通信、输入输出(Input Output，IO)等硬件组件，这些硬件组件本身或者它们的接口可作为故障注入点，故障注入方式可以是硬件组件本身出现了故障或者硬件接口引脚出现了故障。通过在硬件组件上引入故障，然后对注入故障后的系统反馈数据进行分析，来进行系统 可靠性测试。
上述基于硬件的故障注入方式对每一套故障测试系统，都需要构造一套硬件系统。可见，如果进行大量硬件故障注入测试，则需要配置大量的硬件设备，存在测试成本较高以及测试效率较低的问题。另外，在芯片封装、片上系统等场景下，许多硬件组件封装在了一起，硬件故障注入位置不可访问，从而造成了故障注入测试不完备的问题，即硬件故障测试覆盖率较低的问题。
本申请实施例提供了一种测试方法、系统以及装置，可以应用于任何软硬件结合的电子控制系统的测试与验证中，包括汽车电子控制系统、机器人电子控制系统等。在本申请实施例中，使用能够在通用计算机上运行的虚拟化的嵌入式硬件系统取代实际的硬件设备，从而降低了测试成本；对虚拟化的嵌入式硬件系统的故障注入就像对软件故障注入一样灵活，从而提升了硬件故障测试的覆盖率。
另外，在本申请实施例中，通过引入被控对象仿真平台，一方面实现了闭环控制，提升了故障测试反馈精度，另一方面使用被控对象仿真平台替代实际的被控对象，降低了测试成本。同时，在本申请实施例中，通过将虚拟化的嵌入式硬件系统与被控对象仿真平台全部进行云化部署，提升了测试效率以及测试系统的利用率。
下面以汽车电子控制系统为例，对本申请实施例涉及的测试方法、系统和装置进行详细介绍。
图1示出本申请实施例提供的测试系统的结构示意图。如图1所示，本申请实施例提供的测试系统包括故障注入控制台、虚拟化软硬件平台和被控对象仿真平台。其中，故障注入控制台是供测试人员进行操作的测试系统的入口。测试人员登录故障注入控制台后，可以发起故障注入测试。虚拟化软硬件平台可以在测试人员发起的故障注入测试后，进行软硬件故障注入。被控对象仿真平台可以虚拟化软硬件平台完成故障注入后，发起仿真测试验证。
在本申请实施例中，测试人员可以通过终端设备登录故障注入控制台。虚拟化硬件平台和被控对象仿真平台则可以部署在云服务器上，且两者可以部署在同一个云服务器上，也可以部署在不同的云服务器上，对此本申请实施例不做限制。终端设备与云服务器之间可以进行通信，本申请实施例对终端设备与云服务器之间的通信方式不做限制。下面对构成测试系统的各个部分的具体信息分别加以详细描述。
故障注入控制台是供测试人员进行操作的测试系统的入口。故障注入控制台的形式包括但不限于应用程序、小程序或者网页等。故障注入控制台前端提供了和测试人员交互的人机交互界面，后端提供和虚拟化软硬件平台进行交互的功能。一方面将测试人员选择的故障注入配置信息传输给虚拟化软硬件平台；另一方面从虚拟化软硬件平台接收故障测试反馈结果。因此，测试人员用于登录故障注入控制台的终端设备能够提供人机交互界面以及后端通信功能即可。举例来说，本申请实施例涉及的终端设备可以是智能手机、上网本、平板电脑、笔记本电脑、可穿戴电子设备(如智能手环、智能手表等)、TV、虚拟现实设备等等，对此本申请实施例不做限制。
故障注入控制台可以用于采集故障注入配置信息，并将故障注入配置信息发送至虚拟化硬件平台。其中，故障注入配置信息可以用于指示故障注入点和故障类型。
故障注入点可以用于指示注入故障的位置，故障注入点可能位于虚拟化硬件模块， 也可能位于嵌入式软件模块，后文会对虚拟化硬件模块和嵌入式软件模块进行介绍，这里不再赘述。在故障注入点位于虚拟化硬件模块时，表明待注入的故障是硬件故障；在故障注入点位于嵌入式软件模块时，表明待注入的故障是软件故障。故障类型可以用于指示注入故障的类型，不同故障类型的故障，其进行故障注入时采用的注入方式不同。后文会对故障注入点、故障类型和故障注入方式进行介绍，这里不再赘述。
在一种可能的实现方式中，故障注入控制台包括故障配置模块和故障回收模块。其中，故障配置模块可以用于提供人机交互界面，并通过该人机交互界面采集故障注入配置信息。故障回收模块可以用于接收所述故障测试反馈结果，并根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。其中，故障测试反馈结果来自虚拟化软硬件平台。故障分析方式可以参考相关技术，这里不再赘述。
在一个示例中，测试人员可以通过故障注入控制台，选择故障注入点和故障类型；同时还需要配置期望的故障处理结果，以便跟实际的故障测试结果(即来自虚拟化软硬件平台的故障测试反馈结果)进行对比。为了方便测试人员配置，本申请实施例中根据已有经验提供故障模式库(该故障模式库中存储有多个故障注入点，以及多个故障类型)供测试人员选择，当然测试人员也可以手动配置故障注入点和故障类型。
虚拟化软硬件平台可以用于接收所述故障注入配置信息，并按照所述故障类型向所述故障注入点进行故障注入。
图2示出了本申请实施例提供的虚拟化软硬件平台的示例性结构示意图。如图2所示，虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块。
其中，虚拟化硬件模块可以用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统。举例来说，虚拟化硬件模块包括但不限于虚拟化中央处理器(Central Processing Unit，CPU)、虚拟化内存、虚拟化总线、虚拟化板载通信模块，虚拟化IO模块等。在本申请实施例中，硬件虚拟化按照虚拟化层次不同，可以分为全虚拟化、部分虚拟化和操作系统虚拟化等不同的类型，具体采用哪种类型可以根据实际系统的测试需求进行灵活选择，从而提高虚拟化硬件模块运行时的效率。
在一种可能的实现方式中，可以将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
可见，在本申请实施例中可以将整个嵌入式硬件系统的全部进行虚拟化，然后在通用计算机上模拟运行嵌入式硬件系统，嵌入式硬件系统的虚拟化，可以细化到指令集的粒度，为硬件的模拟精度提供了保障。在一个示例中，可以将嵌入式硬件系统的专用指令集动态翻译成通用x86指令集，在通用计算机上模拟出虚拟的硬件嵌入式系统。
在一种可能的实现方式中，将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块可以包括：根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
其中，目标二进制执行指令表示嵌入式硬件系统的二进制执行指令中的任意一部分二进制执行指令。可见，在本申请实施例中可以将嵌入式硬件系统的部分进行虚拟化，然后在通用计算机上模拟运行部分嵌入式硬件系统。
在一种可能的实现方式中，还可以将嵌入式硬件系统的操作系统进行虚拟化，然后在通用计算机上模拟嵌入式硬件系统的操作系统。
以汽车电子控制系统为例对嵌入式硬件系统的虚拟化过程进行说明。在汽车电子控制系统中，以虚拟化电子控制单元(Electronic Control Unit，ECU)资源云作为虚拟化软硬件平台，需要对微控制器单元(Micro Controller Unit，MCU)和主处理单元(Master Process Unit，MPU)、内存、IO和通信等模块进行虚拟化，得到上述虚拟化软硬件平台。
对于MCU系统，因为MCU的硬件和通用计算机硬件差异较大，并且MCU主频较低，对通用计算机来说全虚拟化MCU的计算压力比较小，因此可以采用全虚拟化的方式对MCU进行虚拟化。全虚拟化的MCU是一个完整的独立的运行系统，和实际的硬件MCU可以运行相同的操作系统，因此实际的生成代码可以直接运行在全虚拟化的MCU上面。在实施中，可以利用快速模拟器(Quick EMUlator，QEMU)等开源软件实现MCU的全虚拟化。进一步的，可以采用基于C++语言的用于系统设计的计算机语言(SystemC)与QEMU相结合的方式或者微架构模拟器(microarchitecture simulator，macsim)与模拟器(EMUlator，EMU)相结合的方式，进行MCU内部的运行时序协同。
对于MPU系统，因为MPU的硬件和通用计算机相仿，并且MPU的主频较高，对通用计算机来说全虚拟化MPU的计算压力较大，因此可以采用部分虚拟化的方式对MPU进行虚拟化或者只对MPU的操作系统进行虚拟化。在实施例中，可以采用OpenVZ、Linux虚拟服务器(Linux VServer)、Linux容器(Linux Containers，LXC)等技术来实现MPU的虚拟化。
嵌入式软件模块可以用于表示能够在虚拟化的嵌入式硬件系统上运行的实际的软件。举例来说，嵌入式软件模块包括但不限于基础软件、操作系统和应用软件等软件模块。这些软件模块都是真实的用于实际生成的软件代码。
在本申请实施例中，真实的嵌入式软件模块加上高精度的虚拟化硬件模块，为构造出完整的高精度的故障注入测试提供了基础。
如图2所示，在一种可能的实现方式中，虚拟化软硬件平台还包括硬件故障注入模块和软件故障注入模块。其中，硬件故障注入模块可以用于按照所述故障类型向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入；软件故障注入模块可以用于按照所述故障类型向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
对于软件故障注入，按照故障引入的阶段，可以将软件故障分为：功能故障、算法故障、顺序故障、检验故障以及赋值故障等类型；按照功能和运行环境，可以将软件故障分为：对外接口故障、内部模块间交互故障、配置故障、数据故障、资源类故障、操作系统故障以及驱动故障等类型。由此可见，软件故障涉及的层次和种类较多，故障覆盖率较高。
在一个示例中，可以通过代码注入的方式进行软件故障注入。代码注入即直接改变原代码的某个部分，以产生一个语法正确但语义不正确的故障点，通过向目标代码中某个特定位置打补丁，改变被测代码的行为，从而达到软件故障注入的目的。
在又一示例中，可以通过状态注入的方式进行软件故障注入。状态注入即通过改 变软件系统的运行状态或行为来进行故障注入，通过模拟一个故障所产生的错误，测试整个系统的最终反应。例如：测试系统对数据不一致的容错能力，可以故意修改数据的类型，然后观察系统后续的输出结果，与正常运行的输出结果的差异。
具体的，软件故障注入可以通过以下的技术手段来实现。第一种，基于打桩的软件故障注入：在软件模块中进行打桩，强制软件运行测试用户设置的故障模式，而非正常模式，从而实现故障注入。第二种，基于进程的软故障注入：在软件故障注入模块，发起一个故障注入专用的高优先级的进程，该进程可以修改计算的状态，从而实现故障的注入。第三种，基于调试器的软故障注入：采用调试器提供的工具，通过设置断点的方法，将错误注入到一个正在运行的进程中。举例来说，调试器提供的工具可以为调试器(debugger，dbx)或者gun调试器(gun debugger，gdb)等。第四种，基于消息的软件故障注入：对于通过消息进行通信的软件模块，可以采用基于消息的工具来故意破坏消息的内容、顺序等来产生错误的消息，从而实现软件的故障注入。第五种，基于存储的软件故障注入：利用存储器操作工具，直接修改存储器中，软件模块的数据、状态的值，将故障注入存储系统，通过存储系统引入故障。第六种，基于命令的软件故障注入：采用来自测试人员接口或远程维护端口的命令，改变运行、操作、管理、运维等系统的状态，使工作于故障模式，即实现将故障注入测试软件系统中。
对于硬件故障引入，主要是设法模拟嵌入式硬件本身发生的故障，例如：MCU发生了故障导致代码不能正常运行、内存发生了故障导致数据丢失或无法读写、IO发生了故障导致无法连接外部设备、通信链路发生故障导致无法正常收发数据等。在本申请实施例中，可以通过改造标定协议(CAN Calibration Protocol，CCP)或者通用测量与标定协议(Universal Measurement and Calibration Protocol，XCP)来实现。
其中，CCP是一种基于控制器局域网络总线(Controller Area Network，CAN)总线的匹配标定协议，可以在线检测或者修改ECU内部的变量值。例如，CANape就是一款基于CCP的ECU标定和测试工具。CANape与ECU之间的通信通过一个描述文件来实现，该文件记录了ECU中各参数的详细信息，例如变量在ECU中的存储地址、存储结构、数据类型和转换公式等。通过该文件，即可直接访问和修改ECU中的变量。具体的修改过程是通过CCP协议规定的映射机制来实现的，该协议保证了描述文件中的内容与ECU内部的实际变量之间是同步的。XCP协议是CCP协议的扩展，它能够适配多种底层网络协议和总线类型，例如CAN、以外网(Ethernet)、FlexRay、串行通信接口(Serial Communication Interface，SCI)、串行外设接口(Serial Peripheral Interface，SPI)，通用串行总线(Universal Serial Bus，USB)等。
图3示出本申请实施例提供的硬件故障注入过程的示例性示意图。如图3所示，硬件故障注入模块与虚拟化硬件模块通过虚拟总线连接。硬件故障注入模块把故障注入点加入到CCP协议或者XCP协议的描述文件中，通过CCP协议或者XCP协议，将这些故障注入点映射到CPU、内存、IO或者通信等各个硬件模块的寄存器之中。然后，硬件故障注入模块修改描述文件中的变量，通过修改这些变量，注入所需的各种故障状态和数据(即故障类型)，这些变量就会自动映射到虚拟化硬件模块中，从而实现了硬件故障注入。
CCP协议和XCP协议的本来作用是用来进行ECU内部变量的标定和匹配的，本申请实施例中利用CCP协议和XCP协议中的描述文件到ECU内存地址空间的映射关系，从而达到把故障注入到虚拟ECU资源云中的目的。
在本申请实施例中，软件故障注入模块和相关技术中的基于软件的故障注入方式类似，而硬件故障注入模块的灵活性则大大增加，因为硬件虚拟化后，对硬件的故障注入就像软件故障注入一样灵活，从而大大提升硬件故障注入测试的覆盖率和效率。
在本申请实施例中，如图1或者图2所示，虚拟化硬件模块可以大规模复制，与大规模部署实际嵌入式硬件系统相比，大大降低了成本。同时，利用CCP协议或者XCP协议中的描述文件到ECU内存地址空间的映射关系，巧妙的设计了虚拟化硬件的故障注入技术。另外，在虚拟化硬件模块上，软件可以直接使用实际的生产代码，比纯软件在环(Software In Loop，SIL)仿真验证精度更高。
虚拟化软硬件平台还可以用于在进行故障注入后，生成控制信号，并将所述控制信号发送至所述被控对象仿真平台。
被控对象仿真平台的作用是用高精度仿真的方法来取代实际的被控对象，例如车辆仿真工具云可以用于取代实际的车辆。被控对象仿真平台可以用于：接收来自虚拟化软硬件平台的控制信息；按照所述控制信号对被控罪行进行性能仿真，得到故障测试反馈结果；以及，将故障测试反馈结果发送至虚拟化软硬件平台。之后，虚拟化软硬件平台接收到故障测试反馈结果后，可以将故障测试反馈结果发送至故障注入控制台的故障回收模块进行分析。
在汽车电子控制系统为例，车辆仿真工具主要包括动力学仿真工具和经济性仿真工具。因此，在被控对象仿真平台中进行的性能仿真主要包括动力学仿真和经济性仿真。其中，动力学仿真主要是对车辆加速能力、爬坡能力等进行仿真，经济性仿真主要是对车辆续航能力等进行仿真。
在实施中，可以直接将现有的商用或者开源车辆仿真工具(对应于图1所示的被控对象仿真平台)部署在云端，然后直接和虚拟化ECU资源云(对应于图1所示的虚拟化软硬件平台)对接即可。
在本申请实施例中，虚拟化软硬件平台和被控对象仿真平台可以通过设置统一的仿真时钟或者配置有硬件时钟实现同步，以及通过代码注入或者状态注入的方式实现步长同步设置。因此，在云端部署辆仿真工具与虚拟化ECU资源云之后，可以通过设置统一的仿真时钟或者配置硬件时钟实现车辆仿真工具与虚拟化ECU资源云的同步。其中，配置硬件时钟的方式可以参考硬件故障注入的方式，这里不再赘述。同步步长的设置方式可以参照软件故障注入方式，这里不再赘述。
故障注入测试的目的是为了观察故障注入后，系统的响应与正常工作的系统的响应之间的偏差，从而来评估系统的可靠性。因此，故障反馈的精度和时效性在测试系统中起着很重要的作用。
相关技术中，在基于软件的测试系统中，故障反馈就是软件系统的输出数据或者状态，一般通过日志的方式，记录下来，然后离线分析。在基于硬件的测试系统中，要么采用日志的方式，将故障记录在硬件系统中，然后导出来离线分析；要么将硬件系统与实际的被控系统对接，采集被控系统的反馈数据来进行故障分析。上述所述的 故障反馈记录方式表明，通过记录日志的方式，离线分析数据，则相当于整个故障测试系统是一个开环系统，被测试的软硬件系统没有与实际的被控对象连接起来，缺乏实际被控对象对软硬件故障的真实反馈。而通过与实际被控对象直接连接的方式，又存在测试成本高和效率低的问题。
在本申请实施例中，采用被控对象的仿真系统来取代对实际被控对象，一方面，实现了完整的故障注入、控制、仿真、测试结果反馈闭环系统，从而能够获得实际被控系统在软硬件故障情况下的真实反馈，提升了故障测试反馈结果的准确性；另一方面进一步降低了测试成本。
图4示出本申请实施例提供的测试方法的交互流程图。该方法可以应用于图1所示的测试系统。如图4所示，所述方法包括：
步骤S400，测试人员在故障注入控制台配置故障注入配置信息。
其中，故障注入配置信息可以用于指示故障注入点和故障类型。故障注入点可以用于指示注入故障的位置，故障注入点可能位于虚拟化硬件模块，也可能位于嵌入式软件模块。虚拟化硬件模块可以用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统。举例来说，虚拟化硬件模块包括但不限于虚拟化中央处理器(Central Processing Unit，CPU)、虚拟化内存、虚拟化总线、虚拟化板载通信模块，虚拟化IO模块等。嵌入式软件模块可以用于表示能够在虚拟化的嵌入式硬件系统上运行的实际的软件。举例来说，嵌入式软件模块包括但不限于基础软件、操作系统和应用软件等软件模块。这些软件模块都是真实的用于实际生成的软件代码。
本申请实施例中根据已有经验提供故障模式库(该故障模式库中存储有多个故障注入点，以及多个故障类型)供测试人员选择，当然测试人员也可以手动配置故障注入点和故障类型。
步骤S401，故障注入控制台采集故障注入配置信息。
步骤S402，故障注入控制台将故障注入配置信息发送至虚拟化软硬件平台。
步骤S403，虚拟化软硬件平台根据故障注入配置信息指示的故障类型向故障注入配置信息指示的故障注入点进行故障注入。
虚拟化软硬件平台可以通过硬件故障注入模块向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入；以及通过软件故障注入模块向位于所述嵌入式软件模块的故障注入点进行软件故障注入。软件故障的注入方式和硬件故障的注入方式如前所述，这里不再赘述。
步骤S404，虚拟化软硬件平台在进行故障注入后，生成控制信号。
控制信号可以用于控制被控对象仿真平台进行被控对象的运行。在一个示例中，以汽车电子控制系统为例，控制信号可以为车辆仿真控制信息，此车辆仿真控制信息可以用于控制车辆仿真工具云中仿真的车辆的运行。
步骤S405，虚拟化软硬件平台将控制信号发送至被控对象仿真平台。
步骤S406，被控对象仿真平台按照控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
步骤S407，被控对象仿真平台将故障测试反馈结果发送至虚拟化软硬件平台。
步骤S408，虚拟化软硬件平台将被控对象仿真平台发送至故障注入控制台。
步骤S409，故障注入控制台根据故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
在本申请实施例中，使用能够在通用计算机上运行的虚拟化的嵌入式硬件系统取代实际的硬件设备，从而降低了测试成本；对虚拟化的嵌入式硬件系统的故障注入就像对软件故障注入一样灵活，从而提升了硬件故障测试的覆盖率。
另外，在本申请实施例中，通过引入被控对象仿真平台，一方面实现了闭环控制，提升了故障测试反馈精度，另一方面使用被控对象仿真平台替代实际的被控对象，降低了测试成本。同时，在本申请实施例中，通过将虚拟化的嵌入式硬件系统与被控对象仿真平台全部进行云化部署，提升了测试效率以及测试系统的利用率。
应用示例
下面以汽车的转向系统为例，详细描述硬件故障注入测试在本申请实施例所提出的测试系统中的实现过程。图5示出了本申请实施例提供测试系统的示例性结构示意图。如图5所示，该系统中包括故障注入控制台、虚拟化ECU资源云和车辆仿真工具云。其中，故障注入控制台可以参照图1所示的故障注入控制台，虚拟化ECU资源云可以参照图1所示的虚拟化软硬件平台，车辆仿真工具云可以参照图1所示的被控对象仿真平台，这里不再赘述。
假设方向盘的扭矩传感器测量得到的扭矩数据能够正确的传入ECU，但是由于电磁干扰或者热噪声等原因导致数据在ECU的内存中发生了错误。这种内存数据的故障注入，在真实硬件芯片的运行过程中是很难实现的。在本申请实施例所提出测试系统中，实现方式如下：
首先，测试人员通过故障注入控制台选择位于虚拟硬件模块的注入点，在本应用示例中故障注入点为位于ECU1中的随机访问存储器(Random Access Memory，RAM)上的变量(variable)torque。测试人员可以在人机交互界面，依次选择ECU1、RAM、variable和torque，然后将变量torque的值修改为需要测试的故障注入值，这里假设修改为0。
然后，虚拟化ECU资源云进行故障输入。如图5中的粗线箭头所示，在本申请实施例中，上述故障注入可以利用CCP协议的ECU内存读写功能来实现的。需要说明的是，这里修改的不是物理ECU的内存数据，而是虚拟ECU的内存数据。具体过程如：
步骤一，故障注入控制台将测试人员配置的故障注入配置信息写入CCP协议的CRO(命令接收对象)报文中。CRO报文第一个字节是命令提示符(Command Prompt，CMD)，用于描述报文的目的；CRO报文的第二个字节是命令计数器(CTR)，用于跟踪记录通信；CRO报文的第三个字节至第八个字节用于存放具体的修改数据及其在内存中的地址。在本应用示例中，故障注入控制台需要将变量torque在ECU内存中的地址和值写入到CRO报文中，其具体的写入方法可以参考CCP协议，这里不再赘述。
步骤二，故障注入控制台通过本地CCP驱动模块将CRO报文发送至虚拟化ECU资源云的CCP驱动模块。这一步是通过CCP协议提供的虚拟CAN总线功能来实现的。
步骤三，虚拟化ECU资源云的CCP模块接收到CRO报文后，解析其中的命令和参数，然后执行内存修改指令，即找到指定的内存地址并且将其中的内容修改为0。 在完成修改后，虚拟化ECU资源云通过DTO(数据传输对象)报文把修改结果反馈给故障注入控制台。
接着，车辆仿真工具云进行故障测试。测试人员在完成故障注入后，可以发起仿真测试验证。虚拟ECU资源云开始执行测试人员下发的测试用例。在测试用例的执行过程中，虚拟ECU资源云内存中的torque变量被故障注入功能修改成了0。假设此时测试用例在驾驶员模型中发起了转向指令，因为内存中的torque变量被注入了0，即相当于没有检测到驾驶员的输入扭矩，转向控制算法不会输出转向助力的控制信号，所以在车辆仿真工具云中不会按照预期发生相应的转向动作，从而导致故障的发生。
在故障发生之后，车辆仿真工具云可以生成故障测试反馈结果，并将故障测试反馈结果发送至虚拟化ECU资源云。虚拟化ECU资源云将接收到的故障测试反馈结果发送至故障注入控制台。故障注入控制台可以根据故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
在本申请实施例中，整个测试系统全部云化部署之后，相当于是形成了一套云上的虚拟化的测试台架。因为这套系统是纯软件的系统，不需要任何硬件设备，因此可以在所部署的云平台上进行大规模复制，从而能够满足同时测试多个测试用例，或者以多用户的方式提供给很多测试人员同时使用。
假设在车辆控制系统的测试验证中有10000条测试用例。如果这些测试用例全部都要通过硬件测试台架来完成，台架成本和时间成本会非常高。实际上大多数测试用例(80％甚至更多)不一定非要在硬件测试台架上完成，可以放在本申请实施例所提出的虚拟化测试台架上来完成，此时可以只将剩余的2000条有硬性要求的必须经过硬件测试台架的测试用例，放在硬件测试台架上测试。这样，一方面减轻了对硬件测试台架的压力，另一方面在虚拟化测试台架上对8000条测试用例进行大规模的并行测试，只需要很短的时间就能完成测试，提升了效率。
图6示出本申请实施例提供的测试装置的结构示意图。该装置600可以应用于图1所示的虚拟化软硬件平台。如图6所示，该装置600可以包括：
获取模块601，用于获取故障注入配置信息，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
注入模块602，用于根据所述获取模块601获取的故障类型向所述获取模块601获取的故障注入点进行故障注入。
在一种可能的实现方式中，所述装置还包括：
翻译模块，用于将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
在一种可能的实现方式中，所述翻译模块还用于：
根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
在一种可能的实现方式中，所述装置还包括：
生成模块，用于在进行所述故障注入后，生成控制信号；
第一发送模块，用于将所述控制信号发送至被控对象仿真平台，以使所述被控对 象仿真平台按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
在一种可能的实现方式中，所述装置还包括：
接收模块，用于接收所述故障测试反馈结果；
第二发送模块，用于将所述故障反馈结果发送至故障注入控制台，以使所述故障注入控制台根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
在一种可能的实现方式中，所述注入模块还用于：根据所述故障类型，通过改造标定协议或者通用测量与标定协议的方式向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入。
在一种可能的实现方式中，所述注入模块还用于：根据所述故障类型，通过代码注入或者状态注入的方式向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
在本申请实施例中，使用能够在通用计算机上运行的虚拟化的嵌入式硬件系统取代实际的硬件设备，从而降低了测试成本；对虚拟化的嵌入式硬件系统的故障注入就像对软件故障注入一样灵活，从而提升了硬件故障测试的覆盖率。
另外，在本申请实施例中，通过引入被控对象仿真平台，一方面实现了闭环控制，提升了故障测试反馈精度，另一方面使用被控对象仿真平台替代实际的被控对象，降低了测试成本。同时，在本申请实施例中，通过将虚拟化的嵌入式硬件系统与被控对象仿真平台全部进行云化部署，提升了测试效率以及测试系统的利用率。
图7示出本申请实施例提供的电子设备的结构示意图。该电子设备可以作为终端设备或者云服务器，该电子设备可以用于部署图1所示的故障注入控制台、虚拟化软硬件平台或者被控对象仿真平台。
如图7所示，计算设备可以包括至少一个处理器301，存储器302、输入输出设备303以及总线304。下面结合图7对计算设备的各个构成部件进行具体的介绍：
处理器301是计算设备的控制中心，可以是一个处理器，也可以是多个处理元件的统称。例如，处理器301是一个CPU，也可以是特定集成电路(Application Specific Integrated Circuit，ASIC)，或者是被配置成实施本申请实施例的一个或多个集成电路，例如：一个或多个微处理器(Digital Signal Processor，DSP)，或，一个或者多个现场可编程门阵列(Field Programmable Gate Array，FPGA)。
其中，处理器301可以通过运行或执行存储在存储器302内的软件程序，以及调用存储在存储器302内的数据，执行计算设备的各种功能。
在具体的实现中，作为一种实施例，处理器301可以包括一个或多个CPU，例如图中所示的CPU 0和CPU 1。
在具体实现中，作为一种实施例，计算设备可以包括多个处理器，例如图7中所示的处理器301和处理器305。这些处理器中的每一个可以是一个单核处理器(single-CPU)，也可以是一个多核处理器(multi-CPU)。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。
存储器302可以是只读存储器(Read-Only Memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(Random Access Memory，RAM)或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器

Claims (23)

    (Electrically Erasable Programmable Read-Only Memory，EEPROM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器302可以是独立存在，通过总线304与处理器301相连接。存储器302也可以和处理器301集成在一起。

    输入输出设备303，用于与其他设备或通信网络通信。如用于与以太网，无线接入网(Radio access network，RAN)，无线局域网(Wireless Local Area Networks，WLAN)等通信网络通信。输入输出设备303可以包括基带处理器的全部或部分，以及还可选择性地包括无线射频(Radio Frequency，RF)处理器。RF处理器用于收发RF信号，基带处理器则用于实现由RF信号转换的基带信号或即将转换为RF信号的基带信号的处理。

    在具体实现中，作为一种实施例，输入输出设备303可以包括发射器和接收器。其中，发射器用于向其他设备或通信网络发送信号，接收器用于接收其他设备或通信网络发送的信号。发射器和接收器可以独立存在，也可以集成在一起。

    总线304，可以是工业标准体系结构(Industry Standard Architecture，ISA)总线、外部设备互连(Peripheral Component Interconnect，PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，EISA)总线等。该总线可以分为地址总线、数据总线、控制总线等。为便于表示，图7中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。

    图7中示出的设备结构并不构成对计算设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。

    本申请的实施例提供了一种测试装置，包括：处理器以及用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令时实现上述方法。

    本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。

    本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。

    计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read Only Memory，ROM)、可擦式可编程只读存储器(Electrically Programmable Read-Only-Memory，EPROM或闪存)、静态随机存取存储器(Static Random-Access Memory，SRAM)、便携式压缩盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、数字多功能盘(Digital Video Disc，DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。

    这里所描述的计算机可读程序指令或代码可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。

    用于执行本申请操作的计算机程序指令可以是汇编指令、指令集架构(Instruction Set Architecture，ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或可编程逻辑阵列(Programmable Logic Array，PLA)，该电子电路可以执行计算机可读程序指令，从而实现本申请的各个方面。

    这里参照根据本申请实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本申请的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。

    这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。

    也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。

    附图中的流程图和框图显示了根据本申请的多个实施例的装置、系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方 框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。

    也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行相应的功能或动作的硬件(例如电路或ASIC(Application Specific Integrated Circuit，专用集成电路))来实现，或者可以用硬件和软件的组合，如固件等来实现。

    尽管在此结合各实施例对本申请进行了描述，然而，本领域技术人员通过查看所述附图、公开内容、以及所附权利要求书，可理解并实现所述公开实施例的其它变化。在权利要求中，“包括”(comprising)一词不排除其他组成部分或步骤，“一”或“一个”不排除多个的情况。单个处理器或其它单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施，但这并不表示这些措施不能组合起来产生良好的效果。

    以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

  1. 一种测试方法，其特征在于，所述方法应用于虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统，所述方法包括：
    获取故障注入配置信息，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
    根据所述故障类型向所述故障注入点进行故障注入。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
  3. 根据权利要求2所述的方法，其特征在于，所述将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块，包括：
    根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
  4. 根据权利要求1至3中任意一项所述的方法，其特征在于，所述方法还包括：
    在进行所述故障注入后，生成控制信号；
    将所述控制信号发送至被控对象仿真平台，以使所述被控对象仿真平台按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
  5. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    接收所述故障测试反馈结果；
    将所述故障反馈结果发送至故障注入控制台，以使所述故障注入控制台根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
  6. 根据权利要求1至5中任意一项所述的方法，其特征在于，所述根据所述故障类型向所述故障注入点进行故障注入，包括：
    根据所述故障类型，通过改造标定协议或者通用测量与标定协议的方式向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入。
  7. 根据权利要求1至5中任意一项所述的方法，其特征在于，所述根据所述故障类型向所述故障注入点进行故障注入，包括：
    根据所述故障类型，通过代码注入或者状态注入的方式向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
  8. 一种测试系统，其特征在于，包括：故障注入控制台和虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统；
    所述故障注入控制台，用于采集故障注入配置信息，并将所述故障注入配置信息发送至所述虚拟化软硬件平台，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
    所述虚拟化软硬件平台，用于接收所述故障注入配置信息，并按照所述故障类型向所述故障注入点进行故障注入。
  9. 根据权利要求8所述的系统，其特征在于，所述测试系统还包括：被控对象仿真平台；
    所述虚拟化软硬件平台，还用于在进行故障注入后，生成控制信号，并将所述控制信号发送至所述被控对象仿真平台；
    所述被控对象仿真平台，用于按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
  10. 根据权利要求9所述的系统，其特征在于，
    所述被控对象仿真平台，还用于将所述故障测试反馈结果发送至所述虚拟化软硬件平台；
    所述虚拟化软硬件平台，还用于将所述故障测试反馈结果发送至所述故障注入控制台；
    所述故障注入控制台，还用于根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
  11. 根据权利要求8或10所述的系统，其特征在于，所述故障注入控制台包括故障配置模块和故障回收模块；
    所述故障配置模块，用于提供人机交互界面，并通过所述人机交互界面采集所述故障注入配置信息；
    所述故障回收模块，用于接收所述故障测试反馈结果，并根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
  12. 根据权利要求8至11中任意一项所述的系统，其特征在于，所述虚拟化软硬件平台还包括硬件故障注入模块和软件故障注入模块；
    所述硬件故障注入模块，用于按照所述故障类型向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入；
    所述软件故障注入模块，用于按照所述故障类型向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
  13. 根据权利要求9至11中任意一项所述的系统，其特征在于，所述虚拟化软硬件平台和所述被控对象仿真平台部署在云服务器上。
  14. 根据权利要求9至13中任意一项所述的系统，其特征在于，所述虚拟化软硬件平台和所述被控对象仿真平台设置有统一的仿真时钟或者配置有硬件时钟，且所述虚拟化软硬件平台和所述被控对象仿真平台通过代码注入或者状态注入的方式进行了步长同步设置。
  15. 一种测试装置，其特征在于，所述装置应用于虚拟化软硬件平台，所述虚拟化软硬件平台包括虚拟化硬件模块和嵌入式软件模块，其中，所述虚拟化硬件模块用于表示能够在通用计算机上运行的虚拟化的嵌入式硬件系统，所述装置包括：
    获取模块，用于获取故障注入配置信息，所述故障注入配置信息用于指示故障注入点和故障类型，所述故障注入点位于所述虚拟化硬件模块或者所述嵌入式软件模块；
    注入模块，用于根据所述获取模块获取的故障类型向所述获取模块获取的故障注入点进行故障注入。
  16. 根据权利要求15所述的装置，其特征在于，所述装置还包括：
    翻译模块，用于将嵌入式硬件系统的二进制执行指令动态翻译为能够在通用计算机上运行的二进制执行指令，得到所述虚拟化硬件模块。
  17. 根据权利要求16所述的装置，其特征在于，所述翻译模块还用于：
    根据测试需求，从所述嵌入式硬件系统的二进制执行指令中确定目标二进制执行指令进行所述动态翻译，得到所述虚拟化硬件模块。
  18. 根据权利要求15至17中任意一项所述的装置，其特征在于，所述装置还包括：
    生成模块，用于在进行所述故障注入后，生成控制信号；
    第一发送模块，用于将所述控制信号发送至被控对象仿真平台，以使所述被控对象仿真平台按照所述控制信号对被控对象进行性能仿真，得到故障测试反馈结果。
  19. 根据权利要求18所述的装置，其特征在于，所述装置还包括：
    接收模块，用于接收所述故障测试反馈结果；
    第二发送模块，用于将所述故障反馈结果发送至故障注入控制台，以使所述故障注入控制台根据所述故障测试反馈结果与预先配置的期望故障处理结果，进行故障分析。
  20. 根据权利要求15至19中任意一项所述的装置，其特征在于，所述注入模块还用于：
    根据所述故障类型，通过改造标定协议或者通用测量与标定协议的方式向位于所述虚拟化硬件模块的故障注入点进行硬件故障注入。
  21. 根据权利要求15至19中任意一项所述的装置，其特征在于，所述注入模块还用于：
    根据所述故障类型，通过代码注入或者状态注入的方式向位于所述嵌入式软件模块的故障注入点进行软件故障注入。
  22. 一种测试装置，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令时实现权利要求1至7中任意一项所述的方法。
  23. 一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1至7中任意一项所述的方法。
PCT/CN2022/096367 2022-05-31 2022-05-31 一种测试方法、系统及装置 WO2023230883A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202280007333.XA CN117597669A (zh) 2022-05-31 2022-05-31 一种测试方法、系统及装置
PCT/CN2022/096367 WO2023230883A1 (zh) 2022-05-31 2022-05-31 一种测试方法、系统及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/096367 WO2023230883A1 (zh) 2022-05-31 2022-05-31 一种测试方法、系统及装置

Publications (1)

Publication Number Publication Date
WO2023230883A1 true WO2023230883A1 (zh) 2023-12-07

Family

ID=89026646

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/096367 WO2023230883A1 (zh) 2022-05-31 2022-05-31 一种测试方法、系统及装置

Country Status (2)

Country Link
CN (1) CN117597669A (zh)
WO (1) WO2023230883A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724920A (zh) * 2024-02-07 2024-03-19 四川赛狄信息技术股份公司 一种嵌入式设备的测试方法、装置、上位机及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103529820A (zh) * 2013-09-26 2014-01-22 北京航天自动控制研究所 一种适用于嵌入式设备的故障注入测试系统及测试方法
US20170060655A1 (en) * 2015-08-31 2017-03-02 International Business Machines Corporation Isolating hardware and network failures in a computing environment
CN107025171A (zh) * 2017-03-10 2017-08-08 深圳航天科技创新研究院 一种实现虚拟验证系统故障注入的方法
CN113127331A (zh) * 2019-12-31 2021-07-16 航天信息股份有限公司 一种基于故障注入的测试方法、装置及计算机设备
CN114063472A (zh) * 2021-11-18 2022-02-18 成都邦飞科技有限公司 一种数字化仿真设计系统、方法、存储介质及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103529820A (zh) * 2013-09-26 2014-01-22 北京航天自动控制研究所 一种适用于嵌入式设备的故障注入测试系统及测试方法
US20170060655A1 (en) * 2015-08-31 2017-03-02 International Business Machines Corporation Isolating hardware and network failures in a computing environment
CN107025171A (zh) * 2017-03-10 2017-08-08 深圳航天科技创新研究院 一种实现虚拟验证系统故障注入的方法
CN113127331A (zh) * 2019-12-31 2021-07-16 航天信息股份有限公司 一种基于故障注入的测试方法、装置及计算机设备
CN114063472A (zh) * 2021-11-18 2022-02-18 成都邦飞科技有限公司 一种数字化仿真设计系统、方法、存储介质及电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724920A (zh) * 2024-02-07 2024-03-19 四川赛狄信息技术股份公司 一种嵌入式设备的测试方法、装置、上位机及介质
CN117724920B (zh) * 2024-02-07 2024-04-26 四川赛狄信息技术股份公司 一种嵌入式设备的测试方法、装置、上位机及介质

Also Published As

Publication number Publication date
CN117597669A (zh) 2024-02-23

Similar Documents

Publication Publication Date Title
CN103235756B (zh) 一种面向嵌入式系统分区应用程序软件的仿真测试方法
US9465718B2 (en) Filter generation for load testing managed environments
WO2019216976A1 (en) Analytics for an automated application testing platform
CN114036013B (zh) 一种基于uvm的应答器芯片多模块同步验证平台和验证方法
CN114528792B (zh) 芯片验证方法、装置、电子设备及存储介质
WO2014088398A1 (en) Automated test environment deployment with metric recommender for performance testing on iaas cloud
US7957951B2 (en) Address translation system for use in a simulation environment
US9405315B2 (en) Delayed execution of program code on multiple processors
CN111752772B (zh) 存储设备仿真测试系统及方法
CN108628734B (zh) 一种功能程序调试方法和终端
WO2024130861A1 (zh) 一种云原生的硬件逻辑仿真fpga加速方法及系统
CN108459951A (zh) 测试方法和装置
WO2023230883A1 (zh) 一种测试方法、系统及装置
US9117018B2 (en) Method of debugging software and corresponding computer program product
US20130024178A1 (en) Playback methodology for verification components
CN116340150A (zh) 一种基于uvm的可重用的寄存器性能交互验证系统及其应用
CN112860587B (zh) Ui自动测试方法和装置
CA3144852A1 (en) Automatic generation of integrated test procedures using system test procedures
CN114548027A (zh) 在验证系统中追踪信号的方法、电子设备及存储介质
CN117076337B (zh) 一种数据传输方法、装置、电子设备及可读存储介质
CN116467211B (zh) 一种基于数字化仿真环境的系统级测试验证方法
CN115993937A (zh) 一种多进程固态硬盘仿真环境实现方法和装置
US10481969B2 (en) Configurable system wide tests
Muli et al. Virtual validation-a new paradigm in controls engineering
KR102006212B1 (ko) 제 1 시뮬레이터에서 이용되는 xml 스크립트를 변환하여, 제 2 시뮬레이터에서 이용되는 파이썬 스크립트를 생성하는 장치 및 방법

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202280007333.X

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22944220

Country of ref document: EP

Kind code of ref document: A1