CN115982026A

CN115982026A - Instruction set testing method, device, equipment and storage medium

Info

Publication number: CN115982026A
Application number: CN202211694343.3A
Authority: CN
Inventors: 金辉; 王云浩
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2022-12-28
Filing date: 2022-12-28
Publication date: 2023-04-18

Abstract

The disclosure provides an instruction set testing method, an instruction set testing device, an instruction set testing equipment and a storage medium, wherein program data of an instruction set to be tested is obtained by carrying out static analysis on the instruction set to be tested; carrying out constraint solving on the program data of the instruction set to be tested to generate a path constraint set; generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain a test result of each test case; and determining the vulnerability information of the instruction set to be tested according to the test result of each test case.

Description

Instruction set testing method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for testing an instruction set.

Background

The protocol test refers to a test for evaluating the tested protocol by controlling and observing the external behavior thereof according to the protocol standard, and belongs to a test of communication rules between a software interface and an application server, so that before the software product is on line, a protocol vulnerability of the software product is an essential important link for testing.

The bottom layer instructions in the protocol are generally complex, because the program lacks type information containing semantics and syntax, the test aiming at the instruction set at the present stage generally focuses on the upper layer algorithm, but neglects the test of the instruction set, and the test aiming at the instruction set at the present stage basically stays at the stage of functional test and functional unit test.

Disclosure of Invention

The present disclosure provides an instruction set testing method, apparatus, device and storage medium to at least solve the above technical problems in the related art.

According to a first aspect of the present disclosure, there is provided an instruction set testing method, the method comprising:

performing static analysis on an instruction set to be tested to obtain program data of the instruction set to be tested, wherein the program data is used for displaying programs in the instruction set and execution paths among the programs;

performing constraint solution on the program data of the instruction set to be tested to generate a path constraint set, wherein the path constraint set comprises all execution paths executed by the test case;

generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain a test result of each test case;

and determining the vulnerability information of the instruction set to be tested according to the test result of each test case.

In an embodiment, the generating a plurality of test cases based on the path constraint set includes:

acquiring a path constraint condition of the test case, wherein the path constraint condition is used for limiting a generation range of a test seed, and the test seed is used for deriving a plurality of test cases;

and generating a plurality of test cases based on the path constraint set and the path constraint conditions of the test cases.

In an implementation manner, the obtaining of the path constraint condition of the test case includes:

determining test seeds which are easy to generate vulnerability information according to the type of the instruction set to be tested, and generating a first path constraint condition according to the test seeds which are easy to generate vulnerability information.

In an implementation manner, the generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain the test result of each test case includes:

and executing the instruction set to be tested according to the plurality of test cases based on a test score evaluation standard to obtain a test result of each test case and a test score of each test case, wherein the test score evaluation standard is determined according to a test speed threshold, a test result threshold and a path depth threshold reaching the instruction set to be tested.

receiving feedback information of the test scores of the test cases, generating a second path constraint condition, and determining a plurality of new test cases, wherein the second path constraint condition is a test seed corresponding to the test case with the test score higher than a score threshold; and executing the instruction set to be tested according to the new test cases to perform a new round of test until the number of the test cases which are accumulated to be tested meets the preset test number, and ending the test.

According to a second aspect of the present disclosure, there is provided an instruction set testing apparatus, the apparatus comprising:

the static analysis module is used for carrying out static analysis on an instruction set to be tested to obtain program data of the instruction set to be tested, wherein the program data is used for displaying programs in the instruction set and execution paths among the programs;

the generating module is used for carrying out constraint solving on the program data of the instruction set to be tested and generating a path constraint set, wherein the path constraint set comprises all execution paths executed by the test cases;

the fuzzy test module is used for generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain the test result of each test case;

and the vulnerability determining module is used for determining vulnerability information of the instruction set to be tested according to the test result of each test case.

In an implementation manner, the fuzz testing module is specifically configured to:

In an implementation, the fuzz testing module is specifically configured to:

and executing the instruction set to be tested according to the plurality of test cases based on a fuzzy test and a test score evaluation standard with a scoring mechanism type to obtain a test result of each test case and a test score of each test case, wherein the test score evaluation standard is determined according to a test speed threshold, a test result threshold and a path depth threshold reaching the instruction set to be tested.

In an implementation, the fuzz testing module is specifically configured to:

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods of the present disclosure.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of the present disclosure.

According to the instruction set testing method, the device, the equipment and the storage medium, static analysis is carried out on an instruction set to be tested, so that program data of the instruction set to be tested are obtained, wherein the program data are used for displaying programs in the instruction set and execution paths among the programs; performing constraint solution on the program data of the instruction set to be tested to generate a path constraint set, wherein the path constraint set comprises all execution paths executed by the test case; generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain a test result of each test case; and determining the vulnerability information of the instruction set to be tested according to the test result of each test case, so that vulnerability detection of invalid cases can be avoided, and the detection efficiency of vulnerability detection is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

FIG. 1 is a schematic diagram illustrating an implementation flow of an instruction set testing method according to an embodiment of the present disclosure;

fig. 2A illustrates an implementation flow diagram of an instruction set testing method provided in the second embodiment of the present disclosure;

FIG. 2B is a flowchart illustrating an implementation of an exemplary instruction set testing method according to a second embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of an instruction set testing apparatus provided in a third embodiment of the present disclosure;

fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, features and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

Example one

Fig. 1 is a flowchart of an instruction set testing method provided in an embodiment of the present disclosure, which may be performed by an instruction set testing apparatus provided in an embodiment of the present disclosure, where the apparatus may be implemented in software and/or hardware. The method specifically comprises the following steps:

s110, performing static analysis on the instruction set to be tested to obtain program data of the instruction set to be tested.

The program data is used for displaying programs in the instruction set and execution paths among the programs.

The instruction set under test may be the content of the specific execution in the protocol under test. For example, in protocol calculation, the instruction set to be tested may be a file such as an Advanced Encryption Standard (AES) implemented by the mpc file. The protocol under test refers to a protocol that needs to test for vulnerabilities. The program data may be obtained by static analysis of all programs in the instruction set and the execution paths that can be executed between the programs.

Because many instruction sets to be tested belong to bottom layer instructions, are binary files, not plaintext files, and cannot be directly measured like common codes when testing the vulnerability information of the instruction sets to be tested. Therefore, the embodiment can perform static analysis on the instruction set to be tested, translate the underlying instructions into a simple and standard intermediate language, and perform static analysis on the instruction set on the basis of the intermediate language to obtain the programs and the execution paths among the programs.

Specifically, the process of performing static analysis on the instruction set to be tested in this embodiment may be performing analysis on a static analysis component integrated in the system of the embodiment; analysis can also be performed through software or hardware with a static analysis function outside the system; the static analysis process can also be split, a software or hardware is used for translating the instruction set to be tested into an intermediate file, and then another software or hardware is used for analyzing based on the intermediate file. The program data of the instruction set to be tested obtained in this embodiment is a drawing program dependency graph, a data flow graph, and a program control flow graph obtained by performing static analysis based on the intermediate file obtained by translating the instruction set to be tested. The program dependency graph is drawn as the program itself, the data flow graph is an execution path between programs, for example, the data flow graph may be an operation process between programs, and the program control flow graph may be a plurality of complex execution paths depending on the programs.

And S120, carrying out constraint solving on the program data of the instruction set to be tested to generate a path constraint set.

The path constraint set comprises all execution paths for executing the test case.

Specifically, after the program data is obtained through static analysis, the program data may be analyzed based on the semantics of the translated intermediate language. For example, the embodiment may use a symbol execution component integrated with the system, or may use hardware or software having a symbol execution function outside the system to analyze the symbol execution component. The basic idea of the symbolic execution component is to replace a program variable with an abstract symbol, or represent the value of the program variable as a computational expression consisting of a symbolic value and a constant, and simulate the execution of a program, thereby performing correlation analysis. That is, the procedure of performing constraint solution on the program data of the instruction set to be tested by the symbol execution component may be to use an abstract symbol to replace a specific numerical value to test the instruction set to be tested, output a test result, and compare the test result with a preset result. If the result is correct, the path executed by the symbol is the required constraint path, which is one of the execution paths in the path constraint set.

It should be noted that, the symbolic execution component adopted in the present embodiment not only summarizes the constraint paths corresponding to the above results correctly into the path constraint set, but actually, the symbolic execution component may also expand the constraint paths and merge the constraint paths into the path constraint set. For example, if the constraint path solved by the symbolic execution component constraint is AB, the constraint path AB is expanded to A1A2A3B1B2B3 according to a certain rule and stored in the path constraint set. Thus, the path constraint set generated by the present embodiment covers all execution paths for the instruction set under test.

S130, generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain the test result of each test case.

The test case refers to a case for inputting a command set to be tested to perform testing. The test result refers to a result of the test case after bug detection, and may include, for example, an execution result after the execution of the instruction set to be tested, and further include, for example, an execution path corresponding to the test case in addition to the execution result.

Specifically, in the present embodiment, in order to reduce invalid test cases, the test cases are not generated randomly, but generated on the basis of the path constraint set. The process of generating the test case may also depend on the inside or outside of the system, and the specific generation process is not limited as long as the test case can be generated in this embodiment. After the test cases are generated, the test cases can be sequentially input into the instruction set to be tested to perform the vulnerability test. It should be noted that the vulnerability testing method adopted in this embodiment may be any testing method capable of implementing vulnerability detection in the related art, for example, may be a fuzz test, specifically, for example, a fuzz test with a scoring mechanism or a dynamic instrumentation fuzz test.

Specifically, in this embodiment, a plurality of test cases may be directly generated based on the path constraint set, or other generation conditions may be added, and the generation range of the test cases is constrained based on the path constraint set and the other generation conditions. The other generation conditions may be related to the characteristics of the instruction set to be tested, or may be conditions set by a person based on experience. In addition, the number of test cases generated may be any value. For example, if there is one computation-class protocol, the generation range of the test case is 0 to 1000 as the path constraint set after constraint solution is 0 to 1000, and this embodiment may directly generate a plurality of test cases based on the path constraint set. If research and development personnel obtain that the possibility of the bugs existing in 800-1000 is low according to the experience, therefore, in order to improve the quality of the test cases, other generating conditions can be added, the test cases with low possibility of the bugs are eliminated, and the test cases are generated on the basis of 0-800.

S140, determining vulnerability information of the instruction set to be tested according to the test result of each test case.

In this embodiment, after the test case is input into the instruction set to be tested, the test result corresponding to the test case may be obtained, and then the vulnerability information of the instruction set to be tested may be determined according to the test result. For example, for a computation-based protocol, the embodiment may execute a to-be-tested instruction set of the computation-based protocol on a to-be-tested case, and if an obtained test result is different from a preset result, it indicates that a bug exists in an execution path executed by the test case in the to-be-tested instruction set; for a communication protocol, the embodiment may execute a to-be-tested instruction set of a computation protocol on a to-be-tested case, and if an obtained test result is a communication failure, it indicates that a bug exists in an execution path executed by the test case in the to-be-tested instruction set.

In the instruction set testing method provided by this embodiment, the program data of the instruction set to be tested is obtained by performing static analysis on the instruction set to be tested; carrying out constraint solving on program data of the instruction set to be tested to generate a path constraint set; generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain the test result of each test case; according to the test result of each test case, the vulnerability information of the instruction set to be tested is determined, vulnerability detection of invalid cases can be avoided, and vulnerability detection efficiency is improved.

Example two

Fig. 2A is a flowchart of an instruction set testing method provided in the second embodiment of the present disclosure, where the second embodiment of the present disclosure is based on the above embodiments, where the method, in the second embodiment of the present disclosure, generates a plurality of test cases based on a path constraint set, and executes an instruction set to be tested according to the plurality of test cases to obtain a test result of each test case, includes: generating a plurality of test cases based on a path constraint set and a fuzzy test with a scoring mechanism type, executing a to-be-tested instruction set according to the plurality of test cases based on a test score evaluation standard, and obtaining a test result of each test case and a test score of each test case, wherein the test score evaluation standard is determined according to a test speed threshold, a test result threshold and a path depth threshold reaching the to-be-tested instruction set. The method specifically comprises the following steps:

s210, performing static analysis on the instruction set to be tested to obtain program data of the instruction set to be tested.

And S220, carrying out constraint solving on the program data of the instruction set to be tested to generate a path constraint set.

And S230, generating a plurality of test cases based on the path constraint set and the fuzzy test with the scoring mechanism type, and executing the instruction set to be tested according to the plurality of test cases based on the test score evaluation standard to obtain the test result of each test case and the test score of each test case.

The test score evaluation standard is determined according to a test speed threshold, a test result threshold and a path depth threshold reaching the instruction set to be tested. The test speed threshold value can be determined according to the test time of the fuzz test of the to-be-tested case, a plurality of test speed threshold values can be set in the embodiment, each test speed threshold value corresponds to a different speed range, and different scores correspond to different test speed threshold values; the test result threshold can be determined according to a threshold set by a test result of the fuzz test of the to-be-tested case, if the test result threshold meets the requirement, the set threshold is lower, if the test result threshold does not meet the requirement, the set threshold is higher, and different scores are correspondingly provided for different test result thresholds; the path depth threshold reaching the instruction set to be tested may be determined according to the path depth from the case to be tested to the instruction set to be tested during the fuzzing test, in this embodiment, different path depth thresholds may be set according to different path depths, each path depth threshold corresponds to a different path depth range, and different scores are provided for different path depth thresholds, for example, the deeper the path depth from the case to be tested to the instruction set to be tested is, the higher the coverage rate is, the higher the score is.

In addition, the test score evaluation criterion provided by the embodiment is determined comprehensively according to the test speed threshold, the test result threshold and the path depth threshold reaching the instruction set to be tested, and is not determined by a single element. For example, some test cases have a short test time and a fast test speed, and even though the test result threshold is high, the test score may not be very high because the path into the instruction set to be tested is shallow.

Specifically, the fuzzy test is a representative technology of software vulnerability analysis, and has the advantages of being independent of program source codes and low in system consumption, so that the fuzzy test plays an important role in the field of software vulnerability analysis. The fuzzy test has the advantage of low cost because the internal structure of a program does not need to be known, but has the problems of high test blindness, low test efficiency, incapability of ensuring code coverage rate and the like. In the fuzzy test, a case generation method is also one of the key technologies, and the case generation method directly influences the coverage and vulnerability mining efficiency of the fuzzy test. The present embodiment employs a symbol execution technique to overcome the disadvantage of the fuzz test, because the symbol execution technique has a good path coverage. Specifically, in this embodiment, a symbolic execution technology is used to perform constraint solution on the statically analyzed instruction set to be tested, so as to obtain a path constraint set, and based on the path constraint set and the fuzzy test, a test case with guidance can be generated for the fuzzy test.

Specifically, the present embodiment may adopt a fuzzy test with a scoring mechanism type, and score each to-be-tested case based on the test score evaluation standard, so that the to-be-tested case inputs the to-be-tested instruction set to perform the execution result of the fuzzy test, and the test result and the corresponding test score are included. The quality of the corresponding test case can be reflected through the test score. For example, if the test score for the test case a to execute the instruction set to be tested to perform the fuzz test is 90, and the test score for the test case B to execute the instruction set to be tested to perform the fuzz test is 60, it indicates that the quality of the test case B is higher than that of the test case a.

S240, determining vulnerability information of the instruction set to be tested according to the test result of each test case.

In the embodiment, by means of a way of combining the path constraint set and the fuzzy test, the efficiency of the fuzzy test can be greatly improved, and the detection effect can be obviously improved.

In the embodiment of the present disclosure, generating a plurality of test cases based on the path constraint set includes: acquiring path constraint conditions of the test cases, wherein the path constraint conditions are used for limiting the generation range of test seeds, and the test seeds are used for deriving a plurality of test cases; and generating a plurality of test cases based on the path constraint set and the path constraint conditions of the test cases.

Specifically, in order to improve the generation quality of the test case, the generation range of the test seed may be further limited by setting a path constraint condition. The method comprises the steps of obtaining a test seed with higher quality based on a path constraint set and path constraint conditions of test cases, and generating a plurality of test cases based on the test seed with higher quality.

In the embodiment of the present disclosure, obtaining the path constraint condition of the test case includes: determining test seeds which are easy to generate vulnerability information according to the type of the instruction set to be tested, and generating a first path constraint condition according to the test seeds which are easy to generate vulnerability information.

Because the characteristics of different instruction sets to be tested are different and the weak items are also different, the embodiment can set the first path constraint condition aiming at the characteristics, and the test seeds which are easy to generate vulnerability information are selected in a targeted manner through the first path constraint condition. For example, taking the multi-party secure computation SPDZ protocol in the private computation protocol as an example, since the private computation protocol stores more instruction sets × mpc files in the Programs/sources/files, the files are all specific execution algorithms, and have characteristics of large mathematical computation amount and more modulus taking operations, for the instruction set files to be tested, attention should be paid to data overflow, inversion, command injection, and null pointer usage of the instruction set files to be tested, and protocol correctness and performance tests, etc., and it is easier to find the attack surface, so the first path constraint condition may be set based on the weak items to improve the quality of the test seeds. For another example, when a to-be-tested instruction set of the privacy computation type is tested, the embodiment can test the floating point number and the fixed point number which are sequentially executed in a finite field in a targeted manner, and the part can find the vulnerability information of the to-be-tested instruction set more easily, so that the first path constraint condition can be set based on the above situation to improve the quality of the test seed.

The fuzzy test is carried out on the weak items of the instruction set to be tested in a targeted manner, so that vulnerability information of the instruction set to be tested can be found more quickly, and potential safety hazards are eliminated.

In the embodiment of the present disclosure, obtaining the path constraint condition of the test case includes: receiving feedback information of the test scores of all the test cases, generating a second path constraint condition, and determining a plurality of new test cases; and executing the instruction set to be tested according to the new test cases to perform a new round of test until the number of the test cases which are accumulated to perform the test meets the preset test number, and finishing the test.

The second path constraint condition is a test seed corresponding to a test case with a test score higher than a score threshold; the preset test number refers to the total number of test cases for testing; the score threshold refers to a critical score value used to distinguish whether or not to employ a test seed from which the test case was derived.

Specifically, in order to find more comprehensive vulnerability information, multiple rounds of fuzzy tests are generally set, and each round of fuzzy test can be set with one to-be-tested case for testing, and can also be set with multiple to-be-tested cases for testing. For convenience of understanding, in an exemplary embodiment, the preset number of test cases is 100, and 10 test cases are set for each round of test, in this embodiment, a first batch of 10 test cases are generated through a path constraint set and/or a first path constraint condition, the first batch of 10 test cases are input into an instruction set to be tested one by one for testing, a second batch of 10 test cases are generated, and the input into the instruction set to be tested one by one for testing, until a 100 th test case is generated for testing, and the test is finished.

Specifically, the present embodiment may evaluate the test work before the summary according to the test score of the fuzz test, so as to determine the generation direction of the next test seed. For example, the present embodiment may receive feedback information of the test scores of the test cases, generate the second path constraint condition through the feedback information of the test scores of the test cases, screen out test seeds with higher quality according to the second path constraint condition when a test case is generated in the next round, generate a plurality of new test cases based on the test seeds with higher quality, execute the instruction set to be tested according to the plurality of new test cases to perform a new round of testing, and repeat this procedure until the number of test cases to be tested in the accumulated test satisfies the preset number of tests, and end the testing.

For example, if there is a computational protocol, the path constraint set after constraint solution is 0 to 1000, the preset number of test cases is 100, 10 test cases are set for each test, after the first test, the present embodiment receives feedback information of the test scores of the first test case, and it is known that the test scores of the test cases in the range of 500 to 1000 are higher, and it is found that there is more bug information, and the execution path of the executed instruction set to be tested is deeper, therefore, the present embodiment may set the range of 500 to 1000 as the second path constraint condition, and based on the path constraint set and the second path constraint condition, generate a new test seed in the range of 500 to 1000, generate a new test case based on the new test seed, perform the second test, and so on, after each test, the next test seed is given with guidance information according to the feedback information of the test scores of each test case, thereby facilitating to mine the bug information of the deeper execution path.

Fig. 2B is a flowchart of an exemplary instruction set testing method according to an embodiment of the disclosure. In the embodiment, a privacy computation instruction set is taken as an example for explanation, and as shown in fig. 2B, the test system of the embodiment integrates a static analysis component, a symbol execution component, and a fuzzy test component. Wherein the static analysis component may also be referred to as an instruction translator. Specifically, in this embodiment, the static analysis component performs static analysis on the privacy computation instruction set to obtain program data, the symbol execution component performs constraint solving on the program data to obtain a path constraint set, a test case is generated based on the path constraint set and the fuzzy test to perform privacy computation, a computation result is evaluated and fed back to the symbol execution component, that is, a second path constraint condition is added, so as to generate a test case of a next round based on the path constraint set and the second path constraint condition, and the test is finished until the number of generated test cases reaches a preset test number.

In the embodiment, the quality of the test seeds is considered, each test seed is not treated equally, the time spent on low-quality test seeds is avoided, the second path constraint condition is generated through the feedback information based on the test scores of the test cases, and the second path constraint condition is used for screening out the test seeds with higher quality, so that the coverage rate of the instruction set to be tested is expanded, and the undiscoverable vulnerability information is revealed.

EXAMPLE III

Fig. 3 is a schematic structural diagram of an instruction set testing apparatus provided in an embodiment of the present disclosure, where the apparatus specifically includes:

the static analysis module 310 is configured to perform static analysis on the instruction set to be tested to obtain program data of the instruction set to be tested, where the program data is used to display programs in the instruction set and execution paths between the programs;

the generating module 320 is configured to perform constraint solution on the program data of the instruction set to be tested, and generate a path constraint set, where the path constraint set includes all execution paths executed by the test case;

the fuzzy test module 330 is configured to generate a plurality of test cases based on the path constraint set, and execute the instruction set to be tested according to the plurality of test cases to obtain a test result of each test case;

the vulnerability determining module 340 is configured to determine vulnerability information of the instruction set to be tested according to the test result of each test case.

In one implementation, the fuzz testing module 330 is specifically configured to: acquiring path constraint conditions of the test cases, wherein the path constraint conditions are used for limiting the generation range of test seeds, and the test seeds are used for deriving a plurality of test cases; and generating a plurality of test cases based on the path constraint set and the path constraint conditions of the test cases.

In one implementation, the fuzz testing module 330 is specifically configured to: determining test seeds which are easy to generate vulnerability information according to the type of the instruction set to be tested, and generating a first path constraint condition according to the test seeds which are easy to generate vulnerability information.

In one embodiment, the fuzz testing module 330 is configured to: based on a fuzzy test with a scoring mechanism type and a test score evaluation standard, executing the instruction set to be tested according to a plurality of test cases to obtain a test result of each test case and a test score of each test case, wherein the test score evaluation standard is determined according to a test speed threshold, a test result threshold and a path depth threshold reaching the instruction set to be tested.

In one implementation, the fuzz testing module 330 is specifically configured to: receiving feedback information of the test scores of all the test cases, generating a second path constraint condition, and determining a plurality of new test cases, wherein the second path constraint condition is a test seed corresponding to the test case with the test score higher than a score threshold; and executing the instruction set to be tested according to the new test cases to perform a new round of test until the number of the test cases which are accumulated to perform the test meets the preset test number, and finishing the test.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

FIG. 4 shows a schematic block diagram of an example electronic device 400 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 4, the apparatus 400 includes a computing unit 401 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 402 or a computer program loaded from a storage unit 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The calculation unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

A number of components in device 400 are connected to I/O interface 405, including: an input unit 406 such as a keyboard, a mouse, or the like; an output unit 407 such as various types of displays, speakers, and the like; a storage unit 408 such as a magnetic disk, optical disk, or the like; and a communication unit 409 such as a network card, modem, wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 401 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 401 performs the various methods and processes described above, such as the instruction set test method. For example, in some embodiments, the instruction set testing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 400 via the ROM 402 and/or the communication unit 409. When the computer program is loaded into RAM 403 and executed by computing unit 401, one or more steps of the instruction set testing method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the instruction set testing method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present disclosure, "a plurality" means two or more unless specifically limited otherwise.

The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present disclosure, and all the changes or substitutions should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method for instruction set testing, the method comprising:

2. The method of claim 1, wherein generating a plurality of test cases based on the set of path constraints comprises:

3. The method according to claim 2, wherein the obtaining path constraints of the test cases comprises:

4. The method according to claim 2 or 3, wherein the generating a plurality of test cases based on the path constraint set, and executing the instruction set to be tested according to the plurality of test cases to obtain a test result of each test case comprises:

5. The method according to claim 4, wherein the obtaining path constraints of the test cases comprises:

receiving feedback information of the test scores of the test cases, generating a second path constraint condition, and determining a plurality of new test cases, wherein the second path constraint condition is a test seed corresponding to the test case with the test score higher than a score threshold; and executing the instruction set to be tested according to the new test cases to perform a new round of test until the number of the test cases which are subjected to test cumulatively meets the preset test number, and ending the test.

6. An instruction set testing apparatus, the apparatus comprising:

the fuzzy test module is used for generating a plurality of test cases based on the path constraint set and executing the instruction set to be tested according to the plurality of test cases to obtain the test result of each test case;

7. The apparatus of claim 6, wherein the fuzz testing module is specifically configured to:

8. The apparatus of claim 7, wherein the fuzz testing module is specifically configured to:

and determining test seeds which are easy to generate vulnerability information according to the type of the instruction set to be tested, and generating a first path constraint condition according to the test seeds which are easy to generate vulnerability information.

9. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

10. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-5.