WO2023207973A1 - Compiler test method and apparatus, case generation method and apparatus, and instruction storage structure - Google Patents

Abstract

A compiler test method and apparatus, a device, a storage medium, and a storage structure for a test instruction. The method comprises: on the basis of data which is stored in an instruction database and is related to a test instruction, constructing a test case, and a reference compiling instruction corresponding to the test case; compiling the test case by using a compiler to be tested, so as to obtain a target compiling instruction; and on the basis of a comparison result between the reference compiling instruction corresponding to the test case and the target compiling instruction, determining whether the compiler is abnormal. In the embodiments of the present disclosure, automatic testing of a compiler can be realized, thereby improving the test efficiency and the accuracy of a test result.

Description

Compiler testing methods, use case generation methods, devices and instruction storage structures

Cross-references to related applications

This application claims priority from the Chinese patent application with application number 2022104743224, submitted on April 29, 2022, which is incorporated herein by reference.

Technical field

The present disclosure relates to the field of computer technology, and in particular, to a compiler testing method, a use case generation method, a device, a device, a storage medium, and a storage structure of test instructions.

Background technique

Compilers are usually used to compile programs from one language into another language. The accuracy of the compiler results is a prerequisite for ensuring the normal operation of the program. If the compiler function is abnormal, it will not only cause the program to not obtain the originally intended running results, but may even mislead the debugging (troubleshooting) of subsequent programs. Therefore, after developing a compiler, it is necessary to provide a solution that can test the compiler to detect abnormalities in the compiler in a timely manner.

Contents of the invention

The present disclosure provides a compiler testing method, use case generation method, device, equipment, storage medium and storage structure of test instructions.

According to a first aspect of an embodiment of the present disclosure, a method for testing a compiler is provided. The method includes: constructing a test case based on data related to test instructions stored in an instruction database, and a reference compilation instruction corresponding to the test case. ;Use the compiler to be tested to compile the test case to obtain the target compilation instruction; based on the comparison result of the reference compilation instruction and the target compilation instruction, determine whether there is an exception in the compiler.

In some embodiments, the test cases include functional test cases and/or stability test cases. The functional test cases include a single test instruction for testing the function of the compiler; the stability test case It is spliced together from multiple test instructions and used to test the stability of the compiler.

In some embodiments, the stability test case meets one or more of the following conditions: the compiler compiles the stability test case for longer than a preset time; the test instructions included in the stability test case The quantity is greater than the preset quantity; the data volume of the stability test case is greater than the preset data volume.

In some embodiments, the test cases include functional test cases and stability test cases. Based on the comparison results of the reference compilation instructions corresponding to the test cases and the target compilation instructions, it is determined whether there is an abnormality in the compiler. , including: determining whether there is an abnormality in the function of the compiler based on the comparison result of the reference compilation instruction corresponding to the functional test case and the target compilation instruction corresponding to the functional test case; There is no abnormal situation. Based on the reference compilation instructions corresponding to the stability test case and the stability test case, the The comparison results of the target compilation instructions corresponding to the qualitative test cases are used to determine whether there are abnormalities in the stability of the compiler.

In some embodiments, the test instruction includes multiple components, and the data related to the test instruction includes for one component among the multiple components, the characteristic data corresponding to the component and the characteristic data corresponding to The reference compiled data, the characteristic data corresponding to this component is stored in association with the characteristic data corresponding to the remaining components among the plurality of components except this component.

In some embodiments, the component includes one or more of the following: instruction type, control domain, and instruction address, where the control domain is used to represent the control operations included in the test instruction.

In some embodiments, the test cases include a plurality of functional test cases for testing the compiler function, the test cases are constructed based on the data related to the test instructions stored in the instruction database, and the test cases correspond to The reference compilation instructions include: for one component among the plurality of components, a piece of target characteristic data corresponding to the component is determined from the remaining components among the plurality of components except the component. Associated feature data stored in association with the target feature data; generating test instructions based on the target feature data and the associated feature data, and obtaining the test based on the reference compiled data corresponding to the target feature data and the reference compiled data corresponding to the associated feature data The reference compilation instruction corresponding to the instruction; the test instruction is used as a functional test case, and the reference compilation instruction corresponding to the test instruction is used as the reference compilation instruction corresponding to the functional test case.

In some embodiments, the test cases include stability test cases for testing the stability of the compiler. The test cases are constructed based on the data related to the test instructions stored in the instruction database and the reference compilation corresponding to the test cases. Instructions include: for one of the plurality of components, for a piece of target characteristic data in the component, respectively determine the relationship with the component from the remaining components of the plurality of components except the component. The target feature data is associated with the stored associated feature data; a test instruction is generated based on the target feature data and the associated feature data, and the test instruction correspondence is obtained based on the reference compiled data corresponding to the target feature data and the reference compiled data corresponding to the associated feature data. The reference compilation instructions are used to splice the generated test instructions to obtain the stability test case, and the reference compilation instructions corresponding to the multiple test instructions are used to splice the reference compilation instructions corresponding to the stability test case.

In some embodiments, the data related to the test instructions is stored in a tree structure, and the root node of the tree structure is used to store a piece of target feature data in the instruction type and the reference compiled data corresponding to the target feature data. , the child nodes of the tree structure are used to store the first associated feature data associated with the target feature data in the control domain and the reference compiled data corresponding to the first associated feature data, and the leaves of the tree structure The node is used to store the second associated feature data associated with the target feature data and the first associated feature data and the reference compiled data corresponding to the second associated feature data in the instruction address.

In some embodiments, constructing a test case and a reference compilation instruction corresponding to the test case based on the data related to the test instruction stored in the instruction database includes: obtaining the data from the root node to the leaf node in the tree structure. The characteristic data stored in each node in any path, and the reference compiled data corresponding to the characteristic data; a test instruction is obtained by splicing the characteristic data stored in each node, and the reference compiled data corresponding to the characteristic data stored in each node is spliced to obtain The reference compilation instruction corresponding to the test instruction; constructing a test case based on the obtained test instruction, and constructing the reference compilation instruction corresponding to the test case based on the reference compilation instruction corresponding to the test instruction.

In some embodiments, after determining that there is an exception in the compiler based on a comparison result between the reference compilation instructions corresponding to the test case and the target compilation instructions, the method further includes: outputting a test report, and the test The report includes one or more of the following information: the type of exception, the test instruction in which the exception occurs, and the component in which the exception occurs in the test instruction; wherein the type of exception includes functional abnormality or stability abnormality, and the component Including instruction type, control domain, and instruction address.

According to a second aspect of the embodiment of the present disclosure, a storage structure of a test instruction is provided. The storage structure includes: a tree structure; the tree structure is used to store data related to the test instruction; the data related to the test instruction The data includes the characteristic data corresponding to each component of the multiple components of the test instruction and the reference compiled data corresponding to the characteristic data; the multiple components include instruction type, control domain and instruction address, and the control domain represents The control operations included in the test instructions; wherein, the root node of the tree structure is used to store a piece of target feature data in the instruction type and the reference compilation data corresponding to the target feature data, and the sub-tree of the tree structure The node is used to store the first associated feature data associated with the target feature data in the control domain and the reference compiled data corresponding to the first associated feature data. The leaf nodes of the tree structure are used to store the instruction address. The second associated feature data associated with the target feature data and the first associated feature data and the reference compiled data corresponding to the second associated feature data.

According to the third aspect of the embodiment of the present disclosure, a method for generating test cases is provided. The test cases are used to test the compiler. The test instructions for constructing the test cases are stored through the storage structure described in the second aspect, so The method includes: obtaining the characteristic data stored in each node in any path from the root node to the leaf node in the tree structure, and the reference compiled data corresponding to the characteristic data; splicing the characteristic data stored in each node to obtain A test instruction, splicing the reference compilation data corresponding to each characteristic data to obtain the reference compilation instruction corresponding to the test instruction; generating a test case based on the obtained test instruction, and generating the test case corresponding to the reference compilation instruction corresponding to the test instruction Reference compilation directives.

According to a fourth aspect of an embodiment of the present disclosure, a test device for a compiler is provided, the device including: a building module for constructing test cases based on data related to test instructions stored in an instruction database and the The reference compilation instructions corresponding to the test cases; the compilation module is used to compile the test cases using the compiler to be tested to obtain the target compilation instructions; the determination module is used to compare the reference compilation instructions and the target compilation instructions As a result, it is determined whether there is an exception in the compiler.

According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes a processor, a memory, and computer instructions stored in the memory for execution by the processor. The processor executes the computer instructions. When the instruction is issued, the method mentioned in the first aspect and/or the third aspect may be implemented.

According to a sixth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided. Computer instructions are stored on the storage medium. When the computer instructions are executed, the above mentioned first and/or third aspects are implemented. method.

In the embodiment of the present disclosure, when testing the compiler, an instruction database can be set in advance. Data related to test instructions can be stored in the instruction database, and then test cases can be automatically constructed based on the data related to test instructions stored in the instruction database. , and the reference compiler instructions corresponding to the test case, and then use the compiler to be tested to compile the test case to obtain the target compilation instructions. Based on the comparison results of the target compilation instructions and the reference compilation instructions corresponding to the test case, determine whether there is an exception in the compiler. . In this way, test cases can be automatically constructed based on the data stored in the instruction database, and the compiler can be automatically tested. This can automate the entire testing process, improve testing efficiency, and improve the accuracy of test results.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not restrictive of the disclosure.

Description of the drawings

The accompanying drawings herein are incorporated into and constitute a part of this specification. They illustrate embodiments consistent with the disclosure and, together with the description, serve to explain the technical solutions of the disclosure.

Figure 1(a) is a flow chart of a compiler testing method according to an embodiment of the present disclosure.

Figure 1(b) is a schematic structural diagram of a test instruction according to an embodiment of the present disclosure.

Figure 1(c) is a tree structure diagram for storing test instruction related data according to an embodiment of the present disclosure.

FIG. 1(d) is a schematic path diagram in a tree structure diagram for storing test instruction related data according to an embodiment of the present disclosure.

Figure 2 is a schematic diagram of an instruction database according to an embodiment of the present disclosure.

Figure 3 is a schematic diagram of a test instruction storage method according to an embodiment of the present disclosure.

FIG. 4 is a schematic diagram of the logical structure of a compiler testing device according to an embodiment of the present disclosure.

Figure 5 is a schematic diagram of the logical structure of a device according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects of the disclosure as detailed in the appended claims.

The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure, and to make the above objects, features and advantages of the embodiments of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure are described below in conjunction with the accompanying drawings. The plan is explained in further detail.

Compilers are usually used to compile programs from one language into another language. The accuracy of the compiler results is a prerequisite for ensuring the normal operation of the program. If the function of the compiler is abnormal, not only will the program not be able to obtain the originally intended running results, but it will even be misleading for subsequent debugging of the program. Take the assembler as an example. The assembler is usually used to convert assembly language into machine language. If the function of the assembler is abnormal, the final machine language may have errors, resulting in errors in the final program running results.

For some self-developed compilers, it is usually necessary to test their functions before putting them into use in products. In related technologies, there are relatively few solutions that specifically test the compiler alone. Most of them test the entire system that runs the program, that is, by testing the entire system and comparing the final running results of the program with the expected results to determine Check whether there is an abnormality in the entire system. If there is an abnormality, check whether there is an abnormality in the compilation process. This method cannot independently test the functionality of the compiler. There are also technologies that directly test the compiler manually. For example, manually compare the compiler's compilation results with the correct results. This method is cumbersome, time-consuming, and error-prone.

Based on this, embodiments of the present disclosure provide a compiler testing method that can pre-set instruction data Library, the instruction database can store data related to test instructions, and then can automatically build test cases based on the data related to test instructions stored in the instruction database, as well as the reference compilation instructions corresponding to the test cases, and then use the compiler to be tested to compile This test case obtains the target compilation instruction, and determines whether there is an exception in the compiler based on the comparison result between the target compilation instruction and the reference compilation instruction corresponding to the test case. In this way, test cases can be automatically constructed based on the data stored in the instruction database, and the compiler can be automatically tested. This can automate the entire testing process, improve testing efficiency, and improve the accuracy of test results.

The compiler testing method in the embodiment of the present disclosure can be executed by various electronic devices, such as mobile phones, computers, cloud servers, etc.

The compiler in the embodiment of the present disclosure can be software or hardware with the function of converting one language of the program into another language. For example, the compiler can be a software program with the compilation function, or it can also have the compilation function. hardware equipment. The compiler may be an assembler that converts assembly language into machine language, or a compiler with other types of language conversion capabilities.

Among them, in order to facilitate distinction, the correctly compiled data corresponding to the characteristic data corresponding to each component in the test instruction is called reference compilation data, and the correctly compiled data corresponding to the test instruction or test case to be compiled is called reference compilation instruction. .

As shown in Figure 1(a), the compiler testing method may include the following steps:

S102. Construct test cases and reference compilation instructions corresponding to the test cases based on the data related to the test instructions stored in the instruction database;

In step S102, a test case and a reference compilation instruction corresponding to the test case may be constructed based on the data related to the test instruction stored in the instruction database. Among them, a test case can be a single test instruction or a test document constructed from a large number of test instructions. The data related to the test instruction may be various data that can determine the test instruction, and various data that can determine the reference compilation instruction corresponding to the test instruction. For example, in some scenarios, the data related to the test instruction can be a complete test instruction and the reference compilation instruction corresponding to the test instruction. For example, multiple test instructions and the corresponding code of each test instruction can be directly stored in the instruction data. Refer to the compilation instructions and then build the test cases directly based on the test instructions. In some scenarios, the test instruction can include multiple components, such as instruction type, control domain, instruction address, etc., where the control domain is the control operation included in the instruction. For example, addition instructions usually include rounding operations and saturation processing operations. etc., the data related to the test instruction can be the characteristic data corresponding to each component of the test instruction, that is, the specific content of each component (for example, if the component is an instruction type, the characteristic data can be an addition instruction, a subtraction instruction, etc.) , and the reference compiled data corresponding to the feature data. For example, the instruction database can also store the characteristic data corresponding to each component and the reference compiled data corresponding to the characteristic data, and then construct the test instructions based on these characteristic data, and then based on Build test cases based on the built test instructions, and the reference compilation instructions corresponding to the test cases.

S104. Use the compiler to be tested to compile the test case and obtain the target compilation instruction;

In step S104, after obtaining the test case, the test case can be compiled using the compiler to be tested to obtain the target compilation instruction.

S106. Determine whether there is an exception in the compiler based on the comparison result between the reference compilation instruction corresponding to the test case and the target compilation instruction.

In step S106, the target compilation instruction compiled by the compiler can be compared with the reference compilation instruction corresponding to the test case. If the two are consistent, it means that the compilation result of the compiler is correct. If the two are inconsistent, the compilation result is correct. The compiler result of the compiler is wrong and there is an exception in the compiler.

In some embodiments, the test case may be a test case for testing the functionality of the compiler, hereinafter referred to as a functional test case. Each functional test case can include a single test instruction. Generally, the compiler can be tested separately through multiple functional test cases to detect whether there are abnormalities in the compiler's compilation function, for example, to detect whether the compiler will incorrectly compile certain types of instructions, or to detect certain control operations in the instructions. Incorrect compilation, or incorrect compilation of certain addresses in instructions. Usually, in order to conduct a more comprehensive test on the functionality of the compiler, multiple functional test cases can be constructed. Each functional test case includes a test instruction. These test instructions can cover various instruction types, control operations and instruction addresses. , to ensure the comprehensiveness and completeness of test data.

In some embodiments, the test case may be a test case used to test the stability of the compiler, hereinafter referred to as a stability test case, and each stability test case may be spliced by multiple test instructions. Currently, when testing a compiler, it is common to only test the functionality of the compiler. However, even when the compiler functions normally, there will be scenarios where the amount of data to be compiled is large, causing the compiler to crash and not work properly. Therefore, the stability of the compiler can also be tested to detect compilation Will there be any exceptions when the compiler compiles a large number of instructions at one time? When testing the stability of the compiler, a large number of test instructions can be used to splice together to obtain one or more stability test cases with a large amount of data, which can be used to test the stability of the compiler.

In some embodiments, the test cases may include both functional test cases and stability test cases, that is, testing the above two properties of the compiler.

Since the stability test case is used to test the stability of the compiler, that is, whether the program will crash when the compiler processes a large amount of data at one time. Therefore, the data volume of stability test cases needs to be as large as possible to obtain accurate test results. In some embodiments, the stability test case may meet one or more of the following conditions: for example, the condition may be that the time it takes for the compiler to compile the stability test case is longer than the preset time, thus ensuring that the compiler runs for a longer time. Afterwards, no exception will occur. Among them, the preset duration can be flexibly set based on actual conditions. The condition can also be that the number of test instructions included in the stability test case can be greater than the preset number. For example, stability test cases need to be spliced together with at least 1,000 test instructions. The condition may also be that the data amount of the stability test case is greater than the preset data amount, for example, the data amount of the stability test case is greater than 1GB, etc. Of course, the above are just illustrative examples. In fact, when constructing stability test cases, the evaluation criteria can also be set to other standards, as long as it can ensure that the stability test case data is large and complex, and can accurately consider the stability of the compiler. Just sex.

In some embodiments, the built test cases include functional test cases and stability test cases, that is, the compiler can be tested for functionality and stability. When conducting the above two types of tests on the compiler, you can first conduct a functional test on the compiler, and then further test the stability of the compiler after ensuring that there are no abnormalities in the function of the compiler. For example, when determining whether there is an exception in the compiler based on the comparison results between the reference compilation instructions corresponding to the test cases and the target compilation instructions, you can obtain each functional test case one by one, and use the compiler to compile the functional test case. The target compilation instruction corresponding to the functional test case is obtained, and then the target compilation instruction is compared with the reference compilation instruction corresponding to the functional test case, and based on the comparison result, it is determined whether there is an abnormality in the function of the compiler. Typically, when functionally testing a compiler, multiple functional test cases are used to cover various instruction types, various control operations, and instruction addresses. Therefore, each functional test case can be traversed and the above test steps can be repeated to comprehensively test the functionality of the compiler.

After testing the compiler using all functional tests, if no exceptions are found in the compiler, the stability of the compiler will be further tested. For example, the compiler can be used to compile each stability test case one by one to obtain the target compilation instruction corresponding to the stability test case, and compare the target compilation instruction with the reference compilation instruction corresponding to the stability test case, based on The comparison results determine whether there are any abnormalities in the stability of the compiler.

Generally, each test instruction can be composed of multiple components. In some embodiments, these components can be one or more of the instruction type, control domain, and instruction address. Among them, the instruction address includes the source address of the test instruction and the target address of the test instruction. For example, as shown in Figure 1(b), the test instruction can include three components (components 1-3), which respectively correspond to the instruction type, control domain, and instruction address (source address and target address). Among them, the instruction type can be an addition instruction, a subtraction instruction, a data transfer instruction, an OR operation instruction, an AND operation instruction, etc. The control domain represents the control operations included in the test instruction. Taking the arithmetic processing type as an example, the control operations it contains It can be rounding operations, saturation processing, etc. The source address of the test instruction may be an address where the data to be processed by the instruction is stored, and the target address of the test instruction may be an address where the data processed by the instruction is stored.

When storing test instructions in the instruction database, multiple complete test instructions can be directly stored. When testing the compiler, each test instruction can be directly obtained. However, this storage method will occupy a large amount of storage space. storage, and subsequent expansion or modification of the test instructions in the instruction database is more cumbersome. Therefore, in some embodiments, the data related to the test instructions stored in the instruction database may be one or more pieces of characteristic data corresponding to one or more components that constitute the test instruction and the reference compiled data corresponding to these characteristic data, such as , the component can be an instruction type, and the characteristic data can be an addition instruction, a subtraction instruction, etc. Wherein, the characteristic data corresponding to each component is stored in association with the characteristic data corresponding to the remaining components in the plurality of components. For example, taking the instruction type as an example, the addition instruction has its corresponding control operation, and the data transfer instruction also has its corresponding control operation. Therefore, the associated characteristic data in each component can be associated, which can be used when constructing test instructions. Construct test instructions by associating stored feature data.

For example, assume that the components of the test instruction include instruction type, control domain, and instruction address. The type of test instruction corresponds to multiple pieces of characteristic data. Each piece of characteristic data represents an instruction type. For example, the characteristic data can be addition instructions, subtraction instructions, etc. instructions etc. The control domain of the test instruction can also correspond to multiple pieces of feature data. Each piece of feature data represents a control operation. For example, the feature data can be rounding processing, saturation processing, etc. The instruction address includes the source address of the test instruction and the target address of the test instruction. The instruction address can also correspond to multiple pieces of feature data, and each feature data represents a set of addresses. Therefore, an instruction type list including multiple instruction types and reference compiled data corresponding to each instruction type; a control domain list including multiple control operations, and Reference compilation data corresponding to each control operation; and an instruction address list, which includes multiple sets of addresses, and reference compilation data corresponding to each set of addresses, where each set of addresses includes a source address and a target address. In some embodiments, for each instruction type, such as an addition instruction, the type of control operations involved is certain, for example, it can only be control operations related to addition processing. And after the instruction type and control operation are determined, the type of data it can process is also certain, that is, the data it can process is data within a certain address range, and its corresponding instruction address can also be determined. Therefore, when storing the above data in the instruction database, for each instruction type, the instruction type can be stored in association with its various control operations, source addresses, and target addresses, so that when constructing instructions based on these characteristic data, it can be based on The correlation relationship of the characteristic data selects the associated instruction type, control operation, source address and target address, and constructs the correct test instruction.

In addition, by using the above method to store test instructions in the instruction database, compared with directly storing complete test instructions, the amount of data can be reduced, and it is also possible to add, delete, or modify test instructions stored in the instruction data later when it is necessary to add, delete, or modify them. More convenient.

In some embodiments, the instruction database may store characteristic data corresponding to each component of the test instruction and reference compilation data corresponding to the characteristic data. When constructing functional test cases and reference compilation instructions corresponding to functional test cases based on test instruction-related data stored in the instruction database, multiple components can be traversed. Each piece of characteristic data corresponding to each component. For any piece of characteristic data corresponding to any component, this piece of characteristic data is referred to as the target characteristic data in the following. The target characteristic data can be determined from the remaining components among the multiple components. Associate the stored associated feature data, then generate a test instruction based on the target feature data and the associated feature data, and obtain the reference compilation corresponding to the test instruction based on the reference compilation data corresponding to the target feature data and the reference compilation data corresponding to the associated feature data. instruction. After generating test instructions, each generated test instruction can be used as a functional test case, and the reference compilation instructions corresponding to each test instruction can be used as the reference compilation instructions corresponding to the functional test case, so that the compilation instructions can be obtained. Multiple functional test cases to test the functionality of the processor.

For example, for a piece of target characteristic data in an instruction type (for example, an "addition instruction"), multiple "controls" stored in association with the "addition instruction" can be determined from the two components of the control domain and instruction address stored in the instruction database. Operation" and multiple "addresses", and then a "control operation" and a set of "addresses" can be randomly selected from the multiple "control operations" and multiple "addresses", and the target characteristic data of the above-mentioned "addition instruction" Build a test directive.

By traversing each feature data of each component in the instruction database, multiple test instructions covering various instruction types, control operations and instruction addresses can be constructed, so that the functional test cases constructed are more comprehensive and more accurate. test results. In some embodiments, it is not necessary to traverse each feature data of each component in the instruction database, but only obtain a specified number of test cases according to requirements, which is not limited in this disclosure.

In some embodiments, the instruction database may store characteristic data corresponding to each component of the test instruction and reference compiled data corresponding to the characteristic data. Stability test cases and stability test cases are constructed based on the test instruction-related data stored in the instruction database. When the reference compilation instruction corresponding to the test case is used, for any piece of target feature data in any component, the associated feature data stored in association with the target feature data can be determined from the remaining components in multiple components, and then based on The target characteristic data and associated characteristic data generate test instructions. And based on the reference compilation data corresponding to the target characteristic data and the reference compilation data corresponding to the associated characteristic data, the reference compilation instruction corresponding to the test instruction is obtained. Then the multiple generated test instructions can be used to splice to obtain a stability test case, and the reference compilation instructions corresponding to the multiple test instructions can be used to splice to obtain the reference compilation instructions corresponding to the stability test case.

As a result, multiple test instructions can be randomly constructed based on the interrelated feature data stored in the instruction database, and these test instructions can be used to splice together to obtain stability test cases with a large amount of data.

In some embodiments, since various components of the test instructions are interrelated, for example, the instruction type will affect the selection of the control operation, and the instruction type and the control operation will further affect the selection of the instruction address. That is, the relationship between these multiple components is interrelated. Therefore, the instructions corresponding to the test instructions are stored in the instruction database. When it comes to related data, these data can be stored in a tree structure. For any tree structure, the number of layers can be consistent with the number of components of the test instruction. For example, if the test instruction includes three components: instruction type, control domain, and instruction address, then the tree structure can include three layers, that is, tree structure. The structure includes a root node, child nodes branched from the root node, and leaf nodes branched from each child node. The root node of the tree structure can be used to store any target feature data in the instruction type and the corresponding target feature data. Reference compiled data. The child nodes of the tree structure to the leaf nodes of the tree structure are used to sequentially store the first associated feature data associated with the target feature data in the control domain and the reference compiled data corresponding to the first associated feature data, as well as the instruction address and The second associated feature data associated with the target feature data and the first associated feature data and the reference compiled data corresponding to the second associated feature data.

As shown in Figure 1(c), it is a schematic diagram of using a tree structure to store test instructions. The root node of the tree structure can be used to store a feature data in the instruction type and its corresponding reference compiled data. For example, the characteristic data may be an "addition instruction", and the reference compiled data may be the machine code corresponding to the "addition instruction". Among them, multiple child nodes can be branched from the root node. Each child node is used to store the control operation associated with the "addition instruction" and the reference compiled data corresponding to the control operation. For example, there can be three child nodes, each child node Store "rounding mode" and its machine code, "immediate mode" and its machine code, "register mode" and its machine code, etc. separately, and then for each child node, multiple leaf nodes can be branched out, each leaf node It is used to store the instruction address and its machine code associated with the characteristic data stored in the root node and the child nodes branching out of the leaf node, for example, multiple sets of instructions associated with "addition instructions" and "rounding mode" Address, "source address 1, destination address 1", "source address 2, destination address 2", "source address 3, destination address 3", etc. In some embodiments, if the data related to the test instructions is stored in a tree structure, when building the test case, the feature data stored in each node in any path from the root node to the leaf node in the tree structure can be obtained , and the reference compilation data corresponding to each of these feature data, and then splicing the feature data stored in each node to obtain a test instruction, splicing the reference compilation data corresponding to the feature data stored in each node to obtain the reference compilation instruction corresponding to the test instruction, and then A test case may be constructed based on the obtained test instruction and a reference compilation instruction corresponding to the test case may be constructed based on the reference compilation instruction corresponding to the test instruction. For example, each obtained test instruction can be used as a test case, or multiple test instructions can be spliced into one test case.

For example, as shown in Figure 1(d), the "root node-child node-leaf node" selected in the black box is a path from the root node to the leaf node in the tree structure, and the information stored in each node in the path can be obtained. The characteristic data, namely "addition instruction", "rounding mode", "source address 1, target address 1", is spliced to obtain a test instruction, and the machine code corresponding to these characteristic data is obtained and spliced to obtain the reference corresponding to the test instruction. Compilation directives.

In some embodiments, after determining that there is an exception in the compiler based on the comparison results between the reference compilation instructions corresponding to the test cases and the target compilation instructions, a test report can be further output. In order to facilitate the developer to locate the exception, the test report It can include one or more of the following information: the type of exception, the test instruction in which the exception occurs, and the component where the exception occurs. The type of exception includes functional abnormality or stability abnormality. The components can include instruction type, control domain, and instruction. address.

In order to further introduce the compiler testing method provided by the present disclosure, it will be explained below in conjunction with a specific embodiment.

During the running of the program, the corresponding high-level language needs to be compiled into assembly language, and then the corresponding assembly language is converted into machine code through the assembler, and then the machine code is given to the underlying chip for execution. If the compiler's compilation result is wrong, it will cause the program to run abnormally. The compilation result of the assembler is crucial to the operation of the entire program. Therefore, this embodiment provides an assembler testing method.

For the assembler, the instruction types involved are very complex. In order to achieve more accurate and comprehensive testing of the assembler, it is possible to achieve automation. An instruction database can be set up in advance. The architecture of the instruction database is shown in Figure 2. Since assembly instructions can usually be divided into three components: instruction type, control domain, and instruction address, the instruction type can correspond to an instruction type list. The list includes various instruction types of assembly instructions, and the machine codes corresponding to various instruction types. In the same way, the control domain can also correspond to a control domain list. This list includes various control operations of assembly instructions, as well as various control operations. Corresponding to the machine code, the instruction address can also correspond to an address list, which includes the source address and destination address of the data involved in the instruction, as well as the machine code corresponding to each address. Each piece of feature data in each of the above three lists is stored in association with one or more pieces of feature data in the other lists. For example, the instruction type "addition instruction" in Figure 2 is associated with the control operations "rounding" and "saturation processing", and is associated with the "source address 1, target address 1" and "source address" in the instruction address. Address 2, target address 2" and "source address 3, target address 3" are stored in association. Therefore, when building a test instruction, you can randomly select a control operation "saturation processing" and a set of instruction addresses "source address 1, target address 1" Build a test instruction.

When testing the assembler, you can traverse each feature data in this component for any of the three components: instruction type, instruction control domain, and instruction address. For each piece of characteristic data, you can select from the remaining two components. Determine the associated feature data of the feature data from the feature data of each component, and then select one piece of associated feature data corresponding to the other two components to construct a test instruction. Then you can traverse each feature data of each component and repeat the above steps to get multiple test instructions and the reference machine code corresponding to each test instruction.

Then you can use the assembler to be tested to compile each test instruction one by one to obtain the target compilation instruction. And compare the reference compilation instructions and the target compilation instructions, and determine whether there is an abnormality in the function of the assembler based on the comparison results.

If there is no abnormality in the assembler function, a test document can be obtained by splicing multiple test instructions, wherein the number of test instructions in the test document is greater than the preset number. This test document can then be used to test the stability of the assembler and determine whether there are abnormalities in the assembler.

If an exception occurs in the assembler, an exception test report can be output. The report can indicate the following information: what type of exception is the exception, the test instruction in which the exception occurred, and which component of the test instruction caused the exception, which facilitates developers to locate the exception. and solved.

Among them, it is not difficult to understand that the solutions described in the above embodiments can be combined if there is no conflict, and are not listed one by one in the embodiments of this disclosure.

Usually when testing a compiler, a large number of test instructions are needed. Test instructions generally consist of multiple components. For example, usually test instructions can be composed of three parts: instruction type, control domain, and instruction address. During the storage process, in order to reduce the amount of stored data and facilitate changes to the stored test instructions, the characteristic data corresponding to each component of the test instruction and the reference compiled data corresponding to the characteristic data can be stored separately, and different components can be stored in The associated feature data is stored in an associated manner, so that when generating a test instruction, the associated feature data in each component can be obtained, spliced together to obtain the test instruction, and the reference compilation instruction corresponding to the test instruction. In this way, it is also possible to generate more Comprehensive, covering various types of test instructions.

Since the various components of the test instructions are interrelated, for example, the instruction type will affect the selection of the control operation, and the instruction type and control operation will further affect the selection of the instruction address. In order to avoid splicing unrelated components and obtaining wrong test instructions when generating test instructions based on each stored component, the following provides a storage method for test instructions, as shown in Figure 3, which can include Following steps:

Step 1: Obtain the data related to the test instruction to be stored. The data related to the test instruction includes multiple feature data corresponding to each component of the multiple components of the test instruction, and the reference compilation corresponding to each feature data. Data; the plurality of components include an instruction type, a control domain and an instruction address, and the control domain represents the control operations included in the test instruction;

For example, as shown in Figure 3, the test instruction includes three components: instruction type, control domain, and instruction address. Each component can include multiple pieces of feature data, and each piece of feature data has corresponding reference compiled data. For example, instruction types may include addition instructions, subtraction instructions, data transfer instructions, and other types.

Step 2: Store the data related to the test instructions through a tree structure, where the root node of any tree structure is used to store a piece of target feature data in the instruction type and the reference compiled data corresponding to the target feature data. , the child nodes of the tree structure to the leaf nodes of the tree structure are used to store the control domain in sequence The first associated feature data associated with the target feature data and the reference compiled data corresponding to the first associated feature data, and the third associated feature data associated with the target feature data and the first associated feature data in the instruction address Two associated feature data and reference compiled data corresponding to the second associated feature data.

For example, as shown in Figure 3, the tree structure can include three levels: root node - child node - leaf node. Any target feature data A1 (for example, addition instruction) in the instruction type and its reference compiled data can be stored in a In the root node of the tree structure, then determine the first associated feature data B1, B2, B3 in the control domain that are associated with the target feature data A1 (for example, rounding, saturation processing, immediate mode, etc.), and their respective Reference compiled data is then stored in each child node branched from the root node. Then for each leaf node that branches out from the child node, the second association associated with the target feature data A1 stored in the root node and the first feature data stored in the child node that branches out from the leaf node can be determined from the instruction address. Feature data, such as second associated feature data C1, C2, C3 (for example, three sets of addresses) associated with target feature data A1, first associated feature data B1, and their respective reference compiled data are then stored in the slave In each leaf node branched from the child node where the first associated feature data B1 is located. That is, the characteristic data stored in each node in any path from the root node to the leaf node in the tree structure can constitute a test instruction.

For specific implementation details of the test instruction storage method, please refer to the description of the embodiment in the above compilation method, and will not be repeated here.

The present disclosure also provides a storage structure of test instructions, where the storage structure includes: a tree structure;

The tree structure is used to store data related to test instructions; the data related to test instructions includes multiple pieces of characteristic data corresponding to each component of the multiple components of the test instruction, and a reference corresponding to each piece of characteristic data. Compile data; the plurality of components include an instruction type, a control domain and an instruction address, and the control domain represents the control operations included in the test instruction;

Wherein, the root node of the tree structure is used to store a piece of target feature data in the instruction type and the reference compilation data corresponding to the target feature data, and the child nodes of the tree structure to the leaves of the tree structure The node is used to sequentially store the first associated feature data associated with the target feature data in the control domain and the reference compiled data corresponding to the first associated feature data, as well as the target feature data and the reference compilation data corresponding to the first associated feature data in the instruction address. The second associated feature data associated with the first associated feature data and the reference compiled data corresponding to the second associated feature data.

For specific implementation details of the test instruction storage structure, reference can be made to the description of the embodiment in the above compilation method, and the details will not be repeated here.

In addition, the present disclosure also provides a test case generation method for generating test cases for testing the compiler, in which the test instructions for constructing the test cases can be stored using the above-mentioned storage structure. Specific generation of test cases The process can include the following steps:

Obtain the feature data stored in each node in any path from the root node to the leaf node in the tree structure, and the reference compiled data corresponding to the feature data;

Splice the characteristic data stored in each node to obtain a test instruction, and splice the reference compilation data corresponding to each characteristic data to obtain the reference compilation instruction corresponding to the test instruction;

Generate test cases based on the obtained test instructions, and generate reference compilation instructions corresponding to the test cases based on reference compilation instructions corresponding to the test instructions.

For the specific process of generating test cases, reference may be made to the descriptions in the above embodiments of the compilation method, which will not be described again here.

In this way, test cases for testing the compiler can be automatically generated, errors in the test instructions in the test cases can be avoided, and the efficiency and accuracy of testing the compiler can be improved. Correspondingly, the embodiment of the present disclosure also provides a compiler testing device, as shown in Figure 4, the device includes:

The construction module 41 is used to construct test cases based on the data related to the test instructions stored in the instruction database, and the reference compilation instructions corresponding to the test cases;

The compilation module 42 is used to compile the test case using the compiler to be tested to obtain the target compilation instructions;

The determination module 43 is configured to determine whether there is an exception in the compiler based on the comparison result between the reference compilation instruction and the target compilation instruction. For the specific steps of the above device to execute the compiler testing method, reference can be made to the description in the above method embodiments, which will not be described again here.

Correspondingly, embodiments of the present disclosure also provide a storage device for test instructions, where the device includes:

An acquisition module, used to acquire data related to the test instructions to be stored. The data related to the test instructions include characteristic data corresponding to each component of the multiple components of the test instruction and reference compiled data corresponding to the characteristic data. ; The plurality of components include an instruction type, a control domain and an instruction address, and the control domain represents the control operations included in the test instruction;

A storage module configured to store the data related to the test instructions through a tree structure, where the root node of any tree structure is used to store a piece of target feature data in the instruction type and a reference corresponding to the target feature data. Compiled data, the child nodes of the tree structure are used to store the first associated feature data associated with the target feature data in the control domain and the reference compiled data corresponding to the first associated feature data, the tree structure The leaf node is used to store the second associated feature data associated with the target feature data and the first associated feature data and the reference compiled data corresponding to the second associated feature data in the instruction address.

For the specific steps of the method for storing test instructions executed by the above device, reference may be made to the description in the above method embodiments, which will not be described again here.

Correspondingly, embodiments of the present disclosure also provide a device for generating test cases. The test cases are used to test the compiler. The test instructions for constructing the test cases are stored through the storage method described in the above embodiments. The device include:

An acquisition module, used to acquire the characteristic data stored in each node in any path from the root node to the leaf node in the tree structure, and the reference compiled data corresponding to the characteristic data;

The splicing module is used to splice the characteristic data stored in each node to obtain a test instruction, and splice the reference compilation data corresponding to each characteristic data to obtain the reference compilation instruction corresponding to the test instruction;

A generation module, configured to generate test cases based on the obtained test instructions, and generate reference compilation instructions corresponding to the test cases based on reference compilation instructions corresponding to the test instructions.

For the specific steps of the method for generating test cases performed by the above device, reference may be made to the description in the above method embodiments, which will not be described again here.

Further, an embodiment of the present disclosure also provides an electronic device. As shown in Figure 5, the electronic device includes a processor 51, a memory 52, and computer instructions stored in the memory 52 for execution by the processor 51. When the processor 51 executes the computer instructions, the method described in any one of the above embodiments is implemented.

An embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the foregoing embodiments is implemented.

Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of the present disclosure are essentially or the parts that contribute to the existing technology can be embodied in the form of software products. The computer software products can be stored in storage media, such as ROM/RAM, A magnetic disk, optical disk, etc., includes a number of instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present disclosure.

The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, or a game controller. desktop, tablet, wearable device, or a combination of any of these devices.

Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment. The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated. When implementing the embodiments of the present disclosure, the functions of each module may be integrated into the same device. or implemented in multiple software and/or hardware. Some or all of the modules can also be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

The above are only specific implementations of the embodiments of the present disclosure. It should be noted that those of ordinary skill in the art can make several improvements and modifications without departing from the principles of the embodiments of the present disclosure. Improvements and modifications should also be considered as the protection scope of the embodiments of the present disclosure.

Claims (16)

1. A compiler testing method, characterized in that the method includes:

   Construct test cases and reference compilation instructions corresponding to the test cases based on the data related to the test instructions stored in the instruction database;

   Compile the test case using the compiler to be tested to obtain the target compilation instructions;

   Based on the comparison result between the reference compilation instruction and the target compilation instruction, it is determined whether there is an exception in the compiler.
2. The method of claim 1, wherein the test cases include functional test cases and/or stability test cases, and the functional test cases include a single test instruction for testing the function of the compiler. ; The stability test case is obtained by splicing multiple test instructions and is used to test the stability of the compiler.
3. The method according to claim 2, characterized in that the stability test case meets one or more of the following conditions:

   The time it takes for the compiler to compile the stability test case is longer than the preset time;

   The number of test instructions included in the stability test case is greater than the preset number;

   The data volume of the stability test case is greater than the preset data volume.
4. The method according to claim 2 or 3, characterized in that the test cases include functional test cases and stability test cases, based on the comparison results of the reference compilation instructions corresponding to the test cases and the target compilation instructions. , determine whether there is an exception in the compiler, including:

   Based on the comparison results of the reference compilation instructions corresponding to the functional test case and the target compilation instructions corresponding to the functional test case, determine whether there is an abnormality in the function of the compiler;

   When there is no abnormality in the function of the compiler, the stability of the compiler is determined based on the comparison results of the reference compilation instructions corresponding to the stability test case and the target compilation instructions corresponding to the stability test case. Is there any exception?
5. The method according to any one of claims 1 to 4, characterized in that the test instruction includes multiple components, and the data related to the test instruction includes one component among the multiple components, The characteristic data corresponding to the component and the reference compiled data corresponding to the characteristic data are stored in association with the characteristic data corresponding to the remaining components among the plurality of components except the component.
6. The method according to claim 5, characterized in that the components include one or more of the following: instruction type, control domain, and instruction address, wherein the control domain is used to represent the control included in the test instruction. operate.
7. The method according to claim 5 or 6, characterized in that the test cases include a plurality of functional test cases for testing the compiler function, and the test is constructed based on the data related to the test instructions stored in the instruction database. Use cases and reference compilation instructions corresponding to the test cases include:

   For one component among the plurality of components, for a piece of target characteristic data corresponding to the component, determine the association with the target characteristic data from the remaining components among the plurality of components except the component. Stored associated feature data;

   Generate a test instruction based on the target characteristic data and the associated characteristic data, and obtain the reference compilation instruction corresponding to the test instruction based on the reference compilation data corresponding to the target characteristic data and the reference compilation data corresponding to the associated characteristic data;

   The test instruction is used as a functional test case, and the reference compilation instruction corresponding to the test instruction is used as the reference compilation instruction corresponding to the functional test case.
8. The method according to claim 5 or 6, characterized in that the test case includes a stability test case for testing the stability of the compiler, and the test case is constructed based on the data related to the test instructions stored in the instruction database. And the reference compilation instructions corresponding to the test cases, including:

   For one component among the plurality of components, for a piece of target characteristic data in the component, determine the association with the target characteristic data from the remaining components among the plurality of components except the component. Stored associated feature data;

   Generate a test instruction based on the target characteristic data and the associated characteristic data, and obtain the reference compilation instruction corresponding to the test instruction based on the reference compilation data corresponding to the target characteristic data and the reference compilation data corresponding to the associated characteristic data;

   The stability test case is obtained by splicing the plurality of generated test instructions, and the reference compilation instructions corresponding to the stability test case are obtained by splicing the reference compilation instructions corresponding to the plurality of test instructions.
9. The method according to any one of claims 6 to 8, characterized in that the data related to the test instruction is stored in a tree structure, and the root node of the tree structure is used to store a target in the instruction type. Feature data and reference compiled data corresponding to the target feature data. The child nodes of the tree structure are used to store the first associated feature data associated with the target feature data in the control domain and the corresponding first associated feature data. The reference compiled data, the leaf nodes of the tree structure are used to store the second associated feature data associated with the target feature data and the first associated feature data in the instruction address and the corresponding second associated feature data Reference compiled data.
10. The method according to claim 9, characterized in that said constructing a test case based on the data related to the test instruction stored in the instruction database and the reference compilation instruction corresponding to the test case includes:

    Obtain the feature data stored in each node in any path from the root node to the leaf node in the tree structure and the reference compiled data corresponding to the feature data;

    Splice the characteristic data stored in each node to obtain a test instruction, and splice the reference compilation data corresponding to the characteristic data stored in each node to obtain the reference compilation instruction corresponding to the test instruction;

    Construct a test case based on the obtained test instruction, and construct a reference compilation instruction corresponding to the test case based on the reference compilation instruction corresponding to the test instruction.
11. The method according to any one of claims 1 to 10, characterized in that, after determining that there is an abnormality in the compiler based on a comparison result between the reference compilation instruction corresponding to the test case and the target compilation instruction, The above methods also include:

    Output a test report, which includes one or more of the following information: the type of exception, the test instruction in which the exception occurred, and the component in the test instruction in which the exception occurred;

    The type of abnormality includes functional abnormality or stability abnormality, and the components include instruction type, control domain, and instruction address.
12. A storage structure of test instructions, characterized in that the storage structure includes: a tree structure;

    The tree structure is used to store data related to test instructions; the data related to test instructions includes characteristic data corresponding to each component of the multiple components of the test instruction and reference compiled data corresponding to the characteristic data; The plurality of components include an instruction type, a control domain and an instruction address, and the control domain represents the control operations included in the test instruction;

    Wherein, the root node of the tree structure is used to store a piece of target feature data in the instruction type and the reference compiled data corresponding to the target feature data, and the child nodes of the tree structure are used to store the control domain. The first associated feature data associated with the target feature data and the reference compiled data corresponding to the first associated feature data. The leaf nodes of the tree structure are used to store the target feature data and the reference data in the instruction address. The second associated feature data associated with the first associated feature data and the reference compiled data corresponding to the second associated feature data.
13. A method for generating test cases, characterized in that the test cases are used to test a compiler, and the test instructions for constructing the test cases are stored through the storage structure as claimed in claim 12, and the method includes:

    Obtain the characteristic data stored in each node in any path from the root node to the leaf node in the tree structure, and the reference compiled data corresponding to the characteristic data;

    Splice the characteristic data stored in each node to obtain a test instruction, and splice the reference compilation data corresponding to each characteristic data to obtain the reference compilation instruction corresponding to the test instruction;

    Generate test cases based on the obtained test instructions, and generate reference compilation instructions corresponding to the test cases based on reference compilation instructions corresponding to the test instructions.
14. A compiler testing device, characterized in that the device includes:

    A building module for constructing test cases and reference compilation instructions corresponding to the test cases based on the data related to the test instructions stored in the instruction database;

    A compilation module, used to compile the test case using the compiler to be tested and obtain the target compilation instructions;

    A determination module, configured to determine whether there is an exception in the compiler based on a comparison result between the reference compilation instruction and the target compilation instruction.
15. An electronic device, characterized in that the electronic device includes a processor, a memory, and computer instructions stored in the memory and executable by the processor. When the processor executes the computer instructions, the implementation as claimed in claim 1 -The method described in any one of 11 or 13.
16. A computer-readable storage medium, characterized in that computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed, the method according to any one of claims 1-11 or 13 is implemented.