CN113962068A

CN113962068A - Verification strategy determination method and device, electronic equipment and storage medium

Info

Publication number: CN113962068A
Application number: CN202111177298.XA
Authority: CN
Inventors: 索健; 王正
Original assignee: Beijing Aixin Technology Co ltd
Current assignee: Beijing Aixin Technology Co ltd
Priority date: 2021-10-09
Filing date: 2021-10-09
Publication date: 2022-01-21

Abstract

The application provides a method and a device for determining a verification policy, an electronic device and a storage medium, wherein the method comprises the following steps: obtaining a candidate verification strategy, wherein the candidate verification strategy comprises each item of to-be-verified characteristic of a simulated to-be-verified code; running the simulated to-be-verified code based on the candidate verification strategy, verifying each to-be-verified characteristic in the candidate verification strategy, and generating a verification coverage report; reversely marking the candidate verification strategies according to the verification coverage report to obtain the verified coverage rate of each to-be-verified characteristic; generating a target verification policy based on the verified coverage of each to-be-verified characteristic. In the method and the device, the verification coverage report is reversely marked to the candidate verification strategy, so that the verification result is more visually presented, the optimization and adjustment of the candidate verification strategy are effectively realized, the adjustment efficiency of the candidate verification strategy is improved, and the quality of the target verification strategy is optimized.

Description

Verification strategy determination method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for determining a verification policy, an electronic device, and a storage medium.

Background

With the development of chip technology, the design of chip bus is more complex, and in implementation, the design of chip bus plays an important role in the implementation of chip function, so that the determination of verification strategy has a certain influence on the implementation of chip function.

In the related art, the determination period of the chip verification strategy is long, and the efficiency is low, so how to improve the efficiency of bus verification is a problem to be solved at present.

Disclosure of Invention

The object of the present application is to solve at least to some extent one of the technical problems in the above-mentioned technology.

The application provides a method for determining a verification policy in a first aspect, including: obtaining a candidate verification strategy, wherein the candidate verification strategy comprises each item of to-be-verified characteristic of a simulated to-be-verified code; running the simulated to-be-verified code based on the candidate verification strategy, verifying each to-be-verified characteristic in the candidate verification strategy, and generating a verification coverage report; reversely marking the candidate verification strategies according to the verification coverage report to obtain the verified coverage rate of each to-be-verified characteristic; generating a target verification policy based on the verified coverage of each to-be-verified characteristic.

The method for determining the verification policy provided by the first aspect of the present application further has the following technical features, including:

according to an embodiment of the present application, the reversely marking the candidate verification policies according to the verification coverage report to obtain the verified coverage of each feature to be verified includes: acquiring first identification information of the verified performance and the corresponding verified performance under each to-be-verified characteristic from the verification coverage report; acquiring all second identification information of the to-be-verified performance under each to-be-verified characteristic from the candidate verification strategy; matching the first identification information with the second identification information, and reversely marking the verified performance to each item of characteristic to be verified; obtaining unverified performance of each to-be-verified characteristic, and generating the verified coverage rate of each to-be-verified characteristic based on the verified performance and the unverified performance.

According to an embodiment of the present application, the generating a target verification policy based on the verified coverage of each to-be-verified characteristic includes: determining whether to adjust the candidate verification policy based on the verified coverage of each feature to be verified; in response to determining that no adjustment is required, determining the candidate verification policy as the target verification policy; and responding to the judgment that the adjustment is needed, adjusting the candidate verification strategies according to the unverified performance of each to-be-verified characteristic, and generating the target verification strategy when the to-be-verified performance included by each to-be-verified characteristic in the adjusted candidate verification strategies is verified.

According to an embodiment of the present application, the determining whether to adjust the candidate verification policy based on the verified coverage of each to-be-verified characteristic includes: in response to the to-be-verified performance under each to-be-verified characteristic being verified, determining that the candidate verification policy does not need to be adjusted; and in response to the to-be-verified performance under each to-be-verified characteristic is not verified, determining that the candidate verification strategy needs to be adjusted.

According to an embodiment of the present application, the obtaining of the candidate verification policy includes: constructing a test case, and acquiring each item of to-be-verified characteristic of the simulated to-be-verified code under the environment of the test case; and generating the candidate verification strategy based on each characteristic to be verified.

According to an embodiment of the present application, the running the simulated to-be-verified code based on the candidate verification policy, verifying each to-be-verified characteristic in the candidate verification policy, and generating a verification coverage report includes: running the simulated to-be-verified code based on each to-be-verified characteristic in the candidate verification strategy, and verifying the performance of each to-be-verified characteristic according to a running result to obtain the verified performance of each to-be-verified characteristic; generating the verification coverage report of the candidate verification strategy based on the verified performance under each to-be-verified characteristic and the verification result of the verified performance.

A second aspect of the present application provides a device for determining a verification policy, including: the verification device comprises an acquisition module, a verification module and a verification module, wherein the acquisition module is used for acquiring a candidate verification strategy, and the candidate verification strategy comprises each item of to-be-verified characteristic of a simulated to-be-verified code; the verification module is used for running the simulated to-be-verified codes based on the candidate verification strategies, verifying each to-be-verified characteristic in the candidate verification strategies and generating a verification coverage report; the marking module is used for reversely marking the candidate verification strategies according to the verification coverage reports so as to obtain the verified coverage rate of each to-be-verified characteristic; and the generating module is used for generating a target verification strategy based on the verified coverage rate of each to-be-verified characteristic.

The device for determining the verification policy according to the second aspect of the present application further includes the following technical features:

according to an embodiment of the present application, the marking module is further configured to: acquiring first identification information of the verified performance and the corresponding verified performance under each to-be-verified characteristic from the verification coverage report; acquiring all second identification information of the to-be-verified performance under each to-be-verified characteristic from the candidate verification strategy; matching the first identification information with the second identification information, and reversely marking the verified performance to each item of characteristic to be verified; obtaining unverified performance of each to-be-verified characteristic, and generating the verified coverage rate of each to-be-verified characteristic based on the verified performance and the unverified performance.

According to an embodiment of the present application, the generating module is further configured to: determining whether to adjust the candidate verification policy based on the verified coverage of each feature to be verified; in response to determining that no adjustment is required, determining the candidate verification policy as the target verification policy; and responding to the judgment that the adjustment is needed, adjusting the candidate verification strategies according to the unverified performance of each to-be-verified characteristic, and generating the target verification strategy when the to-be-verified performance included by each to-be-verified characteristic in the adjusted candidate verification strategies is verified.

According to an embodiment of the present application, the generating module is further configured to: in response to the to-be-verified performance under each to-be-verified characteristic being verified, determining that the candidate verification policy does not need to be adjusted; and in response to the to-be-verified performance under each to-be-verified characteristic is not verified, determining that the candidate verification strategy needs to be adjusted.

According to an embodiment of the present application, the obtaining module is further configured to: constructing a test case, and acquiring each item of to-be-verified characteristic of the simulated to-be-verified code under the environment of the test case; and generating the candidate verification strategy based on each characteristic to be verified.

According to an embodiment of the present application, the verification module is further configured to: running the simulated to-be-verified code based on each to-be-verified characteristic in the candidate verification strategy, and verifying the performance of each to-be-verified characteristic according to a running result to obtain the verified performance of each to-be-verified characteristic; generating the verification coverage report of the candidate verification strategy based on the verified performance under each to-be-verified characteristic and the verification result of the verified performance.

An embodiment of a third aspect of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the method of determining a verification policy as provided in the first aspect of the present application.

A fourth aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method for determining a verification policy provided in the first aspect of the present application.

An embodiment of a fifth aspect of the present application provides a computer program product, which when executed by an instruction processor of the computer program product performs the method for determining a verification policy provided in the first aspect of the present application.

According to the verification strategy determining method and device, the code to be verified is simulated based on the candidate verification strategy, and the verification coverage report of the candidate verification strategy is obtained. And reversely marking the verification coverage report into the candidate verification strategies, further acquiring the verified coverage rate corresponding to each to-be-verified characteristic in the candidate verification strategies, giving the verified coverage rate to adjust the candidate verification strategies until the set standard is met, and further generating the target verification strategy. In the application, the verification coverage report is reversely marked to the candidate verification strategy, so that the verification result of the candidate verification strategy can be more visually presented, the optimization and adjustment of the candidate verification strategy can be more effectively realized, the adjustment efficiency of the candidate verification strategy is effectively improved, and the quality of the target verification strategy is optimized

Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a schematic flowchart illustrating a method for determining a verification policy according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating a method for determining a verification policy according to another embodiment of the present application;

FIG. 3 is a flowchart illustrating a method for determining a verification policy according to another embodiment of the present application;

FIG. 4 is a flowchart illustrating a method for determining a verification policy according to another embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for determining a verification policy according to another embodiment of the present application;

fig. 6 is a schematic structural diagram of an apparatus for determining a verification policy according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a verification policy determining apparatus according to another embodiment of the present application;

fig. 8 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

A method, an apparatus, an electronic device, and a storage medium for determining an authentication policy according to embodiments of the present application are described below with reference to the accompanying drawings.

Fig. 1 is a schematic flowchart of a method for determining a verification policy according to an embodiment of the present application, and as shown in fig. 1, the method includes:

s101, obtaining a candidate verification strategy, wherein the candidate verification strategy comprises each item of to-be-verified characteristic of the simulated to-be-verified code.

In the implementation, in order to implement the performance verification of the chip more systematically and completely, before entering the verification link, a corresponding verification strategy needs to be formulated for the to-be-verified characteristics of the chip.

In order to ensure that the to-be-verified performance of the chip can realize more systematic and complete verification, a corresponding verification environment, such as a Universal Verification Methodology (UVM) verification environment, can be established for the chip. And then, based on the built verification environment and the to-be-verified characteristics of the chip which can be realized in the verification environment, a verification strategy of the chip is formulated.

In the embodiment of the application, each parameter of the chip to be verified can be expressed by using a code, and the code is determined as a simulated code to be verified corresponding to the chip to be verified.

In the implementation, the simulated code to be verified has various characteristics, and in order to realize the sufficient verification of the simulated code to be verified, so that the verification effect of the chip characteristics is ensured, a corresponding verification strategy can be formulated based on the various characteristics to be verified.

Furthermore, the verification strategy comprises each to-be-verified characteristic, and the to-be-verified performance of a plurality of simulated to-be-verified codes exists under each to-be-verified characteristic. It can be understood that, in a scenario corresponding to each feature to be verified, all functions simulating a code to be verified can be implemented.

Alternatively, the characteristics to be verified of the code to be verified are simulated, and can be extracted from a design document containing all source data of the chip and the verification environment.

Because the chip has a plurality of characteristics to be verified and different characteristics are realized based on different modules, the characteristics to be verified can be extracted based on different modules on the chip. It can be understood that, according to the characteristics of the chip that can be realized by a certain module on the chip that needs to be verified at present, the relevant expression of the corresponding characteristic to be verified is selected from the design document.

Further, in order to achieve a better chip verification effect, the established verification strategy can be optimized and adjusted, so that a candidate verification strategy can be established based on the to-be-verified characteristic of the simulated to-be-verified code.

Wherein the relevant representation of the characteristic to be verified extracted from the design document may include all of the following information:

partial simulation of the to-be-verified code of the to-be-verified characteristic can be realized;

the part simulates the state which can be realized by the code to be verified, wherein the state can comprise the operational logic, the working mode, the state change and the like of the functional module;

the part simulates the current configuration of the code to be verified and the corresponding configuration under different states;

the part simulates the input rule and the output rule of the code to be verified;

the part simulates the influence of the output of the code to be verified after running on the state of the current code to be verified.

Further, the relevant expression of the feature to be verified may be stored in a relevant position of the design document, and therefore, information extraction needs to be performed from different levels of the design document, such as from the whole design document, or from each chapter in the design document, or from each section in each chapter, or from each paragraph in each section, or from each sentence in each paragraph, and so on.

In the implementation, for the characteristic to be verified in the special scene, there is a possibility that effective information extraction cannot be realized from the design document, so that the content in the design document can be separated, and the individual information editing can be performed according to the characteristic to be verified in the special scene.

It should be noted that, in the design document, there are two relations of orthogonality and non-orthogonality in the to-be-verified characteristics of the simulated to-be-verified code. In the same candidate verification strategy, only the characteristics to be verified under the orthogonal relationship exist. The orthogonal relationship can be understood as a characteristic to be verified with an association relationship, and the non-orthogonal relationship can be understood as a characteristic to be verified without an association relationship.

Further, based on effective extraction of expression information of the characteristics to be verified, a corresponding candidate verification strategy is generated.

S102, operating the simulation to-be-verified code based on the candidate verification strategy, verifying each to-be-verified characteristic in the candidate verification strategy, and generating a verification coverage report.

In order to realize the adjustment optimization of the candidate verification strategies, the simulated to-be-verified codes can be run based on the candidate verification strategies, and each to-be-verified characteristic in the candidate verification strategies can be verified.

And based on the input corresponding to each characteristic to be verified and the operating mode operating code, and according to the operating result of the simulated code to be verified, realizing the verification of each characteristic to be verified.

Optionally, a plurality of functions to be verified may exist under each to-be-verified characteristic, and the code may be run by simulating the to-be-verified code according to different functions to be verified, and corresponding outputs may be generated.

Further, after the verification of the to-be-verified characteristics of all items is completed, a verification coverage report of the candidate verification strategy is generated based on the operation result of the simulated to-be-verified code under each to-be-verified characteristic.

Alternatively, the verification coverage report may be stored based on a set format, such as a table. The parameters of each feature to be verified, the verification result of each feature to be verified, the matching relationship between the parameters of each feature to be verified and the verification result, and the like may be included.

S103, reversely marking the candidate verification strategies according to the verification coverage report to obtain the verified coverage rate of each to-be-verified characteristic.

The verification coverage report includes the verification result of each verified characteristic to be verified, so that there is a possibility that when a certain characteristic to be verified is not verified, the related information thereof will not be presented in the verification coverage report.

In the embodiment of the application, in order to adjust the candidate verification policy according to the verification coverage report, the candidate verification policy may be reversely marked, which may be understood as reversely marking the content in the acquired verification coverage report into the candidate verification policy, and further determining the to-be-verified characteristics that are not verified, and the like from the marked candidate verification policy.

The insufficient verification can be understood as that only part of the functions in the multiple functions that can be realized under the characteristic to be verified are verified. Alternatively, in the verified function under the to-be-verified characteristic, there is a scenario in which verification is unsuccessful, or the like.

Further, after the verification coverage report is reversely marked to the candidate verification policies, the verified coverage rate of each to-be-verified characteristic in the candidate verification policies can be generated based on the marked verification result of each to-be-verified characteristic in the candidate verification policies.

For example, if all the realizable functions of a certain feature to be verified are verified, the verified coverage rate of the feature to be verified is 100%.

For another example, if there are 80 realizable functions under a certain feature to be verified, and it is known after the verification coverage report is reversely marked, if there are 60 verified functions under the feature to be verified, the verified coverage rate of the feature to be verified is 75%.

For another example, when a certain feature to be verified is not verified in the current round of operation of the simulated code to be verified, and the reverse marking result is blank, the verified coverage rate of the feature to be verified is 0%.

It should be noted that the verified coverage may include a code coverage and a function coverage.

The code coverage rate can be obtained based on running codes and non-running codes in the simulated to-be-verified codes corresponding to the to-be-verified characteristics. The functional coverage may be obtained based on the verified and unverified functions of the feature to be verified.

In the implementation, in the to-be-verified performance of the chip, the performance that all functions of the chip need to be completely and effectively verified and covered exists, so that in the simulation of the to-be-verified performance of the to-be-verified code, whether the complete and effective verification and coverage of the to-be-verified performance is completed or not can be judged according to the function coverage rate.

For example, a certain feature to be verified is set to simulate the configuration of the code to be verified.

In order to make the verified chip operate normally, all configurations of the chip need to be verified. Therefore, each configuration of the simulated code to be verified can be verified, and whether the current verification is finished or not is judged according to the verified coverage rate corresponding to the characteristic to be verified.

For another example, a certain feature to be verified is set to simulate the state of the code to be verified.

In order to enable the verified chip to normally operate in different states, all states of the chip need to be verified. Therefore, all states of the code to be verified can be verified, and whether the current verification is finished or not is judged according to the verified coverage rate corresponding to the characteristic to be verified.

For another example, a certain feature to be verified is set as a simulation port for simulating a code to be verified.

In order to enable the verified ports of the chip to normally perform data transmission, the functions of the analog ports of the chip need to be verified. Therefore, all functions of the code simulation port to be verified can be verified, and whether the current verification is finished or not is judged according to the verified coverage rate corresponding to the characteristic to be verified.

And S104, generating a target verification strategy based on the verified coverage rate of each to-be-verified characteristic.

In the embodiment of the application, the availability of the current candidate verification strategy can be judged based on the verified coverage rate of each to-be-verified characteristic.

Optionally, when the verified coverage corresponding to each feature to be verified in the candidate verification policy reaches the set standard, it may be determined that, in the actual chip verification process, complete and effective verification of the function to be verified of the chip may be achieved based on the current candidate verification policy.

Optionally, when the verified coverage rate corresponding to the to-be-verified feature of the partial item in the candidate verification policy does not meet the set standard, the current candidate verification policy needs to be optimized and adjusted until the set standard is met.

And further, determining the candidate verification strategy of which the verified coverage rate of each to-be-verified characteristic reaches the set standard as the target verification strategy.

According to the method for making the verification strategy, the code to be verified is simulated based on the candidate verification strategy, and the verification coverage report of the candidate verification strategy is obtained. And reversely marking the verification coverage report into the candidate verification strategies, further acquiring the verified coverage rate corresponding to each to-be-verified characteristic in the candidate verification strategies, giving the verified coverage rate to adjust the candidate verification strategies until the set standard is met, and further generating the target verification strategy. In the application, the verification coverage report is reversely marked to the candidate verification strategy, so that the verification result of the candidate verification strategy can be more visually presented, the optimization and adjustment of the candidate verification strategy are more effectively realized, the adjustment efficiency of the candidate verification strategy is effectively improved, and the quality of the target verification strategy is optimized.

In the above embodiment, regarding reverse tagging of the verification coverage report to the candidate verification policy, it can be further understood with reference to fig. 2, fig. 2 is a schematic flowchart of a method for making a verification policy according to another embodiment of the present application, as shown in fig. 2, the method includes:

s201, obtaining the verified performance first identification information and the corresponding verified performance under each to-be-verified characteristic from the verification coverage report.

In the embodiment of the application, each to-be-verified characteristic comprises a plurality of to-be-verified performances, wherein each to-be-verified performance has set identification information.

The identification information corresponding to the performance that has been verified under each feature to be verified may be determined as the first identification information. An accurate search and marking of the already verified properties can be achieved by means of the first identification information.

Therefore, the accurate reverse direction of the verification coverage report can be marked into the candidate verification strategy through the first identification information.

Wherein the first identification information has uniqueness.

Alternatively, the first identification information may be stored in an attribute list of each of the characteristics that have been verified, and thus, the first identification information of each of the performances that have been verified may be acquired by reading the attribute list of each of the performances that have been verified.

S202, second identification information of all to-be-verified performances under each to-be-verified characteristic is obtained from the candidate verification strategy.

In the embodiment of the application, each to-be-verified performance under each to-be-verified characteristic in the candidate verification policy has set identification information, and the identification information can be determined as second identification information of each to-be-verified characteristic.

Wherein the second identification information has uniqueness.

Therefore, through the second identification information, accurate search of each item to-be-verified performance under each item to-be-verified characteristic in the candidate verification strategy can be achieved.

Alternatively, the second identification information may be stored in the attribute list of each item of performance to be verified under each item of characteristics to be verified, and thus, the second identification information of each item of characteristics to be verified may be obtained by reading the attribute information list of each item of performance to be verified under each item of characteristics to be verified.

It should be noted that the first identification information and the second identification information are set based on the same format, and matching between the two can be performed.

S203, matching the first identification information with the second identification information, and reversely marking the verified performance to each item of characteristics to be verified.

In the embodiment of the application, matching can be performed based on the format, the parameter, the content and the like in the first identification information and the second identification information, and based on the matching result, the association relationship between the verified performance corresponding to the first identification information and the to-be-verified performance corresponding to the second identification information is determined.

For example, it is set that the first identification information a and the second identification information B are matched, the verified performance a corresponding to the first identification information a and the to-be-verified performance B corresponding to the second identification information B are in a corresponding relationship, and the verification result X of the verified performance a is the verification result corresponding to the to-be-verified performance B.

It is understood that the property a that has been verified is the same as the property b to be verified. Therefore, when performing reverse marking, it is necessary to mark the verification result X of verifying the already verified performance a in the coverage report to the to-be-verified performance b in the candidate policy.

Further, all the first identification information and all the second identification information are matched one by one, and based on the matching result, the verification result of each verified performance in the verification coverage report is marked to the corresponding to-be-verified performance in the candidate policy.

S204, obtaining the unverified performance of each to-be-verified characteristic, and generating the verified coverage rate of each to-be-verified characteristic based on the verified performance and the unverified performance.

Based on the mark of the verification result of each to-be-verified performance under each to-be-verified characteristic, it can be determined that the performance is not verified in the candidate verification policy.

Alternatively, the verification result may be marked at a set position for each item of performance to be verified, and the positions where no verification result is marked are filled with blank data.

Therefore, after the marking of all the verification results in the verification coverage report is completed, if the position of the verification result under a certain to-be-verified performance is blank data, it can be judged that the to-be-verified performance corresponding to the position to which the blank data belongs is not verified.

Further, unverified performance of each to-be-verified characteristic in the candidate verification policy may be obtained. And generating verified coverage of each to-be-verified characteristic based on the verified performance and the unverified performance of each to-be-verified characteristic.

For example, if the number of verified performances is 80 and the number of unverified performances is 20 under the to-be-verified characteristic P, the verified coverage corresponding to the to-be-verified characteristic P is 80%.

For another example, if the verified performance quantity is 0 and the unverified performance quantity is 50 under the to-be-verified characteristic Q, the verified coverage corresponding to the to-be-verified characteristic Q is 0%.

According to the verification strategy determining method, the verification coverage report is reversely marked to the performance to be verified under each characteristic to be verified in the candidate verification strategies, and then the verified coverage rate of each characteristic to be verified is generated. Through the reverse marking, the verified coverage rate of each to-be-verified characteristic in the candidate verification strategy is accurately obtained, so that the adjustment and optimization of the candidate verification strategy are effectively realized, and the quality of the target verification strategy is improved.

Further, regarding generation of the target verification policy, referring to fig. 3, fig. 3 is a schematic flowchart of a method for making a verification policy according to another embodiment of the present application, as shown in fig. 3, where the method includes:

s301, whether the candidate verification strategy is adjusted or not is judged based on the verified coverage rate of each to-be-verified characteristic.

In the embodiment of the application, the verification condition of each to-be-verified characteristic can be determined according to the verified coverage rate of each to-be-verified characteristic.

In order to completely and comprehensively verify the functions of the chip, each item of performance to be verified under each item of characteristic to be verified needs to be verified.

Optionally, in response to that the to-be-verified performance under each to-be-verified characteristic is verified, it is determined that the candidate verification policy does not need to be adjusted.

If the verified coverage rate of each to-be-verified characteristic is one hundred percent, it can be determined that, in the current candidate verification strategy, each to-be-verified performance under each to-be-verified characteristic is verified.

It can be understood that, based on the current candidate verification policy, complete and comprehensive verification can be implemented on the characteristics of the chip in the verification process of the chip, and therefore, no adjustment is required for the current candidate verification policy.

And in response to the fact that the performance to be verified under each characteristic to be verified is not verified, judging that the candidate verification strategy needs to be adjusted.

If the verified coverage report of the to-be-verified characteristic is not one hundred percent, it can be known that the to-be-verified performance which is not verified exists in the to-be-verified characteristic corresponding to the verification result.

It can be understood that, based on the current candidate verification policy, part of the existing characteristics cannot be verified in the verification process of the chip, and therefore, the current candidate verification policy needs to be adjusted.

S302, in response to the judgment that the adjustment is not needed, determining the candidate verification strategy as the target verification strategy.

In the embodiment of the application, when it is determined that the candidate verification policy does not need to be adjusted, it can be understood that the current candidate verification policy can achieve comprehensive verification of the chip function.

Thus, the current candidate authentication policy may be determined as the target authentication policy.

It should be noted that the candidate verification policies may be set based on a part of the function modules, and therefore, the final verification policy may be composed of at least one target verification policy.

And S303, responding to the judgment to be adjusted, adjusting the candidate verification strategies according to the unverified performance of each to-be-verified characteristic, and generating the target verification strategy when the to-be-verified performance included by each to-be-verified characteristic in the adjusted candidate verification strategies is verified.

In the embodiment of the application, the current candidate verification strategy can be adjusted according to the unverified performance of each to-be-verified characteristic.

Optionally, for the to-be-verified characteristics with verified coverage rate less than one hundred percent, effective information extraction may be performed again from the design document, and the optimization adjustment of the current candidate verification policy is realized by enriching the expression information of the extracted to-be-verified characteristics.

Further, the verification is continued for the adjusted candidate verification strategy.

And operating the simulated code to be verified based on the adjusted candidate verification strategy, and acquiring the verified coverage rate of each characteristic in the adjusted candidate verification strategy. And continuously judging the verified coverage rate of each to-be-verified characteristic in the adjusted candidate verification strategy until the verified coverage rate of each to-be-verified characteristic in the adjusted candidate verification strategy is one hundred percent, and stopping adjustment.

Further, generation of the target verification strategy is achieved based on candidate verification strategies with one hundred percent of verified coverage rate of each to-be-verified characteristic.

The method for determining the verification strategy provided by the application judges whether the candidate verification strategy needs to be adjusted according to the verified coverage rate. And determining the candidate verification strategy which does not need to be adjusted as the target verification strategy output. And for the candidate verification strategy needing to be adjusted, adjusting the candidate verification strategy based on the unverified performance of each to-be-verified characteristic, verifying the adjusted candidate verification strategy until the verified coverage rate of each to-be-verified characteristic in the adjusted candidate verification strategy is one hundred percent, stopping adjusting, and generating the target verification strategy based on the candidate verification strategy of which the verified coverage rate of each to-be-verified characteristic is one hundred percent. Through the verified coverage rate, the verification result of each item of to-be-verified performance under each item of to-be-verified characteristic is accurately obtained, the adjustment efficiency of the candidate verification strategy is effectively improved, and the quality of the target verification strategy is optimized.

In the above embodiment, regarding generation of the verification coverage report, as can be further understood with reference to fig. 4, fig. 4 is a schematic flowchart of a method for determining a verification policy according to another embodiment of the present application, and as shown in fig. 4, the method includes:

s401, based on each to-be-verified characteristic in the candidate verification strategy, running the simulation to-be-verified code, and verifying the performance of each to-be-verified characteristic according to the running result to obtain the verified performance of each to-be-verified characteristic.

In the embodiment of the application, the simulated code to be verified can be run based on the running environment and the requirement required by each performance to be verified under each characteristic to be verified, and the corresponding output can be generated based on the input corresponding to each performance to be verified.

Whether the performance to be verified currently verified is successfully verified can be judged based on the output result.

Optionally, a standard of a verification result of each to-be-verified performance under each to-be-verified characteristic may be set, and when an output result of simulating operation of a to-be-verified code meets the set standard, it may be determined that verification of the currently-verified performance is successful.

Further, a simulation code to be verified is run based on each performance to be verified under each characteristic to be verified, each output result and the performance to be verified corresponding to each output result are recorded, and when the current verification is judged to be successful based on the output results, the performance to be verified which is successfully verified is marked as the verified performance under the characteristic to be verified.

S402, generating a verification coverage report of the candidate verification strategy based on the verified performance under each to-be-verified characteristic and the verification result of the verified performance.

In the embodiment of the application, the verification result of the verified performance under each to-be-verified characteristic can be obtained according to the verification coverage report.

Alternatively, a verification coverage report, such as a table, may be generated based on a set format. The setting format defines each to-be-verified characteristic, the verified performance under each to-be-verified characteristic, the related attribute parameters and the filling position of the verification result.

After each to-be-verified characteristic, verified performance under each to-be-verified characteristic, related attribute parameters and a verification result are obtained, filling can be performed in a blank table based on a limited position, so as to generate a verification coverage report of a candidate verification strategy in a set format.

It should be noted that, in the method provided by the present application, the verification coverage report is reversely marked into the candidate verification policy, so that, in order to improve the generation efficiency of the verification coverage report and save related resources, only the related information of the performance of generating the verification result may be filled in the filling process of the blank report.

According to the verification strategy determining method, the verification coverage report of the candidate verification strategy is generated according to the verified performance of each item of to-be-verified characteristic. The data base is provided for the reverse marking of the subsequent verification condition, and the adjustment efficiency of the candidate verification strategy is effectively improved.

In the above embodiment, regarding the acquisition of the candidate verification policy, it can be further understood by referring to fig. 5, where fig. 5 is a schematic flowchart of a method for determining the verification policy according to another embodiment of the present application, and as shown in fig. 5, the method includes:

s501, constructing a test case, and acquiring each to-be-verified characteristic of the simulated to-be-verified code in the test case environment.

Optionally, the verification process of simulating the code to be verified may be started by constructing the test case, and it may be understood that the test case provides a verification environment for simulating the characteristic verification of the code to be verified.

In the environment provided by different test cases, different characteristics can be realized by simulating the code to be verified, so that for a certain constructed test case, each characteristic which can be realized by simulating the code to be verified in the environment is determined from the design document based on the verification environment provided by the test case.

Further, based on the set extraction rule of the characteristics to be verified, the characteristics to be verified which are verified under the current test case environment are determined from all the characteristics.

S502, generating a candidate verification strategy based on each to-be-verified characteristic.

In the embodiment of the present application, each to-be-verified characteristic has a relevant setting, such as an implementation segment of a simulated to-be-verified code corresponding to each to-be-verified characteristic, a working mode of the simulated to-be-verified code corresponding to each to-be-verified characteristic, a state and a configuration corresponding to each to-be-verified characteristic, and the like.

Further, related parameters corresponding to each to-be-verified characteristic are obtained, a corresponding relation between the to-be-verified characteristic and the related parameters is established, and then a candidate verification strategy for simulating each to-be-verified characteristic of the to-be-verified code at present is generated.

According to the verification strategy determining method, each item of to-be-verified characteristic of the simulated to-be-verified code is determined through the test case, and therefore the corresponding candidate verification strategy is determined. The method provides a pre-plan for the verification of the simulated code to be verified, and effectively improves the verification efficiency of the simulated code to be verified.

Corresponding to the determination methods of the verification policies provided in the foregoing embodiments, an embodiment of the present application further provides a determination device of the verification policies, and since the determination device of the verification policies provided in the embodiment of the present application corresponds to the determination methods of the verification policies provided in the foregoing embodiments, the implementation of the determination method of the verification policies is also applicable to the determination device of the verification policies provided in the embodiment of the present application, and will not be described in detail in the following embodiments.

Fig. 6 is a schematic structural diagram of a device for determining a verification policy according to an embodiment of the present application, and as shown in fig. 6, the device 600 for determining a verification policy includes an obtaining module 61, a verifying module 62, a marking module 63, and a generating module 64, where:

the acquiring module 61 is configured to acquire a candidate verification policy, where the candidate verification policy includes each to-be-verified characteristic of the simulated to-be-verified code;

the verification module 62 is configured to run a simulation to-be-verified code based on the candidate verification policy, verify each to-be-verified characteristic in the candidate verification policy, and generate a verification coverage report;

the marking module 63 is configured to reversely mark the candidate verification policies according to the coverage report to obtain verified coverage of each to-be-verified feature;

and a generating module 64, configured to generate a target verification policy based on the verified coverage of each to-be-verified characteristic.

Fig. 7 is a schematic structural diagram of a device for determining a verification policy according to another embodiment of the present application, and as shown in fig. 7, the device 700 for determining a verification policy includes an obtaining module 71, a verifying module 72, a marking module 73, and a generating module 74, where:

the acquisition module 61, the verification module 62, the marking module 63, and the generation module 64 have the same configurations and functions as the acquisition module 71, the verification module 72, the marking module 73, and the generation module 74.

In this embodiment of the present application, the marking module 73 is further configured to: acquiring first identification information of the verified performance and the corresponding verified performance under each to-be-verified characteristic from the verification coverage report; acquiring all second identification information of the to-be-verified performance under each to-be-verified characteristic from the candidate verification strategy; matching the first identification information with the second identification information, and reversely marking the verified performance to each characteristic to be verified; and obtaining unverified performance of each to-be-verified characteristic, and generating verified coverage of each to-be-verified characteristic based on the verified performance and the unverified performance.

In this embodiment of the application, the generating module 74 is further configured to: judging whether to adjust the candidate verification strategy or not based on the verified coverage rate of each to-be-verified characteristic; determining the candidate verification strategy as a target verification strategy in response to determining that no adjustment is needed; and responding to the judgment of the need for adjustment, adjusting the candidate verification strategies according to the unverified performance of each to-be-verified characteristic, and generating the target verification strategy when the to-be-verified performance included by each to-be-verified characteristic in the adjusted candidate verification strategies is verified.

In this embodiment of the application, the generating module 74 is further configured to: in response to the fact that the performance to be verified under each characteristic to be verified is verified, judging that the candidate verification strategy does not need to be adjusted; and in response to the fact that the performance to be verified under each characteristic to be verified is not verified, judging that the candidate verification strategy needs to be adjusted.

In this embodiment of the application, the obtaining module 71 is further configured to: constructing a test case, and acquiring each to-be-verified characteristic of the simulated to-be-verified code under the environment of the test case; and generating a candidate verification strategy based on each characteristic to be verified.

In this embodiment of the application, the verification module 72 is further configured to: running a simulation code to be verified based on each characteristic to be verified in the candidate verification strategy, and verifying the performance of each characteristic to be verified according to a running result to obtain the verified performance of each characteristic to be verified; and generating a verification coverage report of the candidate verification strategy based on the verified performance under each to-be-verified characteristic and the verification result of the verified performance.

The verification strategy making device provided by the application runs and simulates the code to be verified based on the candidate verification strategy, and obtains the verification coverage report of the candidate verification strategy. And reversely marking the verification coverage report into the candidate verification strategies, further acquiring the verified coverage rate corresponding to each to-be-verified characteristic in the candidate verification strategies, giving the verified coverage rate to adjust the candidate verification strategies until the set standard is met, and further generating the target verification strategy. In the application, the verification coverage report is reversely marked to the candidate verification strategy, so that the verification result of the candidate verification strategy can be more visually presented, the optimization and adjustment of the candidate verification strategy are more effectively realized, the adjustment efficiency of the candidate verification strategy is effectively improved, and the quality of the target verification strategy is optimized.

To achieve the above embodiments, the present application also provides an electronic device, a computer readable storage medium and a computer program product.

Fig. 8 is a block diagram of an electronic device according to an embodiment of the present application, and a method for determining a verification policy according to the embodiment of fig. 1 to 5 may be implemented by the electronic device shown in fig. 8.

In order to implement the above embodiments, the present application also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the determination method of the verification policy of the embodiment of fig. 1 to 5.

In order to implement the above embodiments, the present application further provides a computer program product, which when executed by an instruction processor in the computer program product, executes the method for determining the verification policy of the embodiments of fig. 1 to 5.

In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims

1. A method for determining a verification policy, comprising:

obtaining a candidate verification strategy, wherein the candidate verification strategy comprises each item of to-be-verified characteristic of a simulated to-be-verified code;

running the simulated to-be-verified code based on the candidate verification strategy, verifying each to-be-verified characteristic in the candidate verification strategy, and generating a verification coverage report;

reversely marking the candidate verification strategies according to the verification coverage report to obtain the verified coverage rate of each to-be-verified characteristic;

generating a target verification policy based on the verified coverage of each to-be-verified characteristic.

2. The method of claim 1, wherein reversely marking the candidate verification policies according to the verification coverage report to obtain verified coverage of each of the characteristics to be verified comprises:

acquiring first identification information of the verified performance and the corresponding verified performance under each to-be-verified characteristic from the verification coverage report;

acquiring all second identification information of the to-be-verified performance under each to-be-verified characteristic from the candidate verification strategy;

matching the first identification information with the second identification information, and reversely marking the verified performance to each item of characteristic to be verified;

obtaining unverified performance of each to-be-verified characteristic, and generating the verified coverage rate of each to-be-verified characteristic based on the verified performance and the unverified performance.

3. The method of claim 2, wherein generating a target verification policy based on the verified coverage for each of the characteristics to be verified comprises:

determining whether to adjust the candidate verification policy based on the verified coverage of each feature to be verified;

in response to determining that no adjustment is required, determining the candidate verification policy as the target verification policy;

and responding to the judgment that the adjustment is needed, adjusting the candidate verification strategies according to the unverified performance of each to-be-verified characteristic, and generating the target verification strategy when the to-be-verified performance included by each to-be-verified characteristic in the adjusted candidate verification strategies is verified.

4. The method of claim 3, wherein the determining whether to adjust the candidate authentication policy based on the verified coverage for each feature to be authenticated comprises:

in response to the to-be-verified performance under each to-be-verified characteristic being verified, determining that the candidate verification policy does not need to be adjusted;

and in response to the to-be-verified performance under each to-be-verified characteristic is not verified, determining that the candidate verification strategy needs to be adjusted.

5. The method of claim 4, wherein obtaining the candidate authentication policy comprises:

constructing a test case, and acquiring each item of to-be-verified characteristic of the simulated to-be-verified code under the environment of the test case;

and generating the candidate verification strategy based on each characteristic to be verified.

6. The method of claim 1, wherein the running the simulated code-to-be-verified based on the candidate verification policy, verifying the each characteristic-to-be-verified in the candidate verification policy, and generating a verification coverage report comprises:

running the simulated to-be-verified code based on each to-be-verified characteristic in the candidate verification strategy, and verifying the performance of each to-be-verified characteristic according to a running result to obtain the verified performance of each to-be-verified characteristic;

generating the verification coverage report of the candidate verification strategy based on the verified performance under each to-be-verified characteristic and the verification result of the verified performance.

7. An apparatus for determining a verification policy, comprising:

the verification device comprises an acquisition module, a verification module and a verification module, wherein the acquisition module is used for acquiring a candidate verification strategy, and the candidate verification strategy comprises each item of to-be-verified characteristic of a simulated to-be-verified code;

the verification module is used for running the simulated to-be-verified codes based on the candidate verification strategies, verifying each to-be-verified characteristic in the candidate verification strategies and generating a verification coverage report;

the marking module is used for reversely marking the candidate verification strategies according to the coverage reports so as to obtain the verified coverage rate of each to-be-verified characteristic;

and the generating module is used for generating a target verification strategy based on the verified coverage rate of each to-be-verified characteristic.

8. The apparatus of claim 7, wherein the tagging module is further configured to:

9. The apparatus of claim 8, wherein the generating module is further configured to:

10. The apparatus of claim 9, wherein the generating module is further configured to:

11. The apparatus of claim 10, wherein the obtaining module is further configured to:

12. The apparatus of claim 7, wherein the verification module is further configured to:

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.