CN114721895B

CN114721895B - Verification method, platform, equipment and medium for design to be tested

Info

Publication number: CN114721895B
Application number: CN202210399728.0A
Authority: CN
Inventors: 金鑫
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-04-15
Filing date: 2022-04-15
Publication date: 2023-02-21
Anticipated expiration: 2042-04-15
Also published as: CN114721895A

Abstract

The disclosure provides a verification method, a verification platform, verification equipment, verification media and a program product of a design to be tested, and relates to the technical field of artificial intelligence chips, in particular to the field of chip verification. The specific implementation scheme is as follows: acquiring a configuration matrix through a configuration model in the verification platform, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of the design to be tested; determining the current working mode of the design to be tested according to the configuration matrix; and verifying the design to be tested according to the current working mode of the design to be tested. The technical scheme disclosed by the invention not only can effectively verify the function and performance of the design to be tested, but also has stronger universality.

Description

Verification method, platform, equipment and medium of design to be tested

Technical Field

The present disclosure relates to the field of artificial intelligence chip technologies, and in particular, to a verification method, a verification platform, a verification device, a verification medium, and a verification program product for a design to be tested.

Background

The audio, image and other signal processing system is an important data processing unit in an artificial intelligence chip, and the main function of the system is to extract, process and send audio, image and other signals. The verification of the functions and the performances of the signal processing system is an indispensable part in the verification work of the artificial intelligent chip.

Because different intelligent chips have different signals to be processed in different scenes, it is important to provide a verification platform and method with higher universality for different signal processing systems.

Disclosure of Invention

The present disclosure provides a verification method, platform, device, medium, and program product for a design under test.

According to an aspect of the present disclosure, there is provided a verification method of a design under test, including:

acquiring a configuration matrix, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of a design to be tested;

determining the current working mode of the design to be tested according to the configuration matrix;

and verifying the design to be tested according to the current working mode of the design to be tested.

According to another aspect of the present disclosure, there is provided a verification platform for a design under test, including:

the system comprises a configuration model, a configuration matrix and a configuration module, wherein the configuration model is used for configuring a configuration matrix, and the configuration matrix is used for recording the combination of configuration options of different working modes of a design to be tested;

the target signal interface bus agent is used for responding to the enabling determined by the verification platform according to the current working mode of the design to be tested, sending a target signal to the design to be tested and receiving a first receiving value of external data processed by the design to be tested;

the data transmission interface bus agent is used for responding to the enabling determined by the verification platform according to the current working mode of the design to be tested, sending the external data to the design to be tested, and receiving a second receiving value of the target signal after being processed by the design to be tested;

the reference model is used for determining a first expected value for processing the external data and a second expected value for processing the target signal according to the current working mode of the design to be tested and the same processing process as that in the design to be tested;

and the scoring board is used for comparing the first receiving value with the first expected value and comparing the second receiving value with the second expected value according to the current working mode of the design to be tested, and the comparison result of the scoring board is the verification result of the verification platform.

According to another aspect of the present disclosure, there is provided a verification apparatus for a design under test, including:

the device comprises a configuration matrix acquisition module, a configuration matrix acquisition module and a configuration matrix generation module, wherein the configuration matrix acquisition module is used for acquiring a configuration matrix, and the configuration matrix is used for recording the combination of configuration options of different working modes of a design to be tested;

the working mode determining module is used for determining the current working mode of the design to be tested according to the configuration matrix;

and the verification module is used for verifying the design to be tested according to the current working mode of the design to be tested.

According to another aspect of the present disclosure, there is provided an electronic device including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of verifying a design under test as set forth in any of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method of verifying a design under test as set forth in any of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of verifying a design under test of any embodiment of the present disclosure.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of a verification method for a design under test according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a verification method for a design under test according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a verification method for a design under test according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a verification method for a design under test according to an embodiment of the present disclosure;

FIG. 5 is an architecture diagram of a verification platform for a design under test, in accordance with an embodiment of the present disclosure;

FIG. 6 is an architecture diagram of a verification platform for a design under test, in accordance with an embodiment of the present disclosure;

FIG. 7 is an architecture diagram of a verification platform for a design under test, in accordance with an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a verification apparatus for a design under test according to an embodiment of the present disclosure;

FIG. 9 is a block diagram of an electronic device for implementing a verification method of a design under test of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic flow diagram of a verification method for a design to be tested according to an embodiment of the present disclosure, which is applicable to a situation of verifying a design to be tested in an artificial intelligence chip, in particular to a situation of verifying a signal processing system in an artificial intelligence chip, and relates to the technical field of artificial intelligence chips, in particular to the field of chip verification. The method can be executed by a verification device of the design to be tested based on a verification platform of the design to be tested, and the device is implemented in a software and/or hardware manner, and is preferably configured in electronic equipment, such as computer equipment or a server. As shown in fig. 1, the method specifically includes the following steps:

s101, obtaining a configuration matrix, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of the design to be tested.

The design to be tested may be a chip to be verified, or a module or unit in the chip, such as a signal processing system in the chip that processes signals such as audio or images. With the diversification of functions and the improvement of complexity of chip products, the design to be tested can work in various different working modes, so that the method which has stronger universality and can verify different working modes of different designs to be tested is particularly important.

In the embodiment of the disclosure, according to the verification requirement, the configuration options of different working modes of the design to be tested can be established first, and then the combination of the configuration options of different working modes of the design to be tested is recorded by forming the configuration matrix. Therefore, according to different verification requirements, any design to be tested can be fully verified based on different working modes of the design to be tested, and the method has high universality.

And S102, determining the current working mode of the design to be tested according to the configuration matrix.

The working mode of the design to be tested can be changed in real time during the operation period, so that the real-time working mode of the design to be tested can be obtained according to the configuration matrix, and the design to be tested is verified based on the current working mode.

S103, verifying the design to be tested according to the current working mode of the design to be tested.

The process of verifying the design to be tested may be applying an excitation signal to the design to be tested in a manner matching with the current working mode of the design to be tested, then comparing a received value of the design to be tested after processing the excitation signal with a theoretical value obtained by processing the excitation signal with a theoretical model, if the received value does not conform to the theoretical value, it indicates that the design to be tested has a problem, otherwise, the verification is passed.

According to the technical scheme of the embodiment of the disclosure, different working modes of the design to be tested are configured through the configuration matrix, so that the design to be tested is verified according to the current working mode of the design to be tested. Therefore, the method not only can effectively verify the function and performance of the design to be tested, but also can verify different designs to be tested and different working modes thereof by adopting the method of the embodiment of the disclosure, and has stronger universality.

Fig. 2 is a schematic flow chart of a verification method of a design to be tested according to an embodiment of the present disclosure, and the embodiment further performs optimization based on the above embodiment. As shown in fig. 2, the method specifically includes the following steps:

s201, obtaining a configuration matrix, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of the design to be tested.

Taking the system to be tested as a signal processing system as an example, the working modes may include a master mode, a slave mode, a transceiving mode, a single channel mode or a multi-channel mode. The configuration matrix records various real-time working modes of the design to be tested in the running process.

S202, determining the current working mode of the design to be tested according to the configuration matrix.

S203, enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the target signal interface bus agent is used for sending a target signal to the design to be tested, and the data transmission interface bus agent is used for receiving a first receiving value of the target signal after the target signal is processed by the design to be tested.

Specifically, the design to be tested may be connected to the target signal interface bus agent and the data transmission interface bus agent, and the target signal interface bus agent and the data transmission interface bus agent may be enabled to operate in a manner matching the current operating mode of the design to be tested according to the current operating mode of the design to be tested. The target signal interface bus agent is configured to send a target signal to the design to be tested, for example, the target signal interface bus agent sends the target signal to the design to be tested by driving a target signal interface bus connected to the design to be tested. The data transmission interface bus agent is configured to receive a first receiving value of the target signal after the design to be tested is processed, for example, the data transmission interface bus agent receives the first receiving value of the target signal after the design to be tested is processed by monitoring a data transmission interface bus connected to the design to be tested.

In one embodiment, the target signal interface bus agent may include a target signal interface bus master agent and a target signal interface bus slave agent, and the data transfer interface bus agent may include a data transfer interface bus master agent and a data transfer interface bus slave agent. The target signal interface bus master agent and the target signal interface bus slave agent are connected with the design to be tested through the target signal interface bus, and the data transmission interface bus master agent and the data transmission interface bus slave agent are connected with the design to be tested through the data transmission interface bus. Correspondingly, for the target signal interface bus, if the current working mode of the design to be tested is the master mode, the slave agent of the target signal interface bus is enabled, and if the current working mode of the design to be tested is the slave mode, the master agent of the target signal interface bus is enabled. And aiming at the data transmission interface bus, if the current working mode of the design to be tested is the master mode, enabling the slave agent of the data transmission interface bus, and if the current working mode of the design to be tested is the slave mode, enabling the master agent of the data transmission interface bus. That is, no matter the design to be tested works in the master mode or the slave mode, the corresponding target signal interface bus agent can be enabled to realize data transceiving, and therefore the verification of the design to be tested can be successfully completed in different working modes.

And S204, comparing the first receiving value with a first expected value through a score board according to the current working mode of the design to be tested, and determining a verification result of the design to be tested according to the comparison result, wherein the first expected value is obtained after the target signal is processed by a first theoretical model, and the first theoretical model realizes the same processing process as the first receiving value obtained by the design to be tested.

The score board is used for comparing the first receiving value with a first expected value, wherein the first expected value is a theoretical value, namely a value which should be output by the design to be tested theoretically according to the same processing process as the first receiving value obtained by the design to be tested. Specifically, the target signal may be processed by the first theoretical model to obtain the first expected value. If the first receiving value is consistent with the first expected value, the design to be tested is indicated to work normally, verification can be passed, and otherwise, the design to be tested is indicated to have problems.

In one embodiment, the design under test may operate in transparent mode. Therefore, if the current working mode of the design to be tested is the transparent transmission mode, the first received value is the first expected value, and it can also be considered that the first expected value obtained by the processing of the first theoretical model is the same as the first received value. In a word, the signal processing process realized by the first theoretical model is the same as the process of processing the target signal by the design to be tested, the theoretical value of the processing target signal is obtained, and the verification result of the design to be tested can be determined by comparing the received value with the theoretical value.

In addition, the verification method of the embodiment of the present disclosure further includes: the target signal, the first received value, and the first expected value are stored to a memory model. The memory model is a part of the verification platform, the target signal is excitation data provided by the verification platform for verification, the first receiving value is a value obtained by processing the target signal by the design to be tested, the target signal serving as source data and the first receiving value serving as result data are stored through the memory model, and the memory model can play a role in data caching when the data rates of the input direction and the output direction of the design to be tested are asynchronous.

According to the technical scheme, different working modes of the design to be tested are configured through the configuration matrix, and then the corresponding target signal interface bus agent and the corresponding data transmission interface bus agent are enabled in real time according to the current working mode of the design to be tested, so that the bus agents work in a mode matched with the current working mode of the design to be tested. Therefore, the verification method is suitable for different designs to be tested and different working modes of the designs to be tested, and has good universality.

Fig. 3 is a schematic flow chart of a verification method of a design to be tested according to an embodiment of the present disclosure, and the embodiment further performs optimization based on the above embodiment. As shown in fig. 3, the method specifically includes the following steps:

s301, obtaining a configuration matrix, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of the design to be tested.

And S302, determining the current working mode of the design to be tested according to the configuration matrix.

And S303, enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the data transmission interface bus agent is used for sending external data to the design to be tested, and the target signal interface bus agent is used for receiving a second receiving value of the external data processed by the design to be tested.

Specifically, the design to be tested may be connected to the target signal interface bus agent and the data transmission interface bus agent, and the target signal interface bus agent and the data transmission interface bus agent may be enabled to operate in a manner that matches the current operating mode of the design to be tested according to the current operating mode of the design to be tested. The data transmission interface bus agent is configured to send external data to the design to be tested, for example, the data transmission interface bus agent sends the external data to the design to be tested by driving a data transmission interface bus connected to the design to be tested. The target signal interface bus agent is configured to receive a second receiving value of the external data after the design to be tested is processed, for example, the target signal interface bus agent monitors a target signal interface bus connected to the design to be tested, and the external data receives the second receiving value after the design to be tested is processed.

As with the above embodiments, in the embodiments of the present disclosure, the target signal interface bus agent may include a target signal interface bus master agent and a target signal interface bus slave agent, and the data transfer interface bus agent may include a data transfer interface bus master agent and a data transfer interface bus slave agent. The specific operation is the same as the above embodiment, and will not be described herein.

S304, according to the current working mode of the design to be tested, comparing a second receiving value with a second expected value through a score board, and determining a verification result of the design to be tested according to the comparison result, wherein the second expected value is obtained after external data is processed through a second theoretical model, and the second theoretical model realizes the same processing process as the second receiving value obtained by the design to be tested.

The scoring board is used for comparing the second receiving value with a second expected value, wherein the second expected value is a theoretical value, namely a value which should be output by the design to be tested theoretically according to the same processing process of the second receiving value obtained by the design to be tested. Specifically, the external data may be processed by the second theoretical model to obtain the second expected value. If the second receiving value is consistent with the second expected value, the design to be tested is indicated to work normally and can pass verification, otherwise, the design to be tested is indicated to have problems. Similarly, if the current working mode of the design to be tested is the transparent transmission mode, the second received value is the second expected value.

In addition, the verification method of the embodiment of the present disclosure further includes: the external data, the second received value, and the second expected value are stored to the memory model.

According to the technical scheme of the embodiment of the disclosure, different working modes of the design to be tested are configured through the configuration matrix, and then the corresponding target signal interface bus agent and the corresponding data transmission interface bus agent are enabled in real time according to the current working mode of the design to be tested, so that the bus agents work in a mode matched with the current working mode of the design to be tested. Moreover, the data streams in different directions on the design to be tested can be verified. Therefore, the verification method is suitable for different designs to be tested and different working modes of the designs to be tested, and has good universality.

Fig. 4 is a schematic flowchart of a verification method of a design to be tested according to an embodiment of the present disclosure, and the embodiment further performs optimization based on the above embodiment. As shown in fig. 4, the method specifically includes the following steps:

s401, obtaining a configuration matrix through a configuration model, wherein the configuration model is used for configuring the configuration matrix.

The configuration model is a part of the verification platform, and the configuration matrix can be configured through the configuration model according to verification requirements.

S402, writing the configuration matrix into a register model, wherein the register model is used for mirroring the register to be designed.

The read-write operation of the register to be tested can be realized by a front door access mode.

And S403, enabling the register interface bus agent to send the configuration matrix in the register model to the design to be tested so as to configure the working mode of the design to be tested.

The design to be tested can be connected with a register interface bus, the register interface bus agent is enabled to drive the register interface bus, and the configuration matrix in the register model is sent to the design to be tested.

S404, monitoring the change data in the register to be tested through the register interface bus agent, and writing the change data into the register model.

For example, the register interface bus agent monitors the register interface bus to obtain the change data in the register to be designed, and writes the change data into the register model.

Through the above S403 and S404, on one hand, the configuration of the working mode of the design to be tested is realized according to the configuration matrix, and on the other hand, when the change data occurs in the register of the design to be tested, the change data can be written into the register model to mirror the register in the design to be tested in real time, so that the working mode of the design to be tested can be accurately determined in the following process, and thus the verification can be accurately performed.

S405, determining the current working mode of the design to be tested according to the configuration matrix and the change data in the register model.

The register model is a mirror image of a register in the design to be tested and can write the change data of the register in the design to be tested in real time, so that the current working mode of the design to be tested can be determined according to the configuration matrix and the change data in the register model, and the design to be tested can be verified according to the current working mode of the design to be tested.

And S406, verifying the design to be tested according to the current working mode of the design to be tested.

According to the technical scheme of the embodiment of the disclosure, the register in the design to be tested is mirrored through the register model, the configuration matrix is written into the register model, and the configuration matrix in the register model is sent to the design to be tested through the register interface bus agent so as to configure the working mode of the design to be tested. Meanwhile, the change data in the register of the design to be tested can be written into the register module in real time, so that the current working mode of the design to be tested can be obtained according to the configuration matrix and the change data in the register module, and the design to be tested can be verified according to the real-time current working mode of the design to be tested. The embodiment of the disclosure further improves the universality of verification of the design to be tested.

Fig. 5 is an architecture diagram of a verification platform of a design to be tested according to an embodiment of the present disclosure, which is applicable to a situation of verifying a design to be tested in an artificial intelligence chip, especially a situation of verifying a signal processing system in an artificial intelligence chip, and relates to the technical field of artificial intelligence chips, especially to the field of chip verification. As shown in fig. 5, the verification platform of the design to be tested includes:

the configuration model 501 is configured to configure a configuration matrix, where the configuration matrix is used to record a combination of configuration options of different working modes of the design to be tested.

And the target signal interface bus agent 502 is configured to respond to the enable determined by the verification platform according to the current working mode of the design to be tested, send a target signal to the design to be tested, and receive a first receiving value of external data after the design to be tested is processed.

And the data transmission interface bus agent 503 is configured to send external data to the design to be tested in response to the enable determined by the verification platform according to the current working mode of the design to be tested, and receive a second receiving value of the target signal after the processing of the design to be tested.

And the reference model 504 is used for determining a first expected value for processing external data and a second expected value for processing a target signal according to the current working mode of the design to be tested and the same processing process in the design to be tested.

And the scoring board 505 is used for comparing the first receiving value with the first expected value and comparing the second receiving value with the second expected value according to the current working mode of the design to be tested, wherein the comparison result of the scoring board is the verification result of the verification platform.

Specifically, the verification platform may obtain the current working mode of the design to be tested through the configuration model 501, and then enable the data transmission interface bus agent 503 and the target signal interface bus agent 502 according to the working mode, so that the data transmission interface bus agent 503 and the target signal interface bus agent 502 may perform data transceiving with the design to be tested in a manner corresponding to the current working mode of the design to be tested. The operating modes may include a master mode, a slave mode, a transceive mode, a single channel mode, or a multi-channel mode, among others. The working mode of the design to be tested is not limited at all, and the design to be tested can be configured through the configuration model according to the verification requirement.

In the verification process, data streams in different directions on the design to be tested can be aimed at, and configuration can be performed through a receiving and sending mode in a working mode. Based on the configuration of the transceiving mode, the verification platform may enable the target signal interface bus agent 502 to send the target signal to the design to be tested, and enable the data transmission interface bus agent 503 to receive a second receiving value of the target signal after the design to be tested is processed. In addition, the verification platform may also enable the data transmission interface bus agent 503 to send the external data to the design to be tested, and enable the target signal interface bus agent 502 to receive the first receiving value of the external data after the design to be tested is processed. For example, the design to be tested is a signal processing system in the chip, the target signal may be, for example, audio data or image data that needs to be processed by the design to be tested, and the external data is data transmitted by other systems connected to the signal processing system in the chip.

When the first receiving value or the second receiving value is obtained, the scoring board 505 is used for comparing the first receiving value with the first expected value, and comparing the second receiving value with the second expected value, and determining the verification result of the design to be tested according to the comparison result. The target signal interface bus agent 502 and the data transmission interface bus agent 503 may send the source data, i.e., the target signal and the external data, to the reference model 504 through the scoreboard 505, and the reference model 504 performs processing in the same processing manner as the design to be tested, so as to obtain a first expected value for processing the external data and a second expected value for processing the target signal. The scoreboard 505 may obtain the first expected value and the second expected value from the reference model 504 and compare them with the first received value and the second received value, respectively.

The reference model 504 and the score board 505 are processed according to the current working mode of the design to be tested. The reference model 504 implements the same processing procedure according to the current working mode of the design to be tested, and determines a first expected value for processing the external data and a second expected value for processing the target signal. The scoreboard 505 compares the received value with the expected value based on the current operating mode of the design under test. For example, when the current working mode of the design to be tested is the transparent transmission mode, the first receiving value is the first expected value, and similarly, the second receiving value is the second expected value.

According to the technical scheme, different working modes of the design to be tested can be configured through the configuration model, so that the verification platform can enable the target signal interface bus agent and the data transmission interface bus agent according to the current working mode of the design to be tested in real time, and signal sending and receiving between the verification platform and the design to be tested are achieved. Meanwhile, the reference model can also simulate the data processing process of the design to be tested according to the working mode of the design to be tested, so that the verification result is determined through the scoring board. In addition, the bus interface can be flexibly replaced according to the design to be tested. Therefore, the embodiment of the disclosure not only can effectively verify the functions and performances of the to-be-tested designs such as signal processing systems, but also is suitable for various to-be-tested designs with different working modes, and has stronger universality.

Fig. 6 is an architecture diagram of a verification platform of a design under test according to an embodiment of the disclosure, which is further optimized based on the above embodiment. As shown in fig. 6, a target signal interface bus and a data transmission interface bus are connected to a design to be tested 601, and a verification platform of the design to be tested includes a configuration model 602, a data transmission interface bus agent 603, a target signal interface bus agent 604, a reference model 605, and a scoreboard 606. The details of the configuration model 602, the reference model 605 and the score counting board 606 are the same as those of the above embodiments, and are not described herein again.

Specifically, the verification platform acquires the current working mode of the design to be tested through the configuration model 602, and then enables the data transmission interface bus agent 603 and the target signal interface bus agent 604 according to the working mode. The target signal interface bus agent 604 responds to the enabling of the verification platform, sends a target signal to the design to be tested by driving the target signal interface bus, and receives a first receiving value of external data after the processing of the design to be tested by monitoring the target signal interface bus; the data transmission interface bus agent 603 responds to the enable of the verification platform, sends external data to the design to be tested by driving the data transmission interface bus, and receives a second receiving value of the target signal after the processing of the design to be tested by monitoring the data transmission interface bus. Therefore, when the target signal is used as the excitation, the second receiving value can be obtained through the design processing to be tested, and when the external data is used as the excitation, the first receiving value can be obtained through the design processing to be tested, so that the design to be tested can be verified in different data transmission directions.

Correspondingly, the reference model 605 at least includes a first theoretical model and a second theoretical model, and the verification platform obtains a first expected value through the first theoretical model and a second expected value through the second theoretical model. The first theoretical model realizes the processing process which is the same as the processing process for obtaining the first receiving value by the design to be tested, and the second theoretical model realizes the processing process which is the same as the processing process for obtaining the second receiving value by the design to be tested. That is, the first theoretical value is a value that should be theoretically output when the design to be measured processes external data, and the second theoretical value is a value that should be theoretically output when the design to be measured processes the target signal. Then, the score board 606 is used to compare the first received value with the first expected value, or compare the second received value with the second expected value, and the verification result is determined according to the comparison result.

According to the technical scheme of the embodiment of the disclosure, data receiving and sending are realized by driving and monitoring the corresponding target signal interface bus and the data transmission interface bus by utilizing the target signal interface bus agent and the data transmission interface bus agent. And by driving the target signal interface bus and the data transmission interface bus, excitation can be applied to the design to be tested from two data transmission directions, so that bidirectional verification is realized. In addition, the interface bus can be flexibly replaced according to different designs to be tested or different excitation signals. Therefore, the embodiment of the disclosure not only can effectively verify the functions and performances of the to-be-tested designs such as signal processing systems, but also is suitable for various to-be-tested designs with different working modes, and has stronger universality.

Fig. 7 is an architecture diagram of a verification platform of a design under test according to an embodiment of the disclosure, which is further optimized based on the above embodiment. As shown in fig. 7, the design under test is connected with a target signal interface bus, a data transmission interface bus and a register interface bus, and the verification platform of the design under test includes a configuration model 702, a reference model 707 and a score board 708. The details of the configuration model 702, the reference model 707 and the score board 708 are the same as those of the above embodiments, and are not described herein again.

The verification platform of the disclosed embodiment also includes a data transmission interface bus master 703, a data transmission interface bus slave 704, a target signal interface bus master 705, and a target signal interface bus slave 706. The target signal interface bus master agent 705 and the target signal interface bus slave agent 706 are used for responding to the enable determined by the verification platform according to the current working mode of the design to be tested, and implementing data receiving and sending with the design to be tested 701, and include sending signals to the design to be tested 701 by driving the target signal interface bus and receiving data processed by the design to be tested 701 by monitoring the target signal interface bus. Correspondingly, the data transmission interface bus master agent 703 and the data transmission interface bus slave agent 704 are used for responding to the enable determined by the verification platform according to the current working mode of the design to be tested, and implementing data transceiving with the design to be tested 701, including sending data to the design to be tested 701 by driving the data transmission interface bus, and receiving data processed by the design to be tested 701 by monitoring the data transmission interface bus. For a target signal interface bus, when the current working mode of the design to be tested 701 is a master mode, the verification platform enables the target signal interface bus slave agent 706 to realize data transceiving, and when the current working mode of the design to be tested 701 is a slave mode, enables the target signal interface bus master agent 705 to realize data transceiving; for the data transmission interface bus, when the current working mode of the design to be tested 701 is the master mode, the verification platform enables the data transmission interface bus slave agent 704 to implement data transceiving, and when the current working mode of the design to be tested 701 is the slave mode, the data transmission interface bus master agent 703 is enabled to implement data transceiving. Also, the input data as the source data and the received value as the result data obtained after the design under test processing are sent to the memory model 711 to be stored.

The score calculating board 708 acquires the received value of the design to be tested through the memory model 710, acquires the expected value calculated by the theoretical model in the received value through the reference model 707, and obtains a verification result by comparing the received value with the expected value. Moreover, the scoring board 708 may obtain the current working mode of the design to be tested 701 through the configuration model 702, and when the current working mode of the design to be tested 701 is the transparent transmission mode, the received value is the expected value.

In addition, the verification platform of the embodiment of the present disclosure further includes:

a register model 709 for mirroring a register of the design to be tested and obtaining a configuration matrix through the configuration model 702;

the register interface bus agent 710 is configured to read the configuration matrix in the register model 709, and drive the register interface bus to send the configuration matrix to the design to be tested 701, so as to configure the working mode of the design to be tested 701.

The register interface bus agent 710 is further configured to monitor change data in a register of the design to be tested 701 through the register interface bus, and write the change data into the register model 709.

Specifically, the configuration matrix is set in advance through the configuration model 702 according to the verification requirement, and the configuration matrix is written into the register model 709, written into the design to be tested 701 through the register interface bus agent 710 and the register interface bus, and configures the register in the design to be tested 701, so as to implement the configuration of the working mode of the design to be tested. In the working process of the design to be tested 701, the changed data in the register can be obtained through the register interface bus agent 710 in reverse, and is written into the register group model 709, so that the register model 709 can mirror the register in the design to be tested 701 in real time, and thus the reference model 707 can obtain the real-time working mode of the design to be tested 701 through the register model 709, and perform the same processing procedure as the current processing procedure of the design to be tested 701 according to the working mode to obtain the corresponding theoretical expected value of data processing, thereby improving the accuracy of the result of comparing the received value with the expected value by the score board 708, and further improving the accuracy of the verification of the design to be tested.

The working modes of the design to be tested in the embodiment of the present disclosure may include a master mode, a slave mode, a transceiving mode, a single channel mode, a multi-channel mode, and the like. Different working modes can be correspondingly arranged for different designs to be tested, and the configuration can be carried out according to the characteristics and the verification requirements of the designs to be tested, which is not limited in any way by the embodiment of the disclosure.

According to the technical scheme of the embodiment of the disclosure, different working modes of the design to be tested can be configured through the configuration model, and the register in the design to be tested is configured through the register model and the register interface bus agent, so that the configuration of the working modes of the design to be tested is realized, and the register model can accurately mirror the register in the design to be tested. On one hand, the verification platform can enable the target signal interface bus master agent or slave agent and the data transmission interface bus master agent or slave agent in real time according to the current working mode of the design to be tested, so that the precise control of signal transmission and signal reception between the verification platform and the design to be tested can be realized. On the other hand, the reference model can also accurately simulate the data processing process of the design to be tested according to the working mode of the design to be tested through the register model, so that the verification result is determined through the scoring board, and the verification accuracy is improved. In addition, each bus interface can be flexibly replaced according to the design to be tested. Therefore, the embodiment of the disclosure not only can effectively verify the function and performance of the design to be tested, but also is suitable for various designs to be tested with different working modes, and has stronger universality. Particularly, in the case where the design to be measured is a signal processing system in an artificial intelligence chip, the signal processing system needs to process various signals, and therefore, it is particularly important to provide a more general verification platform.

Fig. 8 is a schematic structural diagram of a verification apparatus for a design to be tested according to an embodiment of the present disclosure, which is applicable to a situation of verifying a design to be tested in an artificial intelligence chip, especially a situation of verifying a signal processing system in an artificial intelligence chip, and relates to the technical field of artificial intelligence chips, especially to the field of chip verification. The device can realize the verification method of the design to be tested in any embodiment of the disclosure. As shown in fig. 8, the apparatus 800 specifically includes:

a configuration matrix obtaining module 801, configured to obtain a configuration matrix, where the configuration matrix is used to record combinations of configuration options of different working modes of a design to be tested;

a working mode determining module 802, configured to determine a current working mode of the design to be tested according to the configuration matrix;

and the verification module 803 is configured to verify the design to be tested according to the current working mode of the design to be tested.

Optionally, the configuration matrix obtaining module 801 is specifically configured to:

and acquiring the configuration matrix through a configuration model, wherein the configuration model is used for configuring the configuration matrix.

Optionally, the verification module 803 includes:

the first enabling unit is used for enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the target signal interface bus agent is used for sending a target signal to the design to be tested, and the data transmission interface bus agent is used for receiving a first receiving value of the target signal after the target signal is processed by the design to be tested;

and the first verification unit is used for comparing the first receiving value with a first expected value through a scoring board according to the current working mode of the design to be tested, and determining a verification result of the design to be tested according to the comparison result, wherein the first expected value is obtained after the target signal is processed by a first theoretical model, and the first theoretical model realizes the same processing process as the first receiving value obtained by the design to be tested.

Optionally, the verification module 803 includes:

the second enabling unit is used for enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the data transmission interface bus agent is used for sending external data to the design to be tested, and the target signal interface bus agent is used for receiving a second receiving value of the external data processed by the design to be tested;

and the second verification unit is used for comparing the second receiving value with a second expected value through a score board according to the current working mode of the design to be tested and determining a verification result of the design to be tested according to the comparison result, wherein the second expected value is obtained after the external data is processed by a second theoretical model, and the second theoretical model realizes the same processing process as the second receiving value obtained by the design to be tested.

Optionally, the target signal interface bus agent includes a target signal interface bus master agent and a target signal interface bus slave agent, and the data transmission interface bus agent includes a data transmission interface bus master agent and a data transmission interface bus slave agent.

Optionally, the first enabling unit and the second enabling unit are specifically configured to:

for a target signal interface bus connected with the design to be tested, enabling the target signal interface bus slave agent if the current working mode of the design to be tested is the master mode, and enabling the target signal interface bus master agent if the current working mode of the design to be tested is the slave mode;

and aiming at a data transmission interface bus connected with the design to be tested, if the current working mode of the design to be tested is a master mode, enabling the slave agent of the data transmission interface bus, and if the current working mode of the design to be tested is a slave mode, enabling the master agent of the data transmission interface bus.

Optionally, the apparatus further comprises:

a configuration matrix writing module, configured to write the configuration matrix into a register model, where the register model is used to mirror the register of the design to be tested;

and the working mode configuration module is used for enabling a register interface bus agent to send the configuration matrix in the register model to the design to be tested so as to configure the working mode of the design to be tested.

Optionally, the apparatus further comprises:

and the change data writing module is used for monitoring the change data in the register to be tested through the register interface bus agent and writing the change data into the register model.

Optionally, the working mode determining module 802 is specifically configured to:

and determining the current working mode of the design to be tested according to the configuration matrix and the change data in the register model.

Optionally, the apparatus further comprises:

the first storage module is used for storing the target signal, the external data, the first receiving value and the first expected value into a memory model.

Optionally, the apparatus further comprises:

and the second storage module is used for storing the target signal, the external data, the second receiving value and the second expected value into a memory model.

Optionally, the operating mode includes a master mode, a slave mode, a transceiving mode, a single channel mode, or a multi-channel mode.

Optionally, the design to be tested is a signal processing system in an artificial intelligent chip.

The product can execute the method provided by any embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the execution method.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901 which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 901 performs the respective methods and processes described above, such as the verification method of the design under test. For example, in some embodiments, the verification method of the design under test may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the verification method of the design under test described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the verification method of the design under test in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome. The server may also be a server of a distributed system, or a server incorporating a blockchain.

Artificial intelligence is the subject of research that causes computers to simulate certain human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge map technology and the like.

Cloud computing (cloud computing) refers to a technology system that accesses a flexibly extensible shared physical or virtual resource pool through a network, where resources may include servers, operating systems, networks, software, applications, storage devices, and the like, and may be deployed and managed in a self-service manner as needed. Through the cloud computing technology, high-efficiency and strong data processing capacity can be provided for technical application and model training of artificial intelligence, block chains and the like.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions provided by the present disclosure can be achieved, which is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A verification method of a design under test includes:

acquiring a configuration matrix, wherein the configuration matrix is used for recording the combination of configuration options of different working modes of a design to be tested; the design to be tested is a signal processing system in an artificial intelligent chip;

verifying the design to be tested according to the current working mode of the design to be tested, comprising:

enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the target signal interface bus agent is used for sending a target signal to the design to be tested, and the data transmission interface bus agent is used for receiving a first receiving value of the target signal after the target signal is processed by the design to be tested;

according to the current working mode of the design to be tested, comparing the first receiving value with a first expected value through a scoring board, and determining a verification result of the design to be tested according to the comparison result, wherein the first expected value is obtained after the target signal is processed by a first theoretical model, and the first theoretical model realizes the same processing process as the first receiving value obtained by the design to be tested;

alternatively, the first and second electrodes may be,

enabling a target signal interface bus agent and a data transmission interface bus agent according to the current working mode of the design to be tested, wherein the data transmission interface bus agent is used for sending external data to the design to be tested, and the target signal interface bus agent is used for receiving a second receiving value of the external data after the external data is processed by the design to be tested;

and comparing the second receiving value with a second expected value through a scoring board according to the current working mode of the design to be tested, and determining a verification result of the design to be tested according to the comparison result, wherein the second expected value is obtained after the external data is processed by a second theoretical model, and the second theoretical model realizes the same processing process as the second receiving value obtained by the design to be tested.

2. The method of claim 1, wherein the obtaining a configuration matrix comprises:

3. The method of claim 1, wherein the target signal interface bus agents comprise a target signal interface bus master agent and a target signal interface bus slave agent, and the data transfer interface bus agents comprise a data transfer interface bus master agent and a data transfer interface bus slave agent.

4. The method of claim 3, wherein enabling the target signal interface bus agent and the data transfer interface bus agent according to the current operating mode of the design under test comprises:

for a target signal interface bus connected with the design to be tested, if the current working mode of the design to be tested is a master mode, enabling the target signal interface bus slave agent, and if the current working mode of the design to be tested is a slave mode, enabling the target signal interface bus master agent;

and aiming at a data transmission interface bus connected with the design to be tested, if the current working mode of the design to be tested is a master mode, enabling the data transmission interface bus slave agent, and if the current working mode of the design to be tested is a slave mode, enabling the data transmission interface bus master agent.

5. The method of claim 1, further comprising:

writing the configuration matrix into a register model, wherein the register model is used for mirroring the register of the design to be tested;

enabling a register interface bus agent to send a configuration matrix in the register model to the design to be tested so as to configure the working mode of the design to be tested.

6. The method of claim 5, further comprising:

and monitoring the change data in the register to be tested by the register interface bus agent, and writing the change data into the register model.

7. The method of claim 6, wherein said determining the current operating mode of the design under test from the configuration matrix comprises:

8. The method of claim 1, further comprising:

storing the target signal, the first received value, and the first expected value to a memory model.

9. The method of claim 1, further comprising:

storing the external data, the second received value, and the second expected value to a memory model.

10. The method of claim 1, wherein the operating mode comprises a master mode, a slave mode, a transceive mode, a single channel mode, or a multi-channel mode.

11. A verification platform for a design under test, comprising:

the target signal interface bus agent is used for responding to the enabling of the verification platform determined according to the current working mode of the design to be tested, sending a target signal to the design to be tested and receiving a first receiving value of external data processed by the design to be tested; the design to be tested is a signal processing system in an artificial intelligent chip;

the data transmission interface bus agent is used for responding to the enabling of the verification platform determined according to the current working mode of the design to be tested, sending the external data to the design to be tested and receiving a second receiving value of the target signal after the target signal is processed by the design to be tested;

12. A verification platform according to claim 11, wherein said design under test has a target signal interface bus and a data transfer interface bus connected thereto; accordingly, the method can be used for solving the problems that,

the target signal interface bus agent is specifically used for responding to the enabling determined by the verification platform according to the current working mode of the design to be tested, sending the target signal to the design to be tested by driving the target signal interface bus, and receiving a first receiving value of the external data processed by the design to be tested by monitoring the target signal interface bus;

the data transmission interface bus agent is specifically configured to respond to enabling determined by the verification platform according to the current working mode of the design to be tested, send the external data to the design to be tested by driving the data transmission interface bus, and receive a second receiving value of the target signal after being processed by the design to be tested by monitoring the data transmission interface bus.

13. A verification platform according to claim 11 wherein said reference model comprises at least a first theoretical model and a second theoretical model; accordingly, the method can be used for solving the problems that,

the reference model is specifically used for obtaining the first expected value through the first theoretical model and the second expected value through the second theoretical model according to the current working mode of the design to be tested;

the first theoretical model realizes the same processing procedure with the first receiving value obtained by the design to be tested, and the second theoretical model realizes the same processing procedure with the second receiving value obtained by the design to be tested.

14. A verification platform according to claim 11, wherein said design under test is further connected to a register interface bus; correspondingly, the verification platform further comprises:

the register model is used for mirroring the register of the design to be tested and acquiring the configuration matrix through the configuration model;

and the register interface bus agent is used for reading the configuration matrix in the register model and driving the register interface bus to send the configuration matrix to the design to be tested so as to configure the working mode of the design to be tested.

15. A validation platform according to claim 14, wherein,

the register interface bus agent is also used for monitoring the change data in the register of the design to be tested through the register interface bus and writing the change data into the register model.

16. A verification platform as claimed in claim 11 wherein said target signal interface bus agents include a target signal interface bus master agent and a target signal interface bus slave agent, and said data transfer interface bus agents include a data transfer interface bus master agent and a data transfer interface bus slave agent.

17. The validation platform of claim 16, wherein,

the target signal interface bus master agent is used for responding to the enabling of the verification platform when the current working mode of the design to be tested is a slave mode aiming at the target signal interface bus;

the target signal interface bus slave agent is used for responding to the enabling of the verification platform aiming at the target signal interface bus under the condition that the current working mode of the design to be tested is a master mode;

the data transmission interface bus master agent is used for responding to the enabling of the verification platform when the current working mode of the design to be tested is a slave mode aiming at the data transmission interface bus;

and the data transmission interface bus slave agent is used for responding to the enabling of the verification platform aiming at the data transmission interface bus under the condition that the current working mode of the design to be tested is a master mode.

18. The verification platform of claim 11, further comprising:

a memory model for storing the target signal, external data, a first received value, a first expected value, a second received value, and a second expected value.

19. A verification platform according to claim 14, wherein said read/write operations to said register of said design under test are implemented by front door access.

20. The authentication platform of claim 11, wherein the operational mode comprises a master mode, a slave mode, a transceive mode, a single channel mode, or a multi-channel mode.

21. A verification apparatus for a design under test, comprising:

the device comprises a configuration matrix acquisition module, a configuration matrix acquisition module and a configuration matrix generation module, wherein the configuration matrix acquisition module is used for acquiring a configuration matrix, and the configuration matrix is used for recording the combination of configuration options of different working modes of a design to be tested; the design to be tested is a signal processing system in an artificial intelligent chip;

the verification module is used for verifying the design to be tested according to the current working mode of the design to be tested;

wherein the verification module comprises:

the first verification unit is used for comparing the first receiving value with a first expected value through a scoring board according to the current working mode of the design to be tested, and determining a verification result of the design to be tested according to the comparison result, wherein the first expected value is obtained after the target signal is processed by a first theoretical model, and the first theoretical model realizes the same processing process as the first receiving value obtained by the design to be tested;

22. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of validating a design under test of any one of claims 1-10.

23. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of verifying a design under test according to any one of claims 1 to 10.