WO2020034116A1

WO2020034116A1 - Verification method for ai calculation results, and related products

Info

Publication number: WO2020034116A1
Application number: PCT/CN2018/100626
Authority: WO
Inventors: 牛昕宇; 蔡权雄
Original assignee: 深圳鲲云信息科技有限公司
Priority date: 2018-08-15
Filing date: 2018-08-15
Publication date: 2020-02-20
Also published as: CN110036367A

Abstract

Provided are a verification method for AI calculation results, and related products. The method comprises the following steps: reading input data and parameter data of an n-th layer of AI calculation from storage, and inputting the input data and the parameter data into a calculation engine to perform AI calculation so as to obtain an AI calculation result; extracting a reference result from the storage, compressing the reference result to obtain a reference digest value of a set length, and compressing the AI calculation result to obtain a calculation digest value of a set length, and then inputting the reference digest value and the calculation digest value into a comparison unit; and the comparison unit comparing the reference digest value with the calculation digest value, and if they are consistent, determining that the nth layer of verification is passed, and if they are not consistent, determining that the nth layer of verification fails, and sending alarm information. The present application has the advantage of a high efficiency.

Description

Method for verifying AI operation results and related products

Technical field

The present application relates to the field of computers and artificial intelligence technologies, and in particular, to a method for verifying the results of AI operations and related products.

Background technique

In recent years, the continuous explosion of AI has been inseparable from the development of device computing capabilities and deep learning network structures. When the calculation of the entire network uses floating-point calculation, it will cause a large computational dependency on the CPU. If the data can be converted from floating-point to fixed-point values, and fixed-point calculations can be processed in parallel on hardware devices, network computing capabilities can be improved.

In hardware design, it is often necessary to verify whether a certain network is satisfied, and reference data and reference parameters need to be input into the network for calculation to obtain actual results. Compare the actual results with the reference results one by one. In order to determine whether the hardware meets a certain network calculation. The existing one-to-one comparison method has low efficiency and long time for verifying AI operation results.

Application content

The embodiments of the present application provide a verification of an AI operation result and a related product, which implements verification of an AI operation result by using a digest value, thereby speeding up comparison speed and improving efficiency.

In a first aspect, an embodiment of the present application provides a method for verifying an AI calculation result. The method includes the following steps:

Read the input data and parameter data of the AI calculation of the nth layer from the storage, and input the input data and parameter data to the calculation engine to perform the AI calculation to obtain the AI calculation result;

Extract the reference result from the storage, compress the reference result to obtain the reference digest value of the set length, compress the AI calculation result to obtain the calculated digest value of the set length, and input the reference digest value and the calculated digest value to Comparison unit

The comparison unit compares whether the reference digest value and the calculated digest value are consistent. If they are the same, it is determined that the n-th layer verification is passed. If they are not consistent, it is determined that the n-th layer verification is not passed, and an alarm message is issued.

Optionally, when the inconsistency is determined, the method further includes:

The reference digest value is stored in the memory, and when the calculation result of the nth layer AI is verified again, the reference digest value is directly obtained.

Optionally, when the inconsistency is determined, the method further includes:

A first area having the same value as the reference digest value and a second area having a different value for the calculated digest value are obtained, and a problem existing in the calculation engine is determined according to the analysis of the first area and the second area, and the problem is displayed.

Optionally, the compression algorithm is an MD5 compression algorithm.

In a second aspect, a verification system for AI calculation results is provided. The system includes: a storage, data extraction unit, a calculation engine, and a comparison unit.

A data extraction unit, configured to read input data and parameter data of the AI calculation of the nth layer from the storage, and input the input data and parameter data to the calculation engine;

The calculation engine is configured to perform AI calculation on the input data and parameter data to obtain an AI calculation result;

The data extraction unit is further configured to extract a reference result from the storage, compress the reference result to obtain a reference digest value of a set length, and compress the AI calculation result to obtain a calculated digest value of the set length, and refer to the digest The value and calculation summary value are input to the comparison unit;

The comparison unit is configured to compare whether the reference digest value and the calculated digest value are consistent. If they are consistent, it is determined that the n-th layer verification is passed. If they are not consistent, it is determined that the n-th layer verification is not passed, and an alarm message is issued.

Optionally, when inconsistencies are determined,

The data extraction unit is further configured to store the reference digest value in a memory, and directly obtain the reference digest value when verifying an n-th layer AI calculation result again.

Optionally, when the inconsistency is determined, the system further includes: an analysis unit,

The analysis unit is configured to obtain a first area of the same value of the reference digest value and a second area of the calculated digest value, and a second area of a different value, and determine a problem in the calculation engine according to the first area and the second area analysis, Show the problem.

Optionally, the compression algorithm is an MD5 compression algorithm.

In a third aspect, a computer-readable storage medium is provided that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the method as provided in the second aspect.

In a fourth aspect, a computer program product is provided. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the method provided in the second aspect.

The implementation of the embodiments of the present application has the following beneficial effects:

It can be seen that after obtaining the input data and parameter data in the technical solution provided by this application, the two data are input into the AI calculation engine for calculation to obtain the AI calculation result. After extracting the reference result, the two results are compressed to obtain two summary values. The two summary values are compared to determine whether the verification is successful. Since the comparison does not require comparison of each element value, it has the advantages of reducing the amount of comparison data, improving the comparison efficiency, and reducing the comparison time.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. Those of ordinary skill in the art can obtain other drawings according to the drawings without paying creative labor.

Figure 1 is a schematic diagram of the structure of an AI system.

FIG. 2 is a schematic flowchart of a method for verifying an AI calculation result.

FIG. 3 is a schematic flowchart of another method for verifying an AI calculation result of the present application.

FIG. 4 is a structural diagram of a verification system for an AI calculation result provided by the present application.

detailed description

In the following, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The terms "including" and "having" in the specification and claims of the present application and the accompanying drawings, as well as any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device containing a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

Reference to "an embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they independent or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The electronic device in this application may include: a server, a smart camera device, a smart phone (such as an Android phone, an iOS phone, a Windows Phone phone, etc.), a tablet computer, a handheld computer, a laptop computer, and a mobile Internet device (MID, Mobile Internet Devices) Or wearable devices, etc., the above electronic devices are merely examples, not exhaustive, and include but are not limited to the above electronic devices. For convenience of description, the above electronic devices are referred to as user equipment (UE), Terminal or electronic device. Of course, in practical applications, the above-mentioned user equipment is not limited to the above-mentioned realization form, and may include, for example, a smart vehicle terminal, a computer device, and the like.

Please refer to FIG. 1, which is a schematic diagram of an AI system. As shown in Figure 1, the system includes storage, a computing engine (with detection hardware), and a CPU.

The storage part includes reference data, reference parameters, and reference results of the network model. The reference data may specifically be AI input data, AI weight data, and so on.

The function of the hardware calculation engine is equivalent to the AI network accelerator, which can be implemented by FPGA devices, GPUs or ASICs.

The comparison unit (CPU) reads the values from the two files of the reference result and the actual result, and compares each value. During each network test, the comparison value reached tens of millions. Through comparison unit improvement and optimization, the result file is pre-processed. Both the reference result and the actual result are compressed into a 1024bits summary value by the compression module. In this way, only the two 1024bits digest values need to be compared, and the two 1024bits digest values can be compared to realize the correctness of the AI calculation result.

The verification of the AI calculation results is generally the verification of the AI calculation engine. If the AI calculation engine is modified, it may have deviations in the calculation results. For example, in a possible case, if the AI calculation has 10 steps There are only 9 steps in the AI calculation engine, so the calculation results will have a certain deviation, so after the AI calculation engine changes, a large deviation will occur, so the AI calculation results need to be verified.

Referring to FIG. 2, FIG. 2 is a method for verifying an AI calculation result. The method is implemented in the system shown in FIG. 1. The method shown in FIG. 2 includes the following steps:

Step S201: Read the input data and parameter data of the AI calculation of the nth layer from the storage, and input the input data and parameter data to the calculation engine to execute the AI calculation to obtain the AI calculation result;

Step S202: Extract a reference result from the storage, compress the reference result to obtain a reference digest value of a set length, compress the AI calculation result to obtain a calculated digest value of a set length, and compare the reference digest value with the calculated digest. Enter the value into the comparison unit;

Optionally, the algorithm for the compression processing in step S202 may specifically be an MD5 (MD5 Message-Digest Algorithm) version 5 compression algorithm. Of course, in actual applications, other compression algorithms may also be used. This application It does not limit the specific expression of the above compression algorithm. The above-mentioned set length includes, but is not limited to, values of 128 bits, 256 bits, 1024 bits, and the like.

Step S203: The comparison unit compares whether the reference digest value and the calculated digest value are consistent. If they are consistent, it is determined that the n-th level verification is passed. If they are not consistent, it is determined that the n-th level verification is not passed, and an alarm message is issued.

After obtaining the input data and parameter data in the technical solution provided by the present application, the two data are input into the AI calculation engine for calculation to obtain the AI calculation result. After extracting the reference result, the two results are compressed to obtain two summary values. The summary values are compared to determine whether the verification is successful. Since the comparison does not require individual element values for comparison, it has the advantages of reducing the amount of comparison data, improving the comparison efficiency, and reducing the comparison time.

Optionally, when the above method is determined to be inconsistent, the method may further include:

In this method, the reference digest value is stored, so that in the subsequent comparison process, the reference value does not need to be compressed, and the reference digest value can be directly extracted, which saves the calculation overhead.

This technical solution is to determine the problems of the computing engine based on the first region and the second region, because for the computing engine, if one of its steps is missing, its AI calculation results will have a certain regular behavior, and the missing steps Different, the regularity of the AI calculation results may be different, then by analyzing the first region and the second region, you can roughly obtain the steps (that is, problems) that the computing engine lacks, so as to optimize for subsequent debugging, such as in In the convolution calculation, if there are no subsequent steps, such as the pooling step, the AI calculation results will be the same and different periodically. Then, determining the corresponding first region and the second region can determine that the calculation engine lacks the corresponding step.

Referring to FIG. 3, FIG. 3 is a method for verifying an AI calculation result provided by an embodiment of the present application. The method includes the following steps:

Step S301: Read reference data and reference parameters from the storage unit to the hardware engine

Step S302: Whether the calculation of all the layers of the network is finished. If the last layer, go to step S303. If it is not the last layer, go to step S304.

Step S303: Exit the network.

Step S304: the hardware engine calculates the AI layer

Step S305: the hardware stores the actual result and saves it to a file

Step S306: the CPU reads the reference result and performs preprocessing to obtain a digest value digest_ext

Step S307: The CPU reads the actual result and performs preprocessing to obtain the digest value digest_act

Step S308: Compare whether digest_ext and digest_act are the same. If they are the same, proceed to step S302. If they are not, proceed to step S309.

Step S309: alarm fail and exit the network.

Referring to FIG. 4, FIG. 4 provides an AI calculation result verification system. The system includes: a storage 401, a data extraction unit 402, a calculation engine 403, and a comparison unit 404.

The data extraction unit is further configured to extract a reference result from the storage, compress the reference result to obtain a reference digest value of a set length, and compress the AI calculation result to obtain a calculated digest value of a set length, and refer to the digest. The value and calculation summary value are input to the comparison unit;

The comparison unit is configured to compare whether the reference digest value is consistent with the calculated digest value. If the reference digest value is consistent, it is determined that the n-th layer verification is passed.

Optionally, when inconsistencies are determined,

Optionally, when the inconsistency is determined, the system further includes: an analysis unit 405,

Optionally, the compression algorithm is an MD5 compression algorithm.

An embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to perform verification of any one of the AI calculation results described in the foregoing method embodiments. Part or all of the steps of the method.

An embodiment of the present application further provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the operations described in the foregoing method embodiments. Part or all of the steps in any method of verifying AI calculation results.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all described as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action order. Because according to the present application, certain steps may be performed in another order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for this application.

In the above embodiments, the description of each embodiment has its own emphasis. For a part that is not described in detail in one embodiment, reference may be made to related descriptions in other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are merely schematic

In addition, the processors and chips in the various embodiments of the present application may be integrated in one processing unit, or may exist separately physically, or two or more pieces of hardware may be integrated in one unit. The computer-readable storage medium or computer-readable program may be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application essentially or part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory, Several instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. The foregoing memories include: U disks, Read-Only Memory (ROM), Random Access Memory (RAM), mobile hard disks, magnetic disks, or optical disks and other media that can store program codes.

A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be completed by a program instructing related hardware. The program may be stored in a computer-readable memory, and the memory may include a flash disk. , Read-only memory (English: Read-Only Memory, referred to as ROM), random access device (English: Random Access Memory, referred to as RAM), magnetic disks or optical disks, etc.

The embodiments of the present application have been described in detail above. Specific examples have been used in this document to explain the principles and implementation of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. Persons of ordinary skill in the art may change the specific implementation and application scope according to the idea of the present application. In summary, the content of this description should not be construed as a limitation on the present application.

Claims

A method for verifying an AI calculation result, characterized in that the method includes the following steps:

Read the input data and parameter data of the AI calculation of the nth layer from the storage, and input the input data and parameter data to the calculation engine to perform the AI calculation to obtain the AI calculation result;

Extract the reference result from the storage, compress the reference result to obtain the reference digest value of the set length, compress the AI calculation result to obtain the calculated digest value of the set length, and input the reference digest value and the calculated digest value to Comparison unit

The comparison unit compares whether the reference digest value and the calculated digest value are consistent. If they are consistent, it is determined that the n-th layer verification is passed. If they are not consistent, it is determined that the n-th level verification is not passed, and an alarm message is issued.
The method according to claim 1, wherein, when the inconsistency is determined, the method further comprises:

The reference digest value is stored in the memory, and when the calculation result of the nth layer AI is verified again, the reference digest value is directly obtained.
The method according to claim 1, wherein, when the inconsistency is determined, the method further comprises:

A first area having the same value as the reference digest value and a second area having a different value for the calculated digest value are obtained, and a problem existing in the calculation engine is determined according to the analysis of the first area and the second area, and the problem is displayed.
The method according to any one of claims 1-3, characterized in that:

The compression algorithm is an MD5 compression algorithm.
A verification system for AI calculation results, characterized in that the system includes: a storage, data extraction unit, a calculation engine, and a comparison unit, wherein:

A data extraction unit, configured to read input data and parameter data of the AI calculation of the nth layer from the storage, and input the input data and parameter data to the calculation engine;

The calculation engine is configured to perform AI calculation on the input data and parameter data to obtain an AI calculation result;

The data extraction unit is further configured to extract a reference result from the storage, compress the reference result to obtain a reference digest value of a set length, and compress the AI calculation result to obtain a calculated digest value of a set length, and refer to the digest. The value and calculation summary value are input to the comparison unit;

The comparison unit is configured to compare whether the reference digest value is consistent with the calculated digest value. If the reference digest value is consistent, it is determined that the n-th layer verification is passed.
The system according to claim 5, wherein, when the inconsistency is determined,

The data extraction unit is further configured to store the reference digest value in a memory, and directly obtain the reference digest value when verifying an n-th layer AI calculation result again.
The system according to claim 5, wherein when it is determined that the inconsistency, the system further comprises: an analysis unit,

The analysis unit is configured to obtain a first area of the same value of the reference digest value and a second area of the calculated digest value, and a second area of a different value, and determine a problem in the calculation engine according to the first area and the second area analysis, Show the problem.
The system according to any one of claims 5-7, wherein:

The compression algorithm is an MD5 compression algorithm.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the computer program according to any one of claims 1-4. Methods.
A computer program product, characterized in that the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of claims 1-4 The method described.