CN114356787A

CN114356787A - Automatic testing method and device for deep learning model compiler and storage medium

Info

Publication number: CN114356787A
Application number: CN202210270367.XA
Authority: CN
Inventors: 龚政; 欧阳鹏
Original assignee: Jiangsu Qingwei Intelligent Technology Co ltd
Current assignee: Jiangsu Qingwei Intelligent Technology Co ltd
Priority date: 2022-03-18
Filing date: 2022-03-18
Publication date: 2022-04-15
Anticipated expiration: 2042-03-18
Also published as: CN114356787B

Abstract

The invention discloses an automatic testing method and device for a deep learning model compiler and a storage medium. The automatic testing method of the deep learning model compiler comprises the following steps: determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required for creating a single operator test model; analyzing the case configuration file by using a preset test frame, and determining the model frame type corresponding to the case configuration file; generating an offline model file associated with the model framework type according to the use case configuration file; and carrying out deep learning model compiler test corresponding to the model framework type through the offline model file.

Description

Automatic testing method and device for deep learning model compiler and storage medium

Technical Field

The invention relates to the technical field of automatic testing, in particular to an automatic testing method and device of a deep learning model compiler and a storage medium.

Background

In the testing process of the deep learning model compiler, the most basic testing coverage is to test the correctness of the compiled and post-deployment calculation of the general operators. Usually, when an operator is tested to cover, the following parts are covered: a matrix dimension of an input of an operator; inputting a matrix shape by an operator; the parameters in the calculation formulas of the operators. Usually, hundreds of cases need to be constructed when performing a parameter combination coverage test on operators with more parameters such as convolution, pooling and the like. Other operators without parameters, which only need to cover the dimension of the input matrix and the shape of the input matrix, also need to design 20+ use cases. Usually, tens of thousands of use cases need to be constructed when a compiler covers a single operator use case, and more use cases need to be constructed for a multi-layer cascading test scenario. In some existing AI algorithm models with large use amount, about 50 basic operators exist, and about 1000 full operators exist. A large number of use cases need to be built to enable full testing of the compiler. The existing test scheme has the following disadvantages: 1. the process of building the model is complex and time-consuming. Each use case requires the addition of code to create a corresponding test model. For some complex cascaded operator scenes, scripts are complex to write, and large-scale creation testing is not facilitated. 2. The repetitive workload is large. Operators with the same deep learning frames, such as Caffe, Tensorflow, Pyorch and the like, need to independently create the generated model script due to the inconsistency of parameter names, and further increase the workload. 3. The use case management and maintenance efficiency is low. The parameters and configuration covered by the test case in the technical scheme are written in the code. If centralized unified management is needed, the covering condition of each use case for the parameters needs to be recorded in detail in a form or text recording mode, and corresponding workload is increased.

Aiming at the technical problems of complex and time-consuming process, large workload and low management and maintenance efficiency of use cases of the existing deep learning model compiler test scheme in the prior art, an effective solution is not provided at present.

Disclosure of Invention

The embodiment of the disclosure provides an automatic testing method and device for a deep learning model compiler and a storage medium, so as to at least solve the technical problems of complex and time-consuming process, large workload and low management and maintenance efficiency of use cases in the existing deep learning model compiler testing scheme in the prior art.

According to an aspect of the embodiments of the present disclosure, there is provided an automated testing method for a deep learning model compiler, including: determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required for creating a single operator test model; analyzing the case configuration file by using a preset test frame, and determining the model frame type corresponding to the case configuration file; generating an offline model file associated with the model framework type according to the use case configuration file; and carrying out deep learning model compiler test corresponding to the model framework type through the offline model file.

According to another aspect of the embodiments of the present disclosure, there is also provided a storage medium including a stored program, wherein the method of any one of the above is performed by a processor when the program is executed.

According to another aspect of the embodiments of the present disclosure, there is also provided an automated testing apparatus for a deep learning model compiler, including: the system comprises a first determination model, a second determination model and a third determination model, wherein the first determination model is used for determining a use case configuration file according to a preset data structure and input information of a user, and the data structure is used for storing parameter information required by creating a single operator test model; the second determining module is used for analyzing the case configuration file by using a preset test frame and determining the model frame type corresponding to the case configuration file; the generating module is used for generating an offline model file associated with the model framework type according to the use case configuration file; and the testing module is used for testing the deep learning model compiler corresponding to the model framework type through the offline model file.

According to another aspect of the embodiments of the present disclosure, there is also provided an automated testing apparatus for a deep learning model compiler, including: a processor; a memory coupled to the processor for providing instructions to the processor for processing the following processing steps: determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required for creating a single operator test model; analyzing the case configuration file by using a preset test frame, and determining the model frame type corresponding to the case configuration file; generating an offline model file associated with the model framework type according to the use case configuration file; and carrying out deep learning model compiler test corresponding to the model framework type through the offline model file.

In the embodiment of the disclosure, the code development workload can be reduced by means of configuration files in the deep learning model compiler test process, so that the creation of the use case is more efficient and simpler. The construction process of the test case input model is simplified, and the case writing efficiency is improved. A set of use cases can be adapted to a plurality of deep learning frames, and each use case does not need to be modified and adapted again. And optimizing a use case management mode. The use cases can be managed in batches more conveniently and the use cases can be cut more conveniently. And the technical problems that the existing deep learning model compiler test scheme in the prior art is complex in process, time-consuming, large in workload and low in management and maintenance efficiency of use cases are solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and together with the description serve to explain the disclosure and not to limit the disclosure. In the drawings:

fig. 1 is a hardware block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure;

FIG. 2 is a flowchart of a method for automated testing of a deep learning model compiler according to a first aspect of embodiment 1 of the present disclosure;

fig. 3A is another schematic flowchart of an automated testing method of a deep learning model compiler according to embodiment 1 of the present disclosure;

FIG. 3B is a diagram of a model structure of an offline model file according to embodiment 1 of the present disclosure;

FIG. 4 is a schematic flowchart of an offline model file birthday according to embodiment 1 of the present disclosure;

FIG. 5 is a schematic diagram of an automated testing apparatus of a deep learning model compiler according to embodiment 2 of the present disclosure;

fig. 6 is a schematic diagram of an automated testing apparatus of a deep learning model compiler according to embodiment 3 of the present disclosure.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. It is to be understood that the described embodiments are merely exemplary of some, and not all, of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

According to the present embodiment, there is also provided an automated testing method embodiment of a deep learning model compiler, it is noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.

The method embodiments provided by the present embodiment may be executed in a mobile terminal, a computer terminal, a server or a similar computing device. FIG. 1 illustrates a block diagram of a hardware architecture of a computing device for implementing an automated testing method for a deep learning model compiler. As shown in fig. 1, the computing device may include one or more processors (which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, and a transmission device for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computing device may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computing device. As referred to in the disclosed embodiments, the data processing circuit acts as a processor control (e.g., selection of a variable resistance termination path connected to the interface).

The memory may be configured to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the automated testing method of the deep learning model compiler in the embodiments of the present disclosure, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned automated testing method of the deep learning model compiler of the application program. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory located remotely from the processor, which may be connected to the computing device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device is used for receiving or transmitting data via a network. Specific examples of such networks may include wireless networks provided by communication providers of the computing devices. In one example, the transmission device includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computing device.

It should be noted here that in some alternative embodiments, the computing device shown in fig. 1 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that FIG. 1 is only one example of a particular specific example and is intended to illustrate the types of components that may be present in a computing device as described above.

In the above operating environment, according to the first aspect of the present embodiment, an automated testing method for a deep learning model compiler is provided. Fig. 2 shows a flow diagram of the method, which, with reference to fig. 3, comprises:

s202: determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required for creating a single operator test model;

s204: analyzing the case configuration file by using a preset test frame, and determining the model frame type corresponding to the case configuration file;

s206: generating an offline model file associated with the model framework type according to the use case configuration file;

s208: and carrying out deep learning model compiler test corresponding to the model framework type through the offline model file.

As described in the background, the existing test solutions suffer from the following disadvantages: 1. the process of building the model is complex and time-consuming. Each use case requires the addition of code to create a corresponding test model. For some complex cascaded operator scenes, scripts are complex to write, and large-scale creation testing is not facilitated. 2. The repetitive workload is large. Operators with the same deep learning frames, such as Caffe, Tensorflow, Pyorch and the like, need to independently create the generated model script due to the inconsistency of parameter names, and further increase the workload. 3. The use case management and maintenance efficiency is low. The parameters and configuration covered by the test case in the technical scheme are written in the code. If centralized unified management is needed, the covering condition of each use case for the parameters needs to be recorded in detail in a form or text recording mode, and corresponding workload is increased.

In view of this, an embodiment of the present application provides an automated testing method for a deep learning model compiler, where a computing device determines a use case configuration file according to a preset data structure and input information of a user, where the data structure is used to store parameter information required for creating a single operator test model (S202).

Specifically, the data structure stores key information such as operator input matrix shape information and operator parameters required for creating the single operator test model. And inputting parameters by a user through a preset data structure, wherein the parameters are parameter information of a deep learning model compiler which needs to be tested by the user, and then generating a use case configuration file.

In addition, as shown in fig. 3A, the use case configuration file may also be generated by traversing through a preset rule. And adding the parameters of the model/operator which needs to be tested by a person who needs to write the use case according to the traversal rule. For example: one model has two parameters a and b, with the value ranges True and False respectively. Then the person writing the use case only needs to add two rules to give parameter ranges range _ a = [ True, False ] range _ b = [ True, False ], and then the two parameter combinations are traversed to generate four use cases, namely { a: True, b: True } { a: False, b: False } { a: True, b: False } { a: False, b: True }. Thereby realizing the generation of the use case configuration file.

Further, the computing device analyzes the use case configuration file by using a preset test frame, and determines a model frame type corresponding to the use case configuration file (S204).

Specifically, the use case configuration file is analyzed through the test framework, and the parameter information of the use case configuration file is determined, so that the type of the model framework is determined. As shown therein with reference to fig. 3A, the model framework types may include, for example, tensrflow, Caffe, pitorch, ONNX, etc.

Further, the computing device generates an offline model file associated with the model framework type from the use case configuration file (S206). A deep learning model compiler test corresponding to the model framework type is then performed through the offline model file (S208).

Specifically, by generating an offline model file corresponding to the model framework type, the deep learning model compiler can be tested through the offline model file. Therefore, the automatic test of the deep learning model compiler is completed through the mode.

Therefore, by the mode, the code development workload can be reduced and the creation of the use case is more efficient and simpler by the mode of configuring the file in the test process of the deep learning model compiler. The construction process of the test case input model is simplified, and the case writing efficiency is improved. A set of use cases can be adapted to a plurality of deep learning frames, and each use case does not need to be modified and adapted again. And optimizing a use case management mode. The use cases can be managed in batches more conveniently and the use cases can be cut more conveniently. And the technical problems that the existing deep learning model compiler test scheme in the prior art is complex in process, time-consuming, large in workload and low in management and maintenance efficiency of use cases are solved.

Optionally, before determining the operation of the use case configuration file according to a preset data structure and input information of a user, the method further includes: a data structure is created, wherein the data structure is used to store parameter information required by the single operator test model.

Specifically, a unified data structure is defined to store key information such as operator input matrix shape information and operator parameters required by creating a single operator test model. Thus, each test case can be described by using the configuration file. Therefore, the test case writing efficiency can be greatly improved after the deep learning single operator test case configuration is documented and fully decoupled with codes.

Optionally, the method further comprises: and uniformly packaging the interfaces of the multiple deep learning models.

Specifically, the API interfaces of the creation model of each deep learning framework are uniformly packaged, so that the API interfaces can directly read the configuration files of the use cases to generate corresponding use cases. For example, in both the Caffe and Tensorflow deep learning frameworks, the convolution operator formula is the same, but the names of the same parameters are not the same in both frameworks. The difference can be shielded through uniform interface packaging, and the same use case configuration file can directly generate use case models of different depth learning frames. The model generation interface is further packaged on the basis of test case configuration documentation, so that differences among learning frames with different depths can be effectively shielded, and the effect of further increasing the test case construction efficiency is achieved.

In addition, by defining a data structure and packaging an interface, when a test case model is subsequently added, a configuration file of case creation in batches is needed, a single operator model needed by a corresponding case can be generated without developing new codes, and the workload of related development is reduced. Meanwhile, all use cases can be managed only by maintaining one set of configuration files after use case configuration and code decoupling. Only the specification rules are required to filter the configuration file for new verification requirements and scenarios, and no code modification or clipping is required.

Optionally, before generating the offline model file associated with the model framework type, the method further includes: and constructing a model structure of the offline model file, wherein the model structure is associated with the model framework type.

Specifically, referring to FIG. 3B, a configuration file structure of a use case model file is shown, thereby facilitating later creation of an offline file.

Optionally, the operation of constructing the model structure comprises: configuring the connection sequence of a plurality of operators in the deep learning model; configuring parameter values of a plurality of operators; and configuring matrix information of the model structure, wherein the matrix information comprises the number of the matrixes and the dimension of the matrixes.

Specifically, referring to fig. 3B, the first key configuration is to define the structure of the model to be constructed through the op _ list field, mainly the name of the operator and the connection order in the model. The second key configuration defines the parameters of each layer of operators in create _ params, and writes the corresponding values according to the encapsulated interface names. The third key configuration is to define the number of input matrices of the model and shape of each matrix. Thereby realizing the construction of the model structure by the above configuration.

Optionally, the operation of generating an offline model file associated with the model framework type includes: analyzing the use case configuration file, and determining data information corresponding to the use case configuration file, wherein the data information comprises the number of model input layers and specification information of the input layers; acquiring a creation interface of each input layer, and creating an operator; and connecting a plurality of operators corresponding to the plurality of input layers according to a preset model structure to determine an offline model file.

Specifically, referring to fig. 4, a flowchart for generating an offline model file is shown, and the specific steps are as follows:

1, the test framework reads the model configuration file.

And 2, analyzing the data _ num and input _ shape in the configuration file to obtain the number of the input layers of the model and the specification of the input layers.

And 3, traversing the op _ list, acquiring the API interface created by the corresponding operator through the operator name in the op _ list, and acquiring the parameters required by the corresponding API from the create _ params to create a corresponding operator instance.

And 4, connecting the input nodes and the output nodes of the operators of the operator instances created in the third step according to the corresponding sequence to generate a corresponding operator network according to the configured model structure. And finally, calling an interface corresponding to the deep learning framework to store the offline model.

Therefore, by the mode, the off-line model file corresponding to the model framework type can be generated, and the automatic test of the deep learning model compiler is realized.

Further, referring to fig. 1, according to a second aspect of the present embodiment, there is provided a storage medium. The storage medium comprises a stored program, wherein the method of any of the above is performed by a processor when the program is run.

Therefore, according to the embodiment, the code development workload can be reduced in a mode of configuring the file in the deep learning model compiler test process, and the creation of the use case is more efficient and simpler. The construction process of the test case input model is simplified, and the case writing efficiency is improved. A set of use cases can be adapted to a plurality of deep learning frames, and each use case does not need to be modified and adapted again. And optimizing a use case management mode. The use cases can be managed in batches more conveniently and the use cases can be cut more conveniently. And the technical problems that the existing deep learning model compiler test scheme in the prior art is complex in process, time-consuming, large in workload and low in management and maintenance efficiency of use cases are solved.

In addition, the invention also has the following technical effects:

1. the conventional common method for generating the model file is to generate a corresponding model by calling each API interface of each deep learning framework (coffee/tensorflow/restore/onnx) through a code, which is equivalent to that the code generated by the model needs to be increased every time a use case is added. By means of configuration files, the workload of code development can be reduced, and the creation of use cases is more efficient and simpler.

2. After the use case configuration is documented, more use cases can be created in a mode of generating the configuration files in batches through scripts, and the use case generation efficiency can be further improved.

3. And the use case configuration file is fully decoupled with the code, so that the use case can be managed more conveniently. For example, modification and cutting of use cases, transverse comparison and coverage analysis are more convenient.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Example 2

Fig. 5 shows an automated testing apparatus 500 of a deep learning model compiler according to the present embodiment, the apparatus 500 corresponding to the method according to the first aspect of embodiment 1. Referring to fig. 5, the apparatus 500 includes: a first determination model 510, configured to determine a use case configuration file according to a preset data structure and input information of a user, where the data structure is used to store parameter information required for creating a single operator test model; a second determining module 520, configured to analyze the case configuration file by using a preset test frame, and determine a model frame type corresponding to the case configuration file; a generating module 530, configured to generate an offline model file associated with the model framework type according to the use case configuration file; and the testing module 540 is used for performing deep learning model compiler testing corresponding to the model framework type through the offline model file.

Optionally, before determining the operation of the use case configuration file according to a preset data structure and input information of a user, the apparatus 500 further includes: and the creating module is used for creating a data structure, wherein the data structure is used for storing parameter information required by the single operator test model.

Optionally, the apparatus 500 further comprises: and the packaging module is used for uniformly packaging the interfaces of the multiple deep learning models.

Optionally, before generating the offline model file associated with the model framework type, the apparatus 500 further includes: and the construction module is used for constructing a model structure of the offline model file, wherein the model structure is associated with the model frame type.

Optionally, the building block comprises: the first configuration submodule is used for configuring the connection sequence of a plurality of operators in the deep learning model; the second configuration submodule is used for configuring parameter values of the operators; and the third configuration submodule is used for configuring matrix information of the model structure, wherein the matrix information comprises the matrix number and the matrix dimension.

Optionally, the generating module 530 includes: the first determining submodule is used for analyzing the use case configuration file and determining data information corresponding to the use case configuration file, wherein the data information comprises the number of model input layers and specification information of the input layers; the creating submodule is used for acquiring a creating interface of each input layer and creating an operator; and the second determining submodule is used for connecting a plurality of operators corresponding to the plurality of input layers according to a preset model structure to determine an offline model file.

Example 3

Fig. 6 shows an automated testing apparatus 600 of a deep learning model compiler according to the present embodiment, the apparatus 600 corresponding to the method according to the first aspect of embodiment 1. Referring to fig. 6, the apparatus 600 includes: a processor 610; a memory 620, coupled to the processor 610, for providing instructions to the processor 610 to process the following process steps: determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required for creating a single operator test model; analyzing the case configuration file by using a preset test frame, and determining the model frame type corresponding to the case configuration file; generating an offline model file associated with the model framework type according to the use case configuration file; and carrying out deep learning model compiler test corresponding to the model framework type through the offline model file.

Optionally, before determining the operation of the use case configuration file according to the preset data structure and the input information of the user, the memory 620 is further configured to provide the processor 610 with instructions for processing the following processing steps: a data structure is created, wherein the data structure is used to store parameter information required by the single operator test model.

Optionally, the memory 620 is further configured to provide the processor 610 with instructions to process the following processing steps: and uniformly packaging the interfaces of the multiple deep learning models.

Optionally, before generating the offline model file associated with the model framework type, the memory 620 is further configured to provide the processor 610 with instructions for processing the following processing steps: and constructing a model structure of the offline model file, wherein the model structure is associated with the model framework type.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present invention, it should be understood that the disclosed technical contents can be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An automated testing method for a deep learning model compiler, comprising:

determining a use case configuration file according to a preset data structure and input information of a user, wherein the data structure is used for storing parameter information required by creating a single operator test model;

analyzing the case configuration file by using a preset test frame, and determining a model frame type corresponding to the case configuration file;

generating an off-line model file associated with the model framework type according to the use case configuration file;

and performing deep learning model compiler test corresponding to the model framework type through the offline model file.

2. The method of claim 1, wherein before determining the use case configuration file according to a preset data structure and input information of a user, the method further comprises:

creating a data structure, wherein the data structure is used for storing parameter information required by the single operator test model.

3. The method of claim 1, further comprising:

and uniformly packaging the interfaces of the multiple deep learning models.

4. The method of claim 1, prior to generating an offline model file associated with the model framework type, further comprising:

and constructing a model structure of the offline model file, wherein the model structure is associated with the model framework type.

5. The method of claim 4, wherein the operation of constructing the model structure of the offline model file comprises:

configuring the connection sequence of a plurality of operators in the deep learning model;

configuring parameter values of the plurality of operators;

and configuring matrix information of the model structure, wherein the matrix information comprises the number of the matrixes and the dimension of the matrixes.

6. The method of claim 4, wherein the operation of generating an offline model file associated with the model framework type comprises:

analyzing the use case configuration file, and determining data information corresponding to the use case configuration file, wherein the data information comprises the number of model input layers and specification information of the input layers;

acquiring a creation interface of each input layer, and creating an operator;

and connecting a plurality of operators corresponding to the plurality of input layers according to the preset model structure to determine the offline model file.

7. A storage medium comprising a stored program, wherein the method of any one of claims 1 to 6 is performed by a processor when the program is run.

8. An automated testing apparatus for a deep learning model compiler, comprising:

the system comprises a first determination model, a second determination model and a third determination model, wherein the first determination model is used for determining a use case configuration file according to a preset data structure and input information of a user, and the data structure is used for storing parameter information required by creating a single operator test model;

the second determining module is used for analyzing the case configuration file by using a preset test frame and determining the model frame type corresponding to the case configuration file;

the generating module is used for generating an offline model file associated with the model framework type according to the use case configuration file;

and the testing module is used for testing the deep learning model compiler corresponding to the model framework type through the offline model file.

9. The apparatus of claim 8, wherein before determining the use case configuration file according to a preset data structure and input information of a user, the apparatus further comprises:

and the creating module is used for creating a data structure, wherein the data structure is used for storing parameter information required by the single operator test model.

10. An automated testing apparatus for a deep learning model compiler, comprising:

a processor;

a memory coupled to the processor for providing instructions to the processor for processing the following processing steps: