KR102641240B1 - Apparatus and Method for Artificial Intelligence Inference - Google Patents

Abstract

Embodiments disclose artificial intelligence reasoning devices and methods. The embodiment is an artificial intelligence inference method, which includes converting the learned neural network and parameters into executable code of a high-level language independent of the learning framework, and converting the executable code into a general purpose language (General Purpose Language, GPL) depending on whether accelerated operations are needed. ) It may include separating the code into a domain specific language (Domain Specific Language, DSL) code and generating the separated GPL code and DSL code into a target code optimized for hardware.

Description

Artificial Intelligence Inference Apparatus and Method {Apparatus and Method for Artificial Intelligence Inference}

The embodiment relates to artificial intelligence inference technology that runs a neural network in an embedded system environment.

Artificial neural network-based deep learning technology has been actively researched outside of Korea, and its scope of application is expanding to various embedded environments such as self-driving cars, unmanned vehicles, image processing devices, and factory automation.

Applications using deep learning consist of learning and inference processes. The inference system that actually operates the learned deep learning in an embedded environment manufactures hardware devices specialized for artificial intelligence applications, and develops an inference engine and application system according to the manufactured hardware. It consists of a making process. In the process of manufacturing hardware, accelerators for deep learning processing are installed to increase computational performance, and the inference engine is designed to be optimized for the hardware, including deep learning accelerators.

However, in this case, it is necessary to design an inference system that operates independently of hardware because it can result in high costs in terms of reusability and maintenance of software and code. In particular, in the case of artificial intelligence applications, the hardware environment is selected considering the amount of parallel computing of artificial intelligence. Various acceleration hardware such as CPU, GPU, FPGA, and dedicated accelerators are considered, and various types of accelerators are used simultaneously, not just one type. It can also happen. In this way, because the inference system is designed to be dependent on various hardware acceleration hardware environments, a lot of time and effort is required to build a model optimized for the selected hardware environment each time.

Korean Patent Publication No. 10-2018-0028967

The purpose of the embodiment is to facilitate the implementation of artificial intelligence applications in embedded systems with various hardware environments.

The purpose of the present invention is to minimize changes to the inference engine due to hardware changes in developing an inference engine that accelerates deep learning.

The embodiment is an artificial intelligence inference method, which includes converting an application based on a pre-trained neural network into executable code of a high-level language independent of the learning framework, and converting the executable code into a general-purpose language (General) depending on whether accelerated operations are needed. It includes steps of separating the Purpose Language (GPL) code and Domain Specific Language (DSL) code and generating the separated GPL code and DSL code into target code optimized for hardware.

At this time, the separation step can be generated as GPL code or DSL code depending on whether or not it is a calculation-oriented instruction as a result of analyzing the executable code.

At this time, the separating step may be checking based on the results of lexical analysis and parsing of the executable code to determine whether it is an operation-oriented instruction.

At this time, the step of generating the target code may be generating the GPL code as a target code that runs on the CPU of the hardware.

At this time, the step of generating the target code may be to generate a target code that runs on the CPU or accelerator of the hardware based on the result of analyzing the DSL code or the accelerator configuration state of the hardware.

At this time, the step of generating the target code may be to generate the target code by applying the DSL separation rule if it is advantageous for an acceleration environment as a result of analyzing the DSL code.

At this time, the step of generating the target code may be generating the target code by applying the DSL separation rule if an accelerator exists in the hardware.

At this time, the step of generating the target code may be to apply DSL separation rules for each type of accelerator when the types of a plurality of accelerators in the hardware are different.

At this time, the step of generating the target code may be to apply the DSL separation rule to the plurality of accelerators within the same accelerator environment when a plurality of the same accelerators exist in the hardware.

An embodiment is an artificial intelligence reasoning device, comprising a memory in which at least one program is recorded and a processor for executing the program, where the program is a high-level language independent of a learning framework for applications based on a pre-trained neural network. Converting the executable code into executable code, separating the executable code into General Purpose Language (GPL) code and Domain Specific Language (DSL) code depending on whether accelerated operations are required, and separated GPL code and DSL code Steps can be taken to generate target code optimized for hardware.

At this time, the separation step can be generated as GPL code or DSL code depending on whether or not it is a calculation-oriented instruction as a result of analyzing the executable code.

At this time, the separating step may be checking based on the results of lexical analysis and parsing of the executable code to determine whether it is an operation-oriented instruction.

At this time, the step of generating the target code may be generating the GPL code as a target code that runs on the CPU of the hardware.

At this time, the step of generating the target code may be to generate a target code that runs on the CPU or accelerator of the hardware based on the result of analyzing the DSL code or the accelerator configuration state of the hardware.

At this time, the step of generating the target code may be to generate the target code by applying the DSL separation rule if it is advantageous for an acceleration environment as a result of analyzing the DSL code.

At this time, the step of generating the target code may be generating the target code by applying the DSL separation rule if an accelerator exists in the hardware.

At this time, the step of generating the target code may be to apply DSL separation rules for each type of accelerator when the types of a plurality of accelerators in the hardware are different.

At this time, the step of generating the target code may be to apply the DSL separation rule to the plurality of accelerators within the same accelerator environment when a plurality of the same accelerators exist in the hardware.

The artificial intelligence inference method according to the embodiment includes the steps of converting an application based on a pre-trained neural network into executable code of a high-level language independent of the learning framework, and converting the executable code into a general-purpose language (General) depending on whether accelerated operations are needed. It includes the step of separating the Purpose Language (GPL) code and the Domain Specific Language (DSL) code and the step of generating the separated GPL code and DSL code into target code optimized for hardware, but the separation step includes, As a result of analyzing the execution code, it is generated as GPL code and DSL code depending on whether it is a calculation-oriented instruction. The step of generating target code is to generate GPL code as target code that runs on the CPU of the hardware and analyze the DSL code. Based on the results or the configuration state of the hardware's accelerator, it can be generated as target code that runs on the hardware's CPU or accelerator.

At this time, in the step of generating the target code, if it is advantageous for an acceleration environment as a result of analyzing the DSL code, the target code is generated by applying the DSL separation rule. If an accelerator exists in the hardware, the target code is generated by applying the DSL separation rule. It may be creating.

The present invention proposes an artificial intelligence inference device that is not dependent on various artificial intelligence applications and hardware acceleration environments, and has the advantage of reducing development time and effort and maintenance costs in embedded artificial intelligence development.

1 is a schematic block diagram of an embedded system including an artificial intelligence inference device according to an embodiment.
Figure 2 is a flowchart for explaining an artificial intelligence inference method according to an embodiment.
FIG. 3 is a flowchart for explaining the step (S220) of separating the executable code shown in FIG. 2 into GPL code and DSL code.
FIG. 4 is a flowchart for explaining the step (S232) of generating the DSL code shown in FIG. 2 as a target code.
Figure 5 is a diagram showing the configuration of a computer system according to an embodiment.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various different forms. The present embodiments only serve to ensure that the disclosure of the present invention is complete and that common knowledge in the technical field to which the present invention pertains is not limited. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Although terms such as “first” or “second” are used to describe various components, these components are not limited by the above terms. The above terms may be used only to distinguish one component from another component. Accordingly, the first component mentioned below may also be the second component within the technical spirit of the present invention.

The terms used in this specification are for describing embodiments and are not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” or “comprising” implies that the mentioned component or step does not exclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used in this specification can be interpreted as meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, an artificial intelligence inference device and method operating in various hardware acceleration environments according to embodiments will be described in detail with reference to FIGS. 1 to 5.

At this time, the artificial intelligence inference device can be implemented as an embedded device independent of various hardware acceleration environments. In other words, the present invention proposes a technology that can be easily ported to various artificial intelligence acceleration hardware environments by separating the hardware-independent part into lower layers, rather than newly constructing an artificial intelligence inference device for each type of accelerator.

1 is a schematic block diagram of an embedded system including an artificial intelligence inference device according to an embodiment.

Referring to FIG. 1, the artificial intelligence inference device 100 according to the embodiment is inputted with program codes implementing various artificial intelligence applications 10 based on a pre-trained neural network, and the application code is converted into a hardware system. (20) Ensure that it can be executed optimized for its characteristics.

At this time, the neural network may be a deep learning neural network, and many applications using deep learning neural networks undergo a learning process in advance on the server. At this time, examples of learning frameworks may include tensorflow, caffe, etc. Because these deep learning neural networks require a large amount of computational processing capacity, they require accelerator devices with excellent computational capabilities, such as GPUs or dedicated accelerators. In some cases, two or more same or heterogeneous accelerators may be used.

However, since the learned neural network model and weight data are distributed in a form dependent on the learning framework, the artificial intelligence inference device requires the same environment settings as the learning framework or is converted to a format specialized for the inference engine. The process must be carried out. In other words, in the case of existing inference systems, a system dependent on specific hardware had to be implemented, so a new inference system had to be created every time the acceleration hardware was changed. This significantly reduces the reusability of deep learning acceleration code.

Therefore, the artificial intelligence inference device 100 according to the embodiment is designed with a hardware-independent part and a hardware-dependent part, and is designed to newly build only the hardware-dependent part even if the hardware environment changes.

Accordingly, the artificial intelligence inference device 100 according to the embodiment may include a front-end layer 110, a DSL layer 120, and a target code generation layer 130.

The front-end layer 110 can convert an application based on a pre-trained neural network and parameters into executable code of a high-level language independent of the learning framework. In other words, the artificial intelligence application 10 is converted from code dependent on the artificial intelligence framework to code in a high-level language independent of the framework. That is, the front-end layer 110 is a hardware-independent layer and can commonly process data generated from various learning frameworks.

At this time, the high-level language may be Python. Additionally, it may be a standardized deep learning data conversion format such as NNEF (Neural Network Exchange Format), ONNX (Open Neural Network eXchange format), etc.

The DSL layer 120 can separate the execution code into general purpose language (GPL) code and domain specific language (Domain Specific Language, DSL) code depending on whether accelerated operations are needed. That is, the DSL layer 120 converts the execution code generated by the front-end layer 110 into a hardware-independent artificial intelligence processing routine using the DSL code.

At this time, the DSL layer 120 can be generated as a GPL code or a DSL code depending on whether or not it is a calculation-oriented instruction as a result of analyzing the execution code. A detailed description of this will be provided later with reference to FIG. 3.

The target code generation layer 130 can generate the separated GPL code and DSL code into a target code optimized for hardware.

That is, the artificial intelligence application 10 is executed in the hardware system 20, and an accelerator 22 may be further installed along with the CPU 21. At this time, various types of accelerators, such as GPU, FPGA, and dedicated acceleration chips, may be mounted as the accelerator 22, and a plurality of accelerators of the same type may exist. For example, a GPU and an accelerator chip may be installed in the hardware system 20 at the same time, or two identical GPUs may be installed. That is, at this time, the acceleration environment settings of the hardware system 20 are implemented in a way that optimizes performance by considering size, power consumption, etc. in accordance with the characteristics of artificial intelligence applications.

GPL code, typically including C and C++, can be executed in the CPU 21. Accordingly, the target code generation layer 130 can generate GPL code as a target code that runs on the CPU of hardware.

Additionally, the target code generation layer 130 may generate a target code that is executed in the CPU or accelerator of the hardware based on the result of analyzing the DSL code or the configuration state of the hardware accelerator. DSL code can be executed in the accelerator 22, and the DSL code can be converted into a form specific to the accelerator. Additionally, depending on the characteristics of the DSL code, the DSL code may also be executed in the CPU 21. A detailed description of this will be provided later with reference to FIG. 4 .

Figure 2 is a flowchart for explaining an artificial intelligence inference method according to an embodiment.

Referring to Figure 2, the embodiment is an artificial intelligence inference method, which includes converting an application based on a pre-trained neural network into executable code of a high-level language independent of the learning framework (S210), and accelerating the executable code. Separating into General Purpose Language (GPL) code and Domain Specific Language (DSL) code depending on whether calculation is required (S220, see FIG. 3) and optimizing the separated GPL code and DSL code for hardware It includes a step (S230) of generating the target code.

At this time, in the separation step (S220), GPL code and DSL code can be generated depending on whether the executable code is an operation-oriented instruction or not as a result of analyzing the execution code.

At this time, the separating step (S220) may be an inspection based on the results of lexical analysis and parsing of the executable code to determine whether it is an operation-oriented instruction. A detailed description of this will be provided later with reference to FIG. 3.

At this time, the step of generating the target code (S230) may include the step of generating the GPL code as a target code that runs on the CPU of the hardware (S231).

At this time, the step of generating the target code (S230) may include the step of generating the target code to be executed in the CPU or accelerator of the hardware based on the result of analyzing the DSL code or the accelerator configuration state of the hardware (S232). . That is, the artificial intelligence reasoning device 100 converts the DSL language into target code to be optimized for a specific hardware environment. A detailed description of this will be provided later with reference to FIG. 4 .

Figure 3 is a flow chart to explain the step (S220) of separating the executable code into GPL code and DSL code according to an embodiment.

Referring to FIG. 3, the device 100 performs lexical analysis (S310) and syntactic analysis (S320). Here, lexical analysis refers to dividing each statement in a program into tokens, which are the minimum units. Here, syntactic analysis refers to creating a pastry or syntax tree from tokens created in the lexical analysis stage. At this time, variables, argument values, and array values for the neural network are stored as a result of the syntax analysis using the rules and the command database of the neural network framework.

Then, the device 100 determines whether the execution code as a result of the analysis is an operation-oriented instruction (S330). In other words, it is checked whether it is an operation-oriented instruction or a control-oriented instruction based on predefined rules.

If, as a result of the determination in S330, it is not an operation-oriented instruction, the device 100 generates the executable code as GPL code (S340). In other words, if it is not a part that requires high performance of operations, it is converted to GPL code. For example, when the application is 'face recognition', the codes corresponding to routines such as camera operation, shooting, or image input are not parts that require high performance in computation, so they are generated as GPL code.

On the other hand, if the determination at S330 is that it is not an operation-oriented instruction, the device 100 generates the execution code as a DSL code (S350). In other words, parts that require high performance of deep learning acceleration calculations are converted to DSL code. For example, if the application is 'face recognition', the codes corresponding to the deep learning neural network that is actually executed by receiving the prepared data are generated as DSL codes because high performance of operations is required.

At this time, DSL is defined by grammar and is designed as a language that optimally expresses the BLAS library. An example of a DSL language for deep learning acceleration may be as follows.

C[i,j: M, N] = A(i,k: M,N) *+ B(k,j:M, N)

Figure 4 is a flowchart for explaining the step (S232) of generating a DSL code as a target code according to an embodiment.

Depending on the embodiment, the step of generating the DSL code as a target code (S232) may be to generate the DSL code as a target code by applying the DSL separation rule if it is advantageous for an acceleration environment as a result of analyzing the DSL code.

That is, referring to FIG. 4, the device 100 determines whether the acceleration environment is advantageous as a result of analyzing the DSL code (S410). As a result of the determination in S410, if it is not advantageous for the acceleration environment, the device 100 generates the DSL code as a target code to be executed on the CPU (S420). If it is advantageous for the acceleration environment, the device 100 proceeds to S430.

Additionally, depending on the embodiment, the step of generating the DSL code as the target code (S232) may be to generate the target code by applying the DSL separation rule when an accelerator exists in the hardware.

That is, referring to FIG. 4, the device 100 determines whether an accelerator exists in the hardware (S430). As a result of the determination in S430, if the accelerator does not exist, the device 100 generates the DSL code as a target code to be executed in the CPU (S420). If the accelerator exists, the device 100 proceeds to S440.

Additionally, depending on the embodiment, the step of generating DSL code as a target code (S232) may be to apply DSL separation rules for each type of accelerator when the types of accelerators in the hardware are different.

That is, referring to FIG. 4, the device 100 analyzes the environment of the accelerator (S440) and determines whether a plurality of heterogeneous accelerators of different types exist in the hardware (S450). As a result of the determination in S450, if a plurality of heterogeneous accelerators of different types exist, the device 100 applies the DSL separation rule for each accelerator type (S460).

On the other hand, as a result of the determination in S450, if a plurality of heterogeneous accelerators of different types do not exist or after performing S460, the device 100 proceeds to S470.

In addition, according to the embodiment, the step of generating the DSL code as the target code (S232) may be to apply the DSL separation rule to the plurality of accelerators in the same accelerator environment when a plurality of homogeneous accelerators exist in the hardware.

That is, referring to FIG. 4, the device 100 determines whether a plurality of accelerators of the same type exist in the hardware (S470). As a result of the determination in S470, if a plurality of homogeneous accelerators exist in the hardware, the device 100 applies the DSL separation rule to the plurality of accelerators within the homogeneous accelerator environment (S480).

As described above, in the embodiment, the deep learning execution part is converted to an intermediate language using a DSL language, and the generation from the DSL language to a hardware-optimized target code is separated into a separate layer, making distribution of the inference system easy. . In particular, it has a structure that operates easily even in an environment where there is more than one acceleration hardware. In addition, the artificial intelligence inference device and method according to the embodiment can operate independently of various deep learning accelerator devices (CPU, GPU, FPGA, dedicated accelerator) when deploying a deep learning neural network to an embedded system environment. .

Figure 5 is a diagram showing the configuration of a computer system according to an embodiment.

The artificial intelligence inference device 100 according to the embodiment may be implemented in a computer system 1000 such as a computer-readable recording medium.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input device 1040, user interface output device 1050, and storage 1060 that communicate with each other via bus 1020. You can. Additionally, the computer system 1000 may further include a network interface 1070 connected to the network 1080. The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or storage 1060. The memory 1030 and storage 1060 may be storage media including at least one of volatile media, non-volatile media, removable media, non-removable media, communication media, and information transfer media. For example, memory 1030 may include ROM 1031 or RAM 1032.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical idea or essential features. You will understand that it exists. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

100: Artificial Intelligence Reasoning Device
110: Front-end layer 120: DSL layer
130: Target code generation layer

Claims (20)

In an artificial intelligence inference method performed by an artificial intelligence inference device,
Converting an application based on a pre-trained neural network by the front-end layer of the artificial intelligence inference device into executable code of a high-level language independent of the learning framework;
Separating the execution code into general purpose language (GPL) code and domain specific language (Domain Specific Language, DSL) code depending on whether accelerated operations are required by the DSL layer of the artificial intelligence reasoning device; and
A step of generating the GPL code and DSL code separated by the target code generation layer of the artificial intelligence inference device into target code optimized for the number and type of CPU or accelerator included in the hardware to run the application,
The separation step is,
Analyzing executable code; and
As a result of the analysis, an artificial intelligence inference method comprising: generating execution codes corresponding to operation-oriented instructions, which are codes corresponding to deep learning neural networks, as DSL codes, and generating execution codes that are not operation-oriented instructions as GPL codes. delete The method of claim 1, wherein the analyzing step includes:
An artificial intelligence inference method that checks whether an instruction is an operation-oriented instruction based on the results of lexical analysis and parsing of the executable code. The method of claim 1, wherein generating the target code comprises:
An artificial intelligence inference method that generates GPL code into target code that runs on the hardware's CPU. The method of claim 1, wherein generating the target code comprises:
An artificial intelligence inference method that generates a target code that runs on the hardware's CPU or accelerator based on the results of analyzing the DSL code or the hardware's accelerator configuration state. The method of claim 5, wherein generating the target code includes:
An artificial intelligence inference method that generates target code by applying DSL separation rules if the results of analyzing the DSL code are favorable for an acceleration environment. The method of claim 5, wherein generating the target code includes:
An artificial intelligence inference method that generates target code by applying DSL separation rules when an accelerator exists in hardware. The method of claim 7, wherein generating the target code comprises:
An artificial intelligence inference method that applies DSL separation rules for each accelerator type when hardware has different types of accelerators. The method of claim 7, wherein generating the target code comprises:
An artificial intelligence inference method that applies DSL separation rules to multiple accelerators within a homogeneous accelerator environment when there are multiple homogeneous accelerators in hardware. a memory in which at least one program is recorded; and
Contains a processor that executes a program,
The program is,
A front-end layer that converts applications based on pre-trained neural networks into executable code in a high-level language that is independent of the learning framework;
DSL layer, which separates the executable code into General Purpose Language (GPL) code and Domain Specific Language (DSL) code depending on whether accelerated operations are needed or not; and
Generates separated GPL code and DSL code into target code optimized for the number and type of CPU or accelerator included in the hardware to run the application. Include target code generation rare words,
The DSL layer is,
Analyzing executable code; and
As a result of the analysis, an artificial intelligence inference device that performs the following steps: generating execution codes corresponding to operation-oriented instructions, which are codes corresponding to deep learning neural networks, as DSL codes, and generating execution codes that are not operation-oriented instructions as GPL codes. delete The method of claim 10, wherein the DSL layer is:
An artificial intelligence inference device that checks whether an instruction is an operation-oriented instruction based on the results of lexical analysis and parsing of the executable code. The method of claim 10, wherein the target code generation layer is:
An artificial intelligence inference device that generates GPL code into target code that runs on the hardware's CPU. The method of claim 10, wherein the target code generation layer is:
An artificial intelligence inference device that generates target code that runs on the hardware's CPU or accelerator based on the results of analyzing the DSL code or the hardware's accelerator configuration state. The method of claim 14, wherein the target code generation layer is:
An artificial intelligence inference device that generates target code by applying DSL separation rules if it is advantageous for an acceleration environment as a result of analyzing DSL code. The method of claim 14, wherein the target code generation layer is:
An artificial intelligence inference device that generates target code by applying DSL separation rules when an accelerator exists in hardware. The method of claim 16, wherein the target code generation layer is:
An artificial intelligence inference device that applies DSL separation rules for each accelerator type when the hardware has different types of accelerators. The method of claim 16, wherein the target code generation layer is:
An artificial intelligence inference device that applies DSL separation rules to multiple accelerators within a homogeneous accelerator environment when multiple homogeneous accelerators exist in hardware. In an artificial intelligence inference method performed by an artificial intelligence inference device,
Converting an application based on a pre-trained neural network by the front-end layer of the artificial intelligence inference device into executable code of a high-level language independent of the learning framework;
Separating the execution code into general purpose language (GPL) code and domain specific language (Domain Specific Language, DSL) code depending on whether accelerated operations are required by the DSL layer of the artificial intelligence reasoning device; and
A step of generating the GPL code and DSL code separated by the target code generation layer of the artificial intelligence inference device into target code optimized for the number and type of CPU or accelerator included in the hardware to run the application,
The separation step is,
Analyzing executable code; and
As a result of the analysis, the executable code corresponding to the operation-oriented instructions, which is the code corresponding to the deep learning neural network, is generated as DSL code, and the execution code that is not the operation-oriented instruction is generated as GPL code;
The steps to generate target code are:
An artificial intelligence inference method that generates GPL code as target code that runs on the hardware's CPU and generates it as target code that runs on the hardware's CPU or accelerator based on the results of analyzing the DSL code or the hardware's accelerator configuration state. The method of claim 19, wherein generating the target code comprises:
An artificial intelligence inference method that generates target code by applying DSL separation rules if an acceleration environment is advantageous as a result of analyzing DSL code, and if an accelerator exists in the hardware, applies DSL separation rules to generate target code.

Images