KR102324889B1 - Method and system for executing open-source programs written for personal computer on low-powered devices using artificial neural network - Google Patents

Abstract

A system for performing a certain algorithm may include a first electronic device and a second electronic device for performing a certain algorithm. The first electronic device performs learning on a first artificial neural network by applying first input data of a predetermined algorithm and first output data of an algorithm with respect to the first input data as an input and an output of the first artificial neural network, respectively, to determine weight values of the first artificial neural network and transmit the determined weight values to the second electronic device through a wired or wireless communication interface. The second electronic device stores a second artificial neural network having the same structure as the first artificial neural network and calculates the second output data of the second artificial neural network with respect to the second input data using the weight values to perform a certain algorithm on the second input data.

Description

METHOD AND SYSTEM FOR EXECUTING OPEN-SOURCE PROGRAMS WRITTEN FOR PERSONAL COMPUTER ON LOW-POWERED DEVICES USING ARTIFICIAL NEURAL NETWORK

The present invention relates to a method and system for performing an algorithm in a low-power device, and more particularly, to a method and system for easily performing an algorithm that requires a high-spec memory and a processor in a low-power device using an artificial neural network to perform it. it's about

Research to derive meaningful results by applying artificial intelligence technology to data collected from vehicles for autonomous driving or vehicle control automation is being actively conducted. Currently, most of the latest research open source codes for artificial intelligence or big data are implemented based on the Python language.

However, since a small device or mobile device such as a vehicle data collection device generally has a small memory capacity and is equipped with a low-performance processor running at low power, open source code (e.g., Python code) is not easy to run directly on the device. Therefore, in order to run an externally developed and tested algorithm on a small device, an algorithm previously implemented in programming code such as Python is converted into Embedded C or Assembly Language that can be operated on a small device. must go through This conversion or porting operation takes a lot of development manpower, cost, and time, and thus reduces the efficiency of R&D.

A method and system for enabling execution of a predetermined algorithm execution code developed and tested externally in a small device using an artificial neural network without a separate code conversion or porting operation may be provided.

The technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may be inferred from the following embodiments.

The system includes a first electronic device and a second electronic device for performing a predetermined algorithm, wherein the first electronic device includes first input data of the predetermined algorithm and a first input data of the algorithm on the first input data. By applying the first output data as input and output of the first artificial neural network, respectively, learning for the first artificial neural network is performed to determine the weight values of the first artificial neural network, and the determined weight value through a wired or wireless communication interface to the second electronic device, the second electronic device stores a second artificial neural network having the same structure as the first artificial neural network, and the second artificial neural network for second input data using the weight values The predetermined algorithm may be performed on the second input data by calculating the second output data of .

Each of the first input data and the first output data may be an input value and an output value for a function, API, library, or module implemented as programming code to perform the predetermined algorithm.

The second electronic device may be a terminal for collecting vehicle data and performing the predetermined algorithm on the collected vehicle data, and the first electronic device may be a server computer located outside the second electronic device.

An executable file for calculating the second output data of the second artificial neural network with respect to the second input data is stored in the terminal, the second input data includes the vehicle data, and the terminal includes the executable file can be used to perform the predetermined algorithm on the vehicle data.

A capacity of the first artificial neural network may be smaller than a reference capacity to be stored in the second electronic device.

An apparatus for obtaining information necessary for enabling a predetermined algorithm implemented in a high-level language to be executed in a low-performance device includes: performing an operation on input data of the predetermined algorithm to obtain output data of the predetermined algorithm; By performing learning on the recurrent neural network so that a memory for storing the recurrent neural network and a result of calculating the input data using the recurrent neural network and the result of performing the predetermined algorithm on the input data are the same a processor for determining weight values of the recurrent neural network, and a communication unit for transmitting the weight values to the low-performance device located outside, wherein the structure of the recurrent neural network depends on the performance and capacity of the low-performance device is determined, and each of the input data and the output data may be an input value and an output value for a function, API, library, or module implemented in Python code to perform the predetermined algorithm.

A method and system for enabling an externally developed and tested algorithm execution code to be executed in a small device using an artificial neural network without a separate code conversion or porting operation may be provided. According to the method and system disclosed herein, unnecessary time and cost in the research and development process can be reduced and efficiency can be improved.

1 shows a system for performing a predetermined algorithm in a small device, according to an embodiment.
2 illustrates an artificial neural network for obtaining output data by performing an operation on input data, according to an embodiment.
3 illustrates a Python code that may be implemented as an artificial neural network, according to an embodiment.
4 is a block diagram of an apparatus for obtaining information necessary to enable a low-performance apparatus to execute a predetermined algorithm implemented in a high-level language, according to an embodiment.

Below, some embodiments will be described clearly and in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present invention pertains (hereinafter, those skilled in the art) can easily practice the present invention. will be.

1 shows a system for performing a predetermined algorithm in a small device, according to an embodiment.

Referring to FIG. 1 , a system 1000 may include a first electronic device 1200 and a second electronic device 1400 .

Each of the first electronic device 1200 and the second electronic device 1400 may include a computer, a smart phone, a tablet PC, a data processing device, a device equipped with embedded software, a terminal, and the like. For example, the first electronic device 1200 may be a server computing device, and the second electronic device 1400 may be a vehicle data collection device loaded with embedded software. The first electronic device 1200 may be located outside the second electronic device 1400 .

The second electronic device 1400 may be a small electronic device (or a low-power device) in which a memory and a processor of a lower specification than that of the first electronic device 1200 are mounted. Embedded software may be installed in the small electronic device, and a low-performance processor that does not have a large memory capacity and operates at low power may be mounted. Such a small electronic device is a computing device for performing a specific purpose or function, and although it is miniaturized and easy to move, the memory capacity is not large and a processor of relatively low performance is generally mounted.

For example, the second electronic device 1400 includes Embedded Linux, Raspberry pi, Arduino, Google's Android, Apple's ios, or other low-power/low-performance computing. An operating system used in the device may be loaded. Alternatively, the second electronic device 1400 may be an Internet of Things (IoT) device. The IoT device may include a device that has an accessible wired or wireless interface, communicates with at least one other device through a wired or wireless interface, and transmits or receives data.

In contrast, the first electronic device 1200 may be a general-purpose computer device for performing a general purpose. The first electronic device 1200 may be equipped with a large-capacity memory such as a hard disk. The first electronic device 1200 has a larger memory capacity than the second electronic device 1400 and includes a high-performance processor. Accordingly, the first electronic device 1200 can implement an algorithm having a higher complexity than the second electronic device 1400 and is suitable for processing a large amount of data. According to an embodiment, the first electronic device 1200 may be a server including various types of centralized computing devices or distributed computing devices.

The first electronic device 1200 and the second electronic device 1400 may be connected to each other through a wired or wireless communication interface. The wireless communication interface is a wireless local area network (WLAN) such as Wi-Fi (Wireless Fidelity), a wireless personal area network (WPAN) such as Bluetooth, and a wireless USB (Wireless Universal Serial Bus). ), Zigbee, NFC (Near Field Communication), RFID (Radio-frequency identification), PLC (Power Line communication), or 3G (3rd Generation), 4G (4th Generation), LTE (Long Term Evolution), 5G It may include a modem communication interface connectable to a mobile cellular network, such as (5th Generation).

Each of the first electronic device 1200 and the second electronic device 1400 may include a processor, a memory, and a communication circuit. The processor may include a central processing unit (CPU), a microprocessor, a graphic processing unit (GPU), a digital signal processor (DSP), or a micro controller unit (MCU). Memory includes dynamic random access memory (DRAM), volatile memory such as static random access memory (SRAM), flash memory, read only memory (ROM), phase-change random access memory (PRAM), magnetic random access memory (MRAM), non-volatile memory such as resistive random access memory (ReRAM), and ferroelectrics random access memory (FRAM). For example, the first electronic device 1200 may mount a CPU and a memory, and the second electronic device 1400 may mount a microprocessor and a memory.

In a general R&D environment, algorithms with high complexity (eg, big data processing, algorithms using artificial intelligence technology) are developed in programming languages such as Python, Java, and Matlab. are tested In order to enable the algorithm (or open source program for PC) to be executed by the second electronic device 1400 that is a small electronic device, a language that can be directly compiled by the second electronic device 1400 (eg, embedded C (Embedded C)). C)), it is necessary to convert it to This is because the second electronic device 1400 does not have a large memory capacity and is equipped with a low-performance processor, so it is not easy to execute an algorithm developed in an external environment as it is.

In particular, since most of the open source codes for the latest research on artificial intelligence or big data are implemented with Python code-based Deep Learning Frameworks such as Tensorflow or PyTorch, developers can use Python codes for quick implementation. You have no choice but to implement and test your own ideas or algorithms. However, since the second electronic device 1400, which is a small electronic device, does not use a high-performance computing device, it is impossible to use a Python-based deep learning framework, and the second electronic device 1400 is unable to use a previously implemented algorithm. For this, the algorithm needs to be re-implemented in the embedded C language or assembly language.

For example, when the second electronic device 1400 is a device that collects and processes vehicle data and outputs various types of information necessary for driver habit, vehicle state, or autonomous driving using an algorithm using artificial intelligence technology, An algorithm developed as a Python code in an external environment needs to undergo a code conversion operation into a form operable by the second electronic device 1400 . Significant development manpower, cost, and time are invested in this transformation process.

In order to perform a predetermined algorithm (an algorithm previously implemented externally) in the second electronic device 1400 , the artificial neural network 1420 may be stored in the second electronic device 1400 (the artificial neural network 1420 may be 2 may be stored in the memory of the electronic device 1400). The artificial neural network 1420 may function as a function converter and may be performed by replacing a predetermined algorithm previously implemented externally. The non-linear activation function (Step, Sigmoid, ReLu, etc.) of the artificial neural network 1420 can reveal the relationship between the input and the output, and several non-linear transformations are connected through a plurality of layers, thereby increasing the flexibility of the artificial neural network 1420 and Expressive power can be greatly improved. This means that an artificial neural network with a nonlinear activation function and a hidden layer can express a mapping between an arbitrary input and an output, and can also express a relationship between arbitrary input data and arbitrary output data for a given algorithm. .

Hereinafter, an algorithm means a predetermined function, API (Application Programming Interface), library, module, etc. developed and tested with programming code such as Python, C language, Java, and Matlab, and the algorithm Each of the input data and output data of , may mean an input value and an output value for the function, API, library, or module. An input value and an output value may each have an arbitrary data type.

The artificial neural network 1420 may obtain output data by performing an operation on arbitrary algorithm input data through weight values and operators or operands of nodes. The process of obtaining the output data O by performing the operation defined by the artificial neural network 1420 on the input data I is the process of obtaining the output data O by actually performing a predetermined algorithm implemented in Python on the input data I. can be replaced That is, the operation defined by the artificial neural network 1420 performs substantially the same function as a predetermined algorithm, and obtaining output data O by performing an operation on the input data I using the artificial neural network 1420 is an input It can have the same effect as performing a predetermined algorithm on data I. In this embodiment, there is no need to convert a predetermined algorithm previously implemented in a programming language such as Python into a language executable by the second electronic device 1400 to execute the second electronic device 1400, and the second electronic device The predetermined algorithm may be performed by performing an operation on the input data I and obtaining the output data O using the artificial neural network 1420 , which replaces the predetermined algorithm 1400 has already implemented.

Referring to FIG. 2 , the artificial neural network 1420 includes an input layer (I-1 to In), hidden layers (La-1 to La-n, Lk-1 to Lk-n), and an output layer (RI). can do. The hidden layers La-1 to La-n and Lk-1 to Lk-n may be formed of k layers each including n nodes. Nodes of the artificial neural network 1420 may be connected by weighted synapses. At least one of the nodes of the artificial neural network 1420 may have a nonlinear activation function defined. The second electronic device 1420 may perform an operation on the input data I by using the weight values of the artificial neural network 1420 and the operator/operand of the node to obtain the output data O. A process in which the artificial neural network 1420 performs an operation on the input data I and obtains the output data O is apparent to those skilled in the art, and thus the detailed description thereof will be omitted.

3 illustrates a Python code that may be implemented as an artificial neural network, according to an embodiment.

Referring to FIG. 3 , the function fun1 may perform an algorithm for outputting an output_list having a list data type from an input_list having a list data type. More specifically, the function fun1 may output a list in which 1 is added to the value stored at each index of the input_list as output_list . For example, if input_list is {1, 2, 3, 4}, output_list becomes {2, 3, 4, 5}. The artificial neural network 1420 is designed to output an output_list from an arbitrary input input_list through learning, and the operation of the artificial neural network 1420 may replace the execution of the function fun1 defined in the Python code. Accordingly, the artificial neural network 1420 performs an operation on the input data {1, 2, 3, 4} to output {2, 3, 4, 5}, and the input data {20, 30, 40, 50} It can be learned to output {21, 31, 41, 51} by performing an operation on .

Referring back to FIG. 1 , according to an embodiment, the artificial neural network 1420 may be a recurrent neural network. This is because recurrent neural networks have Turing-Completeness, and any software code can be expressed. However, since the artificial neural network 1420 is stored and calculated by the second electronic device 1400 , the size and structure of the artificial neural network 1420 may be determined according to the performance and capacity of the second electronic device 1400 . For example, the artificial neural network 1420 may be designed to be stored in a memory of 100 MB or less and to obtain an output within 100 ms from the input data I to the output data O operation.

A loss representing the performance of the artificial neural network 1420 is output data obtained by performing an operation defined by the artificial neural network 1420 on input data by a predetermined algorithm (eg, Python) implemented in the input data. code) may be determined to be smaller as the output data obtained when the code is actually applied, and as the loss is smaller, the performance of the artificial neural network 1420 may be determined to be good. Weight values that determine the performance of the artificial neural network 1420 are determined through learning, and in the learning of the artificial neural network 1420, a predetermined result of calculating input data using the artificial neural network 1420 is previously implemented in the input data. It can be performed to be the same as the result obtained by actually performing the algorithm of .

According to an embodiment, the learning of the artificial neural network 1420 is not performed in the second electronic device 1400 , but the learning of the artificial neural network 1220 identical to the artificial neural network 1420 is performed in the first electronic device 1200 . and may be configured to transmit weight values determined through learning in the first electronic device 1200 to the second electronic device 1400 . Since the second electronic device 1400, which is a small electronic device, has insufficient performance to perform learning on the artificial neural network 1420, the first electronic device ( The artificial neural network 1220 may be trained in 1200 to determine weight values of the artificial neural network 1220 , and the determined weight values may be transmitted to the second electronic device 1400 through a wired or wireless communication interface.

To this end, the artificial neural network 1220 having the same structure as the artificial neural network 1420 may be stored in the first electronic device 1200 . The artificial neural network 1220 is a copy of the artificial neural network 1420, and the artificial neural network 1220 and the artificial neural network 1420 have the same number of nodes, number of layers, operators/operators defined in nodes, and edges between nodes.

Learning of the artificial neural network 1220 may be performed by applying input data of a predetermined algorithm and output data actually obtained by applying the predetermined algorithm to the input data as the input and output of the artificial neural network 1220 , respectively. For example, the artificial neural network 1220 that replaces the execution of a predetermined algorithm expressed by the function fun1 of FIG. 3 may be trained using the dataset shown in [Table 1] below.

input data output data {1, 2, 3, 4} {2, 3, 4, 5} {20, 30, 40, 50} {21, 31, 41, 51} {10, 100, 20, 30} {11, 101, 21, 31} {One} {2} {11, 12, 13, 14, 15, 16} {12, 13, 14, 15, 16, 17} {100, 200, 300, 400, 500, 600, 700, 800} {101, 201, 301, 401, 501, 601, 701, 801}

Learning of the artificial neural network 1220 may be performed using a back propagation algorithm that updates synaptic weight values to reduce loss. The weight values determined through learning are transmitted to the second electronic device 1400 through a wired or wireless communication interface, and the second electronic device 1400 applies the weight values received to the artificial neural network 1420 to the artificial neural network 1420 . The algorithm is executed by performing the operation of

According to an embodiment, an executable file for performing an operation of the artificial neural network 1420 is stored in the second electronic device 1400 , and the second electronic device 1400 performs a predetermined algorithm using the executable file. can For example, when the second electronic device 1400 is a terminal for collecting vehicle data and performing a predetermined algorithm on the collected vehicle data, the second electronic device 1400 is a first electronic device that is an externally located server computer. Weight values may be received from the device 1200 and a predetermined algorithm may be performed on vehicle data using an executable file.

4 is a block diagram of a device for acquiring information necessary to enable a small electronic device to execute a predetermined algorithm implemented in a high-level language, according to an embodiment.

Referring to FIG. 4 , a device 4000 may include a processor 4200 , a memory 4400 , and a communication circuit 4600 . The device 4000 may correspond to the first electronic device 1200 of FIG. 1 . The small electronic device may correspond to the second electronic device 1400 of FIG. 1 .

The memory 4200 may store a cyclic artificial neural network for obtaining output data by performing an operation on input data. The recurrent neural network may be designed such that output data obtained by performing an operation defined by the recurrent neural network on input data is the same as output data obtained when a predetermined algorithm is actually performed on the same input data. The structure of the recurrent artificial neural network may be determined according to the performance and capacity of the small electronic device. For example, the recurrent artificial neural network may be stored in a memory of 100 MB or less and designed so that the time to calculate output data from input data is within 100 ms.

The processor 4400 learns about the recurrent neural network so that output data obtained by performing an operation defined by the recurrent neural network on input data becomes the same as output data obtained when a predetermined algorithm is actually performed on the same input data. can be performed. The processor 4400 applies the input data of the predetermined algorithm and the output data obtained when the predetermined algorithm is actually performed to the input data as the input and output of the recurrent artificial neural network, respectively, and performs learning on the recurrent artificial neural network. It is possible to determine the weight values of the neural network.

The communication circuit 4600 may transmit the weight values determined by the processor 4400 to an external or small electronic device. Accordingly, the device 4000 serves as a middleware between the computing device and the small electronic device in which a predetermined algorithm is developed and tested, and the small electronic device uses the received weight values in the memory of the device 4000 . A predetermined algorithm may be performed by performing the same calculation of the recurrent artificial neural network stored in 4200 .

The descriptions are intended to provide exemplary configurations and acts for implementing the present invention. The technical spirit of the present invention will include not only the embodiments described above, but also implementations that can be obtained by simply changing or modifying the above embodiments. In addition, the technical spirit of the present invention will include implementations that can be achieved by easily changing or modifying the embodiments described above in the future.

Claims (5)

a first electronic device for storing a first artificial neural network; and
A second electronic device for storing a second artificial neural network for substituting a predetermined algorithm,
The second electronic device has a smaller memory capacity than the first electronic device,
The first artificial neural network has the same structure as the second artificial neural network, and the structures of the first artificial neural network and the second artificial neural network are determined according to the memory capacity of the second electronic device;
The first electronic device applies first input data of the predetermined algorithm and first output data of the predetermined algorithm to the first input data as inputs and outputs of a first artificial neural network, respectively, to apply the first artificial determining the weight values of the first artificial neural network by performing learning on the neural network, and transmitting the determined weight values to the second electronic device through a wired or wireless communication interface;
The second electronic device is configured to perform an operation of the second artificial neural network based on the weight values on second input data input to the second electronic device to obtain second output data, thereby adding the second input data to the second input data. Substitute a predetermined algorithm,
Each of the first input data and the first output data is an input value and an output value for a function, API, library, or module implemented as programming code to perform the predetermined algorithm. According to claim 1,
The second electronic device is a device in which embedded software is installed,
The first electronic device is a server computer located outside the second electronic device. 3. The method of claim 2,
an executable file for calculating the second output data of the second artificial neural network with respect to the second input data is stored in the second electronic device;
The second input data includes vehicle data,
and the second electronic device performs the predetermined algorithm on the vehicle data using the executable file. delete delete

Images