WO2021109593A1

WO2021109593A1 - Embedded platform deployment method and device, and storage medium

Info

Publication number: WO2021109593A1
Application number: PCT/CN2020/104548
Authority: WO
Inventors: 邓辅秦; 黄永深; 陈颖颖; 彭健烽; 冯华; 李伟科
Original assignee: 五邑大学
Priority date: 2019-12-05
Filing date: 2020-07-24
Publication date: 2021-06-10
Also published as: CN111190605A

Abstract

An embedded platform deployment method and device, and a storage medium, relating to the technical field of embedded development of deep learning. The method comprises the following steps of: communicating with an embedded motherboard; obtaining a hardware environment and a software environment of the embedded motherboard; sending and installing preset software and library file resources to the embedded motherboard on the basis of obtained hardware environment information and software environment information; after receiving a user instruction by means of a human-machine interaction interface, converting formats of training files having different formats, packaging and sending the training files to the embedded motherboard; uploading, by dragging, data to be tested in batches and sending same to the embedded motherboard to enable the embedded motherboard to start testing; and obtaining a speed measurement experiment report and a data analysis report of the embedded motherboard. In the method, the operations of deploying a platform for deep learning training for different types of embedded devices, importing said data in batches, and testing said data in batches are implemented by means of the human-machine interaction interface, thereby greatly reducing the working difficulty of a test personnel and improving the test efficiency.

Description

Deployment method, equipment and storage medium of embedded platform

Technical field

The invention relates to the technical field of embedded development of deep learning, in particular to a deployment method, equipment and storage medium of an embedded platform.

Background technique

With the development of the era of big data, computer vision based on deep learning has received more and more attention from researchers and high-tech companies in related fields around the world. As the use of deep learning on mobile devices becomes more and more common, people are more concerned about embedded The computing power and performance requirements of motherboard platforms are getting higher and higher. The training and deployment of deep learning are very different, and the training phase and the deployment phase are separate and independent of each other. Thanks to the open source lightweight deep learning framework models that have emerged in the past few years, such as the TensorFlow Lite deep learning framework, these models can reduce the amount of deep learning calculations and effectively reduce the burden on embedded development boards. However, there are big differences in the components, structures, connection methods, and materials used between embedded development boards produced by different companies. Compatibility problems are prone to occur when configuring the framework environment. Therefore, if you want to run deep learning smoothly on embedded motherboards, A series of tedious tasks such as downloading source code, patching, finding files, and installing dependent libraries need to be completed through complex command lines. Each step has different and complex commands. In order to complete different steps, it is often necessary to switch between different software. , The operation is very inconvenient.

When using the deep learning model, the above method only tests one image data at a time. Before the test, the user imports the data to be tested from the personal computer into the embedded development board. The traditional transmission method is to use the command line or some software for batch transmission. , After importing the data to be tested, enter a specific command line command to test the model on the embedded development board, but the command line can only be tested once. In order to test multiple pictures, you must write a script file by yourself. Each test is performed. It is necessary to re-enter the command line instructions to complete the data import and test. If the data to be tested contains tens of thousands of pictures, the workload of the above process is very huge.

Summary of the invention

In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a deployment method, equipment and storage medium of an embedded platform, which simplifies the deployment process of deep learning and improves the efficiency of training models.

The technical solutions adopted by the present invention to solve its problems are:

In the first aspect, an embedded platform deployment method of the present invention includes:

Communicate with embedded motherboard;

Acquiring the hardware environment and software environment of the embedded motherboard;

Based on the acquired hardware environment information and software environment information, the preset software and library file resources are sent and installed to the embedded motherboard;

After receiving the user's instruction through the human-computer interaction interface, the training files of different formats are converted into formats, packaged and sent to the embedded motherboard;

Beneficial effects: The deployment method of an embedded platform of the present invention adopts the following steps: communicating with the embedded motherboard, acquiring the hardware environment and software environment of the embedded motherboard, and presetting based on the acquired hardware environment information and software environment information The software and library file resources are sent and installed to the embedded motherboard. After receiving user instructions through the human-computer interaction interface, the training files of different formats are converted and packaged and sent to the embedded motherboard, and the data to be tested is dragged and uploaded in batches. And send it to the embedded motherboard and make it start the test, and obtain the speed test report and data analysis report of the embedded motherboard, so as to realize the deployment of platforms for deep learning training to different types of embedded devices, and batch import Test data and batch test the operation of the data to be tested, thereby greatly reducing the difficulty of the tester's work and improving the test efficiency.

Further, the step of sending and installing preset software and library file resources to the embedded motherboard based on the acquired hardware environment information and software environment information includes:

The software and library file resources include TensorFlow Lite deep learning framework files and software that can realize the speed measurement experiment function and the data analysis report function.

Specifically, the TensorFlow Lite deep learning framework is used to provide a training environment for deep learning models, and the software that can realize the speed measurement experiment function and the data analysis report function is used for the speed measurement and data analysis of the embedded motherboard in the deep learning training process. , And send the training results to the host.

After receiving the user's instruction through the human-computer interaction interface, the software and library file resources are uniformly reinstalled. Before installation, it is determined whether the software and library file resources are already on the embedded motherboard. If not, the software and library are installed If the file exists, the command line interface displays the prompt information that the software and library file resources have been installed, and extracts the prompt information to determine whether the software and library file are successfully installed.

Specifically, by analyzing the software environment of the embedded motherboard, the host obtains the software and library file resources needed in the process of deep learning model training of the embedded motherboard, and then the software obtains the software resource package integrated from the background of the host Grab the corresponding software resources such as the photo library and patches.

After the host receives the user instruction through the ssh and sftp protocol and the human-computer interaction interface, it automatically packages and sends the software resources required by the embedded motherboard in training to the embedded motherboard, so that the user does not need to analyze the embedded motherboard by himself The hardware environment of the motherboard and searching for missing software and library file resources of the embedded motherboard greatly reduce the difficulty of deployment of the deep learning platform. For example, if the embedded motherboard is jetson Nano of the MVIDIA series, the TensorRT software needs to be installed, and the host detects it by itself. If the TensorRT software is not installed, the embedded motherboard will be installed.

Further, the step of converting the trained training files in different formats into .tflite format files after receiving user instructions through the human-computer interaction interface and packaging and sending them to the embedded motherboard includes:

Receive user instructions through the human-computer interaction interface, the instructions include the file format before and after conversion;

Compile and convert the file format of the trained deep learning model file and variable file;

The converted file is packaged and sent to the embedded motherboard.

Specifically, because the TensorFlow lite deep learning framework runs on the embedded motherboard, the user must train a model on the desktop device in advance, and in order to implement the model import, some other types of files are needed, such as Graph Definition, Checkpoints, and Frozen Graph, however, only files in the .tflite format can be run on the embedded motherboard with the TensorFlow lite library installed, so training files in different formats need to be converted to files in the .tflite format. The user only needs to input the pre-conversion and post-conversion file formats in the human-computer interaction interface, and the host can convert training files in different formats into .tflite format files that can be run in the TensorFlow Lite deep learning framework.

The conversion of .tflite format files is divided into three steps: save the trained model files .ckpt and .pb; use freeze_graph tool to generate frozen graphdef files; use toco (Tensorflow Optimizing COnverter) tool to generate the final tflite file.

Further, after the step of receiving user instructions through the human-computer interaction interface, converting the trained training files in different formats into .tflite format files and packaging and sending them to the embedded motherboard, the data to be tested is batch dragged Drag and upload and send to the embedded motherboard and start testing; obtain the speed test experiment report and data analysis report of the embedded motherboard.

Specifically, the user can upload and send multiple sets of data at one time by dragging and dropping the buttons on the human-machine communication interface. After importing the data, the user can complete a large number of tests for the model by clicking on the corresponding interface of the host computer to start running the network model. The embedded motherboard records the time when the model is used and when it ends, and calculates the results of the speed measurement experiment. The data analysis report includes accuracy, recall, sensitivity, specific validity and comprehensive evaluation indicators, and the accuracy is expressed The probability of the true positive sample in the sample that is predicted to be positive, the recall rate indicates how many positive examples in the sample are predicted correctly, and the sensitivity indicates the proportion of all positive examples that are matched, which measures the classifier’s ability to recognize positive examples. The specific validity represents the proportion of all negative cases that are matched, and it measures the classifier’s ability to recognize negative cases. The comprehensive evaluation index is the weighted and average of accuracy and recall. When the value of the comprehensive evaluation index is high, the training model is explained. More effective.

In the second aspect, the present invention provides a deployment method and equipment of an embedded platform,

It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor, so that the at least one control processor A method for deploying an embedded platform as described above can be executed.

In a third aspect, the present invention provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to make a computer execute the method for deploying an embedded platform as described above.

In a fourth aspect, the present invention also provides a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer , Make the computer execute the deployment method of an embedded platform as described above.

Description of the drawings

The present invention will be further described below with reference to the drawings and embodiments;

FIG. 1 is a schematic diagram of the software flow of an embedded platform deployment method according to an embodiment of the present invention.

Detailed ways

This section will describe the specific embodiments of the present invention in detail. The preferred embodiments of the present invention are shown in the accompanying drawings. The function of the accompanying drawings is to supplement the description of the text part of the manual with graphics, so that people can understand the present invention intuitively and vividly. Each technical feature and overall technical solution of the invention cannot be understood as a limitation on the protection scope of the present invention.

In the description of the present invention, if the first and second are described for the purpose of distinguishing technical features, they cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating The precedence of the indicated technical characteristics.

In the description of the present invention, unless otherwise clearly defined, terms such as setting, installation, and connection should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above terms in the present invention in combination with the specific content of the technical solution.

Referring to Fig. 1, step S101, run the software. First, open the software applying the embedded platform deployment method of the present invention.

In step S102, the host communicates with the embedded motherboard through a data network cable.

In some of these embodiments, the host and the embedded motherboard communicate via a data network cable or wirelessly.

In the software in which the embodiment of the present invention applies an embedded platform deployment method, the host computer and the embedded motherboard communicate through a data network cable.

Step S103, the host detects the software environment and hardware environment of the embedded motherboard. After the host is successfully connected to the embedded motherboard, the host starts to detect the software environment and hardware environment of the embedded motherboard, and the hardware information of the embedded motherboard includes the model and operating system type of the embedded motherboard .

In some of the embodiments, the host obtains the hardware environment of the embedded motherboard by manually selecting the hardware environment by the user or by command detection. One is that the user manually selects the hardware environment of the embedded motherboard, the deployment process is less buggy, the software will list commonly used artificial intelligence embedded motherboards (such as Jeston series, RK3399) for users to choose, and different motherboard models use different Installation steps; another way to obtain the hardware environment of the embedded motherboard is to detect through commands, such as detecting CPU information: uname-a or cat/proc/cpuinfo and the general computer name will put the model in: hostname#View computer name.

In the software of an embedded platform deployment method in the embodiment of the present invention, the host acquires the hardware environment of the embedded motherboard through a command detection method, and the host acquires the hardware environment by sending a ping packet to the embedded motherboard. Windows starts The default value of ttl is 128, and the default value of linux and unix systems is 64. There are some special unix systems whose ttl value is 255, or use the parameter -O provided by map.

In step S104, the software analyzes the software environment and the hardware environment and packages the required software and library file resources into a software containing TensorFlow lite and transmits it to the embedded development board system to deploy a deep learning environment. By analyzing the software environment of the embedded motherboard, the software and library file resources needed in the process of deep learning model training of the embedded motherboard are obtained, and then the software grabs the corresponding software resource package from the back-end integrated software resource package of the host. For example, if the embedded motherboard is a jetson Nano of the MVIDIA series, the TensorRT software needs to be installed, and the host detects itself. If the TensorRT software is not installed, it will install the embedded motherboard.

The software and library file resources include TensorFlow Lite deep learning framework files and software that can realize the speed test function and the data analysis report function. The user clicks on the corresponding button of the software to make the host re-install the software and library file resources to the embedded motherboard in a unified manner. Before installation, the host determines whether the software and library file resources are already on the embedded motherboard. If it does not exist, the software and library files are installed. If they exist, the command line interface displays a prompt message that the software and library file resources have been installed. The host extracts the prompt information to determine whether the software and library files are Successful installation.

In step S105, the software on the host converts the trained training files in different formats into .tflite format files and packs them and sends them to the embedded motherboard. Since the TensorFlow lite deep learning framework runs on the embedded motherboard, the user must train a model in advance on the desktop device, and in order to implement the model import, some other types of files are needed, such as Graph Definition, Checkpoints, and Frozen Graph. However, only files in the .tflite format can be run on the embedded motherboard with the TensorFlow lite library installed, so training files in different formats need to be converted to files in the .tflite format. First, the user enters the pre-conversion and post-conversion file formats in the software. The software uses the source code in the TensorFlow Lite deep learning framework to compile and convert the trained deep learning model files and variable files into files in .tflite format. Finally, the software packages the converted files and sends them to the embedded motherboard.

In step S106, the software sends the data to be tested to the embedded motherboard in batches and starts the test. The software provides a file drag and drop function, which replaces the traditional use of a command line interface for file positioning, transmission and other complex instructions.

In some of the embodiments, the user issues a transmission instruction by clicking the corresponding button on the software, or by dragging and dropping the data file to be tested into the corresponding area of the software, so that the host transmits the data to be tested in batches to The embedded motherboard;

In the software using an embedded platform deployment method in the embodiment of the present invention, the user drags and drops the data file to be tested into the corresponding area of the software, so that the host transmits the data to be tested to the embedded motherboard in batches. , And then the user issues a test instruction by clicking the corresponding button to make the embedded motherboard start batch testing of the received data to be tested, so that the user can upload and test multiple sets of data with one click through the software.

In step S107, the embedded motherboard sends the speed measurement experiment report and the data analysis report to the host to complete the deployment of the neural network. While testing the data, the embedded motherboard runs the software that can realize the speed measurement experiment in the software resource package transmitted from the host, so as to perform the speed measurement experiment, record the time from the beginning to the end of the model, and internally calculate the speed measurement result And send it to the host, so as to ensure the real-time performance of the data result. The data analysis report includes accuracy rate, recall rate, sensitivity rate, specific validity and comprehensive evaluation index. The accuracy rate indicates the probability of a true positive sample in the sample that is predicted to be positive, the recall rate indicates how many positive examples in the sample are predicted correctly, and the sensitivity indicates the proportion of all positive examples that are matched, which measures the classifier’s ability to positive examples. Recognition ability, special validity indicates the proportion of all negative examples that are matched, and measures the classifier’s ability to recognize negative examples. The comprehensive evaluation index is the weighted average of accuracy and recall. When the value of the comprehensive evaluation index is high It shows that the training model is more effective. In this way, users can clearly understand the operation of the deep learning model and improve the deep learning model.

It should be noted that, because the deployment method of an embedded platform in this embodiment and the deployment method of an embedded platform described above are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to the present invention. The device embodiment will not be described in detail here.

Specifically, the method and equipment for deploying an embedded platform includes: one or more control processors and memories, and the control processors and memories may be connected by a bus or in other ways.

As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, as in the embodiment of the present invention, an embedded platform deployment method corresponds to The program instructions/modules of the control processor execute the various functional applications and data processing of an embedded platform deployment method device by running the non-transitory software programs, instructions and modules stored in the memory, that is, the above method is realized An embodiment of a method for deploying an embedded platform.

The memory may include a storage program area and a storage data area, where the storage program area can store an operating system and an application program required by at least one function; the storage data area can store data created by using an embedded platform deployment method device. Data etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged with respect to the control processor, and these remote memories may be connected to the deployment method device of the embedded platform through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory, and when executed by the one or more control processors, an embedded platform deployment method in the foregoing method embodiment is executed.

The embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors.

The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a general hardware platform. Those skilled in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by computer programs instructing relevant hardware. The programs can be stored in a computer readable storage medium. At this time, it may include the flow of the embodiment of the above-mentioned method. Wherein, the storage medium can be a magnetic disk, an optical disc, a read-only memory (Read Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.

The above is a detailed description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. Equivalent modifications or replacements are all included in the scope defined by the claims of this application.

Claims

A deployment method of an embedded platform is characterized in that it comprises the following steps:

Communicate with embedded motherboard;

Acquiring the hardware environment and software environment of the embedded motherboard;

Based on the acquired hardware environment information and software environment information, the preset software and library file resources are sent and installed to the embedded motherboard;

After receiving the user instruction through the human-computer interaction interface, the training files of different formats are converted, packaged and sent to the embedded motherboard.
An embedded platform deployment method according to claim 1, characterized in that, based on the acquired hardware environment information and software environment information, the preset software and library file resources are sent and installed to the embedded motherboard The steps include:

The software and library file resources include TensorFlow Lite deep learning framework files and software that can realize the speed measurement experiment function and the data analysis report function.
The deployment method of an embedded platform according to claim 2, characterized in that, based on the acquired hardware environment information and software environment information, the preset software and library file resources are sent and installed to the embedded motherboard The steps include:

After receiving the user's instruction through the human-computer interaction interface, the software and library file resources are uniformly reinstalled. Before installation, it is determined whether the software and library file resources are already on the embedded motherboard. If not, the software and library are installed If the file exists, the command line interface displays the prompt information that the software and library file resources have been installed, and extracts the prompt information to determine whether the software and library file are successfully installed.
The deployment method of an embedded platform according to claim 1, characterized in that, after receiving a user instruction through a human-computer interaction interface, the trained training files in different formats are converted into .tflite format files and The steps of packaging and sending to the embedded motherboard include:

Receive user instructions through the human-computer interaction interface, the instructions include the file format before and after conversion;

Compile and convert the file format of the trained deep learning model file and variable file;

The converted file is packaged and sent to the embedded motherboard.
The deployment method of an embedded platform according to claim 1, characterized in that, after receiving a user instruction through a human-computer interaction interface, the trained training files in different formats are converted into .tflite format files and After the step of packaging and sending to the embedded motherboard, drag and upload the data to be tested in batches and send it to the embedded motherboard to start the test; obtain the speed test experiment report and data analysis report of the embedded motherboard.
A deployment device for an embedded platform, which is characterized in that it includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores which can be executed by the at least one control processor The instructions are executed by the at least one control processor, so that the at least one control processor can execute the method for deploying an embedded platform according to any one of claims 1-5.
A computer-readable storage medium, characterized in that: the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to make a computer execute any one of claims 1-5. An embedded platform deployment method.