CN107480115B - Method and system for format conversion of caffe frame residual error network configuration file - Google Patents
Method and system for format conversion of caffe frame residual error network configuration file Download PDFInfo
- Publication number
- CN107480115B CN107480115B CN201710770739.4A CN201710770739A CN107480115B CN 107480115 B CN107480115 B CN 107480115B CN 201710770739 A CN201710770739 A CN 201710770739A CN 107480115 B CN107480115 B CN 107480115B
- Authority
- CN
- China
- Prior art keywords
- parameter
- configuration file
- template
- parameter data
- opencl
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/151—Transformation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/151—Transformation
- G06F40/157—Transformation using dictionaries or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides a method and a system for format conversion of a caffe frame residual error network configuration file, wherein the method comprises the following steps: acquiring parameter data in a caffe frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names; establishing a corresponding relation between the parameter name and each position in the OpenCL template; filling corresponding parameter values to corresponding positions of the template by using the corresponding relation; and outputting to obtain the target format configuration file. The method adopted by the invention can acquire the parameter data in the caffe frame residual error network configuration file, establish the corresponding relation and fill the parameter data into the OpenCL template, thereby reducing the dependence on manpower, improving the working efficiency and reducing errors compared with the prior art that parameter data are manually filled into an OpenCL format file.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a system for converting a residual network configuration file format of a case (Convolutional neural network framework).
Background
Deep learning, which is a branch of machine learning, is one of the rapidly developing fields in Artificial Intelligence (AI), and can help computers understand a large amount of data in the form of images, sounds, and texts. In recent years, as deep learning open-source tools such as caffe and the like tend to mature, deep learning technology develops rapidly, and deep learning is widely applied in the fields of face recognition, voice recognition, precise medical treatment, unmanned driving and the like. In 2015, a deep learning residual network (Resnet) is in breakthrough progress in ImageNet competition, a network structure of 152 layers is deduced, the error rate of Top-5 is reduced to be about 3.5%, a strong reaction in the deep learning neural field is caused, and the study of the residual network based on a deep learning open source framework such as caffe and the like becomes a hot point for developers to study.
Caffe is a clear and efficient deep learning framework, and supports command lines, Python and MATLAB interfaces based on a C + +/CUDA architecture; the CPU and the GPU can be directly and seamlessly switched, and a set of basic programming framework is provided in the CPU and the GPU, so that algorithms such as deep convolution neural network and deep learning under a GPU parallel framework can be realized. Based on the residual network of the caffe frame, the network model Net is defined in the prototxt text file, mainly defining all the layers and the data flow connected with the layers, and only one training or testing command is needed in the caffe, so that the training or testing of the network model can be completed according to the network defined in the prototxt file. However, since the deep learning requires a large amount of computing power to process a large amount of data, the development of the deep learning method faces a lot of difficulties, such as insufficient expansibility of deep learning software, insufficient computing performance, large energy consumption for recognition on a deep learning line, and the like.
Compared with a CPU, the GPU can improve the calculation speed of a calculation intensive program required by deep learning, but the energy efficiency improvement and application range of the GPU are limited by the characteristics of high energy consumption, small cache and the like of the GPU. An FPGA (Field-Programmable Gate Array) heterogeneous platform has become one of the best solutions for improving the performance of the data center server and reducing the power consumption of the internet company due to its characteristics of low power consumption, programmability, high parallelism and the like. The FPGA heterogeneous Computing platform adopts a high-level comprehensive programming model, researches and optimizes a deep learning neural network based on an OpenCL (Open Computing Language) Language, completes efficient transplantation and deployment of a neural network algorithm on the FPGA platform, and can greatly improve the Computing performance of the deep learning neural network algorithm by fully utilizing board card hardware flow design and task level parallelism.
However, in the development process of realizing deep learning residual error network heterogeneous acceleration based on the FPGA, research and development personnel need to fill configuration parameters of a residual error network to a header file required by an OpenCL code for different network models; for a deeper network model, at least 20 engineers are required to finish the model in 1 day, which consumes a large research and development force and may cause errors to affect the research and development efficiency.
Therefore, how to complete format conversion of a file, reduce dependence on labor, improve work efficiency, and reduce errors is a problem to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of this, the present invention provides a method and a system for converting a format of a residual network configuration file of a caffe framework, which can reduce dependence on labor, improve work efficiency, and reduce errors. The specific scheme is as follows:
in one aspect, the invention provides a method for converting a format of a residual network configuration file of a caffe framework, comprising the following steps:
acquiring parameter data in a caffe frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names;
establishing a corresponding relation between the parameter name and each position in the OpenCL template;
filling corresponding parameter values to corresponding positions of the template by using the corresponding relation;
and outputting to obtain the target format configuration file.
Preferably, the acquiring parameter data in the configuration file includes:
acquiring the configuration file;
extracting parameter data from the configuration file.
Preferably, the obtaining the configuration file includes:
receiving a storage path of the configuration file;
and acquiring the configuration file by using the storage path.
Preferably, the extracting parameter data from the configuration file includes:
reading parameter data of each layer in the configuration file;
storing the parameter data packet.
Preferably, the template is a Mustache template.
Preferably, the establishing of the correspondence between the parameter name and each position in the OpenCL template includes:
storing the parameter data to a python dictionary;
and establishing a corresponding relation between the parameter name and each position in the OpenCL template by using the mapping function of the python dictionary.
Preferably, the storing the parameter data to a python dictionary comprises:
storing the parameter data packet to a python dictionary.
On the other hand, the invention also provides a format conversion system for the residual network configuration file of the caffe framework, which comprises the following steps:
the parameter acquisition module is used for acquiring parameter data in a allowance frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names;
the corresponding establishing module is used for establishing the corresponding relation between the parameter name and each position in the OpenCL template;
the parameter filling module is used for filling corresponding parameter values to corresponding positions of the template by utilizing the corresponding relation;
and the target output module is used for outputting the configuration file with the target format.
Preferably, the parameter obtaining module includes:
a file obtaining unit, configured to obtain the configuration file;
and the parameter extraction unit is used for extracting parameter data from the configuration file.
Preferably, the correspondence establishing module includes:
the dictionary storage unit is used for storing the parameter data to a python dictionary;
and the dictionary mapping unit is used for establishing the corresponding relation between the parameter name and each position in the OpenCL template by utilizing the mapping function of the python dictionary.
The invention provides a format conversion method for a caffe frame residual error network configuration file, which comprises the following steps: acquiring parameter data in a caffe frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names; establishing a corresponding relation between the parameter name and each position in the OpenCL template; filling corresponding parameter values to corresponding positions of the template by using the corresponding relation; and outputting to obtain the target format configuration file.
The method adopted by the invention can acquire the parameter data in the caffe frame residual error network configuration file, establish the corresponding relation and fill the parameter data into the OpenCL template, thereby reducing the dependence on manpower, improving the working efficiency and reducing errors compared with the prior art that the OpenCL format file is filled manually.
The format conversion system for the residual network configuration file of the caffe framework, provided by the invention, also has the beneficial effects, and is not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a format conversion method for a caffe frame residual error network configuration file according to a first embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a format conversion system for a cafe framework residual error network configuration file according to a second embodiment of the present invention;
fig. 3 is a schematic diagram illustrating a parameter obtaining module of a buffer frame residual error network configuration file format conversion system according to a second embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a corresponding building module of a format conversion system for a residual network configuration file of a cafe framework according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a format conversion method for a residual network configuration file of a cafe framework according to a first embodiment of the present invention, where the method includes the following steps:
s11: acquiring parameter data in a residual error network configuration file of a caffe frame, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names.
In a first specific implementation manner of the present invention, in the method for converting the format of the cafe frame residual error network configuration file provided in the embodiment of the present invention, first, parameter data in the cafe frame residual error network configuration file needs to be acquired. For example, in practice, these parameter data are typically stored in a prototxt profile of the residual network, and if this profile is found, the relevant parameter data are stored in this profile.
Further, in order to find the cafe framework residual network configuration file, an installation directory of the cafe may be specified, and the configuration file has a fixed storage path under the installation directory, so that the configuration file may be found. Of course, the profile may also be searched using a search function.
Further, after finding the configuration file, parameter data may be extracted from the configuration file, where the parameter data generally includes parameter names and parameter values, and in case, the parameter names correspond to the parameter values of the parameter data of each layer. When data is extracted, the parameter data of each layer can be classified to form a group of data, and the parameter data is stored in groups, so that the subsequent operation can be facilitated.
S12: and establishing a corresponding relation between the parameter name and each position in the OpenCL template.
In order to fill the parameter data into the corresponding position of the OpenCL template, a corresponding relationship between the parameter name and each position in the OpenCL template may be established first. This correspondence may be specified in the code program or may utilize existing dictionary tools. The front-end template engine of the template can be selected to be a Mustache, that is, the template can be a Mustache template.
For example, a python dictionary may be used to establish a corresponding relationship between the parameter name and each position in the OpenCL template, and specifically, the parameter data is stored in the python dictionary first, and then the mapping function of the python dictionary is used to establish the corresponding relationship between the parameter name and each position in the OpenCL template. The Python dictionary, as a container type, can store any number of Python objects, which may also include other container types, and may contain objects of different types. And has powerful mapping function and can map the data in the mapping function. For example, the parameter name in this embodiment may be used as a key, and the corresponding parameter value may be used as a value corresponding to the key, so that the dictionary is formed. The OpenCL template also has a position corresponding to the key, so that a correspondence relationship is established.
Further, in order to process the parameter data conveniently, the parameter data may be stored in a python dictionary in groups, and specifically may be stored in an array manner, so that one array may be operated at a time, thereby speeding up the data processing process.
S13: and filling the corresponding parameter values to the corresponding positions of the template by utilizing the corresponding relation.
After the corresponding relationship is established, the corresponding parameter values in the parameter data can be filled to the corresponding positions of the template according to the corresponding relationship by using the corresponding relationship. The template defines the format of each network layer configuration parameter data required in the parallel optimization design of the residual error network according to the OpenCL language, the template lacks the parameter data in the residual error network configuration file of the caffe framework, and the template can be realized by using the dictionary function of Python.
S14: and outputting to obtain the target format configuration file.
After parameter values are filled, the template filled with the parameter data is output to obtain a configuration file in a target format, namely a configuration file in an OpenCL language, so that efficient transplantation and deployment of the neural network algorithm on an FPGA platform are completed, and the calculation performance of the deep learning neural network algorithm can be greatly improved by fully utilizing board card hardware flow design and task level parallelism.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a configuration of a cafe framework residual network configuration file format conversion system according to a second embodiment of the present invention.
The invention also provides a format conversion system for the residual error network configuration file of the caffe framework, which comprises the following steps:
the parameter obtaining module 201 is configured to obtain parameter data in a caffe frame residual error network configuration file, where the parameter data includes a parameter name and a parameter value corresponding to the parameter name;
a correspondence establishing module 202, configured to establish a correspondence between the parameter name and each position in the OpenCL template;
a parameter filling module 203, configured to fill the corresponding parameter value into the corresponding position of the template by using the corresponding relationship;
and the target output module 204 is configured to output the obtained target format configuration file.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating a parameter obtaining module of a context residual network configuration file format conversion system according to a second embodiment of the present invention.
Preferably, the parameter obtaining module 201 includes:
a file acquiring unit 2011, configured to acquire the configuration file;
a parameter extracting unit 2012, configured to extract parameter data from the configuration file.
Fig. 4 is a schematic diagram illustrating a corresponding building module of a format conversion system for a residual network configuration file of a cafe framework according to a second embodiment of the present invention.
Preferably, the correspondence establishing module 202 includes:
a dictionary storage unit 2021 for storing the parameter data to a python dictionary;
the dictionary mapping unit 2022 is configured to establish a corresponding relationship between the parameter name and each position in the OpenCL template by using a mapping function of the python dictionary.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The method and the system for format conversion of the residual network configuration file of the caffe framework provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (7)
1. A method for converting the format of a residual network configuration file of a caffe framework is characterized by comprising the following steps:
acquiring parameter data in a caffe frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names;
establishing a corresponding relation between the parameter name and each position in the OpenCL template;
filling corresponding parameter values to corresponding positions of the template by using the corresponding relation;
outputting to obtain a target format configuration file;
the establishing of the corresponding relationship between the parameter name and each position in the OpenCL template includes:
storing the parameter data packet to a python dictionary; and establishing a corresponding relation between the parameter name and each position in the OpenCL template by using the mapping function of the python dictionary.
2. The method of claim 1, wherein the obtaining parameter data in the configuration file comprises:
acquiring the configuration file;
extracting parameter data from the configuration file.
3. The method of claim 2, wherein obtaining the configuration file comprises:
receiving a storage path of the configuration file;
and acquiring the configuration file by using the storage path.
4. The method of claim 2, wherein said extracting parameter data from said configuration file comprises:
reading parameter data of each layer in the configuration file;
storing the parameter data packet.
5. The method of claim 1, wherein the template is a Mustache template.
6. A kind of cafe frame residual error network configuration file format conversion system, characterized by, comprising:
the parameter acquisition module is used for acquiring parameter data in a allowance frame residual error network configuration file, wherein the parameter data comprises parameter names and parameter values corresponding to the parameter names;
the corresponding establishing module is used for establishing the corresponding relation between the parameter name and each position in the OpenCL template;
the parameter filling module is used for filling corresponding parameter values to corresponding positions of the template by utilizing the corresponding relation;
the target output module is used for outputting and obtaining a target format configuration file;
wherein, the correspondence establishing module comprises:
a dictionary storage unit for storing the parameter data in a python dictionary in groups;
and the dictionary mapping unit is used for establishing the corresponding relation between the parameter name and each position in the OpenCL template by utilizing the mapping function of the python dictionary.
7. The system of claim 6, wherein the parameter obtaining module comprises:
a file obtaining unit, configured to obtain the configuration file;
and the parameter extraction unit is used for extracting parameter data from the configuration file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710770739.4A CN107480115B (en) | 2017-08-31 | 2017-08-31 | Method and system for format conversion of caffe frame residual error network configuration file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710770739.4A CN107480115B (en) | 2017-08-31 | 2017-08-31 | Method and system for format conversion of caffe frame residual error network configuration file |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107480115A CN107480115A (en) | 2017-12-15 |
| CN107480115B true CN107480115B (en) | 2021-04-06 |
Family
ID=60604319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710770739.4A Active CN107480115B (en) | 2017-08-31 | 2017-08-31 | Method and system for format conversion of caffe frame residual error network configuration file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107480115B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920177A (en) * | 2018-06-28 | 2018-11-30 | 郑州云海信息技术有限公司 | Mapping method of the deep learning model configuration file to FPGA configuration file |
| CN110689138B (en) * | 2018-12-29 | 2021-03-19 | 中科寒武纪科技股份有限公司 | Operation method, device and related product |
| CN109947464A (en) * | 2019-03-22 | 2019-06-28 | 优信拍（北京）信息科技有限公司 | A kind of configuration update method and device |
| CN112015470B (en) * | 2020-09-09 | 2022-02-01 | 平安科技（深圳）有限公司 | Model deployment method, device, equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11087234B2 (en) * | 2016-01-29 | 2021-08-10 | Verizon Media Inc. | Method and system for distributed deep machine learning |
| CN106355244B (en) * | 2016-08-30 | 2019-08-13 | 深圳市诺比邻科技有限公司 | The construction method and system of convolutional neural networks |
| CN106844693A (en) * | 2017-01-24 | 2017-06-13 | 浙江大学 | A kind of conversion methods of openEHR Template to relational database |
| CN107016175B (en) * | 2017-03-23 | 2018-08-31 | 中国科学院计算技术研究所 | It is applicable in the Automation Design method, apparatus and optimization method of neural network processor |
-
2017
- 2017-08-31 CN CN201710770739.4A patent/CN107480115B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN107480115A (en) | 2017-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107480115B (en) | Method and system for format conversion of caffe frame residual error network configuration file | |
| CN110287480A (en) | A kind of name entity recognition method, device, storage medium and terminal device | |
| CN109635838A (en) | Face samples pictures mask method, device, computer equipment and storage medium | |
| CN109670191A (en) | Calibration optimization method, device and the electronic equipment of machine translation | |
| JP6838161B2 (en) | End-to-end modeling methods and systems | |
| CN103823857B (en) | Space information searching method based on natural language processing | |
| CN110910283A (en) | Method, device, equipment and storage medium for generating legal document | |
| CN110673840A (en) | Automatic code generation method and system based on tag graph embedding technology | |
| CN111368544B (en) | Named entity identification method and device | |
| CN112069319A (en) | Text extraction method and device, computer equipment and readable storage medium | |
| CN115438215A (en) | Image-text bidirectional search and matching model training method, device, equipment and medium | |
| CN113157917B (en) | OpenCL-based optimized classification model establishing and optimized classification method and system | |
| US20220414463A1 (en) | Automated troubleshooter | |
| CN115222950A (en) | Lightweight target detection method for embedded platform | |
| CN107526717B (en) | Method for automatically generating natural language text by structured process model | |
| CN116994021A (en) | Image detection method, device, computer readable medium and electronic equipment | |
| CN111241843B (en) | Semantic relation inference system and method based on composite neural network | |
| CN117093260B (en) | Fusion model website structure analysis method based on decision tree classification algorithm | |
| CN110489348B (en) | Software functional defect mining method based on migration learning | |
| CN116992880A (en) | Building name identification method, device, electronic equipment and storage medium | |
| CN110197175A (en) | A kind of method and system of books title positioning and part-of-speech tagging | |
| CN115271055A (en) | Neural network software and hardware cooperative detection method, device, equipment and storage medium | |
| US20220067576A1 (en) | Automatically labeling functional blocks in pipelines of existing machine learning projects in a corpus adaptable for use in new machine learning projects | |
| CN105808522A (en) | Method and apparatus for semantic association | |
| CN111368091A (en) | Document translation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |