CN112748953A - Data processing method and device based on neural network model and electronic equipment - Google Patents
Data processing method and device based on neural network model and electronic equipment Download PDFInfo
- Publication number
- CN112748953A CN112748953A CN202010633667.0A CN202010633667A CN112748953A CN 112748953 A CN112748953 A CN 112748953A CN 202010633667 A CN202010633667 A CN 202010633667A CN 112748953 A CN112748953 A CN 112748953A
- Authority
- CN
- China
- Prior art keywords
- model
- neural network
- data
- network model
- execution logic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/76—Adapting program code to run in a different environment; Porting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Neurology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides a data processing method and device based on a neural network model and electronic equipment, and relates to the technical field of artificial intelligence and machine learning. The method comprises the following steps: acquiring input data, model parameter data and corresponding model operation of a neural network model; acquiring current operation environment information of a neural network model, and determining a corresponding operation execution logic code; and obtaining output data of the neural network model by running operation execution logic codes based on the input data, the model parameter data and the model operation. According to the technical scheme provided by the disclosure, model operation and operation execution logic codes corresponding to the neural network model are mutually separated, the model operation can be compatible with different operation environments, and the model operation can be executed by combining the corresponding operation execution logic codes, so that the model operation and operation execution logic codes have transportability; the model operation and the operation execution logic code are separated, so that the maintenance and the modification are convenient to respectively carry out.
Description
Technical Field
The application relates to the technical field of artificial intelligence and machine learning, in particular to a data processing method and device based on a neural network model and an electronic device.
Background
The neural network model is a mathematical model that simulates the human actual neural network. The neural network model has wide application in the fields of system identification, pattern recognition, intelligent control and the like. In the development process of a neural network model, the research of a learning algorithm is of great importance.
At present, neural network models proposed by people correspond to learning algorithms. The neural network model learns the internal rules and the expression levels of the sample data through training, and the trained model can be used for recognizing characters, images, sounds and the like. After the neural network model is trained, the neural network model needs to be operated on different equipment and different platforms when in use.
In the prior art, programmers need to write model operation codes and operation execution logic codes of a neural network respectively according to different operation environments (platforms or equipment) for operating a neural network model, and the model operation codes and the operation execution logic codes are not portable and have higher requirements on program coding personnel; moreover, the operation execution logic code is mixed with the model operation code of the neural network, and is difficult to maintain or modify.
Disclosure of Invention
The embodiment of the application provides a data processing method and device based on a neural network model and electronic equipment, and can solve at least one of the problems in the prior art.
The embodiment of the application provides the following specific technical scheme:
the embodiment of the application provides a data processing method based on a neural network model, which comprises the following steps:
acquiring input data, model parameter data and corresponding model operation of a neural network model;
acquiring current operation environment information of a neural network model;
determining an operation execution logic code corresponding to the current operation environment information according to the current operation environment information;
based on the input data, the model parameter data and the model operation, executing logic codes through operation to obtain output data of the neural network model;
wherein the input data comprises any one of:
voice data, text data, image data, video data.
The embodiment of the application provides a data processing device based on a neural network model, and the device comprises:
the first acquisition module is used for acquiring input data of the neural network model, model parameter data and corresponding model operation;
the second acquisition module is used for acquiring the current operation environment information of the neural network model;
the first determining module is used for determining an operation execution logic code corresponding to the current operating environment information according to the current operating environment information;
the second determination module is used for executing the logic code by running operation based on the input data, the model parameter data and the model operation to obtain the output data of the neural network model;
wherein the input data comprises any one of:
voice data, text data, image data, video data.
An embodiment of the present application further provides an electronic device, which includes one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the methods as provided by the embodiments of the present application.
Embodiments of the present application further provide a computer-readable storage medium, which is used for storing a computer program, and when the computer program runs on a processor, the processor can be enabled to execute the method provided by the embodiments of the present application.
The beneficial effect that technical scheme that this application provided brought is:
the application provides a data processing method, a data processing device and electronic equipment based on a neural network model, which are used for acquiring input data, model parameter data, corresponding model operation of the neural network model and current operation environment information of the neural network model, and determining corresponding operation execution logic codes according to the current operation environment information; the model operation and the operation execution logic code corresponding to the neural network model are mutually separated, different operation environments can be compatible under the condition that the model operation of the neural network model is not changed, the model output data is obtained by combining the operation of the corresponding operation execution logic code, and the model operation has portability; the model operation and the operation execution logic code are separated, so that the maintenance and the modification are convenient to respectively carry out.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic diagram of a multi-level implementation of a data processing method based on a neural network model according to an embodiment of the present application;
fig. 2 is a flowchart of a data processing method based on a neural network model according to an embodiment of the present application;
fig. 3 is a schematic diagram of a workflow of a data processing method based on a neural network model according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus based on a neural network model according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.
The execution main body of the technical scheme can be any electronic device running a neural network model, for example, the execution main body can be a server, the server can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and the execution main body can also be a cloud server providing basic cloud computing services such as cloud services, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a CDN (content distribution network), big data and an artificial intelligence platform.
For the trained neural network model, in practical use, the trained neural network model needs to be operated on different platforms and different devices, so that the trained neural network model needs to be compatible with different platforms and different devices. In the prior art, generally, after a programmer understands the structure of a neural network model, model operation and operation execution logic codes are written according to a specific operation environment for operating the neural network model, and if the operation environment is changed, the programmer needs to write a set of operation codes again, so that the code portability is poor; furthermore, the operation code is mixed with the model operation and the operation execution logic code, and thus it is difficult to separately maintain and modify the operation code. In addition, the model operation and the operation execution logic code are executed serially, and the model operation can be performed only after the operation execution logic code is executed, which results in low data processing efficiency.
In order to solve the problem of mixing model operation and operation execution logic codes, the invention separates the model operation and the operation execution logic codes and executes the operation and the operation respectively through different levels. Through a multi-level implementation mode, different running environments can be compatible under the condition that model operation of the neural network model is not changed, the model output data is obtained by combining corresponding operation execution logic code running, and model operation has transportability; the model operation and the operation execution logic code are separated, so that the maintenance and the modification are convenient to respectively carry out; the model operation and the operation execution logic code are executed in parallel, and the data processing efficiency is improved.
The following describes a process implemented by multiple hierarchies according to an embodiment of the present invention.
Fig. 1 is a schematic diagram of a logic architecture of a data processing method based on a neural network model according to an embodiment of the present application, where as shown in the diagram, the architecture may include an encapsulation layer, an operation execution logic processing layer, a model operation processing layer, and a framework layer, and the logic architecture is combined to describe the principle of the method according to the embodiment of the present application.
As shown in fig. 1, the encapsulation layer (also referred to as layer) is used to encapsulate the model operation corresponding to the neural network model, and may also be understood as a class for implementing a common neural network operation, and the function of the encapsulation layer may be understood as similar to that of the layers in tensoflow, and the specific implementation manner is different. And the packaging layer packages the model parameter data and the corresponding model operation.
The Operation execution logic processing layer (also referred to as an Operation layer) is used for acquiring input data, model parameter data and current Operation environment information, and determining an Operation execution logic code corresponding to the current Operation environment information, namely a specific model Operation code, and the function of the Operation execution logic processing layer can be understood to be similar to the tf.nn \ tf.math layer of tensoflow, and the like, and the specific implementation mode is different. The model operation may include: matrix multiplication matmul, matrix connection concat, matrix splitting slice, linear rectification function relu and other functions.
The model operation processing layer (also called Interface (inference/forward prediction) layer) is used for obtaining output data of the neural network model by running operation execution logic codes based on input data, model parameter data and model operation. The model operation Processing layer may rely on a Central Processing Unit (CPU) Library (such as Math Kernel Library (MKL)/Graphics Processing Unit (GPU) Library (such as Control Unit Basic Linear Algebra Subsystems (CUBLAS)), which includes languages and tool sets, and a CPU Library (such as languages and tool sets in Open Multi-Processing (OPENMP), and a GPU Library (such as a Unified computing Device framework (CUDA)), during data Processing, each Device (GPU/CPU) corresponds to an implementation manner of the respective model operation Processing layer, for example, the CU DA corresponds to the implementation mode of the CU DA Interface layer, and the MKL corresponds to the implementation mode of the MKLinterface layer.
The Framework layer (also called Framework layer) is composed of a format conversion component Tensor, a data load component Dataloader, a Session component Session and a Device component Device. The sensor is used for carrying out format conversion on the input data loaded by the Dataloader according to the format of the input data of the neural network model to obtain a converted input data matrix; the Dataloader is used for loading model parameters of the neural network model; session is used to create a job processing thread. The Device is used for managing the matrix operation equipment, and if the equipment is a GPU, the Device management object is the GPU; if the Device is a CPU, the Device management object is a set of operation threads. The Device can run N (an integer greater than or equal to 1) job processing threads created by sessions, the operation inside each job processing thread is executed in sequence, and the operation sequence between the job processing threads is not limited.
Based on the logic structure shown in fig. 1, when a neural network model is operated in a server, the server receives a calculation task of the neural network model sent by a user terminal, and a format conversion component Tensor in a framework layer performs format conversion on input data according to a format of the input data of the neural network model to obtain a converted input data matrix; the Dataloader component loads model parameters of the neural network model, if the model parameters run on a CPU, the Session component Session creates a job processing thread corresponding to a calculation task, and the Device manages the job processing thread; if running on the GPU, the Device manages the GPU. The encapsulation layer encapsulates model parameter data (each model parameter inside the neural network model) and corresponding model operation operations (matrix multiplication matmul, matrix connection concat, matrix splitting slice, linear rectification function relu and the like); the operation execution logic processing layer determines an operation execution logic code corresponding to the GPU or the CPU based on the current operation environment information (operated on the GPU or the CPU); and the model operation processing layer runs operation execution logic codes corresponding to a GPU or a CPU based on the input data matrix, the model parameter data and the model operation and the job processing thread created by the Session to obtain the output data of the neural network model.
An embodiment of the present application provides a data processing method based on a neural network model, as shown in fig. 2, the method includes:
step S101, obtaining input data of a neural network model, model parameter data and corresponding model operation;
the neural network model is a mathematical model simulating a human actual neural network, optionally, the neural network model of the embodiment of the present application may be a deep learning model, and may include but is not limited to: a convolutional neural network model, a generative antagonistic neural network model, a recurrent neural network model and a long-short term memory neural network model. The neural network model in the embodiment of the application is obtained by training an initial neural network model by using a training sample, iterative training is carried out on the initial neural network model by using the training sample, and when a training end condition is met, the neural network model which can be applied on line is obtained, namely, the trained neural network model can be operated in electronic equipment, such as a server.
In practical application, a corresponding training sample can be obtained according to practical application requirements, namely, functions which are to be realized by the neural network model, so that the model can be trained, and the model with a good prediction effect can be obtained.
Wherein the input data comprises any one of:
voice data, text data, image data, video data.
It will be appreciated that the type of input data is determined by the neural network model, and that the type of input data for the model may be the same or different for different neural network models. For example, for a text processing model (e.g., a text translation model), the input data may include text data, and for a speech data processing model (e.g., a speech translation model), the input data may include speech data. That is, different neural network models determine the type of input data from the trained neural network model. If the neural network model can be a speech recognition model (for example, speech data is converted into text), and input data of the model is speech data, the speech data to be translated can be processed through the model, so that a text corresponding to the speech data is obtained (i.e., predicted); the neural network model may also be a text image recognition model, and the input data is image data, and characters in the image can be recognized through the model to obtain a recognition result, or may also be a target detection model, and a target object in the input image data or video data can be detected or tracked through the model.
The model parameter data (which may also be referred to as model parameters, network parameters, or model parameter matrix, etc.) may be each model parameter inside the neural network model, the initial model parameters of the initial neural network model are trained by using training samples, and when a training end condition is met, each trained model parameter data may be obtained. For example, the neural network model may be a deep learning model, and the model parameter data may include a weight matrix and a bias matrix corresponding to each hidden layer of the model. When training of the neural network model is complete, the model parameter data may be saved as part of the neural network model. When the trained neural network model is used for prediction, the model parameter data defines the usable neural network model, that is, when the neural network model is used for prediction, the model input data is operated based on the trained model parameter data to obtain a corresponding operation result. For example, if a hidden layer of the neural network model is a convolutional layer, when input data (which may be model input data or output data of other hidden layers) of the convolutional layer is processed by the convolutional layer, convolution calculation is performed on the input data by convolution parameters of the convolutional layer to obtain output data of the convolutional layer.
The model operation may include an operation that needs to be performed and is determined according to a model structure of the neural network model, and specifically may include various operation operations performed on input data when the trained neural network model is used for prediction, that is, operation operations corresponding to hidden layers included in the model, such as multiplication operations of a convolution parameter matrix of a convolution layer and an input data matrix of the convolution layer. For a constructed neural network model, the model architecture of the model is already fixed, which operation operations need to be executed by the model, and the relationship between different operation operations are also fixed. For example, for a speech translation model, the model includes two parts, i.e. an encoder and a decoder, assuming that the first two hidden layers of the encoder are two convolutional layers which are sequentially cascaded, the operation of the model includes the convolution operations corresponding to the two convolutional layers, and the input data of the operation of the subsequent convolutional layer is the output data of the previous convolutional layer.
Because the neural network model in the technical scheme of the application is trained, the model parameter data and the model operation can be directly obtained from the designated storage space. It can be understood that, in the embodiment of the present application, the model operation includes specific calculation operations and a sequence between calculation operations having a sequential execution order, where, as described above, the convolution operation operations corresponding to the two convolutional layers, and the operation sequence of the convolution operation corresponding to the next convolutional layer is located after the convolution operation corresponding to the previous convolutional layer.
The input data is data to be predicted by using the neural network model, and the input data can be input data when the model is operated on line or test data in a test data set.
In an example, assuming that the neural network model is a speech translation model, data processing from speech to text can be performed by using the speech translation model, input data of the model may be speech data to be processed, and the neural network model outputs corresponding text data after predicting the input speech data.
In another example, the data of the reading application program that converts the text into the emotional reading voice is processed by using the neural network model, the input data may be the text data to be processed, and the neural network model outputs the corresponding emotional reading voice after predicting the input text data.
A specific neural network model and model parameter data are described below by way of an example:
in one possible implementation, the neural network model includes a deep learning model, and the model parameter data includes model weights and model biases.
In practical applications, the neural network model may be a deep learning model. The deep learning model may be a neural network model with a large number of parameters and layers in any of the following four basic network frameworks: unsupervised Pre-trained Networks (UPN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN).
Training the initial deep learning model by using the training sample, and obtaining model parameters of the deep learning model when the training end condition is met, namely model weight and model bias, wherein the specific deep learning model can be determined by the value of the model weight and the value of the model bias.
The model operation of the neural network model is described below by one embodiment.
In one possible implementation, the model arithmetic operations include arithmetic operations of hidden layers of the neural network model.
In practical applications, the model operation may include an operation of each hidden layer determined according to a model structure of the neural network model (i.e., a specific structure of each hidden layer included in the model, i.e., a convolutional layer, a pooling layer, a full-link layer, and the like, and a connection relationship between the hidden layers).
The neural network model may include one hidden layer, or may include two or more hidden layers. The hidden layer is used for abstracting the characteristics of input data to another dimensional space to show more abstract characteristics, and the characteristics can be better linearly divided. The model operation may include an operation performed by each hidden layer to process input data.
In addition, after the input data of the neural network model is acquired, a step of preprocessing the input data is further included, which is specifically seen in the following embodiment.
In one possible implementation, after obtaining the input data of the neural network model, the method may further include:
and carrying out format conversion on the input data according to the format of the input data of the neural network model to obtain a converted input data matrix.
In practical application, for a neural network model, input data of the model usually has corresponding data format requirements, so that after the input data of the neural network model is obtained, format conversion can be performed on the input data according to a data structure required by the neural network model to obtain an input data matrix, the input data matrix is input into the neural network model, and each hidden layer of the neural network model is predicted based on the input data matrix.
In an example, a neural network model is used for data processing from speech to text, input data of the neural network model is speech data, the speech data can be processed into a corresponding feature vector matrix according to an input data format requirement of the model, for example, preliminary speech features (such as mel frequency cepstrum coefficients and the like) can be extracted from the speech data, and the preliminarily extracted speech features can be converted into an input data matrix meeting the format requirement of the model input data by feature embedding or other processing methods. And inputting the input data matrix into the neural network model to obtain predicted text data.
In yet another example, if the image recognition is performed by using a neural network model, and the input data of the neural network model is image data, preliminary image features may be extracted from the image data, and the preliminary extracted image features may be converted into a corresponding input data matrix according to the input data requirements of the model. And inputting the input data matrix into the neural network model, and predicting to obtain an identification result corresponding to the image data.
In the embodiment of the application, the format conversion is carried out on the input data, so that the neural network model can conveniently process the input data, and the efficiency and the accuracy of data processing can be improved.
Step S102, obtaining current operation environment information of a neural network model;
step S103, determining an operation execution logic code corresponding to the current operation environment information according to the current operation environment information;
when the trained neural network model is used on line, it may be run on different devices (e.g. GPU/CPU) and/or different platforms to implement the prediction function, i.e. the forward prediction function of the model, so for different model operating environments, it is necessary to make the neural network model compatible with data processing modes of various different operating environments, where the operating environment information may be hardware or software information corresponding to the devices.
In the prior art, when a neural network model is run on different devices (such as a GPU/CPU) and/or different platforms, generally, a programmer understands the structure of the neural network model, and then writes model operation and operation execution logic codes according to the specific running environment of the neural network model, and if the running environment changes, the programmer needs to write a set of running codes again, and the code portability is poor. In order to solve the above problem, the method provided in this embodiment of the present application may configure corresponding operation execution logic codes for different model operation environments without changing model operation operations, and when a model operates in a certain operation environment, current operation environment information may be obtained, and the corresponding operation execution logic codes may be determined according to the current operation environment information.
When the trained neural network model is operated on a certain operation platform, that is, when the model is loaded on a certain platform (such as a server), the current operation environment information of the platform can be preconfigured and stored in a specified storage space, and when forward prediction is performed based on the model, the server can acquire the current operation environment information in the specified storage space.
The current operating environment information includes what contents, as shown in the following embodiments.
In one possible implementation, the current operating environment information includes at least one of the following information:
processor configuration information, operating system configuration information.
In practical applications, the current operating environment information includes at least one of processor configuration information and operating system configuration information of a device (GPU/CPU) that runs the neural network model. Operating system configuration information may include, but is not limited to, window or Linux configuration information.
After the current operating environment information is acquired, the corresponding operation execution logic code may be determined according to the operating environment information. The operation execution logic code may be code that implements operation execution logic.
Each device corresponds to a respective operation execution logic code base, and the operation execution logic code base stores operation execution logic codes.
In one example, the operation execution logic code Library corresponding to the CPU includes a Math Kernel Library (MKL); the operation execution logic code base corresponding to the GPU comprises a controller Basic Linear Algebra subprogram (CU BLAS); the operation execution logic code library of the CPU depends on languages, tool sets, and the like in an underlying library (Open Multi-Processing, OPENMP), and the operation execution logic code library of the GPU depends on languages, tool sets, and the like in a Unified computing Device framework (CU DA) of the underlying library.
In addition, after the input data is acquired, a step of performing validity check on the input data may be further included. For example, for two matrices that need to be multiplied, the number of rows and columns corresponding to each matrix is determined to meet the requirement for matrix multiplication, and if not, the matrix is illegal, and the illegal data is discarded.
The specific implementation manner of determining the corresponding operation execution logic code according to the current operating environment information is shown in the following embodiments.
In one possible implementation manner, determining, according to the current operating environment information, an operation execution logic code corresponding to the current operating environment information includes:
and determining the operation execution logic code corresponding to the current operation environment according to the current operation environment information and a pre-configured mapping relationship, wherein the mapping relationship is the corresponding relationship between the operation environment information and the operation execution logic code corresponding to the operation environment information.
In practical application, the corresponding relationship between the running environment information and the corresponding operation execution logic code is preconfigured, and the operation execution logic code corresponding to the current running environment can be determined according to the current running environment information and the preconfigured corresponding relationship. Specifically, the corresponding relationship between the processor configuration information and the corresponding operation execution logic code may be preconfigured, and the corresponding operation execution logic code may be determined according to the processor configuration information; or the corresponding relation between the operating system configuration information and the corresponding operation execution logic code can be preset, and the corresponding operation execution logic code can be determined according to the operating system configuration information; and the processor configuration information and the operating system configuration information can be configured, and the corresponding operation execution logic code can be determined according to the configuration processor configuration information and the operating system configuration information.
And step S104, based on the input data, the model parameter data and the model operation, executing logic codes through operation to obtain output data of the neural network model.
Specifically, after the operation execution logic code corresponding to the current operation environment is acquired, the operation execution logic code is operated, the specific structure of the neural network model is determined according to the model parameter data, the input data is input into the neural network model, and the model operation is operated to obtain the output data of the neural network model.
In one example, the convolutional neural network model is utilized to perform image recognition, input data of the convolutional neural network model, namely image data to be recognized, is obtained, model parameter data of the convolutional neural network model and corresponding model operation are obtained, operation execution logic codes corresponding to current operation environment information are corresponding codes in a CU BLAS library corresponding to a GPU, and output data of the convolutional neural network model, namely a prediction result corresponding to the image data to be recognized are obtained by operating the corresponding codes in the CU BLAS library based on the image data to be recognized, the model parameter data of the convolutional neural network model and the model operation.
In one possible implementation, obtaining output data of the neural network model by executing the operation execution logic code based on the input data, the model parameter data, and the model operation includes:
based on the input data, the model parameter data and the model operation, the operation execution logic code is operated to create threads for executing the calculation tasks corresponding to the model operation operations in a parallel processing mode and the created threads are used for executing the corresponding calculation tasks.
In practical application, the server acquires input data, model parameter data, model operation operations, and operation execution logic codes, creates a thread for executing a calculation task corresponding to each model operation, and executes a corresponding calculation task by using the created thread. When creating the thread for executing the calculation task corresponding to each model operation, it is not necessary to wait for the execution result of the calculation task, and when creating the thread for the next calculation task requires the execution result of the previous calculation task (the programmer pre-configures which threads for creating the calculation tasks require the execution result of the previous calculation task), it waits for the execution result, that is, the threads for executing the calculation tasks corresponding to each model operation are created in a parallel processing manner, and the created threads are used for executing the corresponding calculation tasks, thereby improving the efficiency of data processing.
In the prior art, because model arithmetic operations and arithmetic execution logic codes are mixed together and cannot be executed respectively, creating a thread for executing a calculation task corresponding to each model arithmetic operation and executing a corresponding calculation task by using the created thread can only be executed serially, creating the thread of the calculation task corresponding to the model arithmetic operations can create a thread of a next calculation task after an execution result of the calculation task is obtained, and the data processing efficiency is low.
The following describes the data processing flow of the technical solution of the present application in detail by using a specific embodiment.
As shown in fig. 3, the server obtains input data, model parameter data, model operation, and operation execution logic code, and the request receiving module (request receiving shown in the figure) of the server receives a data processing request: the request content is to create a thread corresponding to a computation task, the computation task is multiplied by a GPU (matmul (a, b) shown in the figure) based on BLAS library computation matrixes a and b, a thread creation module (logic thread shown in the figure) creates a thread corresponding to the computation task, sends the computation task (adding task shown in the figure) to a task computation module (operation thread shown in the figure), the task computation module executes the computation task through the created thread, and in the computation task execution process, the thread creation module returns a placeholder to the task computation module, wherein the placeholder represents an operation result corresponding to the multiplication of the matrixes a and b or a data format of the operation result, but not an actual operation result of the multiplication of the matrixes a and b; the request receiving module receives the data processing request again: the request content is to create a thread corresponding to a computation task, the computation task is a multiplication of a computation matrix c and d (matmul (c, d) shown in the figure) by a GPU based on a BLAS library, a thread creation module creates a thread corresponding to the computation task and sends the computation task to a task computation module, the task computation module executes the computation task through the created thread, the thread creation module returns a placeholder to the task computation module, the placeholder represents an operation result corresponding to the multiplication of the matrix c and d or a data format of the operation result, but not an actual operation result of the multiplication of the matrix c and the d, when the computation result of the computation task needs to be returned, a request receiving module requests the computation result (the request receiving module requests the computation result when a programmer configures the conditions which are met in advance), and the task computation result is forwarded to the request receiving module by the thread creation module, the request receiving module waits for a result to be returned.
In this embodiment, creating a thread for executing a computation task corresponding to each model operation and executing a corresponding computation task using the created thread are performed in parallel, and when an execution result of the computation task is required, the execution result is returned, otherwise, creating a thread of a computation task corresponding to a model operation does not need to wait for the execution result of the computation task, thereby improving the efficiency of data processing.
The application provides a data processing method based on a neural network model, which comprises the steps of obtaining input data, model parameter data, corresponding model operation of the neural network model and current operation environment information of the neural network model, and determining corresponding operation execution logic codes according to the current operation environment information, wherein the model operation and the operation execution logic codes corresponding to the neural network model are mutually separated, different operation environments can be compatible under the condition that the model operation of the neural network model is not changed, the operation is combined with the corresponding operation execution logic codes to operate, so that model output data is obtained, and the model operation has transportability; the model operation and the operation execution logic code are separated, so that the maintenance and the modification are convenient to respectively carry out.
Based on the same principle as the method shown in fig. 2, an embodiment of the present disclosure further provides a data processing apparatus 30 based on a neural network model, as shown in fig. 4, where the apparatus 30 includes:
a first obtaining module 31, configured to obtain input data of a neural network model, model parameter data, and corresponding model operation;
a second obtaining module 32, configured to obtain current operating environment information of the neural network model;
a first determining module 33, configured to determine, according to the current operating environment information, an operation execution logic code corresponding to the current operating environment information;
a second determining module 34, configured to execute the logic code by running operation based on the input data, the model parameter data, and the model operation to obtain output data of the neural network model;
wherein the input data comprises any one of: voice data, text data, image data, video data.
In one possible implementation, the current operating environment information includes at least one of the following information:
processor configuration information, operating system configuration information.
In a possible implementation, the data processing apparatus 30 further includes a preprocessing module configured to:
and carrying out format conversion on the input data according to the format of the input data of the neural network model to obtain a converted input data matrix.
In a possible implementation manner, the first determining module 33 is configured to:
and determining the operation execution logic code corresponding to the current operation environment according to the current operation environment information and a pre-configured mapping relationship, wherein the mapping relationship is the corresponding relationship between the operation environment information and the operation execution logic code corresponding to the operation environment information.
In one possible implementation, the neural network model includes a deep learning model, and the model parameter data includes model weights and model biases.
In one possible implementation, the model arithmetic operations include arithmetic operations of hidden layers of the neural network model.
In a possible implementation manner, the second determining module 34 is configured to:
based on the input data, the model parameter data and the model operation, the operation execution logic code is operated to create threads for executing the calculation tasks corresponding to the model operation operations in a parallel processing mode and the created threads are used for executing the corresponding calculation tasks.
The data processing apparatus based on the neural network model according to the embodiment of the present disclosure may execute the data processing method based on the neural network model provided in the embodiment of the present disclosure, and the implementation principle is similar, the actions performed by each module in the data processing apparatus based on the neural network model according to the embodiments of the present disclosure correspond to the steps in the data processing method based on the neural network model according to the embodiments of the present disclosure, and for the detailed functional description of each module of the data processing apparatus based on the neural network model, reference may be specifically made to the description in the corresponding data processing method based on the neural network model shown in the foregoing, and details are not repeated here.
The application provides a data processing device based on a neural network model, which acquires input data, model parameter data, corresponding model operation of the neural network model and current operation environment information of the neural network model, and determines corresponding operation execution logic codes according to the current operation environment information, wherein the model operation and the operation execution logic codes corresponding to the neural network model are mutually separated, different operation environments can be compatible under the condition of not changing the model operation of the neural network model, the operation is combined with the corresponding operation execution logic codes to operate, so that model output data is obtained, and the model operation has transportability; the model operation and the operation execution logic code are separated, so that the maintenance and the modification are convenient to respectively carry out.
The above embodiment introduces a data processing apparatus based on a neural network model from the perspective of a virtual module, and the following introduces an electronic device from the perspective of a physical module, as follows:
an embodiment of the present application provides an electronic device, as shown in fig. 5, an electronic device 9000 shown in fig. 5 includes: a processor 9001 and a memory 9003. Among other things, the processor 9001 and memory 9003 are coupled, such as via a bus 9002. Optionally, the electronic device 9000 can also include a transceiver 9004. Note that the transceiver 9004 is not limited to one in practical use, and the structure of the electronic device 9000 is not limited to the embodiment of the present application.
The processor 9001 may be a CPU, general purpose processor, GPU, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 9001 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of DSPs and microprocessors, or the like.
The bus 9002 may include a pathway to transfer information between the aforementioned components. The bus 9002 may be a PCI bus or an EISA bus, etc. The bus 9002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.
The memory 9003 may be a ROM or other type of static storage device that may store static information and instructions, a RAM or other type of dynamic storage device that may store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to.
The memory 9003 is used to store application code for performing aspects of the present application and is controlled by the processor 9001 for execution. The processor 9001 is configured to execute application program code stored in the memory 9003 to implement any of the method embodiments shown above.
An embodiment of the present application provides an electronic device, where the electronic device includes: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs, when executed by the processors, obtaining input data for a neural network model, model parameter data, and corresponding model arithmetic operations; acquiring current operation environment information of a neural network model; determining an operation execution logic code corresponding to the current operation environment information according to the current operation environment information; based on the input data, the model parameter data and the model operation, executing logic codes through operation to obtain output data of the neural network model; wherein the input data comprises any one of: voice data, text data, image data, video data.
The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a processor, enables the processor to execute the corresponding content in the foregoing method embodiments.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the above-mentioned neural network model-based data processing method or various alternative implementations of the neural network model-based data processing method.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (11)
1. A data processing method based on a neural network model is characterized by comprising the following steps:
acquiring input data, model parameter data and corresponding model operation of a neural network model;
acquiring current operation environment information of the neural network model;
determining an operation execution logic code corresponding to the current operation environment information according to the current operation environment information;
obtaining output data of the neural network model by running the operation execution logic code based on the input data, the model parameter data, and the model operation;
wherein the input data comprises any one of:
voice data, text data, image data, video data.
2. The method of claim 1, wherein obtaining the output data of the neural network model by executing the operation execution logic code based on the input data, model parameter data, and model operation comprises:
and based on the input data, the model parameter data and the model operation, creating a thread for executing a calculation task corresponding to each model operation in a parallel processing mode by running the operation execution logic code, and executing the corresponding calculation task by using the created thread.
3. The method of claim 1, wherein the current operating environment information comprises at least one of:
processor configuration information, operating system configuration information.
4. The method of claim 1, wherein after obtaining input data for the neural network model, the method further comprises:
and carrying out format conversion on the input data according to the format of the input data of the neural network model to obtain a converted input data matrix.
5. The method of claim 1, wherein determining, according to the current operating environment information, an operation execution logic code corresponding to the current operating environment information comprises:
and determining an operation execution logic code corresponding to the current operation environment according to the current operation environment information and a pre-configured mapping relationship, wherein the mapping relationship is the corresponding relationship between the operation environment information and the operation execution logic code corresponding to the operation environment information.
6. The method of any one of claims 1-5, wherein the neural network model comprises a deep learning model, and the model parameter data comprises model weights and model biases.
7. The method of any one of claims 1-5, wherein the model arithmetic operations comprise arithmetic operations of hidden layers of the neural network model.
8. A data processing apparatus based on a neural network model, the apparatus comprising:
the first acquisition module is used for acquiring input data of the neural network model, model parameter data and corresponding model operation;
the second acquisition module is used for acquiring the current operation environment information of the neural network model;
the first determining module is used for determining an operation execution logic code corresponding to the current operation environment information according to the current operation environment information;
a second determining module, configured to obtain output data of the neural network model by running the operation execution logic code based on the input data, the model parameter data, and the model operation;
wherein the input data comprises any one of:
voice data, text data, image data, video data.
9. The apparatus of claim 8, wherein the first determining module, when determining the operation execution logic code corresponding to the current operating environment information according to the current operating environment information, is configured to:
and determining an operation execution logic code corresponding to the current operation environment according to the current operation environment information and a pre-configured mapping relationship, wherein the mapping relationship is the corresponding relationship between the operation environment information and the operation execution logic code corresponding to the operation environment information.
10. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a memory;
one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-7.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium is for storing a computer program which, when run on a processor, causes the processor to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010633667.0A CN112748953B (en) | 2020-07-02 | 2020-07-02 | Data processing method and device based on neural network model and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010633667.0A CN112748953B (en) | 2020-07-02 | 2020-07-02 | Data processing method and device based on neural network model and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112748953A true CN112748953A (en) | 2021-05-04 |
| CN112748953B CN112748953B (en) | 2023-08-15 |
Family
ID=75645281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010633667.0A Active CN112748953B (en) | 2020-07-02 | 2020-07-02 | Data processing method and device based on neural network model and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112748953B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298259A (en) * | 2021-06-10 | 2021-08-24 | 中国电子科技集团公司第十四研究所 | CNN network reasoning framework design method supporting embedded platform multi-core parallel |
| WO2023050809A1 (en) * | 2021-09-30 | 2023-04-06 | 浪潮电子信息产业股份有限公司 | Method for training distributed neural network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040122785A1 (en) * | 2000-12-15 | 2004-06-24 | International Business Machines Corporation | Method, system, and program for converting application program code to executable code using neural networks based on characteristics of the inputs |
| CN109597965A (en) * | 2018-11-19 | 2019-04-09 | 深圳力维智联技术有限公司 | Data processing method, system, terminal and medium based on deep neural network |
| CN109643229A (en) * | 2018-04-17 | 2019-04-16 | 深圳鲲云信息科技有限公司 | The application and development method and Related product of network model |
| CN110569106A (en) * | 2019-08-27 | 2019-12-13 | Oppo广东移动通信有限公司 | Code loading method and device, electronic equipment and computer readable medium |
| CN110659728A (en) * | 2019-09-24 | 2020-01-07 | 上海寒武纪信息科技有限公司 | Neural network optimization method and device, computer equipment and storage medium |
-
2020
- 2020-07-02 CN CN202010633667.0A patent/CN112748953B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040122785A1 (en) * | 2000-12-15 | 2004-06-24 | International Business Machines Corporation | Method, system, and program for converting application program code to executable code using neural networks based on characteristics of the inputs |
| CN109643229A (en) * | 2018-04-17 | 2019-04-16 | 深圳鲲云信息科技有限公司 | The application and development method and Related product of network model |
| CN109597965A (en) * | 2018-11-19 | 2019-04-09 | 深圳力维智联技术有限公司 | Data processing method, system, terminal and medium based on deep neural network |
| CN110569106A (en) * | 2019-08-27 | 2019-12-13 | Oppo广东移动通信有限公司 | Code loading method and device, electronic equipment and computer readable medium |
| CN110659728A (en) * | 2019-09-24 | 2020-01-07 | 上海寒武纪信息科技有限公司 | Neural network optimization method and device, computer equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| "教你用TensorFlow实现神经网络（附代码）", pages 1 - 7, Retrieved from the Internet <URL:《https://developer.aliyun.com/article/272461》> * |
| 胡曙: "基于模糊与神经网络的聚类分析", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》 * |
| 芳芳不是我真名: "神经网络中单层神经元表示逻辑运算", pages 1 - 3, Retrieved from the Internet <URL:《https://blog.csdn.net/u010725283/article/details/78677900》> * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298259A (en) * | 2021-06-10 | 2021-08-24 | 中国电子科技集团公司第十四研究所 | CNN network reasoning framework design method supporting embedded platform multi-core parallel |
| CN113298259B (en) * | 2021-06-10 | 2024-04-26 | 中国电子科技集团公司第十四研究所 | CNN (computer network) reasoning framework design method supporting multi-core parallelism of embedded platform |
| WO2023050809A1 (en) * | 2021-09-30 | 2023-04-06 | 浪潮电子信息产业股份有限公司 | Method for training distributed neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112748953B (en) | 2023-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7169091B2 (en) | A Domain-Specific Language for Generation of Recurrent Neural Network Architectures | |
| CN112288075B (en) | Data processing method and related equipment | |
| Li et al. | Auto-tuning neural network quantization framework for collaborative inference between the cloud and edge | |
| WO2022057776A1 (en) | Model compression method and apparatus | |
| Galluppi et al. | A hierachical configuration system for a massively parallel neural hardware platform | |
| WO2021159714A1 (en) | Data processing method and related device | |
| WO2022068623A1 (en) | Model training method and related device | |
| CN111898636B (en) | Data processing method and device | |
| CN112257471A (en) | Model training method and device, computer equipment and storage medium | |
| CN108171328B (en) | Neural network processor and convolution operation method executed by same | |
| CN110705273B (en) | Information processing method and device based on neural network, medium and electronic equipment | |
| CN113408284A (en) | Training method and device of text processing model, electronic equipment and storage medium | |
| CN113505193A (en) | Data processing method and related equipment | |
| CN112748953B (en) | Data processing method and device based on neural network model and electronic equipment | |
| Djurfeldt et al. | Efficient generation of connectivity in neuronal networks from simulator-independent descriptions | |
| Yang et al. | Performance benchmarking of deep learning framework on Intel Xeon Phi | |
| CN116541492A (en) | Data processing method and related equipment | |
| CN111667069A (en) | Pre-training model compression method and device and electronic equipment | |
| CN113657272B (en) | Micro video classification method and system based on missing data completion | |
| CN114936631B (en) | Model processing method and device | |
| CN112132281A (en) | Model training method, device, server and medium based on artificial intelligence | |
| Gosmann et al. | Automatic optimization of the computation graph in the Nengo neural network simulator | |
| Demidovskij et al. | Designing a neural network primitive for conditional structural transformations | |
| CN111652349A (en) | Neural network processing method and related equipment | |
| CN114333069B (en) | Object posture processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40044525 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |