CN111985624A - Neural network training and deploying method, text translation method and related products - Google Patents

Neural network training and deploying method, text translation method and related products Download PDF

Info

Publication number
CN111985624A
CN111985624A CN202010894158.3A CN202010894158A CN111985624A CN 111985624 A CN111985624 A CN 111985624A CN 202010894158 A CN202010894158 A CN 202010894158A CN 111985624 A CN111985624 A CN 111985624A
Authority
CN
China
Prior art keywords
neural network
sub
training
target
training set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010894158.3A
Other languages
Chinese (zh)
Inventor
高鹏
代季峰
李鸿升
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sensetime Group Ltd
Original Assignee
Sensetime Group Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sensetime Group Ltd filed Critical Sensetime Group Ltd
Priority to CN202010894158.3A priority Critical patent/CN111985624A/en
Publication of CN111985624A publication Critical patent/CN111985624A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The application provides a neural network training and deploying method, a text translation method and related products, wherein the application firstly acquires target neural network and deploying requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information comprises information used for representing the bearing capacity of the target equipment; then based on the deployment requirement information, acquiring a neural network to be deployed, which is deployed on the target equipment, from the trained neural network; and finally, deploying the neural network to be deployed on the target equipment. According to the method and the device, the applicable neural network can be screened from the target neural network and each sub-neural network included by the target neural network for deployment according to the bearing capacity of the target equipment, the defect that the neural network needs to be trained for multiple times according to the bearing capacities of different equipment is overcome, the training and deployment efficiency of the neural network is improved, and training resources are saved.

Description

Neural network training and deploying method, text translation method and related products
Technical Field
The application relates to the technical field of computers and deep learning, in particular to a neural network training and deploying method, a text translation method and a related product.
Background
With the development of deep learning technology, various neural network structures play a key role in tasks such as image processing, natural language processing and the like, and show superior performance. However, in different usage scenarios, the computing capabilities or the carrying capabilities of the devices deploying the neural networks are different, so for the neural networks with the same function, the neural networks with different scales need to be trained respectively according to the computing capabilities of different devices, and after the training is completed, each neural network obtained by training is deployed to a matched device.
According to the training method, parameters among the neural networks cannot be shared, training efficiency is low, and training resources are wasted.
Disclosure of Invention
The embodiment of the application at least provides a neural network training and deploying method, so that the training efficiency of neural networks of different scales is improved.
In a first aspect, an embodiment of the present application provides a neural network deployment method, including:
acquiring target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of target equipment;
acquiring a neural network to be deployed on the target equipment from the trained neural network based on the deployment requirement information;
and deploying the neural network to be deployed on the target equipment.
In this aspect, an applicable neural network can be screened from the target neural network and each sub-neural network included in the target neural network for deployment according to the bearing capacity of the target device, that is, the neural network does not need to be trained for multiple times according to the bearing capacities of different devices to meet the deployment requirements of different devices on various types of neural networks, but the trained target neural network and the plurality of sub-neural networks included in the target neural network are utilized to determine the neural network (which may be the target neural network itself and/or one or more sub-neural networks included in the target neural network) satisfying the deployment requirements for deployment and application, so that resources consumed by training different neural networks are saved, and the deployment efficiency of the neural network is improved.
In one possible implementation, the deployment requirement information includes at least one of:
computing power information of the target device; network speed requirement information.
According to the embodiment, the neural network to be deployed suitable for the target equipment can be screened according to the operational capability information of the target equipment, and the neural network to be deployed can be ensured to be normally used after being deployed on the target equipment. And screening the neural network to be deployed according to the network speed requirement information, wherein the neural network to be deployed meets the calculation speed requirement.
In one possible implementation, before the acquiring the target neural network, the method further includes:
extracting a plurality of sub-neural networks from the neural network to be trained;
training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain the target neural network.
In the embodiment, the neural network to be trained and each sub-neural network extracted from the neural network to be trained are respectively trained by utilizing a first prediction training set formed by the real training set and the output of the first neural network in the neutral state of neural network training, so that the sub-neural networks and the neural network to be trained can be mutually promoted, and the parameter sharing between the neural network to be trained and each sub-neural network is realized. The trained target neural network is cut, and the obtained sub-neural network can be deployed on equipment with corresponding bearing capacity, namely, the neural network suitable for different bearing capacities can be obtained through one-time training, so that the training efficiency of the neural network is improved, and training resources are saved.
In a possible implementation manner, the training the first neural network with the second real training set, and separately training each of the plurality of first sub-neural networks with the second real training set and the first predictive training set to obtain the target neural network includes:
training the first neural network by using a second real training set to obtain a second neural network, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a second sub-neural network corresponding to each first sub-neural network;
acquiring a third real training set, and generating a second prediction training set based on associated samples associated with training samples in the third real training set;
and training the second neural network by using the second prediction training set, and respectively training each second sub-neural network in a plurality of second sub-neural networks by using the third real training set and the second prediction training set to obtain the target neural network.
In the implementation mode, in the process of training the neural network to be trained and each sub-neural network, a new real training set, namely the third real training set, is further combined with the second prediction training set associated with the real training set for training, so that parameter sharing between the neural network to be trained and each sub-neural network can be better realized.
In a possible implementation, the training each of the plurality of first sub-neural networks separately using the second real training set and the first predictive training set includes:
respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set to generate a first target function corresponding to each first sub-neural network;
respectively training each first sub-neural network in the plurality of first sub-neural networks by using the first prediction training set to generate a second objective function corresponding to each first sub-neural network;
for each first sub-neural network in a plurality of first sub-neural networks, training the first sub-neural network based on a first objective function and a second objective function corresponding to the first sub-neural network.
According to the embodiment, the sub-neural network is trained by using the first objective function constructed by the first prediction training set and the second objective function constructed by the second real training set, so that the parameter sharing between the neural network to be trained and the sub-neural network can be realized.
In a possible implementation, the training the first sub-neural network based on the first and second objective functions corresponding to the first sub-neural network includes:
obtaining the number of iterations of the first objective function corresponding to each first sub-neural network, which is respectively trained by using the second real training set, and/or obtaining the number of iterations of the second objective function corresponding to each first sub-neural network, which is generated by respectively training each first sub-neural network in the plurality of first sub-neural networks by using the first prediction training set;
determining a function adjusting proportion based on the obtained iteration times;
generating a target training function based on the first target function, the second target function and the function adjusting proportion;
and training the first sub-neural network by using the target training function.
According to the embodiment, the first objective function and the second objective function are integrated by utilizing the function adjusting proportion determined based on the iteration times, the sub-neural network is trained by utilizing the integrated objective training function, the influence of noise in the output of the first neural network on the neural network training can be reduced, the accuracy of the trained neural network is ensured, and meanwhile, the sub-neural network is trained by utilizing the objective training function integrated by the first objective function and the second objective function, so that the parameter sharing between the neural network to be trained and the sub-neural network can be promoted.
In one possible implementation, the training samples in the third real training set comprise words to be translated and standard translation words;
generating a second predictive training set based on the associated samples associated with the training samples in the third real training set, comprising:
determining a predicted vocabulary appearing after the vocabulary to be translated based on the vocabulary to be translated in the training sample and a plurality of texts comprising the vocabulary to be translated;
combining the prediction vocabulary and the standard translation vocabulary to form an associated sample of the training sample;
and generating the second prediction training set by using the associated sample corresponding to each training sample in the third real training set.
According to the embodiment, a prediction training set is constructed by utilizing words possibly appearing after the predicted words to be translated, the first neural network and each first sub-neural network are trained by utilizing the prediction training set, and parameter sharing between the neural network to be trained and the sub-neural networks can be promoted.
In a second aspect, the present application provides a neural network training method, including:
extracting a plurality of sub-neural networks from the neural network to be trained;
training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
In a third aspect, the present application provides a text translation method, including:
acquiring a text to be translated;
translating the text to be translated through a translation neural network deployed on target equipment to obtain a translation result;
the translation neural network is the neural network to be deployed, and/or the translation neural network comprises a target neural network obtained by the neural network training method and/or at least one sub-neural network in the target neural network.
The translation neural network deployed on the target equipment is used for text translation, so that translated texts can be obtained accurately.
In a fourth aspect, the present application provides a neural network deployment device, comprising:
the acquisition module is used for acquiring the target neural network and the deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of target equipment;
the network screening module is used for acquiring a neural network to be deployed, which is deployed on the target equipment, from the trained neural network based on the deployment requirement information;
and the network deployment module is used for deploying the neural network to be deployed on the target equipment.
In a fifth aspect, the present application provides a neural network training device, including:
the sub-network extraction module is used for extracting a plurality of sub-neural networks from the neural network to be trained;
the first training module is used for training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
the sample prediction module is used for respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and the second training module is used for training the first neural network by using a second real training set and respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
In a sixth aspect, the present application provides a text translation apparatus, comprising:
the text acquisition module is used for acquiring a text to be translated;
the translation module is used for translating the text to be translated through a translation neural network deployed on the target equipment to obtain a translation result;
the translation neural network is the neural network to be deployed, and/or the translation neural network comprises a target neural network obtained by the neural network training method and/or at least one sub-neural network in the target neural network.
In a seventh aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the neural network deployment method, or the neural network training method, or the text translation method as described above.
In an eighth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the neural network deployment method, or the neural network training method, or the text translation method as described above.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments and are incorporated in and constitute a part of the specification will be briefly described below, and the drawings illustrate the embodiments consistent with the present application and together with the description serve to explain the technical solutions of the present application. It is appreciated that the following drawings depict only certain embodiments of the application and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 shows a flowchart of a neural network deployment method provided by an embodiment of the present application;
FIG. 2 is a flow chart illustrating a neural network training method provided by an embodiment of the present application;
FIG. 3 is a flow chart illustrating a method of text translation provided by an embodiment of the present application;
fig. 4 is a schematic structural diagram of a neural network deployment device provided in an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a neural network training device provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram illustrating a text translation apparatus provided in an embodiment of the present application;
fig. 7 shows a schematic diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Aiming at the defects of low neural network training efficiency and high resource consumption caused by the fact that neural networks suitable for different bearing devices need to be trained respectively in the prior art, the application provides a neural network deployment and training method, wherein the neural network to be trained and each sub-neural network extracted from the neural network to be trained are jointly trained by utilizing a prediction training set formed by a real training set and the output of the neural network in a training intermediate state, so that the neural networks to be trained and the neural networks to be trained can be mutually promoted, and parameter sharing between the neural network to be trained and each sub-neural network is realized. The trained target neural network is cut, and the obtained sub-neural network can be deployed on equipment with corresponding bearing capacity, namely, the sub-neural network can be obtained on equipment suitable for different bearing capacities through one-time training, so that the training and deployment efficiency of the neural networks of different scales is improved, and training resources are saved.
The following is a detailed description of the neural network deployment and training method and related products provided by the present application with specific embodiments.
The neural network deployment method is applied to screening the neural network to be deployed from the trained target neural network based on information such as the bearing capacity of the target equipment, and deploying the neural network to be deployed obtained through screening on the target equipment. Specifically, as shown in fig. 1, the neural network deployment method provided by the present application may include the following steps:
s110, acquiring target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of the target equipment.
Here, the target neural network includes a plurality of sub neural networks including a part of a neural network layer in the target neural network and/or a part of neurons in the neural network layer. The network scale of the target neural network is the largest, and equipment with larger bearing capacity is required to bear the load, and the network scale of the sub-neural network is relatively smaller, and can be borne by equipment with smaller bearing capacity. And parameters are shared between the target neural network and each sub-neural network. The network scale corresponds to the structural complexity of the neural network, the larger the network scale, the more complex the structure of the neural network, the more neurons in the neural network layer or the neural network layer are included, the smaller the network scale, the simpler the structure of the neural network, the less neurons in the neural network layer or the neural network layer are included.
The information indicating the bearing capacity of the target device may specifically be computing capacity information of the target device, and in addition, the deployment requirement information may further include network speed requirement information indicating a requirement for a computation speed of the neural network to be deployed. Different neural networks have different calculation speeds due to different network scales, the larger the network scale of the neural network is, the more complex the structure of the neural network is, the slower the calculation speed is, and conversely, the smaller the network scale of the neural network is, the simpler the structure of the neural network is, the faster the calculation speed is. When the neural network to be deployed is screened, not only the bearing capacity of the target equipment but also the requirement of the calculation speed of the neural network to be deployed can be considered. When the requirement on the calculation speed is not high and the bearing capacity of the equipment is strong, a neural network with a large network scale can be selected, and when the calculation speed is high and the bearing capacity of the equipment is weak, a neural network with a small network scale can be selected.
And S120, acquiring the neural network to be deployed on the target equipment from the trained neural network based on the deployment requirement information.
And screening the neural network to be deployed suitable for the target equipment according to the operational capability information of the target equipment so as to ensure that the neural network to be deployed can be normally used after being deployed on the target equipment. And screening the neural network to be deployed according to the network speed requirement information to meet the calculation speed requirement so as to meet the requirement of the corresponding application scene.
S130, deploying the neural network to be deployed on the target equipment.
According to the embodiment, the applicable neural network can be screened from the target neural network and each sub-neural network included in the target neural network for deployment according to the bearing capacity of the target equipment, so that the defect that the neural network needs to be trained for multiple times according to the bearing capacities of different equipment is overcome, the training and deployment efficiency of the neural network is improved, and training resources are saved.
In some embodiments, before performing step S110, steps S1001-S1004 of training a target neural network may also be performed as follows:
s1001, extracting a plurality of sub-neural networks from the neural network to be trained.
Here, the neural network to be trained is the neural network to be trained with the maximum network size, which may be set by a human in advance according to the actual application requirements, corresponding to the maximum feature dimension of the neural network. The maximum network size may be specifically set according to the carrying capacity of the device to be deployed by the neural network, for example, for the neural network that needs to be deployed on a device with a smaller carrying capacity at a mobile phone or the like, the maximum network size is set according to the maximum carrying capacity of the device with a smaller carrying capacity, and is set to be smaller; for a neural network to be deployed on a device with a large carrying capacity such as a server, the maximum network size is set to be large according to the maximum carrying capacity of the device with the large carrying capacity.
In addition to setting the maximum network size, it is also necessary to set the minimum network size when extracting the sub neural network. The neural network corresponding to the minimum network size has the simplest structure compared with other sub-neural networks and the neural network to be trained, and the minimum number of neural network layers or neurons is included. The minimum network size is also set manually in advance according to actual application requirements, and corresponds to the minimum characteristic dimension of the neural network. The minimum network size may be specifically set according to the bearer capability of the device to be deployed by the neural network, for example, for the neural network that needs to be deployed on a device with a smaller bearer capability, such as a mobile phone, the minimum network size is set according to the minimum bearer capability of the device with the smaller bearer capability.
In this step, at the time of extracting the sub neural network, the sub neural network is randomly extracted according to the minimum network size, and the network size of the extracted sub neural network needs to be greater than or equal to the minimum network size. In addition, if the sub-neural network of the minimum network size is not included in the extracted sub-neural networks, it is also necessary to additionally extract the neural network of the minimum network size as one of the word neural networks.
According to the above description, the minimum network size is determined according to the minimum carrying capacity required to be deployed on the device, if the minimum network size is not set, when the sub-neural network is extracted, the sub-neural network with the smaller network size may be extracted, and then the sub-neural network with the smaller network size is trained, but the sub-neural network with the smaller network size is not required to be deployed on the device, so that training the sub-neural network which is not used is seen, the training efficiency of the nerves is reduced, and the training resources are wasted.
The neural network to be trained may be a neural network that performs image processing or a neural network that performs word processing, and for example, the neural network to be trained may be a transform model.
And randomly extracting a plurality of sub-neural networks from the neural network to be trained at each iteration so as to share parameters between the neural network to be trained and each sub-neural network through a plurality of times of training.
S1002, training the neural network to be trained by using a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network.
Here, the first real training set includes a plurality of real training samples, each of which includes a real sample and a standard value corresponding to the real sample, for example, for a neural network to be translated, the training samples may include a word to be translated and a standard translated word after translation, where the word to be translated is a real existing word to be translated and is not a word predicted based on the word to be translated.
The first real training set is pre-acquired before this step is performed. In the step, the neural network to be trained and the plurality of randomly extracted sub-neural networks are respectively trained by utilizing the first real training set, so that the parameter gradient orders of the neural network to be trained and the sub-neural networks are consistent as much as possible, and certain compromise is achieved for shared parameters.
In the above, the neural network to be trained is trained by using the first real training set to obtain the first neural network, and the sub-neural networks are trained by using the first real training set to obtain the first sub-neural network. The first neural network and the first sub-neural networks are not neural networks to be obtained by training, the first neural network and each first sub-neural network have the same attribute or the same defined parameter value, but parameter sharing cannot be realized. The first neural network and the first sub-neural network are neural networks of intermediate states obtained in the training.
S1003, respectively inputting the real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the real samples and the obtained sample processing results.
For the neural network for translation, the real sample can be a vocabulary to be translated; the sample processing result may be a vocabulary translated using the first neural network. In this step, a training set for training each sub-neural network, i.e., the first prediction training set, is formed using the output of the first neural network in the neutral state of neural network training. The first prediction training set is used for training each sub-neural network, so that the mutual influence among the sub-neural networks is eliminated, and the parameter sharing among the neural network to be trained and each sub-neural network is favorably realized.
The plurality of real samples may belong to all or part of the first real training set, or may be samples completely different from the samples in the first real training set, which is not limited in the present application.
S1004, training the first neural network by using a second real training set, and respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain the target neural network.
Here, the second real training set includes a plurality of real training samples, each of which includes a real sample and a standard value corresponding to the real sample, for example, for a neural network to be translated, the training samples may include a word to be translated and a standard translated word after translation, where the word to be translated is a real existing word to be translated and is not a word predicted based on the word to be translated.
The second real training set may be different from or partially identical to the first real training set, and this application does not limit this. The plurality of real samples in step 1003 may belong to all or part of the second real training set, or may be samples completely different from the samples in the first real training set, which is not limited in this application.
The second real training set is pre-acquired before this step is performed. In the step, the first neural network is trained by utilizing the second real training set, the first sub-neural network is trained by utilizing the second real training set and the first prediction training set, and parameter sharing between each sub-neural network and the neural network to be trained can be realized, so that the neural network training efficiency is improved, and the resource consumption is reduced.
In the embodiment, the neural network to be trained and each sub-neural network extracted from the neural network to be trained are respectively trained by utilizing a first prediction training set formed by the real training set and the output of the first neural network in the neutral state of neural network training, so that the sub-neural networks and the neural network to be trained can be mutually promoted, and the parameter sharing between the neural network to be trained and each sub-neural network is realized. The trained target neural network is cut, and the obtained sub-neural network can be deployed on equipment with corresponding bearing capacity, namely, the neural network suitable for different bearing capacities can be obtained through one-time training, so that the training efficiency of the neural network is improved, and training resources are saved.
In some embodiments, in the above step 1004, each of the first sub-neural networks in the plurality of first sub-neural networks is trained by using the second real training set and the first predictive training set, which may be implemented by using the following steps S10041-S10043:
s10041, respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set, and generating a first objective function corresponding to each first sub-neural network.
The first objective function is obtained by using a real training sample to perform supervised learning.
S10042, respectively training each first sub-neural network in the plurality of first sub-neural networks by using the first prediction training set, and generating a second target function corresponding to each first sub-neural network.
The second objective function is obtained by performing supervised learning by using a training sample formed by the output of the first neural network in the intermediate state of neural network training.
The present application does not limit the execution order of S10041 and S10042 described above.
S10043, for each first sub-neural network of the plurality of first sub-neural networks, training the first sub-neural network based on a first objective function and a second objective function corresponding to the first sub-neural network.
Here, specifically, a target training function for training the first sub-neural network may be formed by using the first objective function and the second objective function, and then the first sub-neural network may be trained by using the target training function. The following substeps S100431-S100434 can be used to implement:
s100431, obtaining the iteration count of the first objective function corresponding to each first sub-neural network obtained by training each first sub-neural network in the plurality of first sub-neural networks respectively by using the second real training set, and/or obtaining the iteration count of the second objective function corresponding to each first sub-neural network obtained by training each first sub-neural network in the plurality of first sub-neural networks respectively by using the first prediction training set.
Here, the number of iterations to generate the first objective function and the number of iterations to generate the second objective function are equal, where the number of iterations may alternatively be obtained.
S100432, determining a function adjusting proportion based on the obtained iteration times.
Here, when determining the function adjustment ratio, it may specifically be determined by using the number of iterations and a preset linear function. The linear function can be a monotone increasing function, the value of the function adjusting proportion is larger and larger along with the increase of the iteration times, and the function adjusting proportion is not transformed any more when the iteration times are increased to a certain value. In particular implementations, a simulated annealing algorithm may be utilized to determine the value of the adjustment ratio of this function.
S100433, generating a target training function based on the first target function, the second target function and the function adjusting proportion.
The target training function may be formed by using the following formula:
target training function ═ first target function + function adjustment scale × second target function
As can be seen from the description in step S100432, when the number of iterations is small, the value of the function adjustment ratio is small, the effect of the second objective function on the target training function is small, when the sub-neural network is trained by using the target training function, the effect of the second objective function on the training of the sub-neural network is small, along with the increase of the number of iterations, the effect of the second objective function on the target training function is large, and when the sub-neural network is trained by using the target training function, the effect of the second objective function on the training of the sub-neural network becomes larger and larger. The above design concept of function adjustment ratio is: because the output noise of the first neural network is large when the number of iterations is small, at this time, in order to ensure the training accuracy, the effect of the second objective function generated by using the first prediction training set on the target training function needs to be set to be small. As the number of iterations increases, the output noise of the first neural network becomes smaller, and at this time, the second objective function generated by using the first predictive training set may set the effect on the objective training function to be larger, without affecting the training accuracy on the sub-neural network.
The first objective function and the second objective function are integrated by utilizing the function adjusting proportion and the formula, the sub-neural networks are trained by utilizing the target training function obtained by integration, the influence of noise in the output of the first neural network on the neural network training can be reduced, the accuracy of the neural network obtained by training is ensured, meanwhile, the sub-neural networks are trained by utilizing the first objective function corresponding to the second real training set and the target training function obtained by integrating the second objective function corresponding to the predicted training set, the first neural network is trained by utilizing the second real training set, and parameter sharing between each sub-neural network and the neural network to be trained can be realized.
S100434, training the first sub-neural network by using the target training function.
The target neural network obtained by training by using the method in the embodiment can complete the processing task of the non-machine recombined text. For a scenario with higher requirements, such as translation of a recombined text, after the training of the first neural network with the second real training set and the training of the first sub-neural network with the second real training set and the first predictive training set respectively in step S1004 and before the trained target neural network is obtained, the following steps 1 to 2 may be further included:
step 1, a third real training set is obtained, and a second prediction training set is generated based on the associated samples associated with the training samples in the third real training set.
Here, the third real training set includes a plurality of real training samples, each of which includes a real sample and a standard value corresponding to the real sample, for example, for a neural network to be translated, the training samples may include a word to be translated and a standard translated word after translation, where the word to be translated is a real existing word to be translated and is not a word predicted based on the word to be translated.
Any two of the second real training set, the first real training set and the third real training set may be different or partially the same, and this is not limited in the present application. The plurality of real samples in step 1003 may belong to all or part of the second real training set, may be samples completely different from the samples in the first real training set, or may be samples completely different from the samples in the third real training set, which is not limited in this application.
For the neural network to be translated, the second prediction training set may be generated by using the following substeps 11 to 13:
a substep 11 of determining a predicted vocabulary that occurs after the vocabulary to be translated, based on the vocabulary to be translated in the training sample and a plurality of texts comprising the vocabulary to be translated.
The predicted vocabulary is not a real vocabulary, but a vocabulary which is possibly appeared after the vocabulary to be translated is deduced based on a plurality of texts in which the vocabulary to be translated is positioned.
And a substep 12, combining the prediction vocabulary and the standard translation vocabulary to form an associated sample of the training sample.
And a substep 13, generating the second prediction training set by using the associated samples corresponding to each training sample in the third real training set.
And constructing a prediction training set by utilizing the predicted vocabulary which possibly appears after the vocabulary to be translated, and training the first neural network and each first sub-neural network by utilizing the prediction training set, so that the parameter sharing between the neural network to be trained and the sub-neural networks can be realized.
And 2, training the second neural network by using the second prediction training set, and respectively training each second sub-neural network in a plurality of second sub-neural networks by using the third real training set and the second prediction training set to obtain the target neural network.
The second neural network is obtained by training the first neural network by using a second real training set. The second sub-neural network is obtained by respectively training the first sub-neural network by using the second real training set and the first prediction training set.
The training of the second sub-neural network by using the third real training set and the second predictive training set may be specifically implemented by using the following substeps 21 to 23:
and a substep 21 of training the second sub-neural network by using the third real training set to generate a third objective function.
The third objective function is obtained by using a real training sample to perform supervised learning.
And a substep 22 of training the second sub-neural network by using the second predictive training set to generate a fourth objective function.
The fourth objective function is obtained by using a training sample formed by the vocabulary obtained through prediction to perform supervised learning.
The present application does not limit the execution sequence of the substeps 21 and 22 described above.
And a substep 23 of training the second sub-neural network based on the third objective function and the fourth objective function.
Specifically, a comprehensive training function for training the second sub-neural network may be formed by using the third objective function and the fourth objective function, and then the second sub-neural network may be trained by using the comprehensive training function. Specifically, the comprehensive training function may be formed by using the following sub-formula:
the target training function is the third target function + the adjustment ratio parameter value multiplied by the fourth target function
The adjustment ratio parameter value may be a preset constant, or may be determined by using a simulated annealing method, and the determination manner may be the same as the determination manner of the function adjustment ratio, specifically, a preset linear function may be used to determine the adjustment ratio parameter value based on the number of iterations for generating the third objective function or the fourth objective function; the larger the number of iterations for generating the third objective function or the fourth objective function is, the larger the adjustment ratio parameter value is, and when the number of iterations for generating the third objective function or the fourth objective function increases to a certain value, the adjustment ratio parameter value does not change any more. The linear function for determining the adjustment ratio parameter value and the linear function for determining the adjustment ratio of the function may be different or the same, and the adjustment ratio parameter value and the value of the adjustment ratio of the function may be the same or different, which is not limited in the present application.
The fourth objective function constructed by the second prediction training set and the third objective function constructed by the third real training set are used for training the sub-neural network together, so that parameter sharing between the maximum-scale neural network and the sub-neural network can be realized.
In the above embodiment, in the process of training the neural network to be trained and each sub-neural network, a new real training set, that is, the third real training set, is further combined with the second prediction training set associated with the real training set for training, and more training sets are used for training more times, so that parameter sharing between the neural network to be trained and each sub-neural network can be better achieved.
According to the method and the device, parameters are shared between the target neural network obtained through training and each sub neural network, the required sub neural network can be obtained through directly cutting from the target neural network, the neural network suitable for retraining is not needed, on the premise that the precision of the neural network is guaranteed, the time consumed by training of the neural network is greatly reduced, the training efficiency of the neural network is improved, and the resource consumption is reduced. The target neural network trained in the above embodiment is suitable for various application scenarios, for example, for a text translation scenario, the trained target neural network can be suitable for devices with strong bearing capacity, such as a server, and the like, and after the trained target neural network is installed on the server, text translation can be performed more accurately, meanwhile, the sub-neural network in the target neural network can be suitable for devices with weak bearing capacity, such as a mobile terminal, and after the trained sub-neural network is installed on the mobile terminal, text translation can be performed more accurately, so that the neural networks with different network scales are prevented from being trained for many times, the training efficiency of the neural network is improved, and training resources are saved. For example, for an image recognition scene, the trained target neural network can be suitable for equipment with strong bearing capacity, such as a server, and the like, after the trained target neural network is installed on the server, image recognition can be accurately performed, meanwhile, the sub-neural network in the target neural network can be suitable for equipment with weak bearing capacity, such as a mobile terminal, and after the trained sub-neural network is installed on the mobile terminal, image recognition can be accurately performed, so that the neural networks with different network scales are prevented from being trained for many times, the training efficiency of the neural network is improved, and training resources are saved. For example, for a reading understanding scene, the trained target neural network can be suitable for devices with strong bearing capacity, such as a server, and the like, after the trained target neural network is installed on the server, reading understanding answers can be accurately determined, meanwhile, the sub-neural network in the target neural network can be suitable for devices with weak bearing capacity, such as a mobile terminal, and after the trained sub-neural network is installed on the mobile terminal, reading understanding answers can be accurately determined, multiple times of training of neural networks with different network scales are avoided, training efficiency of the neural network is improved, and training resources are saved.
The present application further provides a neural network training method, which can be applied to a client or a server and is used for training neural networks with different network scales and shared parameters, specifically, as shown in fig. 2, the neural network training method provided by the present application includes the following steps:
s210, extracting a plurality of sub-neural networks from the neural network to be trained.
S220, training the neural network to be trained by using a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network.
And S230, respectively inputting the real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the real samples and the obtained sample processing results.
S240, training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
The training process of the neural network to be trained in this embodiment is the same as the training process of the neural network to be trained in the above embodiments, and is not described again.
The present application further provides a text translation method, applied to a target device, as shown in fig. 3, including:
s310, obtaining a text to be translated.
And S320, translating the text to be translated through a translation neural network deployed on the target equipment to obtain a translation result.
The translation neural network is the neural network to be deployed in the embodiment, and/or the translation neural network comprises a target neural network obtained by the neural network training method and/or at least one sub-neural network in the target neural network.
The translation neural network deployed on the target equipment is used for text translation, so that translated texts can be obtained accurately.
Corresponding to the neural network deployment method, the present application also discloses a neural network deployment apparatus, where each module in the apparatus can implement each step in the neural network deployment method of each embodiment, and can obtain the same beneficial effect, and therefore, the description of the same parts is not repeated here. Specifically, as shown in fig. 4, the neural network deployment apparatus includes:
an obtaining module 410, configured to obtain a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of the target equipment.
And the network screening module 420 is configured to obtain a neural network to be deployed, which is deployed on the target device, from the trained neural network based on the deployment requirement information.
A network deployment module 430, configured to deploy the neural network to be deployed on the target device.
Corresponding to the neural network training method, the present application also discloses a neural network training device, where each module in the device can implement each step in the neural network training method of each embodiment, and can obtain the same beneficial effects, and therefore, the description of the same parts is not repeated here. Specifically, as shown in fig. 5, the neural network training device disclosed in the present application includes:
a sub-network extracting module 510 for extracting a plurality of sub-neural networks from the neural network to be trained.
The first training module 520 is configured to train the neural network to be trained by using a first real training set to obtain a first neural network, and train each sub-neural network of the plurality of sub-neural networks by using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network.
The sample prediction module 530 is configured to input a plurality of real samples into the first neural network for processing, obtain a sample processing result corresponding to each real sample, and generate a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results.
The second training module 540 is configured to train the first neural network by using a second real training set, and train each of the plurality of first sub-neural networks by using the second real training set and the first prediction training set, respectively, to obtain a target neural network, where the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
Corresponding to the text translation method, the present application also discloses a text translation apparatus, where each module in the apparatus can implement each step in the text translation method of each embodiment, and can obtain the same beneficial effect, and therefore, the description of the same part is omitted here. Specifically, as shown in fig. 6, a text translation apparatus disclosed in the present application includes:
the text obtaining module 610 is configured to obtain a text to be translated.
The translation module 620 is configured to translate the text to be translated through a translation neural network deployed on the target device, so as to obtain a translation result. The translation neural network is the neural network to be deployed in the embodiment, and/or the translation neural network comprises a target neural network obtained by the neural network training method in the embodiment and/or at least one sub-neural network in the target neural network.
Corresponding to the neural network deployment method, the neural network training method, or the text translation method in the foregoing embodiments, an embodiment of the present application further provides an electronic device 700, and as shown in fig. 7, a schematic structural diagram of the electronic device 700 provided in the embodiment of the present application includes:
a processor 71, a memory 72, and a bus 73; the memory 72 is used for storing execution instructions and includes a memory 721 and an external memory 722; the memory 721 is also referred to as an internal memory, and is used for temporarily storing the operation data in the processor 71 and the data exchanged with the external memory 722 such as a hard disk, the processor 71 exchanges data with the external memory 722 through the memory 721, and when the electronic device 700 is operated, the processor 71 and the memory 72 communicate with each other through the bus 73, so that the processor 71 executes the following instructions corresponding to the neural network deployment method: acquiring target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of target equipment; acquiring a neural network to be deployed on the target equipment from the trained neural network based on the deployment requirement information; and deploying the neural network to be deployed on the target equipment, wherein the trained target neural network comprises a trained first neural network and a trained sub-neural network. Or instructions that cause the processor 71 to perform the following method corresponding to neural network training: extracting a plurality of sub-neural networks from the neural network to be trained; training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network; respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results; and training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks. Or instructions that cause processor 71 to perform the following corresponding to a text translation method: acquiring a text to be translated; translating the text to be translated through a translation neural network deployed on target equipment to obtain a translation result; the translation neural network is the neural network to be deployed in the embodiment, and/or the translation neural network comprises a target neural network obtained by the neural network training method in the embodiment and/or at least one sub-neural network in the target neural network.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the neural network deployment method, the neural network training method, or the text translation method in the foregoing method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the neural network deployment method, the neural network training method, or the text translation method provided in the embodiment of the present application includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute steps of the neural network deployment method, the neural network training method, or the text translation method in the above method embodiment, which may be referred to in the above method embodiment specifically, and are not described herein again.
The embodiments of the present application also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (14)

1. A neural network deployment method, comprising:
acquiring target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of target equipment;
acquiring a neural network to be deployed on the target equipment from the trained neural network based on the deployment requirement information;
and deploying the neural network to be deployed on the target equipment.
2. The neural network deployment method of claim 1, wherein the deployment requirement information comprises at least one of:
computing power information of the target device; network speed requirement information.
3. The neural network deployment method of claim 1 or 2, further comprising, prior to the obtaining the target neural network:
extracting a plurality of sub-neural networks from the neural network to be trained;
training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain the target neural network.
4. The neural network deployment method of claim 3, wherein the training the first neural network with the second real training set and the training each of the plurality of first sub-neural networks with the second real training set and the first predictive training set to obtain the target neural network comprises:
training the first neural network by using a second real training set to obtain a second neural network, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a second sub-neural network corresponding to each first sub-neural network;
acquiring a third real training set, and generating a second prediction training set based on associated samples associated with training samples in the third real training set;
and training the second neural network by using the second prediction training set, and respectively training each second sub-neural network in a plurality of second sub-neural networks by using the third real training set and the second prediction training set to obtain the target neural network.
5. The neural network deployment method of claim 3 or 4, wherein the training each of the plurality of first sub-neural networks separately using the second real training set and the first predictive training set comprises:
respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set to generate a first target function corresponding to each first sub-neural network;
respectively training each first sub-neural network in the plurality of first sub-neural networks by using the first prediction training set to generate a second objective function corresponding to each first sub-neural network;
for each first sub-neural network in a plurality of first sub-neural networks, training the first sub-neural network based on a first objective function and a second objective function corresponding to the first sub-neural network.
6. The neural network deployment method of claim 5, wherein the training the first sub-neural network based on the first and second objective functions corresponding to the first sub-neural network comprises:
obtaining the number of iterations of the first objective function corresponding to each first sub-neural network, which is respectively trained by using the second real training set, and/or obtaining the number of iterations of the second objective function corresponding to each first sub-neural network, which is generated by respectively training each first sub-neural network in the plurality of first sub-neural networks by using the first prediction training set;
determining a function adjusting proportion based on the obtained iteration times;
generating a target training function based on the first target function, the second target function and the function adjusting proportion;
and training the first sub-neural network by using the target training function.
7. The neural network deployment method of claim 4, wherein the training samples in the third real training set comprise words to be translated and standard translation words;
generating a second predictive training set based on the associated samples associated with the training samples in the third real training set, comprising:
determining a predicted vocabulary appearing after the vocabulary to be translated based on the vocabulary to be translated in the training sample and a plurality of texts comprising the vocabulary to be translated;
combining the prediction vocabulary and the standard translation vocabulary to form an associated sample of the training sample;
and generating the second prediction training set by using the associated sample corresponding to each training sample in the third real training set.
8. A neural network training method, comprising:
extracting a plurality of sub-neural networks from the neural network to be trained;
training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and training the first neural network by using a second real training set, and respectively training each first sub-neural network in a plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
9. A method of text translation, the method comprising:
acquiring a text to be translated;
translating the text to be translated through a translation neural network deployed on target equipment to obtain a translation result;
the neural network to be deployed is the neural network to be deployed according to any one of claims 1 to 7, and/or the neural network to be deployed comprises a target neural network obtained by the neural network training method according to claim 8 and/or at least one sub-neural network in the target neural network.
10. A neural network deployment device, comprising:
the acquisition module is used for acquiring the target neural network and the deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information used for representing the bearing capacity of target equipment;
the network screening module is used for acquiring a neural network to be deployed, which is deployed on the target equipment, from the trained neural network based on the deployment requirement information;
and the network deployment module is used for deploying the neural network to be deployed on the target equipment.
11. A neural network training device, comprising:
the sub-network extraction module is used for extracting a plurality of sub-neural networks from the neural network to be trained;
the first training module is used for training the neural network to be trained by utilizing a first real training set to obtain a first neural network, and respectively training each sub-neural network in the plurality of sub-neural networks by utilizing the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;
the sample prediction module is used for respectively inputting a plurality of real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;
and the second training module is used for training the first neural network by using a second real training set and respectively training each first sub-neural network in the plurality of first sub-neural networks by using the second real training set and the first prediction training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are both trained neural networks.
12. A text translation apparatus, comprising:
the text acquisition module is used for acquiring a text to be translated;
the translation module is used for translating the text to be translated through a translation neural network deployed on the target equipment to obtain a translation result;
the neural network to be deployed is the neural network to be deployed according to any one of claims 1 to 7, and/or the neural network to be deployed comprises a target neural network obtained by the neural network training method according to claim 8 and/or at least one sub-neural network in the target neural network.
13. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine readable instructions when executed by the processor performing the steps of the neural network deployment method of any one of claims 1 to 7, or the neural network training method of claim 8, or the text translation method of claim 9.
14. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the neural network deployment method of any one of claims 1 to 7, or the neural network training method of claim 8, or the text translation method of claim 9.
CN202010894158.3A 2020-08-31 2020-08-31 Neural network training and deploying method, text translation method and related products Pending CN111985624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010894158.3A CN111985624A (en) 2020-08-31 2020-08-31 Neural network training and deploying method, text translation method and related products

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010894158.3A CN111985624A (en) 2020-08-31 2020-08-31 Neural network training and deploying method, text translation method and related products

Publications (1)

Publication Number Publication Date
CN111985624A true CN111985624A (en) 2020-11-24

Family

ID=73440325

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010894158.3A Pending CN111985624A (en) 2020-08-31 2020-08-31 Neural network training and deploying method, text translation method and related products

Country Status (1)

Country Link
CN (1) CN111985624A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541577A (en) * 2020-12-16 2021-03-23 上海商汤智能科技有限公司 Neural network generation method and device, electronic device and storage medium
WO2023115776A1 (en) * 2021-12-24 2023-06-29 上海商汤智能科技有限公司 Neural network reasoning method and apparatus, and computer device, computer-readable storage medium and computer program product

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541577A (en) * 2020-12-16 2021-03-23 上海商汤智能科技有限公司 Neural network generation method and device, electronic device and storage medium
WO2023115776A1 (en) * 2021-12-24 2023-06-29 上海商汤智能科技有限公司 Neural network reasoning method and apparatus, and computer device, computer-readable storage medium and computer program product

Similar Documents

Publication Publication Date Title
CN109871532B (en) Text theme extraction method and device and storage medium
CN111615702B (en) Method, device and equipment for extracting structured data from image
CN108984155B (en) Data processing flow setting method and device
CN108304376B (en) Text vector determination method and device, storage medium and electronic device
CN111985624A (en) Neural network training and deploying method, text translation method and related products
CN111767394A (en) Abstract extraction method and device based on artificial intelligence expert system
CN111870959A (en) Resource recommendation method and device in game
CN111709415B (en) Target detection method, device, computer equipment and storage medium
CN110968664A (en) Document retrieval method, device, equipment and medium
CN112667979A (en) Password generation method and device, password identification method and device, and electronic device
CN114398040A (en) Neural network reasoning method, device, computer equipment and storage medium
CN114818600A (en) Chip verification method and device, electronic equipment and storage medium
JP7063274B2 (en) Information processing equipment, neural network design method and program
CN111324725B (en) Topic acquisition method, terminal and computer readable storage medium
CN116561338A (en) Industrial knowledge graph generation method, device, equipment and storage medium
CN115470328A (en) Open field question-answering method based on knowledge graph and related equipment
CN115115901A (en) Method and device for acquiring cross-domain learning model
CN115099344A (en) Model training method and device, user portrait generation method and device, and equipment
Lysechko et al. Experimental study of optimized face recognition algorithms for resource–constrained
CN114118411A (en) Training method of image recognition network, image recognition method and device
CN113159297A (en) Neural network compression method and device, computer equipment and storage medium
CN111985644A (en) Neural network generation method and device, electronic device and storage medium
CN112825143A (en) Deep convolutional neural network compression method, device, storage medium and equipment
CN113469352A (en) Neural network model optimization method, data processing method and device
CN116778264B (en) Object classification method, image classification method and related equipment based on class reinforcement learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination