CN110889450B

CN110889450B - Super-parameter tuning and model construction method and device

Info

Publication number: CN110889450B
Application number: CN201911179159.3A
Authority: CN
Inventors: 欧阳显斌; 周飞虎; 王洋子豪; 魏杰乾; 赵秀峰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-11-27
Filing date: 2019-11-27
Publication date: 2023-08-11
Anticipated expiration: 2039-11-27
Also published as: CN110889450A

Abstract

The application relates to a super-parameter tuning and model construction method and device, wherein the method comprises the following steps: acquiring a first neural network model obtained after training an initial neural network model; training the first neural network model to obtain a second neural network model; obtaining a model length of the second neural network model relative to the first neural network model according to the difference between the second model capacity evaluation value and the first model capacity evaluation value; screening a growth potential model from the second neural network model according to the modeling length; and obtaining the optimal super-parameters according to the super-parameters in the growth potential model. The application can avoid eliminating the super parameters with growth potential.

Description

Super-parameter tuning and model construction method and device

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method and apparatus for optimizing super parameters, a method and apparatus for constructing a model, a computer readable storage medium, and a computer device.

Background

At present, a neural network model is built mainly in a manual mode. For example, super parameters and common parameters are designed manually, super parameters and common parameters are utilized to construct a neural network model for controlling the game characters to operate in the game environment, and the neural network model is applied to the game, so that the game characters can simulate the operation of real players, and the game characters with artificial intelligence (AI, artificial Intelligence) are realized.

Common parameters such as weight values, bias values and the like in the neural network model can be optimized through iterative training. If the number of convolution layers, the number of channels of the convolution kernel and other super parameters are not automatically optimized through iterative training, the initial super parameters are designed in advance according to experience of a researcher before iterative training, an initial neural network model is built and trained by the super parameters, and then the initial super parameters are optimized according to the performance of the neural network model until the neural network model with ideal performance is obtained. In this regard, an automatic tuning method for super parameters, such as AutoML (super parameter tuning tool) which is more common at present, appears. Because the super-parameter tuning method often needs to search for hundreds of times to obtain the ideal super-parameter, the tuning process needs to consume a large amount of computing resources, and therefore the super-parameter with non-ideal performance needs to be continuously eliminated in the tuning process.

However, the tuning process of the related art often eliminates the super parameters that are not ideal in current performance but are actually more ideal in performance after continuous training, so that the performance of the finally constructed neural network model does not meet the requirements of users due to eliminating the preferable super parameters.

Therefore, the super-parameter tuning method of the related art has the problem that the potential super-parameters are eliminated and the neural network model meeting the requirements of users cannot be constructed.

Disclosure of Invention

Based on this, there is a need to provide a super parameter tuning method and device, a model construction method and device, a computer readable storage medium and computer equipment, aiming at the problem that the super parameter tuning method in the related art eliminates the potential super parameter and cannot construct the neural network model meeting the user requirement.

A super parameter tuning method, comprising:

acquiring a first neural network model obtained after training an initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter;

training the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value obtained according to the super parameter and the second trained parameter;

Obtaining a model length of the second neural network model relative to the first neural network model according to the difference between the second model capacity evaluation value and the first model capacity evaluation value;

screening a growth potential model from the second neural network model according to the modeling length;

and obtaining the optimal super-parameters according to the super-parameters in the growth potential model.

A model building method comprising:

obtaining optimal super parameters according to the super parameters in the growth potential model;

constructing a game role control model by adopting the optimized super parameters; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

A super parameter tuning device, comprising:

the acquisition module is used for acquiring a first neural network model obtained after training the initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter;

the training model is used for training the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value obtained according to the super parameter and the second trained parameter;

the length determining module is used for obtaining the model length of the second neural network model relative to the first neural network model according to the difference between the second model capacity evaluation value and the first model capacity evaluation value;

The screening module is used for screening a growth potential model from the second neural network model according to the model length;

and the optimizing module is used for obtaining optimized super-parameters according to the super-parameters in the growth potential model.

A model building apparatus comprising:

the optimizing module is used for obtaining optimized super parameters according to the super parameters in the growth potential model;

the model construction module is used for constructing a game role control model by adopting the optimized super parameters; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:

According to the above method and device for optimizing the super-parameters, the model construction method and device, the computer readable storage medium and the computer equipment, the first neural network model is obtained by obtaining the first neural network model obtained after training the initial neural network model, the second neural network model is obtained by training the first neural network model, the model formation length of the second neural network model relative to the first neural network model after training is obtained according to the difference between the first model capability evaluation value of the first neural network model and the second model capability of the second neural network model, the growth potential model is selected from a plurality of second neural network models according to the model formation length, the super-parameters in the growth potential model are utilized to obtain the preferred super-parameters, and as the model formation length can reflect the potential of the neural network model, the model capability evaluation value of which meets the user requirement after iterative training, the second neural network model is screened based on the model formation length, the neural network model, which possibly meets the user requirement after the subsequent multi-round training in the super-parameter optimization process is avoided, namely the growth potential is eliminated, the super-parameters are avoided from being further trained, the super-parameters are further saved, and the final neural network is constructed, and the super-parameters are further required, and the super-parameters are further obtained.

Drawings

FIG. 1 is an application environment diagram of a super parameter tuning method in one embodiment;

FIG. 2 is a flow chart of a super parameter tuning method in one embodiment;

FIG. 3 is a flow chart of a super parameter tuning method according to another embodiment;

FIG. 4 is a flow diagram of a model building method in one embodiment;

FIG. 5 is a schematic diagram of a model building flow in one embodiment;

FIG. 6 is a block diagram of an embodiment of a super parameter tuning device;

FIG. 7 is a block diagram showing the construction of a model building apparatus in one embodiment;

FIG. 8 is a block diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

FIG. 1 is a diagram of an application environment of a super parameter tuning method in one embodiment. Referring to fig. 1, the super-parameter tuning method is applied to a model training system. The model training system includes at least a training server 110 and an evaluation server 120. The training server 110 and the evaluation server 120 are connected via a network. The training server 110 and the evaluation server 120 may be implemented as separate servers or as a server cluster composed of a plurality of servers.

The training server 110 is mainly used for performing iterative training on the neural network model constructed based on the super parameters. The evaluation server 120 is mainly used for quantitatively evaluating the neural network model at the training site.

It should be noted that the super-parameter tuning method of the present application can be applied to a scenario of generating a neural network model. The neural network model described above may be applied to, for example, an instant messaging scene, a multimedia play scene, a content recommendation scene, a live broadcast scene, a community social scene, a game scene, a shopping scene, a page browsing scene, a financial service scene, and the like. For example, when the neural network model is applied to a game scene, the neural network model may be used to control operations of moving, deciding, cooperating, etc. of a game character in the game environment to simulate the operations of a real player, so that the game character has artificial intelligence capability, and thus, the function of controlling the game character using the neural network model is also called game AI. For another example, when the neural network model is applied to the content recommendation scene, the neural network model may be used to classify the content type of a certain video content, and recommend the video content to the user according to the classification result, thereby realizing intelligent video recommendation.

The super-parameter tuning method is applied to the application scene, so that the problem that a neural network model meeting the requirements of users cannot be constructed due to the fact that potential super parameters are eliminated in the super-parameter tuning process can be avoided.

In one embodiment, as shown in FIG. 2, a super parameter tuning method is provided. The present embodiment is mainly exemplified by the application of the method to the training server 110 in fig. 1. Referring to fig. 2, the super parameter tuning method specifically includes the following steps:

s202, acquiring a first neural network model obtained after training an initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network has a first model capability assessment value derived from the hyper-parameter and the first trained parameter.

The initial neural network model may be an untrained neural network model constructed according to super parameters.

The first neural network model may be a neural network model obtained after training the initial neural network model.

The hyper-parameters may be parameters of a basic framework for constructing the neural network model, which are manually preset before training. For example, for a convolutional layer of a neural network, the hyper-parameters may be the number of convolutional kernels, the number of channels of the convolutional kernels, the height, width, step size in the horizontal/vertical direction; for a fully connected layer in a neural network, the hyper-parameter may be the number of neurons; for an active layer in a neural network, the hyper-parameters may be the type of active function, internal function parameters of the active function, etc.

The trained parameters may be parameters for constructing a neural network, which are optimized by training, and are also commonly referred to as general parameters. For example, parameters such as weight values, bias values, etc. in the neural network model.

The model capability assessment value may be a numerical value for quantitatively assessing the model capability of the neural network model. The model capabilities may reflect the integrated behavior of the neural network model at the time of actual application. For example, when the neural network model is applied to image classification, the model capability evaluation value may be a classification accuracy of classifying the image; for another example, the model ability evaluation value in the game scene may be specifically a win/lose ratio when the game character plays a game by controlling the neural network model.

In a specific implementation, the training server 110 may first generate multiple sets of super parameters and initial trained parameters, where each set of super parameters and initial trained parameters may construct a corresponding initial neural network model. For a plurality of groups of super parameters, a plurality of corresponding initial neural network models can be constructed.

Then, iterative training is performed on the plurality of initial neural network models, respectively. In the iterative training process, initial trained parameters in the initial neural network model are trained with first trained parameters. For example, sample data is input to the initial neural network model, a loss value is calculated based on the result output by the initial neural network model, and back propagation is performed according to the loss value, so as to continuously adjust the initial trained parameters in the initial neural network model, and obtain new training parameters as the first trained parameters.

It should be noted that, the iterative training process does not adjust the super-parameters in the initial neural network model, so that the super-parameters in the first neural network model obtained by training the initial neural network model are not changed due to training, and the initial trained parameters are adjusted to be the first trained parameters.

After the first neural network model is obtained, the evaluation server 120 may perform quantitative evaluation on the performance of the first neural network model to obtain a model capability evaluation value of the first neural network model.

The model capability of the neural network model is used for reflecting the comprehensive performance of the neural network model in practical application, such as whether the comprehensive performance of various aspects of movement, decision making, cooperation and the like can be accurately performed in a game scene. In order to accurately compare the differences between model capabilities of different neural network models, a quantitative assessment of model capabilities may be made.

For example, the first neural network model may be applied to a specific game application to obtain a model capability evaluation value of the neural network model, and more specifically, the neural network model may control operations such as movement, decision or collaboration of a game character, record a win/loss ratio of the game character, and quantitatively evaluate the model capability of the neural network model based on the win/loss ratio.

For another example, the first neural network model may be applied to an image recognition application to obtain a model capability evaluation value of the neural network model, and more specifically, different images may be classified and recognized by the first neural network model, and the accuracy of classification and recognition is used as the model capability evaluation value to implement quantitative evaluation of the model capability of the neural network model.

It should be further noted that the super parameter and the trained parameter in the neural network model determine an output result in the neural network model, and the output result affects the comprehensive capability of the neural network model, so that the model capability evaluation value of the first neural network model is obtained according to the super parameter and the first trained parameter.

For example, in a game scene, game characters are each an operation of determining how to move, decide, cooperate, etc. in a game environment based on a result output by a neural network model based on input game data, and the neural network model calculates the output result by a super parameter and a trained parameter. For another example, in a video recommendation scenario, the neural network model calculates a correlation between a user and a video by using a super parameter and a trained parameter, and performs video recommendation based on the correlation.

For the sake of distinguishing the description, the model capability evaluation value of the first neural network model described above is named as a first model capability evaluation value.

After the first model capability evaluation value of the first neural network model is obtained, the growth degree of the neural network model constructed based on a certain set of super parameters in the training process can be determined based on the first model capability evaluation value.

S204, training the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value derived from the hyper-parameter and the second trained parameter.

The second neural network model may be a neural network model obtained by training the first neural network model.

It should be noted that, in the training process, the neural network model is usually optimized continuously through multiple rounds of iterative training, and in the iterative training process, each round of training can eliminate the neural network model with non-ideal performance, and part of the neural network model is reserved to enter the next round of training.

In a specific implementation, the training server 110 may perform the next training on the first neural network model to obtain a second neural network model. In a second neural network model obtained by training the first neural network model, the super parameters are not changed due to training, and the first trained parameters are adjusted to the second trained parameters.

After the second neural network model is obtained, the evaluation server 120 may quantitatively evaluate the model capability of the second neural network model to obtain a model capability evaluation value of the second neural network model.

For the sake of distinguishing the description, the model capability evaluation value of the second neural network model described above is named a second model capability evaluation value.

After obtaining the second model capability assessment value of the second neural network model, the growth degree of the neural network model constructed based on a certain set of super parameters in the training process can be determined based on the first model capability assessment value and the second model capability assessment value.

S206, obtaining the model length of the second neural network model relative to the first neural network model according to the difference between the second model capacity evaluation value and the first model capacity evaluation value.

The modeling length is used for reflecting the potential of the neural network model that the model capacity evaluation value accords with the user requirement after iterative training.

In particular implementations, the training server 110 may first determine a difference between the second model capability assessment value and the first model capability assessment value, and use the difference to determine a model length of the second neural network model after training relative to the first neural network model.

For example, the above-described model length may be quantitatively evaluated by a difference between the second model capability evaluation value and the first model capability evaluation value, or may be quantitatively evaluated by a ratio between the second model capability evaluation value and the first model capability evaluation value, or may be quantitatively evaluated by a variance between the second model capability evaluation value and the first model capability evaluation value.

Of course, the above example is merely for illustrating the manner of determining the model length, and in practical applications, those skilled in the art may also evaluate the model length based on the difference between the second model capability evaluation value and the first model capability evaluation value in various manners, for example, may also first calculate the difference between the second model capability evaluation value and the first model capability evaluation value as the evaluation value difference, and then quantitatively evaluate the model length described above in combination of the second model capability evaluation value and the evaluation value difference.

Because the modeling length can reflect the growth potential of the model capacity evaluation value of the neural network model constructed based on a certain set of super parameters after iterative training, it can be understood that the more rounds of iterative training, the more ideal the model capacity evaluation value of the neural network model constructed based on the set of super parameters. Thus, after the model length of each second neural network model is obtained, the second neural network model may be eliminated and retained based on the model length.

And S208, screening a growth potential model from the second neural network model according to the model length.

The growth potential model may be a second neural network model with a model growth degree conforming to a set condition.

In a specific implementation, the training server 110 may sort each second neural network model according to the model length of each second neural network model, reject the second neural network model whose model length does not meet the preset retention condition, and use the second neural network model whose retention lower model length does not meet the preset retention condition as the growth potential model. The remaining growth potential model can be used for the next round of training.

For example, the elimination/retention ratio is preset to be 1:1, for 100 second neural network models, descending order is performed according to the length of the model, after the order, 50 second neural network models are formed, the length of the model does not meet the set condition, 50 second neural network models after the order are eliminated, 50 second neural network models before the order are retained, 50 growth potential models are obtained, and the 50 growth potential models enter the next round of training.

For example, a model growth threshold is set in advance, the model growth of each second neural network model is compared with the preset model growth threshold, and the second neural network model having a model length greater than the model length threshold is used as the growth potential model.

Of course, those skilled in the art may use other ways to reject and retain the plurality of second neural network models according to the modeling length to screen out the second neural network models with growth potential.

S210, obtaining the optimal super-parameters according to the super-parameters in the growth potential model.

The preferred superparameters may be those preferred among a plurality of sets of superparameters for constructing a final neural network model.

In particular implementations, after the growth potential model is obtained, the training server 110 may determine the superparameter used to construct the final neural network model using the superparameter in the growth potential model as the preferred superparameter described above. Therefore, the preferable super parameters are screened out from the generated multiple groups of super parameters, and the neural network model applicable to various application scenes can be constructed based on the preferable super parameters.

For example, the total number of iterative training rounds R for performing iterative training may be preset, and after the iterative training of R rounds, one or more growth potential models obtained may be used as a preferred model, where the super-parameters in the preferred model are the preferred super-parameters described above.

In the above-mentioned super-parametric tuning method, the first neural network model is obtained by obtaining the first neural network model obtained after training the initial neural network model, the second neural network model is obtained by training the first neural network model, the model length of the second neural network model relative to the first neural network model after training is obtained according to the difference between the first model capacity evaluation value of the first neural network model and the second model capacity of the second neural network model, the growth potential model is screened out in a plurality of second neural network models according to the model length, the super-parameters in the growth potential model are utilized to obtain the preferred super-parameters, and as the model length can reflect the potential of the neural network model, the model capacity evaluation value of the neural network model accords with the user requirements after iterative training, the screening of the second neural network model is carried out based on the model length, the neural network model with the potential of the user requirements possibly being eliminated after the subsequent multi-round training in the super-parametric tuning process is avoided, namely the super-parametric tuning is avoided, the super-parametric model with the growth potential is eliminated, the final neural network with the growth potential is maintained, the super-parametric model is continuously trained, and the final neural network with the super-parametric model is obtained, and the final neural network with the super-parametric model is required to be constructed.

In one embodiment, the step S206 may specifically include:

calculating an evaluation value difference between the second model capability evaluation value and the first model capability evaluation value; and calculating the model length according to the evaluation value difference value and a second model capacity evaluation value of the second neural network model.

Wherein the evaluation value difference may be the difference between two model capability evaluation values.

In a specific implementation, the training server 110 may calculate a difference between the second model capability assessment value and the first model capability assessment value as the above-mentioned assessment value difference. Then, the model length is calculated by combining the evaluation value difference and the second model capacity evaluation value of the second neural network model.

For example, the evaluation value difference and the second model capability evaluation value are added, and the sum is taken as the model length.

For another example, different weights are respectively given to the evaluation value difference value and the second model capacity evaluation value and weighted, so as to obtain respective weighted values of the evaluation value difference value and the second model capacity evaluation value, then the two weighted values are added, and the sum is taken as the length of the model.

In the above-mentioned super-parameter tuning method, the evaluation value difference value may reflect the potential of the model capacity evaluation value of the second neural network model after the subsequent iterative training to meet the user requirement, and the second model capacity evaluation value may reflect the current model capacity evaluation value of the trained second neural network model, and the model length is calculated by combining the evaluation value difference value and the second model capacity evaluation value, so that the model length may comprehensively reflect the current model capacity and growth potential of the neural network model, thereby more comprehensively quantifying and evaluating the neural network model, and therefore, the second neural network model may be more accurately screened.

In one embodiment, the calculating the model length according to the evaluation value difference and the second model capability evaluation value of the second neural network model includes:

acquiring a growth weight; obtaining a capacity evaluation value weight according to the growth weight; calculating the product of the evaluation value difference and the growth weight to obtain a weighted growth value; calculating the product of the second model capacity evaluation value and the capacity evaluation value weight to obtain a weighted capacity value; and calculating the sum of the weighted growth value and the weighted capacity value to obtain the model length.

The growth weight may be a weight for calculating a proportion of the difference value of the evaluation value in the modeling length.

The capability evaluation value weight may be a weight for calculating a proportion of the model capability evaluation value in the model length.

In particular implementations, the training server 110 may set a growth weight and its corresponding capability assessment value weight. For example, the growth weight α may be set, and accordingly, the capacity evaluation value weight is (1- α).

Then, calculating the product of the difference value of the evaluation value and the growth weight as the weighted growth value; calculating the product of the second model capability evaluation value and the capability evaluation value weight to be used as the weighted capability value; and calculating the sum of the weighted growth value and the weighted capacity value to obtain the model length.

For example, determining the growth weight alpha and the capability assessment value weight as (1-alpha), and after the j-th round of iterative training, the second neural network model has a second model capability assessment value w _j After the first neural network model is subjected to the j-1 th round of iterative training, the first neural network model has a first model capacity evaluation value w _j-1 The evaluation value difference f (w _j ，w _j-1 )＝w _j -w _j-1 Modeling length zw _j ＝α*f(w _j ，w _j-1 )+(1-α)w _j 。

In the above super-parameter tuning method, the corresponding weights are respectively assigned to the evaluation value difference value and the second model capacity evaluation value, and the model length is obtained by weighting and summing the respective weights, so that the proportion of the evaluation value difference value and the second model capacity evaluation value in the model length can be adjusted according to different user requirements, and the flexibility of super-parameter tuning is improved.

In one embodiment, the second neural network model has n numbers, n is greater than or equal to 2, and the step S208 may specifically include:

sorting n second neural network models in descending order according to the length of the models; according to a preset model retention proportion, retaining m second neural network models before sequencing in n second neural network models to obtain m growth potential models; wherein n is more than m and is more than or equal to 1.

In particular implementations, the training server 110 may sort the n second neural network models in descending order according to the model length, i.e., from high to low according to the model length.

And according to a preset model retention proportion, eliminating part of the second neural network models from the n second neural network models, and retaining the m second neural network models before sequencing to serve as the growth potential models.

For example, the preset model retention ratio is 1:1, i.e., half of the neural network model is eliminated. For 100 second neural network models, 50 second neural network models after the model growth degree ordering are eliminated, and 50 second neural network models after the model growth degree ordering are reserved, so that 50 growth potential models are obtained.

In one embodiment, the step S210 may specifically include:

determining the current training round number j for training the first neural network model; j is more than or equal to 1;

when the current training round number j reaches a preset iteration training total round number R, sorting m growth potential models in a descending order according to the second model capacity evaluation value; r is more than or equal to j is more than or equal to 1;

extracting p growth potential models before sequencing as preferred models; wherein m is greater than or equal to p is greater than or equal to 1

And taking the super-parameters in the preferred model as the preferred super-parameters.

The current training number of rounds may be the number of rounds of training the first neural network model.

The total number of iterative training wheels can be a preset total number of wheels which need to be subjected to iterative training.

In a specific implementation, the training server 110 may preset the total number of rounds of iterative training on the neural network model, which is the total number of rounds of iterative training R.

After each time training is carried out on the neural network model, the current training round number j is updated. For example, the first training round, j, is updated to 1, and the second training round, j, is updated to 2.

Then judging whether the current training round number j reaches a preset total round number R of iterative training, if not, representing that the iterative training is not completed, and starting the training of the next round; if yes, the iteration training is completed.

After the iterative training is completed, the m growth potential models may be ranked in descending order according to the second model capability assessment value, that is, the m growth potential models are ranked from high to low according to the second model capability assessment value, and p growth potential models before the ranking are used as the preferred models. The superparameter in the priority model may then be the preferred superparameter.

In the above-mentioned super-parameter tuning method, if the current training round number reaches the preset iteration training total round number, the growth potential models may be sorted in descending order according to the second model capability evaluation value, and the growth potential model with the front sorting may be extracted as the preferred model, so as to obtain the super-parameter in the preferred model as the preferred super-parameter. Because the training of the neural network model is completed, the length of the model is not needed to be considered, the second model capacity evaluation value is directly adopted to determine the optimal model, the calculated amount is reduced on the premise of ensuring the optimal super-parameters, and the super-parameter tuning efficiency is improved.

In one embodiment, when the current training round number j does not reach the total number of iterative training rounds R, step S210 may further include:

and taking m growth potential models as the updated first neural network models, and returning to the step of training the first neural network models to obtain second neural network models so as to perform the next training until the current training round number j reaches the iterative training total round number R.

In a specific implementation, when the current training round number j does not reach the total iterative training round number R, that is, the next round of iterative training needs to be performed, m growth potential models can be used as updated first neural network models, and the step S204 is returned to perform training on the updated first neural network models until the current training round number j reaches the total iterative training round number R, that is, until the iterative training is finished and a preferred model is obtained.

In one embodiment, after the step S204, the following steps may be further included:

judging whether n second neural network models accord with the preset simultaneous training model quantity b or not;

if yes, taking the second neural network model as the growth potential model, and executing the step of obtaining preferable super parameters according to the super parameters in the growth potential model;

and if not, executing the step of obtaining the model length of the second neural network model relative to the first neural network model according to the difference between the second model capability evaluation value and the first model capability evaluation value.

Wherein the number of simultaneous training models may be the number of neural network models that are simultaneously trained that are supportable by the computing resources of the training server 110.

In a specific implementation, the computing resource used by the training server 110 for performing iterative training may be first determined, and according to the computing resource, it is determined that the training server 110 may train b neural network models simultaneously, thereby determining the number b of simultaneous training models.

After the n second neural network models are obtained, it may be determined whether the n second neural network models are smaller than the simultaneous training model number b.

When the number n of the second neural network models is smaller than or equal to the number b of the simultaneous training models, that is, the number n of the second neural network models accords with the preset number b of the simultaneous training models, it indicates that the computing resource of the current training server 110 can support the simultaneous training of the n second neural network models, at this time, all n second neural network models can be used as growth potential models without eliminating the n second neural network models, and the step S210 is performed, that is, the step of obtaining the preferred hyper-parameters according to the hyper-parameters in the growth potential models is performed.

When the number n of second neural network models is greater than the number b of simultaneous training models, that is, the number b of simultaneous training models is not satisfied, it indicates that the computing resources of the current training server 110 cannot support simultaneous training of the n second neural network models, and thus need to be eliminated, so the step S206 may be performed, that is, the step of obtaining a model length of the second neural network model relative to the first neural network model according to the difference between the second model capability evaluation value and the first model capability evaluation value is performed, and eliminating the n second neural network models according to the model length, so as to screen out m growth potential models.

It should be noted that, when the number n of the second neural network models is less than or equal to the number b of the simultaneous training models, the computing resource of the training server 110 may support to train the n second neural network models simultaneously, without waiting for training of the next batch of second neural network models after the training of the b second neural network models is completed, so that no waiting time is generated no matter whether the n second neural network models are eliminated or not.

In the above-mentioned super-parameter tuning method, when the n second neural network models conform to the preset number b of simultaneous training models, the n second neural network models are used as growth potential models, and the step of obtaining the preferred super-parameters according to the super-parameters in the growth potential models is directly executed, so that the second neural network models are not required to be eliminated.

As shown in fig. 3, in one embodiment, a super parameter tuning method is provided, referring to fig. 3, before step S202 described above, further including:

S302, obtaining a super-parameter group number N according to the simultaneous training model number b, the iterative training total wheel number R and the model retention proportion; n is more than or equal to N is more than or equal to 2.

In a specific implementation, the training server 110 may calculate the super-parameter set N according to the number of simultaneous training models b, the total number of iterative training rounds R, and the model retention ratio.

For example, assume that the model retention ratio is 1:1, can pass through n=2 ^R-2 * b, calculating the formula to obtain a super-parameter groupNumber N. Assuming 5 rounds of iterative training, i.e. the total number of iterative training rounds r=5, the computational resources of the training server 110 cannot support simultaneous training of 5 second neural network models, i.e. the number of simultaneous training models b=5, whereby n=2 ^R-2 *b＝2 ⁵ ^-2 * 5=40, i.e. 40 sets of hyper-parameters need to be generated.

S304, generating N groups of the super parameters and N groups of initial trained parameters.

In particular implementations, training server 110 may generate N sets of hyper-parameters and corresponding N sets of trained parameters. More specifically, there is a current need to generate a neural network model that includes k1 hyper-parameters and k2 trained parameters, each set of hyper-parameters consisting of k1 hyper-parameters and each set of trained parameters consisting of k2 trained parameters.

In practical application, the value range of each super parameter c can be [ -1.0,1.0], so that values can be randomly obtained within the range of [ -1.0,1.0] to randomly generate a plurality of super parameters c, thereby forming a group of super parameters.

S306, constructing N initial neural network models by adopting N groups of the super parameters and N groups of the initial trained parameters.

In specific implementation, N groups of super parameters and N groups of initial trained parameters can be respectively adopted to construct N neural network models, and the N neural network models are used as the initial neural network models.

For example, assuming k1=2, N sets of superparameters and corresponding N initial neural network models may be obtained, respectively [ c ] _{i_1} ,c _{i_2} ],init_checkpoint_i，i＝1，2…N。

S308, training the N initial neural network models to obtain N first neural network models.

In a specific implementation, training is performed on the N initial neural network models respectively, so as to obtain N first neural network models.

In one embodiment, the step S204 may specifically include:

determining the model training step number corresponding to the current training wheel number j; the model training step number and the current training wheel number j are in positive correlation; and training the first neural network model according to the model training step number to obtain the second neural network model.

The model training step number may be a training step number for training the neural network model. The number of training steps, also commonly referred to as Batch Size, is used to determine the number of samples needed to calculate the gradient during training.

In a specific implementation, the training server 110 may pre-determine an optimal model training total number of steps, allocate a model training step number for each round of training according to the iterative training total number of rounds R, so that the sum of the model training step numbers of each round of training is equal to the model training total number of steps, and make the model training step number allocated by each round of training form a positive correlation with the training round number.

For example, the optimal total model training step number stepmax=93 w (ten thousand), the total iterative training round number r=5, and the model training step number of the first round is assumed to be 3w, by the formula StepMax/(2) ^R+1 -1) whereby the number of training steps for each round of training can be obtained as [3w,2 x 3w,2 respectively ² *3w，2 ³ *3w，2 ⁴ *3w]。

When each round of iterative training is performed, the model training step number corresponding to the current training round number j can be determined first, and then the first neural network model is trained according to the model training step number corresponding to the current training round number j, so that a second neural network model is obtained.

For example, at round 3 of iterative training, the corresponding model training step number is 12w, and therefore, the first neural network is trained according to the model training step number of 12 w. At the 4 th round of iterative training, the corresponding model training step number is 24w, so that the first neural network is trained according to the model training step number of 24 w.

When training the neural network model, the larger the training step number is, the larger the occupied computing resource is needed, and the more ideal the training quality is. Because part of the neural network model is eliminated after each round of training, as the number j of training rounds increases, the fewer the neural network models are kept, the fewer the required computing resources are, and accordingly, the larger training steps can be adopted for training, so that the training quality is ensured.

In the above super-parameter tuning method, the first neural network model is trained according to the model training step number by determining the model training step number having a positive correlation with the current training step number to obtain the second neural network model, so that the training quality of the neural network model is ensured on the premise of avoiding increasing the demand of computing resources, and the problem that the preferable super-parameter cannot be obtained due to the non-ideal training quality of the neural network model is avoided.

As shown in fig. 4, in one embodiment, a model building method is provided. Referring to fig. 4, the model construction method may include the steps of:

s402, acquiring a first neural network model obtained after training an initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter;

s404, training the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value obtained according to the super parameter and the second trained parameter;

s406, obtaining a model length of the second neural network model relative to the first neural network model according to the difference between the second model capability evaluation value and the first model capability evaluation value;

s408, screening a growth potential model from the second neural network model according to the model length;

s410, obtaining a preferable super-parameter according to the super-parameter in the growth potential model;

S412, constructing a game role control model by adopting the preferable super parameters; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

The game character control model may be a neural network model for controlling game characters to perform moving operations, decision operations, and cooperation operations in a game environment.

Since steps S402 to S410 have been described in detail in the above embodiments, they are not described in detail herein. After the preferred hyper-parameters are obtained, the game character control model may be constructed using the preferred hyper-parameters, for step S412. The game character control model is used in game application to control the game character to execute moving operation, decision making operation, cooperation operation, etc. in game environment. For example, the game character is controlled to move forward and backward, the game character is controlled to attack and defend, and the like, and the game character is controlled to cooperate with other game characters in forward and backward operations.

In the above model construction method, the first neural network model is obtained by obtaining the first neural network model obtained after training the initial neural network model, the second neural network model is obtained by training the first neural network model, the length of the second neural network model relative to the first neural network model after training is obtained according to the difference between the first model capacity evaluation value of the first neural network model and the second model capacity of the second neural network model, the growth potential model is selected in a plurality of second neural network models according to the length of the model, the super-parameters in the growth potential model are utilized to obtain the preferred super-parameters, and as the length of the model can reflect the potential of the neural network model which meets the requirements of users after iterative training, the second neural network model is screened based on the length of the model, the neural network model which possibly meets the requirements of users after the subsequent training for a plurality of rounds in the super-parameter tuning process is avoided, that is avoided, the super-parameters with the growth potential are eliminated, the neural network with the growth potential are kept, the super-parameters in the final role model is further controlled to continue to be controlled based on the preferred role operation, the game role operation is controlled to continue to obtain the preferred role operation, the game operation is controlled to be more accurate, the game operation is controlled, the game role operation is controlled based on the preferred role operation is performed, and the game operation is more required to be controlled, and the game operation is better, and the game operation condition is controlled.

In order to facilitate a thorough understanding of the present application by those skilled in the art, the following description will be provided with reference to the specific example of fig. 5. FIG. 5 is a schematic diagram of a model building process in one embodiment, which may include the steps of:

s502, determining the number b of simultaneous training models, the total number R of iterative training wheels and the model retention proportion;

s504, determining a super-parameter group number N according to the number b of simultaneous training models, the total number R of iterative training wheels and the model retention proportion;

s506, generating N groups of super parameters, and respectively constructing N initial neural network models according to the N groups of super parameters;

s508, training the N initial neural network models to obtain N first neural network models; carrying out quantitative evaluation on the model capacity of the N first neural network models to obtain N first model capacity evaluation values;

s510, training N first neural network models respectively to obtain N second neural network models; carrying out quantitative evaluation on the model capacity of the N second neural network models to obtain N second model capacity evaluation values;

s512, judging whether the number n of the current second neural network models is smaller than the number b of the simultaneous training models; if not, executing S514; if yes, determining the current second neural network model as a growth potential model, and executing S518;

S514, calculating an evaluation value difference value of the second model capability evaluation value and the first model capability evaluation value, and calculating a model length according to the evaluation value difference value and the second model capability evaluation value;

s516, reserving the second neural network model with the front sequence in the N second neural network models according to the length of the model forming, and obtaining N growth potential models;

s518, determining the current training wheel number j, judging whether the current training wheel number j reaches the iterative training total wheel number R, and if not, executing S520; if yes, then execute S522;

s520, taking the n growth potential models as updated first neural network models, and returning to S510;

s522, determining a preferred model in the n growth potential models according to the second model capacity evaluation value, and taking the super-parameters in the preferred model as preferred super-parameters;

and S524, constructing a target neural network model according to the preferable super parameters.

It should be understood that, although the steps in the flowcharts of fig. 2, 4, and 5 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2, 4, and 5 may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

As shown in fig. 6, in one embodiment, there is provided a super parameter tuning device 600, including:

an obtaining module 602, configured to obtain a first neural network model obtained after training the initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter;

a training model 604, configured to train the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value obtained according to the super parameter and the second trained parameter;

a length determining module 606, configured to obtain a model length of the second neural network model relative to the first neural network model according to a difference between the second model capability assessment value and the first model capability assessment value;

a screening module 608, configured to screen a growth potential model from the second neural network model according to the model length;

The optimizing module 610 is configured to obtain a preferred superparameter according to the superparameter in the growth potential model.

In one embodiment, the growth determination module 606 is specifically configured to:

calculating an evaluation value difference between the second model capability evaluation value and the first model capability evaluation value;

and calculating the model length according to the evaluation value difference value and a second model capacity evaluation value of the second neural network model.

In one embodiment, the second neural network model has n, n being equal to or greater than 2, and the filtering module 608 is specifically configured to:

In one embodiment, the optimizing module 610 is specifically configured to:

determining the current training round number j for training the first neural network model; j is more than or equal to 1; when the current training round number j reaches a preset iteration training total round number R, sorting m growth potential models in a descending order according to the second model capacity evaluation value; r is more than or equal to j is more than or equal to 1; extracting p growth potential models before sequencing as preferred models; wherein m is more than or equal to p is more than or equal to 1, and the super parameter in the preferred model is used as the preferred super parameter.

In one embodiment, when the current training round number j does not reach the total number of iterative training rounds R, the optimizing module 610 is further specifically configured to:

In one embodiment, the apparatus is further specifically configured to:

judging whether n second neural network models accord with the preset simultaneous training model quantity b or not; if yes, taking the second neural network model as the growth potential model, and executing the step of obtaining preferable super parameters according to the super parameters in the growth potential model; and if not, executing the step of obtaining the model length of the second neural network model relative to the first neural network model according to the difference between the second model capability evaluation value and the first model capability evaluation value.

In one embodiment, further comprising:

the group number determining module is used for obtaining a super-parameter group number N according to the simultaneous training model number b, the iterative training total wheel number R and the model retention proportion; n is more than or equal to N is more than or equal to 2;

the parameter generation module is used for generating N groups of the super parameters and N groups of initial trained parameters;

the initial model building module is used for building N initial neural network models by adopting N groups of the super parameters and N groups of the initial trained parameters;

the training module 604 is further configured to train the N initial neural network models to obtain N first neural network models.

In one embodiment, the training module 604 is specifically configured to:

As shown in fig. 7, in one embodiment, there is provided a model building apparatus 700 including:

an obtaining module 702, configured to obtain a first neural network model obtained after training an initial neural network model; the first neural network includes a super parameter and a first trained parameter; the first neural network is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter;

A training model 704, configured to train the first neural network model to obtain a second neural network model; the second neural network includes the super-parameter and a second trained parameter; the second neural network has a second model capability assessment value obtained according to the super parameter and the second trained parameter;

a length determining module 706, configured to obtain a model length of the second neural network model relative to the first neural network model according to a difference between the second model capability assessment value and the first model capability assessment value;

a screening module 708, configured to screen a growth potential model from the second neural network model according to the model length;

the optimizing module 710 is configured to obtain a preferred super parameter according to the super parameter in the growth potential model;

a model construction module 712 for constructing a game character control model using the preferred hyper-parameters; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

For the specific limitation of the above-mentioned super parameter tuning and model construction device, reference may be made to the limitation of the above-mentioned super parameter tuning and model construction method, and the description thereof will not be repeated here. The above-mentioned super parameter tuning and each module in the model building device can be implemented in whole or in part by software, hardware and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

The super parameter tuning and model construction device provided by the above can be used for executing the super parameter tuning and model construction method provided by any embodiment, and has corresponding functions and beneficial effects.

It should be noted that the artificial intelligence (Artificial Intelligence, AI) in the above embodiments is a theory, method, technique and application system that simulates, extends and expands human intelligence using a digital computer or a machine controlled by a digital computer, senses environment, acquires knowledge and uses the knowledge to obtain an optimal result. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Computer Vision (CV) is a science of studying how to "look" a machine, and more specifically, to replace human eyes with a camera and a Computer to perform machine Vision such as recognition, following and measurement on a target, and further perform graphic processing, so that the Computer is processed into an image more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, and map construction, among others, as well as common biometric recognition techniques such as face recognition, fingerprint recognition, and others.

Key technologies to the speech technology (Speech Technology) are automatic speech recognition technology (ASR) and speech synthesis technology (TTS) and voiceprint recognition technology. The method can enable the computer to listen, watch, say and feel, is the development direction of human-computer interaction in the future, and voice becomes one of the best human-computer interaction modes in the future.

Natural language processing (Nature Language processing, NLP) is an important direction in the fields of computer science and artificial intelligence. It is studying various theories and methods that enable effective communication between a person and a computer in natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Thus, the research in this field will involve natural language, i.e. language that people use daily, so it has a close relationship with the research in linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic questions and answers, knowledge graph techniques, and the like.

Machine Learning (ML) is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

The automatic driving technology generally comprises high-precision map, environment perception, behavior decision, path planning, motion control and other technologies, has wide application prospect,

with research and advancement of artificial intelligence technology, research and application of artificial intelligence technology is being developed in various fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, automatic driving, unmanned aerial vehicles, robots, smart medical treatment, smart customer service, etc., and it is believed that with the development of technology, artificial intelligence technology will be applied in more fields and with increasing importance value.

The scheme provided by the embodiment of the application relates to the technology of artificial intelligence such as machine learning, and the like, and is specifically described by the following embodiments: training the first neural network model in a machine learning mode to obtain a second neural network model.

FIG. 8 illustrates an internal block diagram of a computer device in one embodiment. The computer device may be specifically the training server 110 of fig. 1. As shown in fig. 8, the computer device includes a processor, a memory, a network interface, an input device, and a display screen connected by a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by a processor, causes the processor to implement a super parameter tuning, model building method. The internal memory may also store a computer program which, when executed by the processor, causes the processor to perform the super-parameter tuning, model building method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in FIG. 8 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, the super-parameter tuning and model building device provided by the application can be implemented as a form of a computer program, and the computer program can be run on a computer device as shown in fig. 8. The memory of the computer device may store the various program modules that make up the super-parametric tuning, model building means, such as training model 604 shown in fig. 6. The computer program comprising the program modules causes the processor to carry out the steps of the super parameter tuning, model building method according to the embodiments of the application described in the present specification.

For example, the computer device shown in fig. 8 may perform training on the first neural network model through the training model 604 in the super parameter tuning device shown in fig. 6, to obtain a second neural network model.

In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the above-described super-parametric tuning, model building method. The step of the super parameter tuning and model construction method may be the step of the super parameter tuning and model construction method in each of the above embodiments.

In one embodiment, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the above-described super-parametric tuning, model building method. The step of the super parameter tuning and model construction method may be the step of the super parameter tuning and model construction method in each of the above embodiments.

Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims

1. A method for optimizing super parameters, comprising:

acquiring a plurality of first neural network models obtained after training a plurality of initial neural network models respectively; each initial neural network model is constructed based on a set of super parameters and a set of initial trained parameters; the initial neural network models correspond to a plurality of groups of super parameters; the first neural network model comprises super parameters and first trained parameters; the first neural network model is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter; the first model capability evaluation value is used for reflecting the performance of the first neural network model for controlling the game role to play in the game scene, and is the winning or losing proportion when the first neural network model is used for controlling the game role to play; in the process of controlling a game character to play a game through the first neural network model, the first neural network model is used for determining an operation executed by the game character in a game environment based on a result output by input game data;

Training the first neural network models respectively to obtain a plurality of second neural network models; the second neural network model includes the hyper-parameters and second trained parameters; the second neural network model has a second model capability assessment value obtained from the hyper-parameters and the second trained parameters; the second model capability assessment value is used for reflecting the performance of the second neural network model for controlling the game role to play in the game scene, and is the winning or losing proportion when the second neural network model is used for controlling the game role to play; in controlling a game character to play a game through the second neural network model, the second neural network model is used for determining an operation performed by the game character in a game environment based on a result output by input game data;

for each second neural network model, obtaining a model length of the second neural network model relative to the first neural network model according to a difference between a second model capability evaluation value of the second neural network model and a first model capability evaluation value corresponding to the second neural network model;

Screening a growth potential model from the plurality of second neural network models according to the respective model length of the plurality of second neural network models;

obtaining optimal super parameters according to the super parameters in the growth potential model; the preferred hyper-parameters are used to construct a game character control model; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

2. The method of claim 1, wherein the deriving the model length of the second neural network model relative to the first neural network model based on the differences between the second model capability assessment value of the second neural network model and the first model capability assessment value corresponding to the second neural network model comprises:

calculating an evaluation value difference of a second model capacity evaluation value of the second neural network model and a first model capacity evaluation value corresponding to the second neural network model;

and calculating the model length of the second neural network model relative to the first neural network model according to the evaluation value difference value and a second model capacity evaluation value of the second neural network model.

3. The method of claim 2, wherein the calculating the model length of the second neural network model relative to the first neural network model based on the evaluation value difference and a second model capability evaluation value of the second neural network model comprises:

acquiring a growth weight;

obtaining a capacity evaluation value weight according to the growth weight;

calculating the product of the evaluation value difference and the growth weight to obtain a weighted growth value;

calculating the product of a second model capacity evaluation value of the second neural network model and the capacity evaluation value weight to obtain a weighted capacity value;

and calculating the sum of the weighted growth value and the weighted capacity value to obtain the model length of the second neural network model relative to the first neural network model.

4. The method of claim 1, wherein the second neural network model has n, n being equal to or greater than 2, the screening growth potential models among the plurality of second neural network models based on respective model length of the plurality of second neural network models, comprising:

sorting n second neural network models in descending order according to the respective model length of the plurality of second neural network models;

According to a preset model retention proportion, retaining m second neural network models before sequencing in n second neural network models to obtain m growth potential models; wherein n is more than m and is more than or equal to 1.

5. The method of claim 4, wherein the deriving the preferred hyper-parameters from the hyper-parameters in the growth potential model comprises:

extracting p growth potential models before sequencing as preferred models; wherein m is more than or equal to p is more than or equal to 1;

6. The method of claim 5, wherein when the current training round number j does not reach the iterative training total round number R, further comprising:

and taking m growth potential models as a plurality of updated first neural network models, and returning to the step of respectively training the plurality of first neural network models to obtain a plurality of second neural network models so as to perform the next training until the current training round number j reaches the iterative training total round number R.

7. The method of claim 5, further comprising, after the training the plurality of first neural network models to obtain a plurality of second neural network models, respectively:

and if not, executing the difference between the second model capacity evaluation value according to the second neural network model and the first model capacity evaluation value corresponding to the second neural network model to obtain the model length of the second neural network model relative to the first neural network model.

8. The method of claim 7, further comprising, prior to the obtaining the plurality of first neural network models that result from training the plurality of initial neural network models, respectively:

obtaining a super-parameter group number N according to the simultaneous training model number b, the iterative training total wheel number R and the model retention proportion; n is more than or equal to N is more than or equal to 2;

Generating N groups of the super parameters and N groups of initial trained parameters;

constructing N initial neural network models by adopting N groups of the super parameters and N groups of the initial trained parameters;

and training the N initial neural network models to obtain N first neural network models.

9. The method of claim 5, wherein training the first neural network model to obtain a second neural network model comprises:

determining the model training step number corresponding to the current training wheel number j; the model training step number and the current training wheel number j are in positive correlation;

and training the first neural network model according to the model training step number to obtain the second neural network model.

10. A method of modeling, comprising:

11. A super parameter tuning device, comprising:

the acquisition module is used for acquiring a plurality of first neural network models obtained after training the plurality of initial neural network models respectively; each initial neural network model is constructed based on a set of super parameters and a set of initial trained parameters; the initial neural network models correspond to a plurality of groups of super parameters; the first neural network model comprises super parameters and first trained parameters; the first neural network model is provided with a first model capacity evaluation value obtained according to the super parameter and the first trained parameter; the first model capability evaluation value is used for reflecting the performance of the first neural network model for controlling the game role to play in the game scene, and is the winning or losing proportion when the first neural network model is used for controlling the game role to play; in the process of controlling a game character to play a game through the first neural network model, the first neural network model is used for determining an operation executed by the game character in a game environment based on a result output by input game data;

The training module is used for respectively training the plurality of first neural network models to obtain a plurality of second neural network models; the second neural network model includes the hyper-parameters and second trained parameters; the second neural network model has a second model capability assessment value obtained from the hyper-parameters and the second trained parameters; the second model capability assessment value is used for reflecting the performance of the second neural network model for controlling the game role to play in the game scene, and is the winning or losing proportion when the second neural network model is used for controlling the game role to play; in controlling a game character to play a game through the second neural network model, the second neural network model is used for determining an operation performed by the game character in a game environment based on a result output by input game data;

a length determining module, configured to obtain, for each of the second neural network models, a model length of the second neural network model relative to the first neural network model according to a difference between a second model capability evaluation value of the second neural network model and a first model capability evaluation value corresponding to the second neural network model;

The screening module is used for screening a growth potential model from the plurality of second neural network models according to the respective model length of the plurality of second neural network models;

the optimizing module is used for obtaining optimized super parameters according to the super parameters in the growth potential model; the preferred hyper-parameters are used to construct a game character control model; the game character control model is used for controlling a game character to perform at least one of a moving operation, a decision operation and a cooperation operation in a game environment.

12. The apparatus of claim 11, wherein the growth determination module is specifically configured to:

calculating an evaluation value difference of a second model capacity evaluation value of the second neural network model and a first model capacity evaluation value corresponding to the second neural network model; and calculating the model length of the second neural network model relative to the first neural network model according to the evaluation value difference value and a second model capacity evaluation value of the second neural network model.

13. The apparatus according to claim 12, wherein the growth determination module is specifically configured to:

acquiring a growth weight; obtaining a capacity evaluation value weight according to the growth weight; calculating the product of the evaluation value difference and the growth weight to obtain a weighted growth value; calculating the product of a second model capacity evaluation value of the second neural network model and the capacity evaluation value weight to obtain a weighted capacity value; and calculating the sum of the weighted growth value and the weighted capacity value to obtain the model length of the second neural network model relative to the first neural network model.

14. The apparatus of claim 11, wherein the second neural network model has n, n being ≡2; the screening module is specifically used for:

sorting n second neural network models in descending order according to the respective model length of the plurality of second neural network models; according to a preset model retention proportion, retaining m second neural network models before sequencing in n second neural network models to obtain m growth potential models; wherein n is more than m and is more than or equal to 1.

15. The apparatus of claim 14, wherein the optimizing module is specifically configured to:

determining the current training round number j for training the first neural network model; j is more than or equal to 1; when the current training round number j reaches a preset iteration training total round number R, sorting m growth potential models in a descending order according to the second model capacity evaluation value; r is more than or equal to j is more than or equal to 1; extracting p growth potential models before sequencing as preferred models; wherein m is more than or equal to p is more than or equal to 1; and taking the super-parameters in the preferred model as the preferred super-parameters.

16. The apparatus of claim 15, wherein when the current training round number j does not reach the iterative training total round number R, the optimizing module is further configured to:

17. The apparatus of claim 15, wherein the apparatus is further configured to:

judging whether n second neural network models accord with the preset simultaneous training model quantity b or not; if yes, taking the second neural network model as the growth potential model, and executing the step of obtaining preferable super parameters according to the super parameters in the growth potential model; and if not, executing the difference between the second model capacity evaluation value according to the second neural network model and the first model capacity evaluation value corresponding to the second neural network model to obtain the model length of the second neural network model relative to the first neural network model.

18. The apparatus of claim 17, wherein the apparatus further comprises:

the training module is further configured to train the N initial neural network models to obtain N first neural network models.

19. The apparatus according to claim 15, wherein the training module is specifically configured to:

20. A model building apparatus, comprising:

21. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 10.

22. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 10.