CN113723692A

CN113723692A - Data processing method, apparatus, device, medium, and program product

Info

Publication number: CN113723692A
Application number: CN202111028726.2A
Authority: CN
Inventors: 刘涛; 卢道和; 罗锶; 黄叶飞; 边元乔; 商市盛
Original assignee: WeBank Co Ltd
Current assignee: WeBank Co Ltd
Priority date: 2021-09-02
Filing date: 2021-09-02
Publication date: 2021-11-30

Abstract

The application provides a data processing method, a device, equipment, a medium and a program product, which are characterized in that training data and a first model to be trained are obtained; then, training the first model to be trained respectively according to the plurality of characteristic parameter groups and training data to determine a plurality of models to be selected, wherein each model to be selected corresponds to the characteristic parameter group; judging whether at least one model to be selected meets the preset requirement or not according to the training data; if not, acquiring adjustment training data, and carrying out dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement; if so, determining at least one target processing model from the multiple models to be selected according to the preset use requirement, and processing the acquired data to be processed by using the target processing model to determine a target processing result. The method solves the technical problems that the service quantity of the service interface is mild and/or sudden fluctuation can not be found or predicted in time in the prior art.

Description

Data processing method, apparatus, device, medium, and program product

Technical Field

The present application relates to the field of computer data monitoring, and in particular, to a data processing method, apparatus, device, medium, and program product.

Background

With the continuous development of internet technology, many network services, such as online shopping, become essential service contents in daily work and life of people. These network services are typically supported or serviced by a customized service management system.

The traffic prediction model in the existing traffic management system generally serves for the preparation work of activities such as scene analysis, service expansion and the like of each service only by predicting the traffic. However, it is also important for a traffic management system to monitor and predict traffic, discover or predict changes in traffic, and allocate resources to support smooth operation of traffic. However, in the prior art, when some service activities are released, the traffic volume may fluctuate greatly in a short time, or continuously fluctuate slightly and slightly, but the accumulated excess amount is excessive, which may cause system interface blockage, further cause system paralysis, cause the previous traffic volume prediction result to have no effect on the actual service, seriously affect the user experience, and cause failure in service activity promotion.

In summary, in the prior art, there are technical problems that the traffic of the service interface continuously fluctuates with a small amplitude and a small temperature within a period of time, but the cumulative influence exceeds the capability range of the traffic prediction model, and/or the traffic suddenly fluctuates for a short time and cannot be found or predicted in time, that is, the traffic prediction model cannot have comprehensive and stable performance.

Disclosure of Invention

The application provides a data processing method, a device, equipment, a medium and a program product, which are used for solving the technical problems that the service volume of a service interface is mild and/or sudden fluctuation cannot be found or predicted in time in the prior art.

In a first aspect, the present application provides a data processing method, including:

acquiring training data and a first model to be trained;

respectively training a first model to be trained according to the plurality of characteristic parameter groups and training data to determine a plurality of models to be selected, wherein each model to be selected corresponds to the characteristic parameter group;

judging whether at least one model to be selected meets the preset requirement or not according to the training data;

if not, acquiring adjustment training data, and carrying out dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement;

if so, determining at least one target processing model from the multiple models to be selected according to the preset use requirement, and processing the acquired data to be processed by using the target processing model to determine a target processing result.

In a second aspect, the present application provides a data processing apparatus comprising:

the acquisition module is used for acquiring training data and a first model to be trained;

a processing module to:

In a third aspect, the present application provides an electronic device comprising:

a memory for storing program instructions;

and the processor is used for calling and executing the program instructions in the memory to execute any one of the possible item storage information determination methods provided by the first aspect.

In a fourth aspect, the present application provides a storage medium, in which a computer program is stored, the computer program being configured to execute any one of the possible data processing methods provided in the first aspect.

In a fifth aspect, the present application further provides a computer program product comprising a computer program, which when executed by a processor, implements any one of the possible data processing methods provided in the first aspect.

The application provides a data processing method, a device, equipment, a medium and a program product, which are characterized in that training data and a first model to be trained are obtained; then, training the first model to be trained respectively according to the plurality of characteristic parameter groups and training data to determine a plurality of models to be selected, wherein each model to be selected corresponds to the characteristic parameter group; judging whether at least one model to be selected meets the preset requirement or not according to the training data; if not, acquiring adjustment training data, and carrying out dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement; if so, determining at least one target processing model from the multiple models to be selected according to the preset use requirement, and processing the acquired data to be processed by using the target processing model to determine a target processing result. Before traffic is predicted each time, adaptive training adjustment is carried out according to a plurality of actual characteristic parameter sets and training data so as to select the most appropriate model from a plurality of models for prediction, the trained model is the most practical requirement, and the technical problems that the traffic of a service interface is mild and/or sudden fluctuation cannot be found or predicted in time in the prior art are solved. The method achieves the technical effects of accurately predicting the traffic through multi-dimensional analysis, simultaneously keeping enough sensitivity to sudden changes and mild changes of the traffic and improving the robustness of the traffic management system.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic view of an application scenario of a data processing method according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a data processing method provided in the present application;

FIG. 3 is a diagram illustrating the detailed steps of one implementation of S203 in the embodiment shown in FIG. 2;

FIG. 4 is a schematic flow chart diagram of another data processing method provided in the practice of the present application;

fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device provided in the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, including but not limited to combinations of embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any inventive step are within the scope of the present application.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The following explanations are made for terms to which this application refers:

a neural network model: the system consists of an input layer, a hidden layer and an output layer, wherein each layer comprises a plurality of neuron nodes, and the neuron nodes between the layers adopt a full interconnection mode. By simulating the reflection process of human neurons on external signals, a large number of input/output mapping relations can be learned and stored, after the neural network model is subjected to repeated iterative learning training, the weights among layers and the number of neuron nodes are adjusted, and finally the neural network model with certain preset performance requirements is established.

Time series analysis model: for example, the ARIMA model (differential Integrated Moving Average Autoregressive model) is a method and theory for mathematical modeling by fitting curves and parameter estimation based on time series data, and its essential feature is the dependency between adjacent data, which is essentially to predict the value of a future time series based on the time series of known intervals.

The combined prediction model refers to a new prediction model obtained by regarding each single model as an information set representing different segments and re-modeling the information sets through a proper combination algorithm, and compared with the single prediction model, the combined prediction model has higher accuracy, better stability and stronger adaptability to the environment.

The inventor of the present application finds that, for the existing service management system, the service volume prediction model can only generate a sufficiently rapid response to the drastic fluctuation, but in the practical situation, the drastic fluctuation in a short time and the entanglement of the temperature and the fluctuation coexist, only the abrupt change of the service volume is concerned, and the accumulation of the moderate change is ignored, so that the long-term stability of the system is affected, or the maximum resource supply must be kept all the time, which causes resource waste, that is, the existing technology has the technical problems that the service volume of the service interface continuously fluctuates in a small range within a period of time, but the accumulated influence exceeds the capability range of the service volume prediction model, and/or the abrupt fluctuation cannot be timely found or predicted, and the two cannot be considered at the same time.

In order to solve the above problems, the inventive concept of the present application is:

and retraining the traffic prediction model before performing traffic prediction each time. And the characteristic parameters of each model which is manually adjusted and used for training are changed into the allowable adjustment range of each characteristic parameter, namely all values in the value range are automatically combined to form a plurality of characteristic parameter groups, each characteristic parameter group is trained once to obtain a plurality of models to be selected, then the models to be selected are tested, the data for testing can be recent data or historical data in the same period, then the errors between the prediction result and the real result of each model to be selected are compared, at least one model to be selected is adjusted and trained according to the errors to further reduce the errors, and then at least one model to be selected with the errors in the preset range is selected as a final target model according to a preset screening rule. Therefore, the service volume prediction model applied in each prediction can be adaptively adjusted according to different dimensions, for example, the service volume prediction models adopted in different seasons, different months, different regions and the like can be changed, so that the supervision of the service volume interface of the whole service management system is not a fixed mode any more, but is intelligently adjusted according to multiple dimensions, the robustness of the system is better, the timely adjustment and response to sudden changes of the service volume are realized, and the use experience of a user is improved.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 1 is a schematic view of an application scenario of a data processing method according to an embodiment of the present application. As shown in fig. 1, a plurality of users 10 place service orders through applications on various types of user terminals 11, the user terminals 11 transmit the service orders to an interface 121 of an order service management system 12, and then the interface 121 allocates order queues to a server 122 for order processing. The existing order management system has a single monitoring dimension for the change of the traffic volume of the interface 121, and cannot analyze and predict in multiple dimensions, so that a large error exists between the order service prediction of the interface 121 and the actual situation, and operation and maintenance personnel of the order service management system cannot adjust the resource allocation of the system in time, so that the problems that the order processing is not timely enough, a data transmission channel is blocked, and finally the system is paralyzed or crashed are caused. The data processing method provided by the application can perform multi-dimensional and variable model dynamic prediction aiming at the interface 121 so as to solve the problems.

The following describes detailed steps of the data processing method described in the present application, which is applied in a scenario of order quantity prediction of order business.

Fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application. As shown in fig. 2, the specific steps of the data processing method include:

s201, obtaining training data and a first model to be trained.

In this step, at a preset time point, the method includes: before the current data is predicted, or after the previous operation period, such as the previous day, or a certain day in a week, a month or a quarter is finished, executing an obtaining instruction, and obtaining historical business data of the business management system in a preset time period as training data of a business volume prediction model. For example, historical business data, such as the historical number of orders, N days before the current date is obtained, N being greater than or equal to 1.

In one possible design, after obtaining the historical service data, the historical service data is also required to be preprocessed, including: classification processing and deburring processing.

The classification processing comprises multi-dimensional division of the historical service data according to different time granularities, regions and user group types. Such as: and respectively summarizing the historical order number according to the hour granularity, the day granularity, the week granularity, the month granularity and the season granularity. Meanwhile, for data with the same attribute label, multidimensional division can be further performed, for example: the historical order quantities are also summarized for data of the same label (such as shirt and trousers) according to hour granularity, day granularity, week granularity, month granularity and season granularity.

The despun treatment comprises the following steps: and if the difference value between the data of the current sample point and the previous sample point and the difference value between the data of the current sample point and the next sample point are larger than a preset threshold value, determining replacement data according to the data of the previous sample point and the data of the next sample point so as to replace the data of the current sample point.

Optionally, an average value of data of a previous sample point and a next sample point is taken as data of the current sample point, as shown in formula (1):

wherein x is_nFor data of the current sample point, x_n-1For data of the previous sample point, x_n+1Data of the latter sample point.

The first model to be trained may be an original traffic prediction model in the traffic management system, or may be a newly-built traffic prediction model. Namely, the data processing method provided by the application can be used as a patch program or an upgrade program to upgrade an original service management system, and can also be used as a traffic detection module of a newly-built service management system.

S202, training the first model to be trained respectively according to the plurality of characteristic parameter sets and the training data to determine a plurality of models to be selected.

In this step, the feature parameter group includes at least one feature parameter, the feature parameter corresponds to the first model to be trained, and types and/or values of the feature parameters in different feature parameter groups are different.

Each model to be selected corresponds to a characteristic parameter group, and the model to be trained comprises a time series analysis model, such as: AR model (Autoregressive model: abbreviated as AR)_(p)) And MA model (moving average model: abbreviated as MA_(q)) ARMA Model (Auto-Regressive Moving Average Model, abbreviated as ARMA)_(p,q,)) ARIMA model (abbreviated as ARIMA)_(p,q,d))。

It should be noted that:

(1) autoregressive model AR_(P)：

Defining the presence of a noise sequence ε_tAnd the real number sequence a₁，a₂，...，a_q(a_qNot equal to 0) so that the values of polynomial a (z) are all outside the unit circle as shown in equation (2):

referred to as p-order differential equations. The formula (3) is called a p-order autoregressive model AR for short_(P)And (4) modeling.

Wherein the coefficient a ═ a₁，a₂，...，a_t) Referred to as AR_(P)Auto-regressive coefficient of model, { X_iThe set is called a stationary solution.

(2) Moving average model MA_(q)：

Defining the presence of a noise sequence ε_tAnd the real number sequence b₁，b₂，...，b_q(b_qNot equal to 0) such that the polynomial is as shown in equation (4):

the formula (5) is called q-order sliding model, MA for short_(q),

When the solution for polynomial a (z) is not in a unit circle, then equation (6) is said to be a reversible model,

wherein A (z) ≠ 0 and | z | < 1.

For reversible MA_(q)Equation (7) can be obtained

Where B is called the push-back operator.

(3) Autoregressive moving average ARMA_(p，q)Model:

defining a white noise sequence [ epsilon ]_tThere is no identical root for the real coefficient polynomials φ (z) and θ (z), and b is satisfied₀＝1，a_qb_qNot equal to 0 and satisfies formula (8) and formula (9)

This is called a difference equation, as shown in equation (10):

is an autoregressive moving average model called time series { X }_tIs ARMA_(p，q)。

(4) Difference integration moving average autoregressive model:

ARMA_(p，q，d)is ARMA_(p，q)Can be expressed as formula (11):

where L is a hysteresis operator.

The characteristic parameters of each time series analysis model include: the three main characteristic parameters of p, q and d and other sub-characteristic parameters corresponding to the main characteristic parameters.

In this embodiment, the method is different from a training mode for a time series analysis model in the prior art, that is, the existing time series analysis model mostly implements modeling training through analysis software such as pass, and since the analysis software generally embeds a data packet of a time series analysis algorithm, three parameters, p, q, and d, are mainly determined by artificial dependent analysis software according to a sample set in the process of determining the model, and values of p and q depend on an autocorrelation coefficient and a partial autocorrelation coefficient in the sample set of training data. That is, in the prior art, after determining the value of at least one of p, q, and d of a model to be trained through analysis software, a user needs to manually set corresponding sub-feature parameters therein to establish a corresponding time series model.

However, the artificially determined values of the characteristic parameters have certain subjectivity, and the model cannot be dynamically and automatically established according to the sample set. As the sample data changes, the old predictive model may no longer be suitable for the new environment, and the sample data needs to be re-collected to re-establish the predictive model for the new data. And each time the user needs to manually set, a large amount of operation and maintenance work exists, the operation and maintenance cost is increased, and even the model updating cannot be implemented sometimes.

In order to overcome the defects of the time series analysis algorithm, the time series analysis algorithm for dynamic modeling training in the threshold range is provided, so that insufficient subjective experience or subjective blindness during characteristic parameter selection is eliminated, more abundant models to be selected are obtained, and more abundant and flexible model materials are provided for subsequent combination and dynamic adjustment training.

Specifically, the inventor of the present application finds that a time series analysis model is very superior for representing periodic characteristic values in a training set, i.e., training data, because parameters p and q in the model have a certain range and the order range of a difference d is certain, the ranges of p, q and d are preset, and a prediction model, i.e., a candidate model, is respectively established according to each of p, q and d. That is, within the respective value ranges of p, q, and d, after all the values of the characteristic parameters are arranged and combined according to a preset value mode, the obtained total number of the arrangement and combination is the total number of the characteristic parameter group. If the value ranges of p and q are (0, 7), and the value range of d is [0,1], if the value mode is that the minimum granularity of p and q is 1, and the minimum granularity of p and q is 0.1, the total number of the feature parameter sets is 490, and correspondingly, the number of the trained candidate models is also 490.

Preferably, the candidate model established according to each p, q, d is autoregressive moving average (ARMA)_(p，q)And (4) modeling.

S203, judging whether at least one model to be selected meets the preset requirement or not according to the training data.

In this step, if no, step S204 is executed, and if yes, step S205 is executed.

Fig. 3 is a schematic diagram illustrating a specific step of one implementation of S203 in the embodiment shown in fig. 2. As shown in fig. 3, in this embodiment, the specific steps include:

s2031, determining a processing error corresponding to each model to be selected according to preset detection data in the training data by using each model to be selected respectively.

In the step, each model to be selected is utilized to process the historical service in the training data within a preset time period so as to determine a first processing result corresponding to each model to be selected; and comparing each first processing result with the corresponding real result of the historical service to determine the processing error.

Specifically, each model to be selected is used for predicting data of consecutive T days, and the optional T is less than or equal to 4, and a difference value between a predicted value of the number of orders of each model to be selected for the consecutive T days and an actual value of the number of orders is counted, namely a processing error.

S2032, determining at least one model to be adjusted according to the processing error and a preset screening mode.

In the step, all models to be selected are sorted according to the sequence of the processing errors from small to large so as to determine a sorting result; and determining at least one model to be adjusted according to the sequencing result and a preset screening mode.

Optionally, the preset screening method includes: and extracting each model to be selected arranged at a preset position.

In this embodiment, the model to be selected corresponding to the minimum processing error is determined as the model to be adjusted, that is, the model to be selected ranked first is used as the model to be adjusted.

In one possible design, before performing this step, the method further includes:

preprocessing the processing error, comprising: and calculating the average value of the processing errors of each model to be selected in a preset time period, taking the average value as the representative value of the processing errors, averaging the error values of each day if the model to be selected predicts the order quantity of the continuous T days, and taking the obtained average value as the processing errors of the model to be selected.

For example, the average of the processing errors of each candidate model is calculated and formed into a data set { ave }₁₁，ave₂₂，…ave_ij}。

S2033, if the first processing error corresponding to the model to be adjusted meets the preset error requirement, determining that the model to be selected meets the preset requirement.

In this step, the preset error requirement includes: the first processing error is within a preset error range.

Specifically, a maximum error threshold M is preset, that is, a preset error range is [ -M, M]Or [0, M]Or [ -M,0 [ ]]. Then from the data set { ave }₁₁，ave₂₂，....ave_ijSelecting a model to be selected corresponding to the processing error falling within the preset error range, taking the model to be selected as a model to be combined, and entering the step S205 to participate in the combination of the target models; and the model to be selected with the processing error out of the preset range is used as the model to be adjusted, and the step S204 is entered for dynamic adjustment training.

Optionally, in order to avoid the too long time for the dynamic adjustment training, the number requirement of the models to be combined may be set, and if the number of the models to be selected falling within the error range does not meet the number requirement of the models to be combined, the dynamic adjustment training is performed on the models to be selected outside the preset error range.

In the present embodiment, the data set { ave } is selected₁₁，ave₂₂，....ave_ijMinimum value min (ave) of_ij). If min (ave)_ij)<And M, the corresponding model to be selected is the model to be combined, and the dynamic adjustment training is not performed on the rest models to be selected. If min (ave)_ij)>And M, dynamically adjusting and training the corresponding model to be selected.

It should be noted that, in this step, a model most suitable for the current scene is selected from a plurality of candidate models, so that it is avoided that too many candidate models are combined finally to obtain a prediction model that is too generic and lacks pertinence, and a phenomenon that an error between a prediction result and a real result is too large is also avoided.

And S204, acquiring adjustment training data, and performing dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement.

In this step, the model to be selected, which is screened in S203 and needs to be dynamically adjusted and trained, is trained in a targeted manner, so as to reduce the processing error. In this process, the values of the characteristic parameters determined in S202 are unchanged, and dynamic adjustment training is performed by acquiring data different from that in S201.

It should be noted that, in this step, the number of the candidate models is not enough due to insufficient representativeness of the training data in S201, or insufficient data amount of the training data, so that the prediction model finally obtained at the time of combination is not enough for various situations, that is, the stability and robustness of the model are not enough. After dynamic adjustment training is adopted, the data volume of training data is enlarged, so that more models to be selected can be used for final combination, the final prediction model can be predicted accurately in time when the face of mild accumulation type change and sudden and abrupt type change is faced, operation and maintenance personnel can adjust resources in time, adverse effects caused by the changes can be coped with, and the stability of the system is improved.

Optionally, the time period corresponding to the training data in S201 is lengthened, or more historical synchronization data is selected, or new historical data of an adjacent time period is introduced to supplement the training data.

And training the model to be selected again according to the newly acquired adjustment training data to obtain a new model to be selected, and repeating S203 to carry out detection on the processing error. If the processing error falls within the preset error range after the dynamic adjustment training, no training is performed, and the process proceeds to step S205; if the processing error is still outside the preset error range after the dynamic adjustment training, S204 is executed again to perform the dynamic adjustment training.

In one possible design, in order to avoid the situation that the training time is too long or the situation is trapped in the dead loop, the maximum training time N can be set, and each model to be selected is dynamically adjusted and trained to be put into the data set { t }₁，t₂，....t_jManagement is performed. And executing S203-S204 circularly as long as the training times are less than or equal to N.

Before each dynamic adjustment training, judging whether the dynamic adjustment training of the model to be adjusted meets the requirement of preset times;

if so, acquiring adjustment training data to perform dynamic adjustment training on the model to be adjusted; re-determining the processing error of the model to be adjusted; judging whether the processing error meets the preset error requirement again, wherein in each dynamic adjustment training, the time period corresponding to the adjustment training data is different from the time period corresponding to the last dynamic adjustment training;

if not, no dynamic adjustment training is carried out on the model to be adjusted, and the model to be adjusted in the model to be selected is determined to meet the preset requirement.

In particular, when min (ave)_ij)>M and t_j<And N, increasing K training sets on the basis of the original training data to obtain the adjusted training data so as to perform modeling training again and generate a new model to be selected.

When min (ave)_ij)>M and t_jWhen N is equal, then select min (ave)_ij) The corresponding candidate model is taken as the model to be combined and proceeds to step S205.

The optimized time series analysis model can dynamically adjust model parameters according to the training data, characteristic parameters and the threshold range of the training control parameters (such as the maximum error threshold M and the maximum training times N), and the optimal processing model is dynamically selected according to different training data in each prediction processing of historical order data, so that the prediction efficiency is improved, and the prediction precision is obviously improved.

S205, according to preset use requirements, at least one target processing model is determined from the multiple models to be selected, and the obtained data to be processed is processed by using the target processing model to determine a target processing result.

In this embodiment, the preset use requirements include use requirements of a combined prediction model, a plurality of candidate models are obtained according to S203 to S204, the candidate models meeting processing errors serve as the candidate models, then a combination algorithm is used to add corresponding weight values to each candidate model, each candidate model has different characteristic response conditions to real operation data, some candidate models respond to sudden changes rapidly, some candidate models respond to mild changes rapidly but situations which may cause severe channel blockage, after the plurality of candidate models are combined, the obtained target model takes into account both sudden changes and mild changes of the data to be processed, the prediction accuracy of the system is improved, and operation and maintenance personnel can guarantee safe and stable operation of the system.

Specifically, the inventor of the present application finds that a single prediction model often extracts effective information from a certain angle and ignores global feature information, and the prediction model does not have robustness due to the situations of incomplete feature value extraction and the like. And a single prediction model is also constrained by conditions such as parameters of the model, and the like, so that the single prediction model is not adaptive to some sudden data, and the phenomena such as sudden change of prediction results and the like are easy to occur.

The inventor of the application discovers that some models are suitable for medium-short term prediction, some models can show excellent prediction capability when predicting long-term data, and some models are sensitive to mutation of the data by analyzing different single prediction models. The purpose of using the combined prediction model is to combine the single models, and adjust the weight of each single model in the whole combined model by utilizing the advantages of each single model, so as to realize result prediction. The commonly used combined prediction weight determination method generally comprises a subjective experience method, an arithmetic mean method, a weighted mean method, a multiple regression method and the like, each weight adjustment algorithm is suitable for different scenes, and the exponential variable weight combined prediction algorithm supports elastic expansion for different types of scenes and can better adapt to scenes such as data mutation and the like.

The combined prediction algorithms are comprehensively compared, and the exponential variable weight combined prediction algorithm is adopted in the embodiment to determine the weight of each prediction single item.

And processing the acquired data to be processed by using the target processing model to determine a target processing result. In this embodiment, an order quantity prediction of an order interface of an order management system is taken as an example to be described:

the interface level monitoring of the existing order management system basically depends on keywords output by a program in a log, whether the transaction is successful is judged by scanning the keywords, and the monitoring of important interfaces also depends on configured unified system level monitoring. The heartbeat detection and the activity of the interface are regularly carried out when the unified system level monitoring is used, even if the interface monitoring is configured, the sudden change phenomena such as sudden drop or sudden increase of the interface TPS (Transactions Per Second) can be easily monitored, the condition of mild change of the success rate of the interface is not very sensitive, and the availability detection and the identification of the link between the cross systems cannot be supported by the traditional system level monitoring.

The combined prediction model, i.e. the target model in this embodiment, is used for the prediction of the order quantity, aiming at solving the cross-system full link availability check when the order interface TPS is in a mildly changing scenario. The historical order data is selected as a training set, namely training data, the order quantity of adjacent time periods is predicted by learning modeling, the predicted value and the real value are compared, the difference between the predicted value and the real value is compared with the predicted order threshold value, if the difference exceeds the threshold value range, an alarm is triggered, so that abnormity can be found at the first time, and the problem of production accidents is reduced. The setting of the threshold value is dynamically adjusted according to the order placing quantity and the predicted value of the historical real order.

For example, order threshold T for predicted time n_n：

The prediction of the order threshold value depends on the difference set of the real quantity and the predicted quantity of the historical orders counted by the latitude in hours, and the order threshold value at the j moment of the ith day is defined as t_ijBy { t_ijRepresents historical order threshold data sets. And (3) according to the periodic rule characteristic value presented by the data set, selecting partial data of historical time i in the threshold data set as a training set, and predicting the order threshold value of the current time i by using a combined prediction model.

For predicting order quantity D at time n_n：

The prediction of the total number of orders at the current moment depends on the historical real order number counted by the dimension of hours, and the total number of orders at the nth moment at the mth day is defined as d_mnBy { d_mnRepresents a historical order data set counted in the hourly dimension, and is selected as d_mnAnd (4) using the partial order number at the historical time n in the data set as a training set to predict the total order number at the current time n.

Calculating the total quantity D of the predicted orders at the nth time_nAnd the total number of real orders at the nth timeP_nDifference of course_n＝|T_n-D_n| compare sub_nAnd T_nSize, if sub_n＞T_nAnd triggering an alarm, and checking whether the whole transaction link is abnormal by human intervention.

According to the logic, the order association interface can be continuously monitored, if the monitoring with finer granularity is needed to be adjusted, for example, the adjustment triggers the check every ten minutes, the training set is adjusted, the subsequent function can be expanded and applied to the monitoring of other interfaces, and the monitoring can be realized only by replacing the training set with other interface data needing to be monitored.

The embodiment of the application provides a data processing method, which comprises the steps of obtaining training data and a first model to be trained; then, training the first model to be trained respectively according to the plurality of characteristic parameter groups and training data to determine a plurality of models to be selected, wherein each model to be selected corresponds to the characteristic parameter group; judging whether at least one model to be selected meets the preset requirement or not according to the training data; if not, acquiring adjustment training data, and carrying out dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement; if so, determining at least one target processing model from the multiple models to be selected according to the preset use requirement, and processing the acquired data to be processed by using the target processing model to determine a target processing result. Before traffic is predicted each time, adaptive training adjustment is carried out to select the most appropriate model from a plurality of models for prediction, and the technical problems that the traffic of a service interface is mild and/or sudden fluctuation cannot be found or predicted in time in the prior art are solved. The method achieves the technical effects of accurately predicting the traffic through multi-dimensional analysis, simultaneously keeping enough sensitivity to sudden changes and mild changes of the traffic and improving the robustness of the traffic management system.

Fig. 4 is a schematic flow chart of another data processing method provided in the present application. As shown in fig. 4, the data processing method includes the specific steps of:

s401, training data and a first model to be trained are obtained.

For a detailed explanation of this step, reference may be made to the description in S201, which is not described herein again.

It should be noted that the application scenario of the embodiment is an application of the order quantity in the aspects of report analysis and business development and preparation.

S402, training the first model to be trained respectively according to the plurality of characteristic parameter groups and the training data to determine a plurality of models to be selected.

The specific implementation principle and noun explanation of this step refer to S202, which is not described herein again.

And S403, determining a processing error corresponding to each first model to be selected according to preset detection data in the training data by using each first model to be selected.

In the step, each model to be selected is utilized to process the historical service in the training data within a preset time period so as to determine a first processing result corresponding to each model to be selected;

and comparing each first processing result with the corresponding real result of the historical service to determine the processing error.

Specifically, each model to be selected is used for predicting data of consecutive T days, and the optional T is less than or equal to 4, and a difference value between a predicted value of the number of orders of each model to be selected for the consecutive T days and an actual value of the number of orders is counted, namely a processing error. Then, calculating an average value of the processing errors of each model to be selected in a preset time period, namely, taking the average value of the processing errors of the predicted value and the true value of each day, taking the average value as a representative value of the processing errors, for example, taking the average value of the error values of each day of the model to be selected as the processing errors of the model to be selected for the number of orders of the consecutive T days, and taking the obtained average value as the processing error of the model to be selected.

And S404, judging whether the processing error meets a preset error requirement.

In this step, the preset error requirement includes: the processing error is within a preset error range.

And when the processing error is within the preset error range, determining that the model to be selected meets the preset requirement, that is, the step S408 can be executed.

When the processing error is outside the preset error range, step S405 is performed.

Specifically, a maximum error threshold M is preset, and whether the processing error corresponding to each model to be selected is less than or equal to M is determined.

S405, judging whether the dynamic adjustment training of the model to be selected meets the requirement of the preset times.

In this step, the preset number of times requirement includes: the number of times of dynamic adjustment training is less than the maximum preset number of times.

If yes, circularly executing the steps S406-S407 and S403; if not, no dynamic adjustment training is performed on the model to be selected, and it is determined that the model to be selected can be used for final combination, and step S408 is performed.

In one possible design, the preset number of requirements includes: the number of times of dynamic adjustment training is greater than or equal to a first preset number of times and less than a second preset number of times. If the dynamic adjustment training is required to be performed for 2-3 times, when it is found that the model to be selected is not subjected to the dynamic adjustment training, and when the first adjustment training is performed, and S406 is executed, the increase amplitude of the time period corresponding to the acquired adjustment training data is different from the increase amplitudes of the 2 nd time and the third time. For example, the 1 st increase is the historical order data increased by 5 days, and the 2 nd and third increases are the historical order data increased by 10 days.

And S406, acquiring the adjustment training data.

In this step, the time period corresponding to the training data is lengthened, or more historical synchronization data are selected, or new historical data of adjacent time periods are introduced to supplement the training data.

And S407, performing dynamic adjustment training on the model to be selected according to the adjustment training data.

In the step, the adjusted training data is used for learning and training the model to be selected again, and a new model to be selected is established to replace the old model which does not meet the preset error requirement.

Re-executing S403 to re-determine the processing error of the model to be selected; and judging whether the processing error meets the preset error requirement again.

It should be noted that, in each dynamic adjustment training, the time period corresponding to the adjustment training data acquired in step S406 is different from the time period corresponding to the last dynamic adjustment training.

And S408, obtaining a second model to be trained.

In this step, the second model to be trained is of a different type than the first model to be trained.

Optionally, the first model to be trained is a time series analysis model, and the second model to be trained is a neural network model.

In this embodiment, the second model to be trained is specifically a Back Propagation (BP) neural network model.

And S409, training a second model to be trained according to the training data.

In the step, error function values corresponding to different nodes are monitored in real time; and determining the learning rate of the current node and the propagation direction of the training result according to the change condition of the first error and the second error, the learning rate of the previous node and a preset adjustment coefficient, wherein the first error is an error function value corresponding to the current node, and the second error is an error function value corresponding to the previous node.

Specifically, the BP neural network is composed of an input layer, a hidden layer and an output layer, each layer comprises a plurality of neurons, and the neurons between the layers are all interconnected. The difficult problem of using the BP network as a prediction model is how to reasonably determine the number of layers of the hidden layer and the number of nodes in each layer. The learning process of the BP network algorithm can be divided into a forward propagation process and a backward propagation process, wherein the forward propagation process is to calculate the actual output value of a single neuron according to known input information and the determined number of layers and nodes of the hidden layer. The back propagation process is mainly a process of calculating errors of actual values and output values in a layer-by-layer backward recursion mode and readjusting weights of all layers through a determined prediction model under the condition that the values of the output layers are not consistent with the actual values.

Taking three layers of BP networks as an example, assume that the number of nodes of an input layer, a hidden layer and an output layer is n, q and m respectively, and the total number of training inputs is p.

x_piRepresenting the ith input value in the p training set;

v_kirepresenting the weight from the ith node of the input layer to the kth node of the hidden layer;

ω_jkrepresenting the weights from the k-th node of the hidden layer to the j-th node of the output layer. The output of the kth node of the hidden layer is shown in equation (12):

the j node output value of the output layer is shown in formula (13):

the activation function f is shown in equation (14):

f(x)＝1/(e^(-x)+1) (14)

the global error function can be expressed as equation (15):

wherein E is_PError value, t, representing the p-th sample_pjThe ideal output value for the p-th sample. The weight adjustment formula is shown in formula (16) and formula (17):

the weight adjustment formula (16) of each neuron of the output layer is as follows:

eta is the learning rate, and the threshold range is generally set to be between 0.1 and 0.3.

The weight adjustment formula (17) of each neuron in the hidden layer is as follows:

because the error convergence speed of the common BP neural network algorithm is related to the learning rate, the phenomenon of insufficient search easily occurs when the learning rate is set to be too small, and the situations of increased iteration times of weight adjustment and model oscillation occur when the learning rate is set to be too large, so that the system uses different learning rates for different nodes when the BP neural network algorithm is used for reversely adjusting weights of all layers, namely, the learning rate is dynamically adjusted according to the model. In the process of reverse adjustment, a smaller learning rate eta is randomly selected, if the derivation result of two continuous neurons in the same layer is of the same sign, the university learning rate needs to be adjusted, then the search learning is continued according to the original direction, and otherwise, the learning rate needs to be decreased.

In one possible design, when the first error is smaller than the second error, whether the first variation trend is the same as the second variation trend is judged;

if so, increasing the learning rate of the current node according to the learning rate of the previous node and a preset adjustment coefficient, and keeping the propagation direction unchanged;

if not, reducing the learning rate of the current node according to the learning rate of the previous node and a preset adjustment coefficient, and keeping the propagation direction unchanged;

and when the first error is larger than the second error, reducing the learning rate of the current node according to the learning rate of the previous node and a preset adjusting coefficient, and reversing the propagation direction.

In particular, if the error function E [ omega (m)]＜E[ω(m-1)]When is coming into contact with

And

the sign of the derivation result is the same, η (m) ═ η (m-1) × 1.3 is adjusted, and the search direction at this time is consistent with the original direction.

When in use

And

when the signs of the derivation results are different, η (m) ═ η (m-1)/1.3 is adjusted, and the search direction is consistent with the original direction.

If the error function E [ omega (m) ] > E [ omega (m-1) ], eta (m) ═ eta (m-1)/1.3, and the search direction is adjusted to the original reverse direction.

In another possible design, when the first error is greater than the minimum adjacent error, the learning rate of the current node is reduced according to the learning rate of the previous node and a preset adjustment coefficient, and the propagation direction is kept unchanged;

when the first error is larger than or equal to the second error and the first error is smaller than or equal to the minimum adjacent error, reducing the learning rate of the current node according to the learning rate of the previous node and a preset adjusting coefficient, and keeping the propagation direction unchanged;

wherein the minimum adjacent error is determined by the second error and a preset error control coefficient.

In particular, if the error function E [ omega (m)]＞e_mE[ω(m-1)]Then, then

η (m) ═ η (m-1)/1.3, and in this case, only the learning rate is changed while the weight and the search direction are kept unchanged. (e)_mRepresenting a minimum error rate, i.e., a preset error control coefficient).

If the error function E [ omega (m-1)]≤E[ω(m)]≤e_mE[ω(m)]If η (m) is η (m-1), the learning rate and the search direction are kept unchanged, and the search is continued to adjust the weight. (e)_mRepresenting a minimum error rate, i.e., a preset error control coefficient).

Steps S408 to S409 and S401 to S407 may be performed in synchronization, and there is no requirement for the execution order.

S410, combining the model to be combined and the model to be selected by utilizing a preset combination algorithm to determine a target processing model.

The research finds that the dynamic adaptability of the artificial neural network (such as a BP neural network model) algorithm to a training set with non-structural change is superior to that of other algorithms, and compared with other algorithms, the artificial neural network has the problems of long training time, difficulty in determining the number of hidden layers and the number of nodes and the like, but the artificial neural network algorithm serving as a nonlinear prediction algorithm has the advantages of memory training set, dynamic optimization model, strong robustness, strong autonomous learning capability and the like.

The research on a time series analysis model (such as an ARIMA model) shows that the prediction model has an ideal prediction value for a training set with periodic characteristics, although the model has certain limitation, the time series analysis model has the advantages of lower complexity and smaller requirement on the data volume of a historical training set compared with other prediction models, and the method is generally suitable for medium-short term prediction with a more stable time series.

The commonly used method for determining the weight of the combined prediction model generally comprises a subjective experience method, an arithmetic mean method, a weighted mean method, a multiple regression method and the like, each algorithm for adjusting the weight is suitable for different scenes, and the exponential variable weight combined prediction algorithm supports elastic expansion for different types of scenes and can better adapt to scenes such as data mutation and the like. The combined prediction algorithms are comprehensively compared, and the system adopts the exponential variable weight combined prediction algorithm to determine the weight of each prediction single item.

S411, processing the acquired data to be processed by using the target processing model to determine a target processing result.

In the embodiment, as the business develops, planning and adjusting work must be completed in advance for the existing scheme to adapt to the development scenario of the future business, whether in terms of hardware or software, such as in terms of production machine resources.

The resources of the general early production, including the production management system, can support the normal development of the existing business. However, as the traffic increases, bottleneck problems may occur in the later stages of production resources, system performance, etc., so the production resources must be adjusted as early as possible to optimize the system performance to avoid production accidents.

For example, the supply chain end can inform the merchant of adjusting the goods staging work as soon as possible according to the development trend of future services, so as to avoid direct economic loss caused by sold-out or lost goods.

Many times, the development condition of future business is analyzed through some reports which can be displayed, such as the sales condition of a certain kind of commodities in a certain time period in the future; or the strategic target of the next stage is established according to the current business data, if the report is obtained by manually analyzing historical data, the final result can be deviated due to missing related characteristic values or data deviation with personal subjective consciousness, and a more ideal result can be obtained by using the combined prediction model.

Specifically, the implementation method in this step includes: predicting total number D of orders on the nth day_n

The prediction of the total number of orders in the nth day depends on the historical real total number of orders counted according to the dimension of the day, and the total number of orders in the mth day is defined as d_mBy { d_mRepresenting historical order data set counted by dimension of day, and selecting { d }_mAnd (4) taking the historical order number in the data set as a training set, namely training data, establishing a prediction model, namely a target model, according to the mode of S401-410, and completing the prediction of the total order number of the nth day after the model is established. If the sales condition of a certain commodity in a future time period needs to be predicted, only a data set of the certain commodity needs to be selected as a training set according to the label, the training set, namely training data, can be classified according to different dimensions such as day, week, month and quarter, and after classification, a prediction model, namely a target model, is established according to the mode of S401-410 for prediction.

It should be noted that, because the error convergence rate of the general BP neural network algorithm is related to the learning rate, when the learning rate is set too small, the search is likely to be insufficient, and if the learning rate is set too large, the number of times of iteration for adjusting the weight is increased, and the model oscillates, so that different learning rates are used for different nodes when the weight of each layer is reversely adjusted by using the BP neural network algorithm, that is, the learning rate is dynamically adjusted according to the model. The phenomenon that the fixed learning rate of a common BP neural network algorithm is easy to cause insufficient search or model oscillation is solved, and the training of the BP neural network is more efficient, stable and accurate.

Fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. The data processing device 500 may be implemented by software, hardware or a combination of both.

As shown in fig. 5, the data processing apparatus 500 includes:

an obtaining module 501, configured to obtain training data and a first model to be trained;

a processing module 502 for:

In one possible design, determining whether at least one candidate model meets a preset requirement according to training data includes:

determining a processing error corresponding to each model to be selected according to preset detection data in the training data by using each model to be selected;

determining at least one model to be adjusted according to the processing error and a preset screening mode;

and if the first processing error corresponding to the model to be adjusted meets the preset error requirement, determining that the model to be selected meets the preset requirement.

In one possible design, the processing module 502 is configured to:

processing historical services in a preset time period in training data by using each model to be selected respectively so as to determine a first processing result corresponding to each model to be selected;

In one possible design, the processing module 502 is configured to:

sequencing all the models to be selected according to the sequence of the processing errors from small to large so as to determine a sequencing result;

and determining at least one model to be adjusted according to the sequencing result and a preset screening mode.

In one possible design, when the candidate model does not meet the preset requirement, the processing module 502 is configured to:

judging whether the dynamic adjustment training of the model to be adjusted meets the requirement of preset times or not;

In one possible design, the preset screening method includes: and extracting each model to be selected arranged at a preset position.

Optionally, the preset error requirement includes: the first processing error is within a preset error range.

In one possible design, the processing module 502 is configured to:

and if the processing error meets the preset error requirement, determining that the model to be selected meets the preset requirement.

In one possible design, the processing module 502 is configured to:

comparing each first processing result with a real result corresponding to the historical service to determine a processing error;

and when the processing error is within the preset error range, determining that the model to be selected meets the preset requirement.

In one possible design, the processing module 502 is further configured to:

when the processing error is out of the preset error range, judging whether the dynamic adjustment training of the model to be selected meets the preset frequency requirement;

if so, acquiring adjustment training data to perform dynamic adjustment training on the model to be selected; re-determining the processing error of the model to be selected; judging whether the processing error meets the preset error requirement again, wherein in each dynamic adjustment training, the time period corresponding to the adjustment training data is different from the time period corresponding to the last dynamic adjustment training;

if not, no dynamic adjustment training is carried out on the model to be selected, and the model to be selected is determined to meet the preset requirement.

In one possible design, the preset number of requirements includes: the number of times of dynamic adjustment training is less than the maximum preset number of times.

In one possible design, the processing module 502 is configured to:

determining at least one target processing model according to the sequencing result and a preset use requirement, wherein the preset use requirement comprises the following steps: and extracting each model to be selected arranged at a preset position.

Optionally, the feature parameter group includes at least one feature parameter, the feature parameter corresponds to the first model to be trained, and types and/or values of the feature parameters in different feature parameter groups are different.

In one possible design, the obtaining module 501 is further configured to obtain a second model to be trained;

the processing module 502 is further configured to:

training a second model to be trained according to the training data to determine a model to be combined, wherein the type of the second model to be trained is different from that of the first model to be trained, and the second model to be trained is a neural network model;

combining the model to be combined and the model to be selected by using a preset combination algorithm to determine a target processing model;

in the training process of a second model to be trained of the training data, error function values corresponding to different nodes are monitored in real time; and determining the learning rate of the current node and the propagation direction of the training result according to the change condition of the first error and the second error, the learning rate of the previous node and a preset adjustment coefficient, wherein the first error is an error function value corresponding to the current node, and the second error is an error function value corresponding to the previous node.

In one possible design, the processing module 502 is further configured to:

when the first error is smaller than the second error, judging whether the first variation trend is the same as the second variation trend;

In one possible design, the processing module 502 is further configured to:

when the first error is larger than the minimum adjacent error, reducing the learning rate of the current node according to the learning rate of the last node and a preset adjusting coefficient, and keeping the propagation direction unchanged;

It should be noted that the apparatus provided in the embodiment shown in fig. 5 can execute the method provided in any of the above method embodiments, and the specific implementation principle, technical features, term explanation and technical effects thereof are similar and will not be described herein again.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 6, the electronic device 600 may include: at least one processor 601 and memory 602. Fig. 6 shows an electronic device as an example of a processor.

A memory 602 for storing programs. In particular, the program may include program code including computer operating instructions.

The memory 602 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The processor 601 is configured to execute computer-executable instructions stored in the memory 602 to implement the methods described in the above method embodiments.

The processor 601 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

Alternatively, the memory 602 may be separate or integrated with the processor 601. When the memory 602 is a device independent from the processor 601, the electronic device 600 may further include:

a bus 603 for connecting the processor 601 and the memory 602. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. Buses may be classified as address buses, data buses, control buses, etc., but do not represent only one bus or type of bus.

Alternatively, in a specific implementation, if the memory 602 and the processor 601 are integrated into a single chip, the memory 602 and the processor 601 may communicate via an internal interface.

An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium may include: various media that can store program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and in particular, the computer-readable storage medium stores program instructions for the methods in the above method embodiments.

An embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the method in the foregoing method embodiments.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A data processing method, comprising:

acquiring training data and a first model to be trained;

respectively training the first model to be trained according to a plurality of characteristic parameter groups and the training data to determine a plurality of models to be selected, wherein each model to be selected corresponds to the characteristic parameter group;

judging whether at least one model to be selected meets a preset requirement or not according to the training data;

if not, acquiring adjustment training data, and performing dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement;

if so, determining at least one target processing model from the multiple to-be-selected models according to a preset use requirement, and processing the acquired to-be-processed data by using the target processing model to determine a target processing result.

2. The data processing method according to claim 1, wherein the determining whether at least one of the candidate models meets a preset requirement according to the training data comprises:

determining a processing error corresponding to each model to be selected according to preset detection data in the training data by using each model to be selected respectively;

3. The data processing method according to claim 2, wherein the determining, by using each candidate model and according to preset detection data in the training data, a processing error corresponding to each candidate model comprises:

processing historical services in a preset time period in the training data by using each model to be selected respectively so as to determine a first processing result corresponding to each model to be selected;

and comparing each first processing result with the real result corresponding to the historical service to determine the processing error.

4. The data processing method according to claim 2, wherein the determining at least one model to be adjusted according to the processing error and a preset screening manner comprises:

5. The data processing method according to claim 2, wherein if not, acquiring adjustment training data, and performing dynamic adjustment training on the model to be selected until the model to be selected meets the preset requirement, including:

judging whether the dynamic adjustment training of the model to be adjusted meets the requirement of preset times;

if so, acquiring the adjustment training data to perform the dynamic adjustment training on the model to be adjusted; re-determining the processing error of the model to be adjusted; judging whether the processing error meets the preset error requirement again, wherein in each dynamic adjustment training, the time period corresponding to the adjustment training data is different from the time period corresponding to the last dynamic adjustment training;

and if not, the dynamic adjustment training is not carried out on the model to be adjusted, and the model to be adjusted in the model to be selected is determined to meet the preset requirement.

6. The data processing method according to claim 1, wherein the determining whether at least one of the candidate models meets a preset requirement according to the training data comprises:

and if the processing error meets a preset error requirement, determining that the model to be selected meets the preset requirement.

7. The data processing method according to claim 6, wherein the determining, by using each candidate model and according to preset detection data in the training data, a processing error corresponding to each candidate model comprises:

comparing each first processing result with a real result corresponding to the historical service to determine the processing error;

if the processing error meets a preset error requirement, determining that the model to be selected meets the preset requirement comprises:

and when the processing error is within a preset error range, determining that the model to be selected meets the preset requirement.

8. The data processing method according to claim 7, wherein if not, adjustment training data is obtained, and dynamic adjustment training is performed on the model to be selected until the model to be selected meets the preset requirement:

when the processing error is out of the preset error range, judging whether the dynamic adjustment training of the model to be selected meets the requirement of preset times;

if so, acquiring the adjustment training data to perform the dynamic adjustment training on the model to be selected; re-determining the processing error of the model to be selected; judging whether the processing error meets a preset error requirement again, wherein in each dynamic adjustment training, the time period corresponding to the adjustment training data is different from the time period corresponding to the last dynamic adjustment training;

and if not, the dynamic adjustment training is not carried out on the model to be selected, and the model to be selected is determined to meet the preset requirement.

9. The data processing method according to any one of claims 1 to 8, wherein the set of feature parameters includes at least one feature parameter, the feature parameter corresponds to the first model to be trained, and types and/or values of the feature parameter are different in different sets of feature parameters.

10. The data processing method according to claim 1, wherein the determining at least one target processing model from the plurality of candidate models according to a preset usage requirement comprises:

obtaining a second model to be trained, and training the second model to be trained according to the training data to determine a model to be combined, wherein the type of the second model to be trained is different from that of the first model to be trained, and the second model to be trained is a neural network model;

combining the model to be combined and the model to be selected by using a preset combination algorithm to determine the target processing model;

in the training process of the training data and the second model to be trained, error function values corresponding to different nodes are monitored in real time; determining the learning rate of the current node and the propagation direction of a training result according to the change condition of a first error and a second error, the learning rate of the previous node and a preset adjustment coefficient, and continuing training the second model to be trained according to the learning rate and the propagation direction, wherein the first error is an error function value corresponding to the current node, and the second error is an error function value corresponding to the previous node.

11. The data processing method of claim 10, wherein determining the learning rate of the current node and the propagation direction of the training result according to the variation of the first error and the second error, the learning rate of the previous node, and a preset adjustment coefficient comprises:

if so, increasing the learning rate of the current node according to the learning rate of the previous node and the preset adjustment coefficient, and keeping the propagation direction unchanged;

if not, reducing the learning rate of the current node according to the learning rate of the previous node and the preset adjustment coefficient, and keeping the propagation direction unchanged;

and when the first error is larger than the second error, reducing the learning rate of the current node according to the learning rate of the last node and the preset adjusting coefficient, and reversing the propagation direction.

12. The data processing method of claim 10, wherein determining the learning rate of the current node and the propagation direction of the training result according to the variation of the first error and the second error, the learning rate of the previous node, and a preset adjustment coefficient comprises:

when the first error is larger than the minimum adjacent error, reducing the learning rate of the current node according to the learning rate of the last node and the preset adjusting coefficient, and keeping the propagation direction unchanged;

when the first error is greater than or equal to the second error and the first error is less than or equal to the minimum adjacent error, reducing the learning rate of the current node according to the learning rate of the previous node and the preset adjustment coefficient, and keeping the propagation direction unchanged;

13. A data processing apparatus, comprising:

a processing module to:

14. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the data processing method of any one of claims 1 to 12.

15. A computer-readable storage medium, having stored thereon computer-executable instructions for implementing a data processing method according to any one of claims 1 to 12 when executed by a processor.

16. A computer program product comprising a computer program which, when executed by a processor, implements the data processing method of any one of claims 1 to 12.