CN111382874B - Method and device for realizing update iteration of online machine learning model - Google Patents

Method and device for realizing update iteration of online machine learning model Download PDF

Info

Publication number
CN111382874B
CN111382874B CN201811619035.8A CN201811619035A CN111382874B CN 111382874 B CN111382874 B CN 111382874B CN 201811619035 A CN201811619035 A CN 201811619035A CN 111382874 B CN111382874 B CN 111382874B
Authority
CN
China
Prior art keywords
machine learning
learning model
model
service
samples
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811619035.8A
Other languages
Chinese (zh)
Other versions
CN111382874A (en
Inventor
高晓伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
4Paradigm Beijing Technology Co Ltd
Original Assignee
4Paradigm Beijing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 4Paradigm Beijing Technology Co Ltd filed Critical 4Paradigm Beijing Technology Co Ltd
Priority to CN201811619035.8A priority Critical patent/CN111382874B/en
Publication of CN111382874A publication Critical patent/CN111382874A/en
Application granted granted Critical
Publication of CN111382874B publication Critical patent/CN111382874B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Disclosed are a method and apparatus for implementing update iterations of an online machine learning model, the method comprising: predicting the business sample by a model group consisting of at least one machine learning model working in series; one or more machine learning models of the model group including the machine learning model arranged at the forefront are set as follows: allowing the service samples with the predicted value larger than the reference threshold of the model to pass, rejecting the service samples with the predicted value smaller than the lower detection threshold of the model and smaller than the reference threshold, and allowing the service samples with the predicted value smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass with the probability corresponding to the model; acquiring service feedback information of service samples allowed to pass through by the model group, and forming training samples according to the service feedback information and the corresponding service samples; training a new machine learning model based on training samples generated within a predetermined time after the last machine learning model is put on line, and concatenating the trained new machine learning model into a model group.

Description

Method and device for realizing update iteration of online machine learning model
Technical Field
The present disclosure relates generally to the field of Artificial Intelligence (AI), and more particularly, to a method and apparatus for implementing update iterations of an online machine learning model.
Background
In the field of machine learning, machine learning models are often trained by providing empirical data to machine learning algorithms to determine the ideal parameters that make up the machine learning model, and the trained machine learning models can be applied in a variety of business fields to provide corresponding predictions in the face of new business samples. The construction and application of machine learning models is not a matter of kick, and over time, models need to be iterated and updated with new data. On one hand, more abundant data can be accumulated along with the time, and newer data can reflect the latest trend and service change; on the other hand, with the development of the service, new data sources are possible for better depicting service samples by the model, which are the internal reasons for triggering the model upgrade iteration.
In the scenario of making business judgment by using a machine learning model, newly accumulated data is composed of a sample after the model judgment is allowed to pass and performance information of the sample in actual business. Therefore, if the new model trained by the newly accumulated data is directly used for replacing the old model, the accuracy of the service judgment is reduced due to the fact that the new model does not see the samples which are "bad" (the samples which are refused to pass through by the old model judgment), and the risk is brought to the actual service.
Disclosure of Invention
Exemplary embodiments of the present disclosure provide a method of implementing update iterations of an online machine learning model, the method comprising: predicting a business sample from a model set of at least one serially-operated machine learning model, wherein one or more machine learning models of the model set comprising the foremost machine learning model are arranged to operate in the following manner: allowing service samples with predicted values larger than a reference threshold corresponding to the model to pass, rejecting service samples with predicted values smaller than a lower detection threshold corresponding to the model and smaller than the reference threshold, allowing service samples with predicted values smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass with probability corresponding to the model, and setting other machine learning models in the model set to work in the following modes: allowing service samples with predicted values larger than or equal to a reference threshold corresponding to the model to pass through, and rejecting service samples with predicted values smaller than the reference threshold; acquiring service feedback information of service samples allowed to pass through by the model group, and forming training samples according to the service feedback information and the corresponding service samples; training a new machine learning model based on training samples generated in a predetermined time after a last machine learning model is put on line in a current model group, and concatenating the trained new machine learning model to the last machine learning model to put the new machine learning model on line; respectively reducing a reference threshold and a scout threshold for each of the one or more machine learning models, wherein the reduced scout threshold is less than the reduced reference threshold; when the reference threshold value of one machine learning model in the model group is changed to be smaller than or equal to a preset early warning threshold value, the machine learning model is disconnected.
According to an exemplary embodiment of the present disclosure, the training the new machine learning model based on training samples generated in a predetermined time after the last machine learning model is on-line in the current model group may include: obtaining training samples according to the principle that the distribution of the training samples is consistent with the distribution of service samples arranged after the last machine learning model is on line in the current model group; the new machine learning model is trained based on the acquired training samples.
According to an exemplary embodiment of the present disclosure, when the one or more machine learning models include only the machine learning model arranged at the front, the training samples acquired following the principle that the distribution of the training samples is consistent with the distribution of the service samples arranged after the last machine learning model is on line in the current model group may satisfy the following formula:
wherein M (N+1) train Training samples for training the new machine learning model; d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group; n is a positive integer, representing the number of machine learning models in the current model set; i represents an i-th machine learning model in the current model group in the order from front to back; p is p 1 Is a probability corresponding to the foremost machine learning model; comp (Comp) i Representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample; prob 1 Representing the foremost rowThe machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and uses a probability p from service samples input to the forefront machine learning model, wherein the predicted value of the service samples is smaller than or equal to the corresponding reference threshold value and larger than or equal to the corresponding lower detection threshold value when the service samples are processed through the forefront machine learning model 1 And forming a training sample by the passed service sample according to the acquired service feedback information and the corresponding service sample.
According to an exemplary embodiment of the present disclosure, the training samples obtained following a principle that a distribution of training samples is consistent with a distribution of service samples arranged after a last machine learning model is on line in a current model group may satisfy the following formula:
wherein M (N+1) train Training samples for training the new machine learning model; d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group; n is a positive integer, representing the number of machine learning models in the current model set; i represents an i-th machine learning model in the current model group in the order from front to back; p is p i Is a probability corresponding to the i-th machine learning model; comp (Comp) i Representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample; prob i Representing that the ith machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and with probability p from service samples input to the ith machine learning model, the predicted value of the ith machine learning model being processed by the ith machine learning model i The passing service sample and the corresponding service sample form a training sample according to the acquired service feedback information; if the ith machine learning model has only the reference threshold, then Prob will be i Set to 0 and set p i Set to 1.
According to an example embodiment of the present disclosure, the step of respectively lowering the reference threshold and the scout threshold of each of the one or more machine learning models may include: after the new machine learning model is brought online, for each machine learning model of the one or more machine learning models, a baseline threshold value of the machine learning model is lowered by a predetermined value, and a scout threshold value of the machine learning model is lowered by a predetermined value.
According to an example embodiment of the present disclosure, the step of respectively lowering the reference threshold and the scout threshold of each of the one or more machine learning models may include: after the new machine learning model is online, for each machine learning model of the one or more machine learning models, setting a reference threshold for the machine learning model equal to a scout threshold for the machine learning model when the new machine learning model is online, and reducing the scout threshold for the machine learning model by a predetermined value.
According to an exemplary embodiment of the present disclosure, the method for implementing update iteration of an online machine learning model may further include: training an initialized machine learning model based on a preset initialized training sample, and selecting the initialized machine learning model as a machine learning model arranged at the front.
An exemplary embodiment of the present disclosure provides an apparatus for implementing update iterations of an online machine learning model, the apparatus comprising: a service unit for configuring a model group consisting of at least one machine learning model working in series, predicting a business sample via the model group, and configuring one or more machine learning models including the machine learning model arranged at the front in the model group to work in the following manner: allowing service samples with predicted values larger than a reference threshold corresponding to the model to pass, rejecting service samples with predicted values smaller than a lower detection threshold corresponding to the model and smaller than the reference threshold, allowing service samples with predicted values smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass with probability corresponding to the model, and setting other machine learning models in the model set to work in the following modes: allowing service samples with predicted values larger than or equal to a reference threshold corresponding to the model to pass through, and rejecting service samples with predicted values smaller than the reference threshold; the training sample generation unit is used for acquiring service feedback information of the service samples allowed to pass through by the model group and forming training samples according to the service feedback information and the corresponding service samples; and a model training unit for training a new machine learning model based on training samples generated in a predetermined time after the last machine learning model is put on line in the current model group, and concatenating the trained new machine learning model after the last machine learning model to put the new machine learning model on line; the service unit is further configured to reduce a reference threshold and a lower detection threshold of each machine learning model in the one or more machine learning models after the new machine learning model is online, where the reduced lower detection threshold is smaller than the reduced reference threshold; and the machine learning module is used for turning off the machine learning module when the reference threshold value of one machine learning module in the module group is changed to be smaller than or equal to the preset early warning threshold value.
According to an exemplary embodiment of the present disclosure, the training sample generation unit may be configured to acquire the training samples following a principle that a distribution of the training samples is consistent with a distribution of traffic samples in the current model group arranged after the last machine learning model is on line, and the model training unit may be configured to train the new machine learning model based on the training samples generated in the current model group arranged within a predetermined time after the last machine learning model is on line.
According to an exemplary embodiment of the present disclosure, wherein when the one or more machine learning models include only the machine learning model arranged at the front, the training sample constituted by the training sample generating unit may satisfy the following formula:
wherein M (N+1) train Training samples for training the new machine learning model; d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group; n is a positive integer, representing the number of machine learning models in the current model set; i represents an i-th machine learning model in the current model group in the order from front to back; p is p 1 Is a probability corresponding to the foremost machine learning model; comp (Comp) i Representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample; prob 1 Representing that the forefront machine learning model selects a service sample input to the forefront machine learning model, wherein the predicted value of the service sample is smaller than or equal to a corresponding reference threshold value and larger than or equal to a corresponding lower detection threshold value and has probability p when the service sample is processed by the forefront machine learning model 1 And forming a training sample by the passed service sample according to the acquired service feedback information and the corresponding service sample.
According to an exemplary embodiment of the present disclosure, wherein the training samples constituted by the training sample generating unit may satisfy the following formula:
wherein M (N+1) train Training samples for training the new machine learning model; d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group; n is a positive integer, representing the number of machine learning models in the current model set; i represents an i-th machine learning model in the current model group in the order from front to back; p is p i Is a probability corresponding to the i-th machine learning model; comp (Comp) i Representing input to the ith machine learning modelSelecting a service sample with a predicted value larger than a corresponding reference threshold value from the service samples processed by the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample; prob i Representing that the ith machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and with probability p from service samples input to the ith machine learning model, the predicted value of the ith machine learning model being processed by the ith machine learning model i The passing service sample and the corresponding service sample form a training sample according to the acquired service feedback information; if the ith machine learning model has only the reference threshold, then Prob will be i Set to 0 and set p i Set to 1.
According to an exemplary embodiment of the present disclosure, the service unit may be configured to, for each machine learning model of the one or more machine learning models, decrease a reference threshold of the machine learning model by a predetermined value and decrease a scout threshold of the machine learning model by a predetermined value after the new machine learning model is online.
According to an exemplary embodiment of the present disclosure, the service unit may be configured to set, for each machine learning model of the one or more machine learning models, a reference threshold value of the machine learning model to be equal to a scout threshold value of the machine learning model when the new machine learning model is on-line, and to decrease the scout threshold value of the machine learning model by a predetermined value after the new machine learning model is on-line.
According to an exemplary embodiment of the present disclosure, the apparatus for updating iteration of the online machine learning model may further include: an initialization unit trains an initialization machine learning model based on a predetermined initialization training sample, and selects the initialization machine learning model as a forefront machine learning model.
Exemplary embodiments of the present disclosure provide a computer readable storage medium having stored thereon computer instructions that, when executed by at least one computing device, cause the at least one computing device to perform the method of implementing the update iteration of an online machine learning model of any of the above.
Exemplary embodiments of the present disclosure provide a system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the method of implementing the update iteration of the online machine learning model of any one of the above.
According to the technical scheme, after the new model is on line, the old model is not on line immediately, the new model and the old model form a model group to work cooperatively, the old model gradually reduces the influence on the prediction result of the whole model group in a threshold value undershooting mode, and finally the old model is off line. According to the technical scheme, the risk brought by the independent online of the new model is avoided.
Further, in the technical scheme of the invention, the data distribution of the training sample for training the new model is kept consistent with the data distribution of the service sample to be processed after the model is applied online as much as possible, so that the application effect of the new model is improved.
Drawings
These and/or other aspects and advantages of the present disclosure will become more apparent and more readily appreciated from the following detailed description of the embodiments of the disclosure, taken in conjunction with the accompanying drawings, wherein:
FIG. 1 is a block diagram illustrating an apparatus 10 implementing update iterations of an online machine learning model in accordance with an exemplary embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating an iterative method of updating an online machine learning model with the apparatus 10 according to an exemplary embodiment of the present disclosure;
fig. 3-6 are schematic diagrams illustrating a method of operation of the apparatus 10 of fig. 1, according to an exemplary embodiment of the present disclosure;
FIG. 7 is a diagram illustrating a method of offline machine learning models in a model set by the apparatus 10 of FIG. 1 in accordance with an exemplary embodiment of the present disclosure;
fig. 8 is a schematic diagram illustrating a method of operation of the apparatus 10 of fig. 1 according to another exemplary embodiment of the present disclosure.
Detailed Description
In order that those skilled in the art will better understand the present disclosure, exemplary embodiments of the present disclosure are described in further detail below with reference to the accompanying drawings and detailed description.
Here, machine learning is an inevitable product of AI research developed to a certain stage, which is directed to improving the performance of the system itself by means of computation using experience. In computer systems, "experience" is usually present in the form of "data" from which "models" can be generated by means of machine learning algorithms, i.e. by providing experience data to the machine learning algorithm, a model can be generated based on these experience data, which model provides corresponding decisions, i.e. predictions, in the face of new situations. Whether the machine learning model is trained or predicted using the trained machine learning model, the data needs to be converted into machine learning samples that include various features. Machine learning may be implemented in the form of "supervised learning", "unsupervised learning", or "semi-supervised learning", it being noted that the exemplary embodiments of the present disclosure are not particularly limited to specific machine learning algorithms. In addition, it should be noted that other means such as statistical algorithms may be incorporated in the training and application of the model.
Fig. 1 is a block diagram illustrating an apparatus 10 implementing update iterations of an online machine learning model according to an exemplary embodiment of the present disclosure.
As shown in fig. 1, an apparatus 10 implementing an update iteration of an online machine learning model according to an exemplary embodiment of the present disclosure may include: a service unit 200, a training sample generation unit 300 and a model training unit 400.
Referring to fig. 1, a service unit 200 is configured to configure a model group consisting of at least one machine learning model working in series, and predict a service sample via the model group. The model group receives a service request comprising a service sample set and provides the service sample set which is allowed to pass through and a prediction result corresponding to the service sample set. Each machine learning model in the model set may give a prediction result for each sample in the set of business samples. For example, each machine learning model may give a predictive value corresponding to each sample in the set of business samples and determine from the predictive value whether the sample passes to provide a business sample that is allowed to pass. For ease of understanding, taking the risk assessment scenario as an example herein, a larger predicted value indicates a lower risk of the corresponding business sample, and conversely, a smaller predicted value indicates a greater risk of the corresponding business sample. In other embodiments, the predictive value "large" may also be used to indicate that the corresponding business sample risk "low". It should be noted that the "large", "small" illustrated herein are merely examples, and the present disclosure is not limited thereto. For convenience of explanation, only a case where the larger the predicted value is, the lower the risk of the corresponding service sample is indicated will be explained hereinafter. A service unit 200 for configuring one or more machine learning models of the model group including the foremost machine learning model to operate as follows: allowing service samples with predicted values larger than a reference threshold corresponding to the model to pass, rejecting service samples with predicted values smaller than a lower detection threshold corresponding to the model and smaller than the reference threshold, and allowing the service samples with predicted values smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass according to the probability corresponding to the model; and wherein the other machine learning models in the model set are arranged to operate as follows: and allowing the service samples with the predicted value larger than or equal to the reference threshold corresponding to the model to pass through, and rejecting the service samples with the predicted value smaller than the reference threshold.
The training sample generating unit 300 is configured to obtain service feedback information of service samples allowed to pass through by the model set, and form training samples according to the service feedback information and the corresponding service samples. The allowed service samples may be allowed service samples in the current model set after being arranged on line in the last machine learning model and judged by the model set.
The model training unit 400 is configured to train a new machine learning model based on training samples generated in a predetermined time after the last machine learning model is on line in the current model group, and concatenate the trained new machine learning model after the last machine learning model to line the new machine learning model. The model training unit 400 may train a new machine learning model based on the training samples constituted by the training sample generating unit 300 by a machine learning method. The method for training the machine learning model can comprise the steps of data stitching for performing model training, feature extraction, model training by using a preset machine learning algorithm, parameter tuning and the like. As an example, the machine learning algorithm employed may be various machine learning methods such as a neural network, a support vector machine, a decision tree, a logistic regression, etc., however, the machine learning algorithm is not particularly limited. Also, the machine learning model may be any type of machine learning model, such as, but not limited to, a Logistic Regression (LR) model, a Support Vector Machine (SVM), a gradient boosting decision tree, or a deep neural network, etc.
In addition, the service unit 200 is further configured to respectively reduce, after the new machine learning model is online, a reference threshold and a lower detection threshold of each machine learning model in the one or more machine learning models, where the reduced lower detection threshold is smaller than the reduced reference threshold; and the machine learning module is used for turning off the machine learning module when the reference threshold value of one machine learning module in the module group is changed to be smaller than or equal to the preset early warning threshold value.
Furthermore, the apparatus 10 for updating iterations of an online machine learning model may further include an initialization unit (not shown) that trains an initialized machine learning model based on a predetermined initialization training sample and selects the initialized machine learning model as the top-ranked machine learning model. That is, the initialized machine learning model described herein is the first machine learning model in the model set, and only the initialized machine learning model is included in the model set at the beginning.
Further, according to an example embodiment, the model set of at least one serially-operated machine learning model may be deployed on a cloud or local, e.g., public cloud, private cloud, or hybrid cloud, and may provide machine learning services related to business samples to entities desiring to obtain the respective predictions (e.g., banks, businesses, schools, etc. desiring to obtain the predictions). Alternatively, the model set may be deployed locally, e.g. in a local system of the content operator.
In addition, it should be noted that, although the apparatus 10 is described above as being divided into units (e.g., the service unit 200, the training sample generation unit 300, and the model training unit 400) for performing the respective processes for convenience of description, it is apparent to those skilled in the art that the processes performed by the respective units described above may be performed without any specific division of units or without explicit demarcation between the respective units. Furthermore, the apparatus 10 described above with reference to fig. 1 is not limited to include the above-described units, but may add/subtract some units as needed, and may also be a combination of the above units. And according to the deployment condition of the model group, each unit of the device 10 can be centralized in part or distributed in deployment. For example, when the model set is deployed at the cloud, the service unit 200 is deployed at the cloud together with the model set, and the training sample generating unit 300 and the model training unit 400 may be deployed at the cloud or locally.
Fig. 2 is a flowchart illustrating an iterative method of updating an online machine learning model with the apparatus 10 according to an exemplary embodiment of the present disclosure. A method of implementing an update iteration of an online machine learning model by the apparatus 10 will be described in detail below with reference to fig. 1 and 2.
Referring to fig. 1 and 2, in step S1, the service unit 200 configures a model group consisting of at least one machine learning model working in series to predict service samples, where the model group includes a dual-threshold machine learning model (having a reference threshold and a lower detection threshold) and a single-threshold machine learning model (only the reference threshold), the dual-threshold machine learning model at least includes a machine learning model arranged at the forefront in the model group, the dual-threshold machine learning model allows service samples having a prediction value greater than the reference threshold corresponding to the present model to pass, denies service samples having a prediction value less than the lower detection threshold corresponding to the present model, and allows service samples having a prediction value less than or equal to the reference threshold and greater than or equal to the lower detection threshold to pass with a probability corresponding to the present model. The single-threshold machine learning model allows service samples with predicted values greater than or equal to a reference threshold corresponding to the model to pass through and rejects service samples with predicted values less than the reference threshold. This will be described in detail below with reference to fig. 3.
In step S2, the training sample generating unit 300 acquires service feedback information of the service samples allowed to pass by the model set in step S1, and constructs training samples according to the service feedback information and the corresponding service samples. The model is based on the product of training samples, which may be inconsistent in data distribution with the business samples to be processed after future model applications, which would have unpredictable consequences for the model's effectiveness. Thus, the training sample generation unit 300 may constitute a training sample for training a new machine learning model following the principle that the distribution of training samples for training the new machine learning model is consistent with the distribution of traffic samples processed by the new model as a row in the model group after the last machine learning model is on line.
In an embodiment in which the dual-threshold machine learning model includes only the machine learning model arranged in the front of the model group, the training sample constituted by the training sample generating unit 300 may satisfy the expression (1):
wherein M (N+1) train Training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p 1 is a probability corresponding to the foremost machine learning model;
Comp i representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample;
Prob 1 representing that the forefront machine learning model selects a service sample input to the forefront machine learning model, wherein the predicted value of the service sample is smaller than or equal to a corresponding reference threshold value and larger than or equal to a corresponding lower detection threshold value and has probability p when the service sample is processed by the forefront machine learning model 1 And forming a training sample by the passed service sample according to the acquired service feedback information and the corresponding service sample.
According to another aspect of the embodiments of the present disclosure, when the dual-threshold machine learning model includes not only the machine learning model arranged at the front, but also one or more machine learning models of other orders, the training sample constituted by the training sample generating unit 300 may satisfy the formula (2):
wherein M (N+1) train Training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p i is a probability corresponding to the i-th machine learning model;
Comp i representing selection of a service sample from a service sample input to an ith machine learning model via the ith machineThe method comprises the steps that when a learning model is processed, a service sample with a predicted value larger than a corresponding reference threshold value is obtained, and a training sample is formed according to the obtained service feedback information and the corresponding service sample;
Prob i Representing that the ith machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and with probability p from service samples input to the ith machine learning model, the predicted value of the ith machine learning model being processed by the ith machine learning model i The passing service sample and the corresponding service sample form a training sample according to the acquired service feedback information;
if the ith machine learning model has only the reference threshold, then Prob will be i Set to 0 and set p i Set to 1.
Because the training samples comprise the service samples which are possibly killed by mistake, the new machine learning model not only can correct the previous misplaced samples, but also can correctly identify the previously misplaced samples, and cannot be corrected due to service termination.
Next, in step S3, a new machine learning model is trained based on training samples generated in a predetermined time after the last machine learning model is on line in the current model group, and the trained new machine learning model is concatenated after the last machine learning model to line the new machine learning model.
The iterative updating of the model according to the above-described embodiments of the present disclosure is actually followed by an improvement in the value of the two-part sample: some samples that are judged to pass by the current machine learning model but that are actually in bad business, i.e., misplaced, for which the correct decisions should be rejected to eliminate the loss caused by the some samples; another part is samples that are judged to be rejected by the current machine learning model, but which are likely to actually appear as good samples, i.e., false positive samples, for which the correct decisions should be passed to obtain the potential benefit of this part of the samples. Thus, the new machine learning model generated in accordance with embodiments of the present disclosure can reduce the probability of misplacing samples as well as the probability of miskilling samples. On the other hand, the machine learning model performs a screening function on the full volume of samples after being online, so that the training samples corresponding to the allowed service samples which are accumulated later do not comprise service samples which are lower than the lower detection threshold or the reference threshold, and in this case, the new machine learning model is independently online, which brings risks. While the model training unit 400 according to the embodiment of the present disclosure may further concatenate the trained new machine learning model in the model group in the service unit 200 to line the new machine learning model after the last machine learning model. Because the new machine learning model is used in series, rather than directly, the misplacement loss can be reduced, and the false killing loss can be reduced.
After the new machine learning model is online, in step S4, the service unit 200 reduces the reference threshold and the scout threshold of each of the two-threshold machine learning models, respectively, wherein the reduced scout threshold is smaller than the reduced reference threshold.
In step S5, when the reference threshold of one machine learning model in the model group is changed to be equal to or smaller than the preset early warning threshold, the service unit 200 turns off the machine learning model.
Furthermore, the apparatus 10 includes training an initialization machine learning model based on a predetermined initialization training sample, and selecting the initialization machine learning model as the top-ranked machine learning model.
In summary, as the apparatus 10 of the exemplary embodiments of the present disclosure can explore the threshold via the dual-threshold machine learning model, so as to robustly allow traffic samples that would otherwise be rejected to pass through with affordable risk, and to have an opportunity to observe their performance via subsequent traffic feedback information; meanwhile, a training sample is formed on the basis of the allowed service sample and the service feedback information corresponding to the allowed service sample according to the principle that the service sample distribution is consistent so as to train a new machine learning model, and the new machine learning model is used in series. Accordingly, the apparatus 10 may robustly iterate updating an online machine learning model, increasing the frequency of updating the machine learning model and improving the flexibility of the machine learning model, thereby avoiding machine learning models from lagging the change in business and reducing the risk of model changes.
Fig. 3 to 6 are schematic diagrams illustrating an operation method of the apparatus 10 of fig. 1 according to an exemplary embodiment of the present disclosure, wherein the dual-threshold machine learning model of the model group includes only an example of the foremost machine learning model and a detailed operation method thereof.
Referring to fig. 3, the horizontal coordinate axis is a time line, the first model M1 is on line at a time point t1, the first model M1 is trained by an initialization unit (not shown) based on the service sample D1 (t 1) and the service feedback information corresponding thereto, and the service sample after the time t1 is scored after the first model M1 is on line, that is, a prediction result is given for the service sample. For example, a prediction value corresponding to each of the traffic samples may be given for each sample. Here, the training sample of the first model M1 may further include a data record acquired from the outside. When applying the model according to the prior art only a threshold value is specified, which represents a business acceptable risk level. The higher the model scoring sample, the better the risk performance, the lower the score sample, and the worse the risk performance. The model application according to the prior art therefore proceeds with a full pass of samples above the threshold and a full rejection of samples below the threshold. In contrast, in the embodiment of the present invention, the first model M1 has the reference threshold value set to Tr1 and the lower detection threshold value set to Tr2 (wherein Tr2 <Tr 1). During the time point t1 to the time point t2, referring to step S1 (see fig. 2), the first model M1 gives a prediction result for the traffic sample D (t 2), which is a full amount of samples processed by the first model M1 during the time point t1 to the time point t2 after the first model M1 is on line. The first model M1 determines that the traffic sample D2 (t 2) having a predicted value greater than Tr1 passes, determines that the traffic sample D4 (t 2) having a predicted value less than Tr2 does not pass, and allows the traffic sample D3 (t 2) having a predicted value equal to or less than Tr1 and equal to or greater than Tr2 (i.e., within the section s) to pass with the probability p. Therefore, the traffic sample determined to pass by the first model M1 includes two partsPart is the traffic sample D2 (t 2) that is fully passed and part is the traffic sample D3 that is allowed to pass with probability p pass (t 2). The reason for allowing the probability p is that the samples in the traffic samples D3 (t 2) are at a slightly higher risk than the samples in the traffic samples D2 (t 2), and during the model iteration, it is desirable to recover the value of the misplaced samples in the traffic samples D3 (t 2), and it is undesirable to take over the risk of misplaced samples in the whole traffic samples D in the traffic samples D3 (t 2), so that the way allowing the probability p is allowed to pass is chosen, i.e. a certain amount of samples is randomly extracted from the traffic samples D3 (t 2) (i.e. traffic samples D3 pass (t 2)). In the case of a random and sufficient sample size, the probability p of passing traffic samples D3 among traffic samples D3 (t 2) is allowed to pass traffic samples D3 pass (t 2) may represent to some extent the risk performance of the D3 (t 2) overall business sample. In this way, the risk performance of the service sample D3 (t 2) can be observed at a lower risk cost in hopes of better discriminating such service samples by accumulation of service feedback information.
The timing of training the new model may be determined according to predetermined conditions and/or may be manually specified. As an example, the predetermined condition may be an effect of a current machine learning model (e.g., the first model M1) or a model group, a predetermined period of time, an amount of traffic samples allowed to pass, and the like. As an example, the effect of the current machine learning model or set of models may be determined by measuring key metrics of the machine learning model, e.g., the effect of the model may be measured using an evaluation value for the model evaluation metrics. For example, the evaluation index of the machine learning model may be AUC (ROC (subject work feature, receiver Operating Characteristic) area under curve, area Under ROC Curve), precision, recall, accuracy, MAE (mean absolute error ), log-loss function (loglos), or the like. As an example, for a risk assessment scenario, the predetermined period of time may be one month or several months, and the amount of traffic samples allowed to pass may be 1, 5, or 10 ten thousand traffic samples. The predetermined time period or the number of traffic samples allowed to pass is merely an example, and the present disclosure is not limited thereto. In addition, these key indicators may also be used in combination.
Referring to fig. 3 and 4, it is assumed that the conditions for training the second model M2 are met at the time point t 2. In step S2 (see fig. 2), if the traffic sample D3 pass (t 2) is directly mixed with the traffic sample D2 (t 2) and based thereon training samples are formed, and then the true distribution ratio of the traffic samples D2 (t 2) and D3 (t 2) cannot be correctly reflected in the sample size. Thus, in constructing the training samples for training the second model M2, the training sample generation unit 300 acquires the traffic samples D2 (t 2) and D3 through which the first model M1 is allowed to pass pass (t 2) service feedback information corresponding to them. Here, the service sample D2 (t 2) and the service feedback information corresponding thereto correspond to D (t 2) ·comp in expression (1) 1 Service sample D3 pass (t 2) and the corresponding service feedback information correspond to D (t 2) Prob in formula (1) 1 . The training sample generation unit 300 allows the training sample for training the second model M2 to satisfy the following equation:
M2 train ＝D(t2)·Comp 1 +D(t2)·Prob 1 /p
copying the traffic sample D3 by the inverse 1/p of the probability p pass (t 2) and the service feedback information corresponding thereto, i.e., D (t 2). Prob 1 And/p to restore the original distribution of the portion of the sample.
After training to obtain the second model M2, if the first model M1 is directly replaced by the second model M2, since the training sample of the second model M2 is derived from the data after the first model M1 is online, the part of the training sample, which is rejected by the first model M1 and is considered to be the worst performance of the service sample D4 (t 2), cannot be seen at all, so that the direct replacement mode may not identify the service sample similar to the service sample D4 (t 2) enough, and thus a great impact is caused to the model effect and service. Thus, preferably, in step S3 (see fig. 2), a plurality of models may be used in series, and after the second model M2 is trained, the second model M2 is used in series behind the first model M1 to line the second model M2, so that the second model M2 works as if it is protected by the first model M1. As an example, after the second model M2 is online, a traffic sample similar to the traffic sample D4 (t 2) previously rejected by the first model M1 will still be rejected by the first model M1. During the time point t2 to the time point t3, the service sample may be processed through the first model M1 and then processed through the second model M2, and if the service sample is allowed to pass through both models M1 and M2, the service sample is determined to pass through, and if the service sample is rejected by either model M1 or M2, the service sample is determined to be rejected.
After the second model M2 is brought on-line, referring to fig. 5, in step S4 (see fig. 2), the threshold value of the first model M1 is adjusted. As an example, the reference threshold of the first model M1 is adjusted from Tr1 to Tr2, the downsampling threshold of the first model M1 is from Tr2 to Tr3 (Tr 3< Tr 2), and the second model M2 may have only the reference threshold. In addition, the interval s and the interval s' can be selected to be suitable values according to actual situations in actual application, and can be the same or different from each other. In the present embodiment, the adjusted reference threshold value is set to be equal to the lower detection threshold value when the second model M2 is on line in the threshold adjustment of the first model M1 (i.e., the reference threshold value of the first model M1 is adjusted from Tr1 to Tr 2), however, the concept of the present disclosure is not limited thereto, and in another embodiment, the adjusted reference threshold value may be set according to the actual situation (e.g., greater than or less than Tr 2). In yet another embodiment, the adjusted reference threshold may be set to a value between Tr1 and Tr2 (i.e., less than Tr1 and greater than Tr 2) to further improve risk prevention capability.
Subsequently, as an example, during the time point t2 to the time point t3, the model group composed of the first model M1 and the second model M2 predicts the traffic data D (t 3) with reference to the description of the foregoing step S1 (see fig. 2).
In detail, referring to fig. 5, the traffic sample D2 (t 3) having a predicted value greater than Tr2 given by the first model M1 is determined to pass through in full, the traffic sample D2 (t 3) allowed to pass through by the first model M1 is processed by the second model M2, the traffic sample D25 (t 3) having an evaluation value greater than the reference threshold of the second model M2 given by the second model M2 is finally determined to pass through, and the traffic sample D26 (t 3) having an evaluation value less than or equal to the reference threshold of the second model M2 given by the second model M2 is finally determined to be rejected; the first model M1 allows for pre-predictionThe traffic sample D3 (t 3) having a measured value of Tr2 or less and Tr3 or more (i.e., within the section s ') passes with probability p', and the traffic sample D3 is allowed to pass through pass (t 3) after the second model M2 is processed, the service sample D35 (t 3) with the evaluation value greater than the reference threshold value of the second model M2 given by the second model M2 is finally judged to pass, and the service sample D36 (t 3) with the evaluation value less than or equal to the reference threshold value of the second model M2 given by the second model M2 is finally judged to be refused; the traffic sample D4 (t 3) whose predicted value is smaller than Tr2 given by the first model M1 is determined as rejected. The traffic data D5 (t 3) represents traffic samples (i.e., traffic samples D25 (t 3) and D35 (t 3)) for which the evaluation value by the second model M2 is greater than the reference threshold value, and the traffic data D6 (t 3) represents traffic samples (i.e., traffic samples D26 (t 3) and D36 (t 3)) for which the evaluation value by the second model M2 is equal to or less than the reference threshold value. Wherein the probability p 'is the probability when the traffic sample D3 (t 3) is processed via the first model M1 during the time point t2 to the time point t3, and the probability p' may be the same as or different from the probability p at the time point t1 to the time point t 2.
Referring to fig. 6, as an example, after the second model M2 is on-line for a predetermined time, for example, at a time point t3, the condition for training the third model M3 is met, and with continued reference to step S2 (see fig. 2), the training sample generating unit 300 acquires the traffic sample D5 (t 3) (i.e., traffic samples D25 (t 3) and D35 (t 3)) including the model group consisting of the first model M1 and the second model M2, and the traffic feedback information corresponding thereto. The service sample D25 (t 3) and the service feedback information corresponding thereto correspond to D (t 3) ·comp in expression (1) 1 ·Comp 2 The service sample D35 (t 3) and the service feedback information corresponding thereto correspond to D (t 3) Prob in formula (1) 1 ·Comp 2 . The training sample generation unit 300 may use the training sample for training the third model M3 to satisfy the following equation:
as can be seen with reference to fig. 6, in the manner described above with reference to step S3 (see fig. 2), after training the third model M3, the third model M3 is used in series behind the second model M2 to put the third model M3 on line. The traffic sample actually processed by the third model M3 is a traffic sample (e.g., a traffic sample similar to the traffic sample D5 (t 3)) determined to pass by the first model M1 and the second model M2. The third model M3 does not process other traffic samples that have been rejected by the previous machine learning model (e.g., traffic samples similar to traffic samples D4 (t 3) and D6 (t 3)).
Subsequently, referring to the aforementioned step S4 (see fig. 2), the threshold value of the first model M1 is again adjusted. As an example, the reference threshold value of the first model M1 is adjusted from Tr2 to Tr3, the downsampling threshold value of the first model M1 is further downscaled from Tr3 by a predetermined section (not shown), and the second and third models M2 and M3 may have only the reference threshold value.
Subsequently, as an example, during a predetermined time after the time point t3, the model group consisting of the first model M1, the second model M2, and the third model M3 predicts the traffic data D (t 3) with reference to the description of the foregoing step S1.
Subsequent model training is also referred to a similar method, with the newly trained model being concatenated at the end and the machine learning model ranked at the front being thresholded down until the first model M1 comes down line, see in particular fig. 7.
Fig. 7 is a schematic diagram illustrating a method of offline machine learning models in a model set by the apparatus 10 of fig. 1, according to an exemplary embodiment of the present disclosure.
Referring to fig. 7, the model group may include a first model M1, a second model M2, a third model M3 … …, and an nth model Mn. When the reference threshold value of the machine learning model (i.e., the first model M1) arranged at the front is changed to be equal to or less than the preset pre-warning threshold value (for example, the preset threshold value or 0), the first model M1 in fig. 7 has no screening effect on the sample in the on-line service, and can be formally taken off line at that time. At the same time, a threshold undershoot is performed on the model immediately following it (e.g., the second model M2 in the figure), i.e., a reference threshold (not shown) of the second model M2 and a undershoot threshold (not shown) smaller than the reference threshold thereof are set, and the second model M2 starts to operate in the manner of the first model M1, thereby starting a new cycle.
The above embodiment describes an example in which the dual-threshold machine learning model includes only one machine learning model arranged at the front, and each time the threshold undershoot is performed on the basis of the one machine learning model arranged at the front in the model group, the single-model threshold undershoot is performed until the threshold undershoot of the machine learning model reaches the early warning threshold (for example, a preset threshold or 0), and then the threshold undershoot of the next machine learning model is started. Additionally, alternatively, the dual-threshold machine learning model may include an example of multiple machine learning models, and multiple machine learning models may be simultaneously thresholded, as described in detail below with reference to FIG. 8.
Fig. 8 is a schematic diagram illustrating a method of operation of the apparatus 10 of fig. 1, wherein the dual-threshold machine learning model in the model set includes 2 machine learning models, according to another exemplary embodiment of the present disclosure.
The present embodiment is substantially the same as the operation method described above with reference to fig. 3 to 7, except that the dual-threshold machine learning model includes a first model M1 and a second model M2. The differences from the foregoing operation methods will be mainly described next. As shown in fig. 8, when the second model M2 is on-line, a reference threshold value of the second model M2 and a lower detection threshold value smaller than the reference threshold value thereof may be set. The service samples D2 and D5 are service samples determined to pass by the first model M1 and the second model M2, respectively, the service sample D3 is a service sample having a lower detection threshold value equal to or greater than the first model M1 and a reference threshold value equal to or less than the first model M1, and D3 pass Is the traffic sample in D3 that is passed by the first model M1 with the corresponding probability of passing. The service sample D6 is a service sample which is greater than or equal to the lower detection threshold of the second model M2 and less than or equal to the reference threshold of the second model M2, D6 pass Is the traffic sample in D6 that is passed by the second model M2 with the corresponding probability of passing. Wherein the corresponding passing probability of the first model M1 is p 1 The corresponding probability of passing of the second model M2 is p 2 . That is, the first model M1 is represented by the probability p 1 Allow partial samples D3 in D3 pass By the second model M2 with probability p 2 Allow part of sample D6 in D6 pass Through the device. The traffic samples D4 and D7 are determined as full denial for the first model M1 and the second model M2, respectivelyTraffic samples. After the second model M2 is online, referring to step S1, the service sample D is predicted by a model group consisting of the first model M1 and the second model M2.
When training the third model M3, training samples for training the third model M3 are composed of the traffic samples D5 and D6 with reference to step S2 pass And their corresponding service feedback information. According to the above formula (2), a part of the traffic sample D5 determined to be fully passed by the second model M2 is derived from the traffic sample D2 (denoted by "D25" in fig. 8) determined to be fully passed by the first model M1, and the traffic sample D25 and the traffic feedback information corresponding thereto correspond to D Comp in the formula (2) 1 ·Comp 2 . Another part of the traffic sample D5 originates from the probability p determined by the first model M1 1 Passing traffic sample D3 pass (denoted by "D35" in FIG. 8), the service sample D35 and the service feedback information corresponding thereto correspond to D.Prob in formula (2) 1 ·Comp 2 . Likewise, the traffic sample D6 determined by the second model M2 as the probability of passing pass Part of the traffic samples D2 (denoted by "D26" in fig. 8) determined to pass through the first model M1, and the traffic samples D26 and the corresponding traffic feedback information correspond to D Comp in the formula (2) 1 ·Prob 2 . Another part of the traffic samples D6 originates from the traffic samples D3 determined by the first model M1 as probability passing pass (represented by "D36" in FIG. 8), the service sample D36 and the service feedback information corresponding thereto correspond to D.Prob in the formula (2) 1 ·Prob 2 . The training samples for training the third model M3 may satisfy the following equation:
after training the third model M3, the third model M3 is connected in series behind the second model M2, in the manner described above with reference to step S3 (see fig. 2), then the thresholds of the first model M1 and the second model M2 are adjusted again with reference to step S4 (see fig. 2), and further the traffic samples after the third model M3 is on-line are predicted with reference to step S1 described above. As an example, only the two machine learning models arranged at the forefront in the model group are subjected to the threshold value undershoot in the present embodiment, and therefore, only two machine learning models among the first model M1, the second model M1, the third model M3 to the nth model Mn in the model group have the undershoot threshold value and the probability corresponding thereto in the subsequent step. For a machine learning model having only a reference threshold, the corresponding Prob function may be set to 0 and the corresponding probability to 1, so that the condition to be satisfied by the training sample when training a new machine learning model may be expressed using the general formula (2).
Similarly, the training data formation mode of the dual-threshold machine learning model comprising three models and more models during simultaneous threshold undershoot can be analogically deduced, and when a new machine learning model is trained, the principle that the distribution of training samples is consistent with the distribution of service samples arranged after the last machine learning model in the current model set is kept.
The present disclosure proposes a set of methods to ensure the robustness of business effects in the course of model replacement after machine learning models are applied online. The machine learning model is obtained based on training samples, and the training samples in the process of constructing the model are consistent with the service sample distribution applied after the model is online, so that the machine learning model is a fundamental guarantee of the steady model effect. The present disclosure provides a system and method for updating and iterating a machine learning model in machine learning application practice from the perspective of ensuring that data distribution in a model training period and an application period is consistent. In the scene of sensitive cost of mispassing such as wind control, anti-fraud, etc., the method has higher application value.
An iterative method and apparatus for updating an online machine learning model according to an exemplary embodiment of the present application have been described above with reference to fig. 1 to 8. However, it should be understood that: the apparatus shown in fig. 1 and its elements may be configured as software, hardware, firmware, or any combination thereof, respectively, that performs a particular function. For example, these means or units may correspond to application specific integrated circuits, to pure software code, or to units of a combination of software and hardware. Furthermore, one or more functions performed by these apparatuses or units may also be uniformly performed by components in a physical entity device (e.g., a processor, a client, a server, or the like).
Furthermore, the above-described method may be implemented by a program recorded on a computer readable medium, for example, according to an exemplary embodiment of the present application, a computer readable storage medium may be provided, on which computer instructions are stored, which when executed by at least one computing device, cause the at least one computing device to perform the method of implementing an update iteration of an online machine learning model as described in any one of the above-described embodiments. For example, in one embodiment of the invention, a computer program may be executed which performs the following method steps: acquiring an initial machine learning model group comprising at least one machine learning model through an automatic machine learning mode; continuously acquiring prediction data; monitoring whether distribution state changes exceeding a preset threshold value occur in the continuously acquired prediction data; in the event of a change in the distribution state beyond the threshold, the initial set of machine learning models is automatically updated.
The computer program in the above-described computer readable medium may be run in an environment deployed in a computer device such as a client, a host, a proxy device, a server, etc., and it should be noted that the computer program may also be used to perform additional steps other than the above-described steps or to perform more specific processes when the above-described steps are performed, and the contents of these additional steps and further processes have been mentioned in the description of the related method with reference to fig. 2, so that a repetition will not be repeated here.
It should be noted that the automatic machine learning device according to the exemplary embodiment of the present application may completely rely on the execution of a computer program to implement the corresponding functions, i.e., each unit corresponds to each step in the functional architecture of the computer program, so that the entire device is called by a dedicated software package (e.g., lib library) to implement the corresponding functions.
Alternatively, the apparatus or elements shown in fig. 1 may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the corresponding operations may be stored in a computer-readable medium, such as a storage medium, so that the processor can perform the corresponding operations by reading and executing the corresponding program code or code segments.
For example, exemplary embodiments of the present application may also be implemented as a system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform a method of implementing an update iteration of an online machine learning model as described in any of the embodiments above. For example, in one embodiment, a method may be performed that includes the steps of: acquiring an initial machine learning model group comprising at least one machine learning model through an automatic machine learning mode; continuously acquiring prediction data; monitoring whether distribution state changes exceeding a preset threshold value occur in the continuously acquired prediction data; in the event of a change in the distribution state beyond the threshold, the initial set of machine learning models is automatically updated.
In particular, the computing devices may be deployed in servers or clients, as well as on node devices in a distributed network environment. Further, the computing device may be a PC computer, tablet device, personal digital assistant, smart phone, web application, or other device capable of executing the above-described set of instructions.
Here, the computing device need not be a single computing device, but may be any device or collection of circuits capable of executing the above-described instructions (or instruction set) alone or in combination. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with locally or remotely (e.g., via wireless transmission).
In the computing device, the processor may include a Central Processing Unit (CPU), a Graphics Processor (GPU), a programmable logic device, a special purpose processor system, a microcontroller, or a microprocessor. By way of example, and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
Some of the operations described in the method for providing an online prediction service using a machine learning model according to the exemplary embodiment of the present application may be implemented in software, some of the operations may be implemented in hardware, and furthermore, the operations may be implemented in a combination of software and hardware.
The processor may execute instructions or code stored in one of the storage components, wherein the storage component may also store data. Instructions and data may also be transmitted and received over a network via a network interface device, which may employ any known transmission protocol.
The memory component may be integrated with the processor, for example, RAM or flash memory disposed within an integrated circuit microprocessor or the like. Further, the storage component may comprise a stand-alone device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The storage component and the processor may be operatively coupled or may communicate with each other, such as through an I/O port, network connection, etc., such that the processor is able to read files stored in the storage component.
In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be connected to each other via buses and/or networks.
Operations involved in a method of performing automatic machine learning according to exemplary embodiments of the present application may be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams may be equally integrated into a single logic device or operate at non-exact boundaries.
The method and system for providing online prediction services using a machine learning model according to the present application have been described above in connection with exemplary embodiments, which can be widely applied to any machine learning scenario in which data is not independently and co-distributed, for example, such as online content (such as news, advertisements, music, etc.) recommendation, credit card fraud detection, abnormal behavior detection, smart marketing, smart investment advisor, network traffic analysis, etc.
While exemplary embodiments of the present application are described above, it should be understood that: the above description is illustrative only and is not exhaustive. The present application is not limited to the exemplary embodiments disclosed and many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present application. The scope of the application should, therefore, be determined with reference to the appended claims.

Claims (16)

1. A method of implementing an update iteration of an online machine learning model, the method for providing an online prediction service, the method comprising:
predicting a business sample from a model set consisting of at least one machine learning model working in series; wherein one or more machine learning models of the model set comprising the foremost machine learning model are arranged to operate as follows: allowing service samples with predicted values larger than a reference threshold corresponding to the model to pass, rejecting service samples with predicted values smaller than a lower detection threshold corresponding to the model and smaller than the reference threshold, and allowing service samples with predicted values smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass with probability corresponding to the model; and wherein the other machine learning models in the model set are arranged to operate as follows: allowing service samples with predicted values larger than or equal to a reference threshold corresponding to the model to pass through, and rejecting service samples with predicted values smaller than the reference threshold;
Acquiring service feedback information of service samples allowed to pass through by the model group, and forming training samples according to the service feedback information and the corresponding service samples;
training a new machine learning model based on training samples generated in a predetermined time after a last machine learning model is put on line in a current model group, and concatenating the trained new machine learning model to the last machine learning model to put the new machine learning model on line;
respectively reducing a reference threshold and a scout threshold for each of the one or more machine learning models, wherein the reduced scout threshold is less than the reduced reference threshold;
when the reference threshold value of one machine learning model in the model group is changed to be smaller than or equal to the preset early warning threshold value, the machine learning model is disconnected,
wherein the set of models predicts traffic samples, the predictions including at least one of online content recommendation, credit card fraud detection, abnormal behavior detection, intelligent marketing, intelligent investment advisor, network traffic analysis.
2. The method of claim 1, wherein training a new machine learning model based on training samples generated within a predetermined time after a last machine learning model line in a current model set comprises:
The training samples are obtained by following the principle that the distribution of the training samples is consistent with the distribution of the service samples arranged after the last machine learning model is on line in the current model group,
the new machine learning model is trained based on the acquired training samples.
3. The method of claim 2, wherein when the one or more machine learning models include only the foremost machine learning model,
the training samples obtained following the principle that the distribution of the training samples is consistent with the distribution of the service samples arranged after the last machine learning model is on line in the current model group satisfy the following formula:
M(N+1) train training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p 1 is a probability corresponding to the foremost machine learning model;
Comp i representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample;
Prob 1 Representing that the forefront machine learning model selects a service sample input to the forefront machine learning model, wherein the predicted value of the service sample is smaller than or equal to a corresponding reference threshold value and larger than or equal to a corresponding lower detection threshold value and has probability p when the service sample is processed by the forefront machine learning model 1 And forming a training sample by the passed service sample according to the acquired service feedback information and the corresponding service sample.
4. The method of claim 2, wherein the training samples obtained following a principle that a distribution of training samples is consistent with a distribution of traffic samples in the current model set arranged after a last machine learning model is online satisfy the following formula:
M(N+1) train training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p i is a probability corresponding to the i-th machine learning model;
Comp i representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample;
Prob i Representing that the ith machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and with probability p from service samples input to the ith machine learning model, the predicted value of the ith machine learning model being processed by the ith machine learning model i The passing service sample and the corresponding service sample form a training sample according to the acquired service feedback information;
if the ith machine learning model has only the reference threshold, then Prob will be i Set to 0 and set p i Set to 1.
5. The method of claim 1, wherein the step of separately lowering the benchmark threshold and the peeping threshold for each of the one or more machine learning models comprises:
after the new machine learning model is brought online, for each machine learning model of the one or more machine learning models, a baseline threshold value of the machine learning model is lowered by a predetermined value, and a scout threshold value of the machine learning model is lowered by a predetermined value.
6. The method of claim 1, wherein the step of separately lowering the benchmark threshold and the peeping threshold for each of the one or more machine learning models comprises:
After the new machine learning model is online, for each machine learning model of the one or more machine learning models, setting a reference threshold for the machine learning model equal to a scout threshold for the machine learning model when the new machine learning model is online, and reducing the scout threshold for the machine learning model by a predetermined value.
7. The method of any of claims 1-6, wherein the method further comprises:
training an initialized machine learning model based on a preset initialized training sample, and selecting the initialized machine learning model as a machine learning model arranged at the front.
8. An apparatus for enabling update iterations of an online machine learning model, the apparatus for providing an online prediction service, the apparatus comprising:
a service unit for configuring a model group consisting of at least one machine learning model working in series, and predicting a service sample via the model group; and for configuring one or more machine learning models of the model set, including the foremost machine learning model, to operate as follows: allowing service samples with predicted values larger than a reference threshold corresponding to the model to pass, rejecting service samples with predicted values smaller than a lower detection threshold corresponding to the model and smaller than the reference threshold, and allowing service samples with predicted values smaller than or equal to the reference threshold and larger than or equal to the lower detection threshold to pass with probability corresponding to the model; and wherein the other machine learning models in the model set are arranged to operate as follows: allowing service samples with predicted values larger than or equal to a reference threshold corresponding to the model to pass through, and rejecting service samples with predicted values smaller than the reference threshold;
The training sample generation unit is used for acquiring service feedback information of the service samples allowed to pass through by the model group and forming training samples according to the service feedback information and the corresponding service samples; and
a model training unit for training a new machine learning model based on training samples generated in a predetermined time after the last machine learning model is put on line in the current model group, and concatenating the trained new machine learning model after the last machine learning model to put the new machine learning model on line;
the service unit is further configured to reduce a reference threshold and a lower detection threshold of each machine learning model in the one or more machine learning models after the new machine learning model is online, where the reduced lower detection threshold is smaller than the reduced reference threshold; and is used for putting off the machine learning model when the reference threshold value of one machine learning model in the model group is changed to be less than or equal to the preset early warning threshold value,
wherein the predictions performed by the service units include at least one of online content recommendation, credit card fraud detection, abnormal behavior detection, smart marketing, smart investment advisor, network traffic analysis.
9. The apparatus of claim 8, wherein,
the training sample generation unit is used for acquiring training samples according to the principle that the distribution of the training samples is consistent with the distribution of service samples arranged after the last machine learning model is on line in the current model group; and is also provided with
The model training unit is used for training the new machine learning model based on training samples generated in a preset time after the last machine learning model is on line in the current model group.
10. The apparatus of claim 9, wherein when the one or more machine learning models include only the foremost machine learning model,
the training sample constituted by the training sample generating unit satisfies the following formula:
M(N+1) train training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p 1 is a probability corresponding to the foremost machine learning model;
Comp i Representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample;
Prob 1 representing that the forefront machine learning model selects a service sample input to the forefront machine learning model, wherein the predicted value of the service sample is smaller than or equal to a corresponding reference threshold value and larger than or equal to a corresponding lower detection threshold value and has probability p when the service sample is processed by the forefront machine learning model 1 And forming a training sample by the passed service sample according to the acquired service feedback information and the corresponding service sample.
11. The apparatus of claim 9, wherein the training samples constituted by the training sample generation unit satisfy the following formula:
M(N+1) train training samples for training the new machine learning model;
d is a full traffic sample input to the model group within a preset time after the last machine learning model is on line with the row in the current model group;
n is a positive integer, representing the number of machine learning models in the current model set;
i represents an i-th machine learning model in the current model group in the order from front to back;
p i Is a probability corresponding to the i-th machine learning model;
Comp i representing that a service sample with a predicted value larger than a corresponding reference threshold value when processed by the ith machine learning model is selected from service samples input to the ith machine learning model, and forming a training sample according to the acquired service feedback information and the corresponding service sample;
Prob i representing that the ith machine learning model selects a corresponding reference threshold value or less and a corresponding lower detection threshold value or more and with probability p from service samples input to the ith machine learning model, the predicted value of the ith machine learning model being processed by the ith machine learning model i The passing service sample and the corresponding service sample form a training sample according to the acquired service feedback information;
if the ith machine learning model has only the reference threshold, then Prob will be i Set to 0 and set p i Set to 1.
12. The apparatus of claim 8, wherein,
the service unit is configured to, after the new machine learning model is online, reduce, for each machine learning model of the one or more machine learning models, a reference threshold value of the machine learning model by a predetermined value, and reduce a scout threshold value of the machine learning model by a predetermined value.
13. The apparatus of claim 8, wherein,
the service unit is configured to, after the new machine learning model is online, set, for each machine learning model of the one or more machine learning models, a reference threshold value of the machine learning model to be equal to a scout threshold value of the machine learning model when the new machine learning model is online, and reduce the scout threshold value of the machine learning model by a predetermined value.
14. The apparatus of any of claims 8-13, wherein the apparatus further comprises:
an initialization unit trains an initialization machine learning model based on a predetermined initialization training sample, and selects the initialization machine learning model as a forefront machine learning model.
15. A computer-readable storage medium having stored thereon computer instructions that, when executed by at least one computing device, cause the at least one computing device to perform the method of implementing the update iteration of the online machine learning model of any one of claims 1 to 7.
16. A system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the method of implementing the update iteration of the online machine learning model of any one of claims 1 to 7.
CN201811619035.8A 2018-12-28 2018-12-28 Method and device for realizing update iteration of online machine learning model Active CN111382874B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811619035.8A CN111382874B (en) 2018-12-28 2018-12-28 Method and device for realizing update iteration of online machine learning model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811619035.8A CN111382874B (en) 2018-12-28 2018-12-28 Method and device for realizing update iteration of online machine learning model

Publications (2)

Publication Number Publication Date
CN111382874A CN111382874A (en) 2020-07-07
CN111382874B true CN111382874B (en) 2024-04-12

Family

ID=71220048

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811619035.8A Active CN111382874B (en) 2018-12-28 2018-12-28 Method and device for realizing update iteration of online machine learning model

Country Status (1)

Country Link
CN (1) CN111382874B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111950738B (en) * 2020-08-10 2023-09-15 中国平安人寿保险股份有限公司 Machine learning model optimization effect evaluation method, device, terminal and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO904807D0 (en) * 1990-05-21 1990-11-05 Int Control Automation Finance PROCEDURE FOR DETERMINING A TITRATION CURVE FOR ADAPTIVE PH CONTROL.
CN105469142A (en) * 2015-11-13 2016-04-06 燕山大学 Neural network increment-type feedforward algorithm based on sample increment driving
CN107508866A (en) * 2017-08-08 2017-12-22 重庆大学 Reduce the method for the transmission consumption of mobile device end neural network model renewal
CN107798409A (en) * 2016-08-30 2018-03-13 中兴智能交通股份有限公司 A kind of crowd massing Forecasting Methodology based on time series models
EP3317785A1 (en) * 2015-07-01 2018-05-09 Services Petroliers Schlumberger Fluid relationship tracking to support model dependencies
CN108023876A (en) * 2017-11-20 2018-05-11 西安电子科技大学 Intrusion detection method and intruding detection system based on sustainability integrated study
CN108921300A (en) * 2018-06-21 2018-11-30 第四范式（北京）技术有限公司 The method and apparatus for executing automaton study

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO904807D0 (en) * 1990-05-21 1990-11-05 Int Control Automation Finance PROCEDURE FOR DETERMINING A TITRATION CURVE FOR ADAPTIVE PH CONTROL.
EP3317785A1 (en) * 2015-07-01 2018-05-09 Services Petroliers Schlumberger Fluid relationship tracking to support model dependencies
CN105469142A (en) * 2015-11-13 2016-04-06 燕山大学 Neural network increment-type feedforward algorithm based on sample increment driving
CN107798409A (en) * 2016-08-30 2018-03-13 中兴智能交通股份有限公司 A kind of crowd massing Forecasting Methodology based on time series models
CN107508866A (en) * 2017-08-08 2017-12-22 重庆大学 Reduce the method for the transmission consumption of mobile device end neural network model renewal
CN108023876A (en) * 2017-11-20 2018-05-11 西安电子科技大学 Intrusion detection method and intruding detection system based on sustainability integrated study
CN108921300A (en) * 2018-06-21 2018-11-30 第四范式（北京）技术有限公司 The method and apparatus for executing automaton study

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Statistical Model Updating and Validation Applied to Nonlinear Transient Structural Dynamics;Madrid;《European COST F3 Conference on System Identification & Structural Health Monitoring》;20000609(第2000期);1-10 *
考虑失效特征的废旧零部件再制造成本预测模型研究;张旭刚;敖秀奕;张华;江志刚;;中国机械工程;20181106(第21期);78-83 *

Also Published As

Publication number Publication date
CN111382874A (en) 2020-07-07

Similar Documents

Publication Publication Date Title
US10943186B2 (en) Machine learning model training method and device, and electronic device
CN111652380B (en) Method and system for optimizing algorithm parameters aiming at machine learning algorithm
US10459827B1 (en) Machine-learning based anomaly detection for heterogenous data sources
CN110188910B (en) Method and system for providing online prediction service by using machine learning model
US11036615B2 (en) Automatically performing and evaluating pilot testing of software
US10719769B2 (en) Systems and methods for generating and communicating application recommendations at uninstall time
WO2021005891A1 (en) System, method, and program
US20230205516A1 (en) Software change analysis and automated remediation
CN111898675A (en) Credit wind control model generation method and device, scoring card generation method, machine readable medium and equipment
CN111078500A (en) Method and device for adjusting operation configuration parameters, computer equipment and storage medium
WO2022251004A1 (en) Hierarchical neural network-based root cause analysis for distributed computing systems
CN111382874B (en) Method and device for realizing update iteration of online machine learning model
WO2021236423A1 (en) Identifying claim complexity by integrating supervised and unsupervised learning
US10636044B2 (en) Projecting resource demand using a computing device
LU102633B1 (en) Ticket troubleshooting support system
CN111737319B (en) User cluster prediction method, device, computer equipment and storage medium
US11748138B2 (en) Systems and methods for computing a success probability of a session launch using stochastic automata
CN112764957A (en) Application fault delimiting method and device
US20240046068A1 (en) Information processing device for improving quality of generator of generative adversarial network (gan)
US11513862B2 (en) System and method for state management of devices
US20220414495A1 (en) System and method for determining expected loss using a machine learning framework
CN111815442B (en) Link prediction method and device and electronic equipment
US20240220660A1 (en) System and method for managing ai models based on downstream use of inferences
US20240220661A1 (en) System and method for remediating poisoned training data used to train artificial intelligence models
CN116186398A (en) Template recommendation method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant