CN114386619A - Machine learning model management method, device, computer equipment and storage medium - Google Patents
Machine learning model management method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114386619A CN114386619A CN202111666441.1A CN202111666441A CN114386619A CN 114386619 A CN114386619 A CN 114386619A CN 202111666441 A CN202111666441 A CN 202111666441A CN 114386619 A CN114386619 A CN 114386619A
- Authority
- CN
- China
- Prior art keywords
- index
- model
- value
- detection
- multidimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a machine learning model management method, a device, a computer device and a storage medium. The method comprises the following steps: carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index; acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index; and updating the first model according to the multidimensional decision index and the value decision index so as to solve the problems of high cost, low efficiency and the like of machine learning model management.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a machine learning model management method, a machine learning model management device, computer equipment and a storage medium.
Background
With the rapid development of artificial intelligence technology, the application scenarios of machine learning models are more and more abundant. However, the machine learning model has the problems of poor interpretability, difficult running quality evaluation, unreasonable feature selection and the like, so that the machine learning model needs to be managed. At present, the model is generally managed in a manual mode based on expert rules. However, when the manual management method is used for explaining the model, the relationship between the model prediction result and the index feature cannot be revealed, and reasonable judgment conditions cannot be provided for updating the model, so that the problems of high cost, low efficiency and the like exist.
Disclosure of Invention
In view of the above, it is necessary to provide a machine learning model management method, apparatus, computer device and storage medium for improving the problem of low machine learning model management performance.
In one aspect, a machine learning model management method is provided, and the machine learning model management method includes:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
In one embodiment, the multidimensional detection is performed on a first model and a first feature of the first model, and the step of obtaining a multidimensional detection index includes:
acquiring first real data, performing first detection on the first characteristic, counting a missing value of the first characteristic in the first real data, and acquiring a first detection index;
and performing second detection on the first feature to obtain a first median of the first feature in the first real data and a second median of the feature in training data, and obtaining a second detection index according to the first median and the second median.
In one embodiment, the step of comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index includes:
judging whether the first detection index is larger than a first threshold value, if so, acquiring a first forward decision index, and if not, acquiring a first reverse decision index;
and judging whether the second detection index is larger than a second threshold value, if so, acquiring a second forward decision index, and if not, acquiring a second reverse decision index.
In one embodiment, the multidimensional detection is performed on a first model and a first feature of the first model, and the step of obtaining a multidimensional detection index includes:
acquiring first real data in a first period, and performing third detection on the first model according to the first real data to acquire a stability index of the first model;
and acquiring a third detection index according to the median of the stability index.
In one embodiment, the step of comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index includes:
and judging whether the third detection index is larger than a third threshold value, if so, acquiring a third forward decision index, and if not, acquiring a third reverse decision index.
In one embodiment, the step of obtaining a value detection indicator of the second model, comparing the value detection indicator with a preset value threshold, and obtaining a value decision indicator includes:
acquiring second real data in a second period, and acquiring a value detection index of the second model according to the second real data;
and judging whether the value detection index is larger than the value threshold value, if so, obtaining a value forward decision index, and if not, obtaining a value reverse decision index.
In one embodiment, the step of updating the first model comprises:
deleting the first characteristic to obtain a third characteristic;
deleting the second characteristics to obtain fourth characteristics;
and fusing the third feature and the fourth feature to serve as the feature of the first model.
In another aspect, there is provided a machine learning model management apparatus including:
the first acquisition module is used for carrying out multi-dimensional detection on a first model and first characteristics of the first model to obtain a multi-dimensional detection index, and comparing the multi-dimensional detection index with a preset multi-dimensional threshold to obtain a multi-dimensional decision index;
the second acquisition module is used for acquiring a second feature from a new sample, fusing the second feature with the first feature, acquiring a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold value to acquire a value decision index;
and the updating module is used for updating the first model according to the multidimensional decision index and the value decision index.
In another aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the following steps when executing the computer program:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
In yet another aspect, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, performs the steps of:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
According to the machine learning model management method, the machine learning model management device, the computer equipment and the storage medium, the first model and the first characteristic are subjected to multi-dimensional detection to obtain the multi-dimensional decision index, the second characteristic and the first characteristic are fused to obtain the second model, and the first model is updated according to the multi-dimensional decision index and the value decision index of the second model, so that the problems of high cost, low efficiency and the like of machine learning model management are solved.
Drawings
FIG. 1 is a diagram of an application environment of a method for machine learning model management in one embodiment;
FIG. 2 is a flow diagram that illustrates a method for machine learning model management, according to one embodiment;
FIG. 3 is a schematic diagram of a process for obtaining a multi-dimensional detection index according to an embodiment;
FIG. 4 is a schematic diagram of a process for obtaining a multi-dimensional decision index according to an embodiment;
FIG. 5 is a schematic diagram of a process for obtaining a multi-dimensional detection index according to an embodiment;
FIG. 6 is a schematic diagram of a process for obtaining a multi-dimensional decision index according to an embodiment;
FIG. 7 is a schematic diagram of a process for obtaining a value decision metric according to an embodiment;
FIG. 8 is a schematic flow chart illustrating updating a first model in one embodiment;
FIG. 9 is a block diagram showing the structure of a machine learning model management apparatus according to an embodiment;
FIG. 10 is a diagram showing an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The storage cluster interconnection method provided by the present application can be applied to an application environment as shown in fig. 1, in which a terminal 102 communicates with a server 104 through a network, for example, the storage cluster interconnection method provided by the present application can be applied to a financial service platform, handle and control potential fraud risks, credit risks and legal risks which may be generated in the stages of sales, loan and payment, perform multidimensional detection on a first model and a first feature to obtain a multidimensional decision index, fuse a second feature with the first feature to obtain a second model, update the first model according to the multidimensional decision index and a value decision index of the second model, so as to explain various features of a risk assessment model, establish a quantitative standard to quantitatively handle and control the operation quality of the model, and solve the problems of high cost, and the like of machine learning model management, Low efficiency and the like. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, or sub-servers, and the server 104 may be implemented by an independent server, or a server cluster formed by a plurality of servers, or a cloud computing platform.
In one embodiment, as shown in FIG. 2, there is provided a machine learning model management method, comprising the steps of:
s1: carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
s2: acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
s3: and updating the first model according to the multidimensional decision index and the value decision index.
Through the steps, the problem of low management performance of the machine learning model can be improved.
For the risk control machine learning model in the financial industry, which has a plurality of features, it is necessary to analyze the plurality of features and evaluate and monitor the operation state of the model, in step S1, illustratively, the first model and the first feature are subjected to multi-dimensional detection to obtain a multi-dimensional detection Index, a multi-dimensional decision Index is obtained according to the multi-dimensional detection Index, for example, the quality and the variation of the first feature can be detected, when the detection result is greater than a preset threshold, it is considered that the first feature is abnormal, information that the operation quality of the model is abnormal can be obtained in real time, the Stability of the model can be comprehensively evaluated, real data stored in a certain period is divided according to a unit of the period, for example, 10 days of real data are obtained, the real data are divided according to each day, and PSI (power Stability Index) is performed on the real data of each day, sample stability index), analyzing the stability of the model according to the PSI value of each day, and if the PSI value is larger than a preset threshold value, considering that the score of the model changes greatly and the running quality is reduced, giving an alarm to remind a user to record the running state of the model and take measures to perfect the running state.
In the operation process of the machine learning model, the feature structure of the current model does not necessarily adapt to the current application environment, and therefore the feature of the model needs to be updated reasonably, in step S2, illustratively, the second feature is fused with the first feature to obtain the Value detection index of the second model, the Value detection index is compared with the Value threshold to obtain the Value decision index, for example, the second feature can be mined in a new sample of a new data source, the second feature is fused with the first feature to form the second model with a new feature composition structure, the second model is used while the first model is used, the predicted data in a certain period is obtained through the second model and compared with the real data, the performance of the second model is analyzed based on IV (Information Value), the degree of distinction between different labels of different feature indexes is evaluated, and obtaining a value detection index, comparing the value detection index with a value threshold, and when the value detection index is larger than the value threshold, if the second characteristic is added into the model when the model is updated, the efficiency of the model can be improved, namely, reasonable guidance can be provided for the characteristic update of the model through the method, and the efficiency of the model is effectively improved.
In other implementation processes, after the attributes and states of each feature are obtained, a real-time attribution mechanism of a production sample can be established from the aspect of the feature index by adopting a Charapril value method, the model is explained, the running state of the model is monitored from the aspect of the data source quality layer, the relation between the prediction result and the index feature is disclosed, and the interpretability of the model is improved.
After the decision indexes of the various features are obtained, the original model may be updated, and in step S3, it is exemplarily illustrated that the first model is updated according to the multidimensional decision index and the value decision index, for example, when the features change and the score of the model decreases, the latest sample of the similar period corresponding to the existing data may be added based on the existing data to further train the model, in some implementation processes, when the IV value of the second model is greater than the threshold, the first feature of the original first model may be partially deleted, and then a part of the second feature is fused with the updated first feature to form a new feature structure, and the new feature structure replaces the original old feature structure, so as to reasonably update the original model.
In monitoring the operation quality of the model, a plurality of indicators of the characteristic may be detected and evaluated, and in some embodiments, as shown in fig. 3, the multidimensional detection is performed on the first model and the first characteristic of the first model, and the step of obtaining the multidimensional detection indicator includes:
s11: acquiring first real data, performing first detection on the first characteristic, counting a missing value of the first characteristic in the first real data, and acquiring a first detection index;
s12: and performing second detection on the first feature to obtain a first median of the first feature in the first real data and a second median of the feature in training data, and obtaining a second detection index according to the first median and the second median.
Through the steps, a plurality of indexes of the characteristics can be detected, so that the reasonability and the effectiveness of monitoring the running quality of the model are improved.
As shown in fig. 3, in step S11, it is exemplarily illustrated that first real data is obtained, a first detection is performed on the first feature, missing values of the first feature in the first real data are counted, a first detection index is obtained, for example, real data of a day may be obtained, missing values of the first feature in the real data are calculated, and in some implementations, a single-value ratio of the first feature in the real data may also be obtained as one of the multi-dimensional detection indexes.
As shown in fig. 3, in step S12, it is exemplarily illustrated that the second detection is performed on the first feature, a first Median of the first feature in the first real data and a second Median of the feature in the training data are obtained, the second detection index is obtained, for example, the real data and the predicted data of one week can be obtained, the first Median of the first feature in the real data and the second Median of the predicted data are obtained, the second Median is subtracted from the first Median by using an MSR (Median Shift Ratio) calculation method, and the second detection index is obtained by dividing the subtraction result by the second Median.
After obtaining the multidimensional detection index of the first feature, the multidimensional detection index needs to be compared with a multidimensional threshold to obtain a multidimensional decision index, as shown in fig. 4, in some embodiments, the step of comparing the multidimensional detection index with a preset multidimensional threshold to obtain the multidimensional decision index includes:
s21: judging whether the first detection index is larger than a first threshold value, if so, acquiring a first forward decision index, and if not, acquiring a first reverse decision index;
s22: and judging whether the second detection index is larger than a second threshold value, if so, acquiring a second forward decision index, and if not, acquiring a second reverse decision index.
Through the steps, a multidimensional decision index can be obtained, so that the operation quality of the model can be monitored.
As shown in fig. 4, in step S21, it is exemplarily illustrated that whether the first detection index is greater than the first threshold is determined, for example, the first threshold may be set to 90%, when the first detection index, that is, the missing value proportion of the first feature, is greater than 90%, the first forward decision index is obtained, that is, the quality of the feature is considered to be degraded, and the user may be prompted to pay attention to the state, in some implementations, the first threshold may also be set to 80%, a single value proportion of the first feature is obtained, and when the single value proportion is greater than 80%, the quality of the feature is considered to be degraded, and the user may be prompted to pay attention to the state, so as to monitor the operation quality of the model.
As shown in fig. 4, in step S22, it is exemplarily described that whether the second detection index is greater than the second threshold is determined, if yes, the second forward decision index is obtained, and if not, the second reverse decision index is obtained, for example, the second threshold may be set to 500%, and when the second detection index, that is, the MSR coefficient of the first feature is greater than 500%, the second forward decision index is obtained, and it is considered that the variation ratio of the feature is large to negatively affect the operation quality of the model, and a user may be reminded to pay attention to the state, so as to monitor the operation quality of the model.
In some embodiments, the stability indicator of the first model may also be directly evaluated, as shown in fig. 5, the multidimensional detection is performed on the first model and the first feature of the first model, and the step of obtaining the multidimensional detection indicator further includes:
s31: acquiring first real data in a first period, and performing third detection on the first model according to the first real data to acquire a stability index of the first model;
s32: and acquiring a third detection index according to the median of the stability index.
Through the steps, the stability index of the first model can be directly evaluated, and the rationality of monitoring the operation quality of the model is further improved.
As shown in fig. 5, in step S31, it is exemplarily illustrated that first real data in a first period is obtained, a third detection is performed on the first model, and a stability indicator of the first model is obtained, for example, a PSI indicator is used to evaluate the overall distribution variation of the model, and periodic data in a certain period, for example, real data for 10 consecutive days is selected, and a stability indicator value of the real data of the first model on each day is calculated by using the PSI.
As shown in fig. 5, in step S32, it is exemplarily illustrated that the third detection index is obtained according to the median of the stability index, for example, the real data of the last 20 days can be obtained, the median is obtained according to the PSI value of the real data of the first feature in each day, and the median is used as the third detection index, so as to enrich the rationality and diversity of monitoring the operation quality of the model.
In some embodiments, after obtaining the third detection index, the third detection index needs to be compared with a third threshold, as shown in fig. 6, the step of comparing the multidimensional detection index with a preset multidimensional threshold includes the steps of:
s41: and judging whether the third detection index is larger than a third threshold value, if so, acquiring a third forward decision index, and if not, acquiring a third reverse decision index.
Through the steps, the third detection index can be compared with the third threshold value, and more parameter bases are provided for the operation quality of the evaluation model.
As shown in fig. 6, in step S41, it is exemplarily described that it is determined whether the third detection index is larger than the third threshold, for example, the third threshold may be set to 0.15, and when the third detection index is larger than 0.15, it is considered that the model score has a large variation, and the user may be prompted to pay attention to the state and take measures to improve the state.
In some embodiments, a new feature may be selected, and a value evaluation may be performed on a model formed by the new feature, as shown in fig. 7, a value detection indicator of the second model is obtained, the value detection indicator is compared with a preset value threshold, and the step of obtaining a value decision indicator further includes:
s51: acquiring second real data in a second period, and acquiring a value detection index of the second model according to the second real data;
s52: and judging whether the value detection index is larger than the value threshold value, if so, obtaining a value forward decision index, and if not, obtaining a value reverse decision index.
Through the steps, new features can be selected, and value evaluation is performed on the model formed by the new features, so that a reasonable guidance suggestion is provided for iterative updating of the model.
As shown in fig. 7, in step S51, for example, second real data in a second period is obtained, a value detection index of a second model is obtained according to the second real data, for example, performance analysis is performed on the second model by using an IV index, in some implementations, for a two-class machine learning model, a good label and a bad label of the model are distinguished according to a characteristic index, reflow data containing the good label and the bad label in a period of time is assumed to be accumulated after the model is online, the reflow data is equally divided into k parts according to the order of the library falling time, and an IV value of the k parts of data is weighted and calculated as:
wherein i is an index, k represents the number of copies, IVi is the IV value of the ith data block, adjustedAverageIV is the weighted average information value, and adjustedAverageIV is used as the value detection index of the second model.
As shown in fig. 7, in step S52, it is exemplarily illustrated that whether the value detection indicator is greater than the value threshold is determined, for example, the value threshold may be set to 0.1, and when the value detection indicator is greater than 0.1, the value forward decision indicator is output, and it is considered that the performance of the second model with the second feature is improved, and the second feature may be considered to be fused with the first feature, so as to provide a rationality guidance suggestion for the update iteration of the model.
In some embodiments, the first model may be updated according to a multidimensional decision index and a value decision index, as illustrated in fig. 8, the step of updating the first model includes:
s61: deleting the first characteristic to obtain a third characteristic;
s62: deleting the second characteristics to obtain fourth characteristics;
s63: and fusing the third feature and the fourth feature to serve as the feature of the first model.
It will be appreciated that the first, second, third and fourth features each comprise a plurality of features, the first, second, third, fourth, etc. being used for descriptive purposes only and not to be construed as indicating or implying any relative importance, e.g., the first, second, third and fourth features each being a feature of a plurality/lot or set of features.
Through the steps, the first model can be updated according to the multi-dimensional decision index and the value decision index, the cost generated by manually maintaining the model is reduced, and the efficiency is improved.
As shown in fig. 8, according to the first decision index, the second decision index, and the third decision index, the first feature may be deleted, the feature that negatively affects the operation quality of the model may be deleted, the feature that cannot obtain a larger tag degree of distinction may be deleted from the second feature according to the value decision index, the updated first feature may be fused with the updated second feature, and the feature structure of the first model may be updated.
In one embodiment, as shown in fig. 9, there is provided a machine learning model management apparatus including:
the first acquisition module is used for carrying out multi-dimensional detection on a first model and first characteristics of the first model to obtain a multi-dimensional detection index, and comparing the multi-dimensional detection index with a preset multi-dimensional threshold to obtain a multi-dimensional decision index;
the second acquisition module is used for acquiring a second feature from a new sample, fusing the second feature with the first feature, acquiring a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold value to acquire a value decision index;
and the updating module is used for updating the first model according to the multidimensional decision index and the value decision index.
In the first obtaining module, it is exemplarily illustrated that multidimensional detection is performed on the first model and the first feature to obtain a multidimensional detection index, and a multidimensional decision index is obtained according to the multidimensional detection index, for example, the quality and the variation of the first feature can be detected, when the detection result is greater than a preset threshold, it is considered that the first feature has an abnormal condition, that is, information that the operation quality of the model is abnormal can be obtained in real time, the stability of the model can also be comprehensively evaluated, real data stored in a certain period is divided according to the unit of the period, for example, 15 days of real data is obtained, the real data is divided according to each day, PSI calculation is performed on the real data of each day, the stability of the model is analyzed according to the PSI value of each day, if the PSI value is greater than the preset threshold, the score of the model is considered to be changed greatly, the operation quality is reduced, and an alarm can be given to remind a user to record the running state of the model and take measures to perfect the running state.
In the second obtaining module, it is exemplarily illustrated that the second feature is fused with the first feature to obtain a value detection indicator of the second model, the value detection indicator is compared with a value threshold to obtain a value decision indicator, for example, second features can be mined from a new data sample, the second features are fused with the first features to form a second model with a new feature composition structure, the performance of the second model is analyzed based on IV, the discrimination of different feature indexes among different labels is evaluated, a value detection index is obtained, the value detection index is compared with a value threshold, when the value detection index is larger than the value threshold value, the efficiency of the model can be improved if the second characteristic is added into the model when the model is updated, therefore, reasonable guidance can be provided for the feature update of the model, and the efficiency of the model is effectively improved.
In the updating module, it is exemplarily illustrated that the first model is updated according to the multidimensional decision index and the value decision index, for example, when the feature changes and the score of the model decreases, the model may be further trained by adding the latest samples of the similar period corresponding to the existing data based on the existing data.
The device can be applied to a financial service platform, potential fraud risks, credit risks and legal risks possibly generated in the stages of sale, loan and payment are controlled, multidimensional decision indexes are obtained by carrying out multidimensional detection on the first model and the first characteristics, the second characteristics and the first characteristics are fused to obtain a second model, the first model is updated according to the multidimensional decision indexes and the value decision indexes of the second model, all characteristics of the risk assessment model are explained, quantitative criteria are established to quantitatively control the operation quality of the model, and the problems of high cost, low efficiency and the like of machine learning model management are solved.
For specific limitations of the machine learning model management apparatus, reference may be made to the above limitations of the storage cluster interconnection method, which is not described herein again. The modules in the machine learning model management device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data managed by the machine learning model. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a machine learning model management method.
Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A machine learning model management method, comprising:
carrying out multidimensional detection on a first model and first characteristics of the first model to obtain a multidimensional detection index, and comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index;
acquiring a second feature from a new sample, fusing the second feature with the first feature to acquire a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold to acquire a value decision index;
and updating the first model according to the multidimensional decision index and the value decision index.
2. The machine learning model management method according to claim 1, wherein the step of performing multidimensional detection on the first model and the first feature of the first model to obtain a multidimensional detection index includes:
acquiring first real data, performing first detection on the first characteristic, counting a missing value of the first characteristic in the first real data, and acquiring a first detection index;
and performing second detection on the first feature to obtain a first median of the first feature in the first real data and a second median of the feature in training data, and obtaining a second detection index according to the first median and the second median.
3. The machine learning model management method of claim 2, wherein the step of comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index comprises:
judging whether the first detection index is larger than a first threshold value, if so, acquiring a first forward decision index, and if not, acquiring a first reverse decision index;
and judging whether the second detection index is larger than a second threshold value, if so, acquiring a second forward decision index, and if not, acquiring a second reverse decision index.
4. The machine learning model management method according to claim 1 or 2, wherein the step of performing multidimensional detection on the first model and the first feature of the first model to obtain a multidimensional detection index includes:
acquiring first real data in a first period, and performing third detection on the first model according to the first real data to acquire a stability index of the first model;
and acquiring a third detection index according to the median of the stability index.
5. The machine learning model management method of claim 4, wherein the step of comparing the multidimensional detection index with a preset multidimensional threshold to obtain a multidimensional decision index comprises:
and judging whether the third detection index is larger than a third threshold value, if so, acquiring a third forward decision index, and if not, acquiring a third reverse decision index.
6. The machine learning model management method of claim 1, wherein the step of obtaining a value detection indicator of the second model, comparing the value detection indicator with a preset value threshold, and obtaining a value decision indicator comprises:
acquiring second real data in a second period, and acquiring a value detection index of the second model according to the second real data;
and judging whether the value detection index is larger than the value threshold value, if so, obtaining a value forward decision index, and if not, obtaining a value reverse decision index.
7. The machine learning model management method of claim 1, wherein the step of updating the first model comprises:
deleting the first characteristic to obtain a third characteristic;
deleting the second characteristics to obtain fourth characteristics;
and fusing the third feature and the fourth feature to serve as the feature of the first model.
8. A machine learning model management apparatus, comprising:
the first acquisition module is used for carrying out multi-dimensional detection on a first model and first characteristics of the first model to obtain a multi-dimensional detection index, and comparing the multi-dimensional detection index with a preset multi-dimensional threshold to obtain a multi-dimensional decision index;
the second acquisition module is used for acquiring a second feature from a new sample, fusing the second feature with the first feature, acquiring a value detection index of a second model and the second model, and comparing the value detection index with a preset value threshold value to acquire a value decision index;
and the updating module is used for updating the first model according to the multidimensional decision index and the value decision index.
9. Computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the machine learning model management apparatus method of any one of claims 1 to 7 are implemented by the processor when executing the computer program.
10. Computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the machine learning model management apparatus method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666441.1A CN114386619A (en) | 2021-12-31 | 2021-12-31 | Machine learning model management method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666441.1A CN114386619A (en) | 2021-12-31 | 2021-12-31 | Machine learning model management method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114386619A true CN114386619A (en) | 2022-04-22 |
Family
ID=81200114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111666441.1A Pending CN114386619A (en) | 2021-12-31 | 2021-12-31 | Machine learning model management method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114386619A (en) |
-
2021
- 2021-12-31 CN CN202111666441.1A patent/CN114386619A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021164465A1 (en) | Intelligent early warning method and system | |
| CN109657917B (en) | Risk early warning method and device for evaluation object, computer equipment and storage medium | |
| US11748227B2 (en) | Proactive information technology infrastructure management | |
| US20120078678A1 (en) | Method and system for estimation and analysis of operational parameters in workflow processes | |
| CN110389970B (en) | User intention prediction method, device, computer equipment and storage medium | |
| CN109886554B (en) | Illegal behavior discrimination method, device, computer equipment and storage medium | |
| CN109992473B (en) | Application system monitoring method, device, equipment and storage medium | |
| CN111145006A (en) | Automobile financial anti-fraud model training method and device based on user portrait | |
| CN111861729A (en) | Behavior scoring system and method based on lstm | |
| CN117196322B (en) | Intelligent wind control method, intelligent wind control device, computer equipment and storage medium | |
| CN114187009A (en) | Feature interpretation method, device, equipment and medium of transaction risk prediction model | |
| JPWO2020157927A1 (en) | Diagnostic system and diagnostic method | |
| CN116664306A (en) | Intelligent recommendation method and device for wind control rules, electronic equipment and medium | |
| CN114386619A (en) | Machine learning model management method, device, computer equipment and storage medium | |
| CN114495137B (en) | Bill abnormity detection model generation method and bill abnormity detection method | |
| CN113011596A (en) | Method, device and system for automatically updating model and electronic equipment | |
| CN114676167B (en) | User persistence model training method, user persistence prediction method and device | |
| Yeh et al. | Predicting failure of P2P lending platforms through machine learning: The case in China | |
| HB et al. | Machine Learning Approaches for Company Share Rate Prediction. | |
| Abimbola et al. | Predicting the Trend of Dollar/Naira Exchange Rate Using Regression Model and Support Vector Machine | |
| ZXhang et al. | LON: SMT SCOTTISH MORTGAGE INVESTMENT TRUST PLC | |
| CN117974321A (en) | Financial product risk management and control method based on rule engine | |
| Shrivastava et al. | INFRANET: Forecasting intermittent time series using DeepNet with parameterized conditional demand and size distribution | |
| CN117473268A (en) | Threshold prediction method, system, equipment and storage medium | |
| CN117744033A (en) | Data fluctuation early warning method and device, nonvolatile storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |