CN112508202B - Method and device for adjusting model stability and electronic equipment - Google Patents
Method and device for adjusting model stability and electronic equipment Download PDFInfo
- Publication number
- CN112508202B CN112508202B CN202110168787.2A CN202110168787A CN112508202B CN 112508202 B CN112508202 B CN 112508202B CN 202110168787 A CN202110168787 A CN 202110168787A CN 112508202 B CN112508202 B CN 112508202B
- Authority
- CN
- China
- Prior art keywords
- value
- model
- shap
- ndps
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method, a device and electronic equipment for adjusting model stability, wherein the method comprises the following steps: receiving alarm information that the samples have distribution difference; acquiring SHAP values of all characteristics in the current model sample; converting the SHAP value to an NDPS value; the NDPS values represent normalized weight offsets of features in different samples; and adjusting the model according to the NDPS value. According to the method, when the samples have distribution difference, SHAP fingers of all characteristics in the current samples of the model are converted into NDPS values capable of representing normalized weight offsets of the characteristics in different samples, and then the model is adjusted according to the NDPS values, so that the stability of the model is improved. Compared with the prior art, the method and the device have the advantages that the model does not need to be retrained again through a new sample, the stability of the model can be rapidly and efficiently improved, and manpower and time are saved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a device for adjusting model stability and electronic equipment.
Background
As a branch and implementation method of artificial intelligence, machine learning is widely applied to the fields of machine vision, speech recognition, natural language processing, data mining, risk control and the like. And (4) predicting and deciding the data by using the model according to the sample data model through machine learning.
Since the model is developed by sample data of a specific period, the population (test sample) of the model test may change with time, for example, the population (development sample) is white-collar population, and then the model is mainly applied to the population of test students. Due to the fact that different crowd samples have distribution differences, the model with poor stability cannot well distinguish new crowds, and the model needs to be updated.
At present, the model is updated mainly by acquiring new crowd sample data to retrain the model so as to improve the stability of the model. However, this method requires new sample data of the population to be collected, cleaned, labeled, and the like, which is time-consuming, labor-consuming, and inefficient.
Disclosure of Invention
The invention aims to solve the technical problems that in the prior art, for new and old samples with distribution differences, the model needs to be retrained again through the new samples to improve the stability of the model, the time and energy of developers are wasted, and the efficiency is low.
In order to solve the above technical problem, a first aspect of the present invention provides a method for adjusting model stability, which is used for improving model stability when there is a distribution difference in a sample, and the method includes:
receiving alarm information that the samples have distribution difference;
acquiring SHAP values of all characteristics in the current model sample;
converting the SHAP value to an NDPS value; the NDPS values represent normalized weight offsets of features in different samples;
and adjusting the model according to the NDPS value.
According to a preferred embodiment of the present invention, the converting the SHAP value into an NDPS value includes:
sorting the SHAP values of all the characteristics in the current sample according to a specified sequence to obtain the actual sorting value rank of the ith characteristicact,i ；
Obtaining the reference SHAP value of each feature in the model, and sequencing the reference SHAP values of each feature according to a specified sequence to obtain the reference sequencing value rank of the ith featureBZ,i；
The NDPS value of the ith feature is obtained by the following formula:
wherein: and N is the total number of the characteristics of the model.
According to a preferred embodiment of the present invention, after receiving the alarm information that there is a distribution difference in the sample, the method further includes:
determining a target alarm type corresponding to the alarm information according to a preset alarm rule;
acquiring a target model adjustment strategy corresponding to the target alarm type according to the corresponding relation between the alarm type and the model adjustment strategy;
the adjusting the model according to the NDPS value specifically includes:
and adjusting the model according to the NDPS value and the target model adjusting strategy.
According to a preferred embodiment of the present invention, the preset alarm rules are used to classify the alarm information into different types according to different fluctuation indexes.
According to a preferred embodiment of the present invention, the fluctuation index includes at least one of a stability index of the model and a throughput index of the model.
According to a preferred embodiment of the present invention, the adjusting the model according to the NDPS value and the target model adjustment strategy includes:
inputting the NDPS value into a target model adjustment strategy;
outputting the characteristics to be adjusted of the target model adjustment strategy;
and adjusting the characteristic to be adjusted.
In order to solve the above technical problem, a second aspect of the present invention provides an apparatus for adjusting model stability, which is used for improving model stability when there is a distribution difference in a sample, the apparatus comprising:
the receiving module is used for receiving alarm information of samples with distribution differences;
the acquisition module is used for acquiring SHAP values of all the characteristics in the current model sample;
a conversion module, configured to convert the SHAP value into an NDPS value; the NDPS values represent normalized weight offsets of features in different samples;
and the adjusting module is used for adjusting the model according to the NDPS value.
According to a preferred embodiment of the invention, the conversion module comprises:
a first sorting module, configured to sort the SHAP values of the features in the current sample according to a specified order to obtain an actual sorting value rank of the ith featureact,i；
A second sorting module, configured to obtain a reference SHAP value of each feature in the model, and sort the reference SHAP values of each feature according to a specified order to obtain a reference sorted value rank of the ith featureBZ,i；
A scaling module for obtaining the NDPS value of the ith feature by the following formula:
wherein: and N is the total number of the characteristics of the model.
According to a preferred embodiment of the invention, the device further comprises:
the determining module is used for determining a target alarm type corresponding to the alarm information according to a preset alarm rule;
the sub-acquisition module is used for acquiring a target model adjustment strategy corresponding to the target alarm type according to the corresponding relation between the alarm type and the model adjustment strategy;
the adjusting module is specifically configured to adjust the model according to the NDPS value and the target model adjusting policy.
According to a preferred embodiment of the present invention, the preset alarm rules are used to classify the alarm information into different types according to different fluctuation indexes.
According to a preferred embodiment of the present invention, the fluctuation index includes at least one of a stability index of the model and a throughput index of the model.
According to a preferred embodiment of the invention, the adjustment module comprises:
the input module is used for inputting the NDPS value into a target model adjustment strategy;
the output module is used for outputting the characteristics to be adjusted of the target model adjustment strategy;
and the sub-adjusting module is used for adjusting the characteristics to be adjusted.
To solve the above technical problem, a third aspect of the present invention provides an electronic device, comprising:
a processor; and
a memory storing computer executable instructions that, when executed, cause the processor to perform the method described above.
To solve the above technical problems, a fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs which, when executed by a processor, implement the above method.
According to the method, when the distribution difference exists in the samples, SHAP fingers of all characteristics in the current samples of the model are converted into NDPS values capable of representing normalized weight offsets of the characteristics in different samples, the influence of the distribution difference of the samples on the characteristic weights is eliminated, so that the characteristics which cause instability of the model are locked, and the model is adjusted according to the NDPS values, so that the stability of the model is improved. Compared with the prior art, the method and the device have the advantages that the model does not need to be retrained again through a new sample, the stability of the model can be rapidly and efficiently improved, and manpower and time are saved.
Drawings
In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects obtained more clear, the following will describe in detail the embodiments of the present invention with reference to the accompanying drawings. It should be noted, however, that the drawings described below are only illustrations of exemplary embodiments of the invention, from which other embodiments can be derived by those skilled in the art without inventive step.
FIG. 1 is a schematic flow chart of a method of adjusting model stability according to the present invention;
FIG. 2 is a diagram illustrating a corresponding relationship between preset alarm rules, alarm types, and model adjustment strategies according to the present invention;
FIG. 3 is a schematic structural framework of an apparatus for adjusting the stability of a model according to the present invention;
FIG. 4 is a block diagram of an exemplary embodiment of an electronic device in accordance with the present invention;
FIG. 5 is a schematic diagram of one embodiment of a computer-readable medium of the present invention.
Detailed Description
Exemplary embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention may be embodied in many specific forms, and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art.
The structures, properties, effects or other characteristics described in a certain embodiment may be combined in any suitable manner in one or more other embodiments, while still complying with the technical idea of the invention.
In describing particular embodiments, specific details of structures, properties, effects, or other features are set forth in order to provide a thorough understanding of the embodiments by one skilled in the art. However, it is not excluded that a person skilled in the art may implement the invention in a specific case without the above-described structures, performances, effects or other features.
The flow chart in the drawings is only an exemplary flow demonstration, and does not represent that all the contents, operations and steps in the flow chart are necessarily included in the scheme of the invention, nor does it represent that the execution is necessarily performed in the order shown in the drawings. For example, some operations/steps in the flowcharts may be divided, some operations/steps may be combined or partially combined, and the like, and the execution order shown in the flowcharts may be changed according to actual situations without departing from the gist of the present invention.
The block diagrams in the figures generally represent functional entities and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The same reference numerals denote the same or similar elements, components, or parts throughout the drawings, and thus, a repetitive description thereof may be omitted hereinafter. It will be further understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these elements, components, or sections should not be limited by these terms. That is, these phrases are used only to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the present invention. Furthermore, the term "and/or", "and/or" is intended to include all combinations of any one or more of the listed items.
Referring to fig. 1, fig. 1 is a flowchart of a method for adjusting model stability according to the present invention, where the method is used to improve model stability when there is a distribution difference between samples. As shown in fig. 1, the method includes:
s1, receiving alarm information that the samples have distribution differences;
in the invention, the sample distribution difference specifically means that the developed sample and the test sample of the model have distribution difference. For example, the development sample is a white-collar crowd, and the test sample is a blue-collar crowd, and generally, the feature distribution of the white-collar crowd and the blue-collar crowd is different.
The invention can determine whether the sample has distribution difference by monitoring and analyzing the fluctuation index of the model, and if the sample has the distribution difference, alarm information is sent out according to the preset alarm rule. The fluctuation index may be a stability index of the model, or may be an index related to the service, such as throughput of the model. Among them, the stability index (PSI) can measure the distribution difference between the test sample and the development sample, and is the most common model stability evaluation index. The throughput of the model refers to the current number of samples of the model.
After receiving alarm information with distribution difference of samples, a target model adjustment strategy corresponding to the alarm information can be obtained, and the method specifically comprises the following steps:
s11, determining a target alarm type corresponding to the alarm information according to a preset alarm rule;
the preset alarm rule is used for dividing alarm information into different types according to different fluctuation indexes. Therefore, the fluctuation index in the alarm information is analyzed firstly; and determining the target alarm type according to the fluctuation index in the alarm information.
For example, as shown in fig. 2, if the fluctuation index is a PSI absolute value and the fluctuation of the PSI absolute value is greater than a first preset fluctuation range, it is determined that the target alarm type corresponding to the alarm information is a first alarm type; the PSI absolute value fluctuation refers to the difference between the current PSI value of the model and the standard PSI value of the model; namely, when the difference between the current PSI value of the model and the standard PSI value of the model exceeds a first preset fluctuation range (for example, exceeds 20% of the standard PSI value), the target alarm type corresponding to the alarm information is the first alarm type.
If the fluctuation index is a PSI relative value and the fluctuation of the PSI relative value is larger than a second preset fluctuation range, determining that the target alarm type corresponding to the alarm information is a second alarm type; the PSI relative value fluctuation refers to a difference value between a PSI value of the model in a first time period and a PSI value of the model in a second time period; for example, the difference between the PSI value of the model at yesterday and the PSI value of the model at the previous day exceeds a second preset fluctuation range (for example, exceeds the PSI value at yesterday by 15%), and the target alarm type corresponding to the alarm information is a second alarm type.
If the fluctuation index is a service index and the fluctuation of the service index is larger than a third preset fluctuation range, determining that the target alarm type corresponding to the alarm information is a third alarm type; the service index fluctuation refers to a difference value between a service index value in the third time period and a service index value in the fourth time period. The third time period may be the same as or different from the first time period, and the fourth time period may be the same as or different from the second time period, which is not limited in the present invention. The service index is determined according to actual service, such as wind control service, and the model throughput can be selected as the service index. For example, the difference between the throughput of the model of yesterday and the throughput of the model of the previous day exceeds a third preset fluctuation range (for example, exceeds 10% of the throughput of the model of yesterday), and the target alarm type corresponding to the alarm information is the third alarm type.
S12, obtaining a target model adjustment strategy corresponding to the target alarm type according to the corresponding relation between the alarm type and the model adjustment strategy;
in the invention, the corresponding relation between the preset alarm rule, the alarm type and the model adjusting strategy can be configured in advance and stored in a designated position (such as a server).
According to the method, SHAP fingers of all characteristics in a current sample of the model are converted into NDPS values capable of expressing normalized weight offsets of the characteristics in different samples, the influence of sample distribution differences on the characteristic weights is eliminated, and the model is adjusted according to the NDPS values, so that the corresponding relation between the alarm type and the model adjusting strategy is configured according to the NDPS values of the characteristics of the model. In a correspondence relationship between an alarm type and a model adjustment policy, as shown in fig. 2, a first model adjustment policy corresponding to the first alarm type is: comparing a reference SHAP value and a DNSP value of the ith feature of the model, and taking the feature that the deviation between the reference SHAP value and the DNSP value is greater than a first threshold value as a feature to be adjusted; the second model adjustment strategy corresponding to the second alarm type is as follows: comparing the DNSP value of the ith model characteristic in the first time period with the DNSP value of the second time period, and taking the characteristic that the deviation of the DNSP value of the first time period and the DNSP value of the second time period is greater than a second threshold value as the characteristic to be adjusted; the third model adjustment strategy corresponding to the third alarm type is as follows: and sorting the DNSP values of the features in the descending order, and taking the feature M before sorting as the feature to be adjusted. The first threshold, the second threshold and the third threshold may be preset according to actual needs.
S2, acquiring SHAP values of all features in the current model sample;
where SHAP is a "model interpretation" package developed by Python, which can interpret the output of any machine learning model. The SHAP constructs an additive interpretation model under the inspiration of cooperative game theory, and all the characteristics are regarded as 'contributors'. For each prediction sample, the model generates a prediction value, and the SHAP value is the weight (contribution) assigned to the prediction value by each feature in the sample. Specifically, the model prediction value can be loaded into the SHAP packet, and the weight of each feature to the prediction value is output, the value range of each feature weight is from negative infinity to positive infinity, and the sum is a probability value between 0 and 1.
S3, converting the SHAP value into an NDPS value;
and the NDPS value represents normalized weight offset of the features in different samples and is used for eliminating the influence of sample distribution difference on the feature weight.
Specifically, the converting the SHAP value into the NDPS value includes:
s31, sorting the SHAP values of the features in the current sample according to the designated sequence to obtain the actual sorting value rank of the ith featureact,i；
Among them, the designated order is preferably in the order from large to small.
S32, obtaining the reference SHAP value of each feature in the model, and sorting the reference SHAP values of each feature according to the designated sequence to obtain the reference sorting value rank of the ith featureBZ,i；
The reference SHAP value of each feature refers to the SHAP value of each feature in the model development sample. Namely, the model loads the predicted values of the development samples into the SHAP packet, and outputs the weight of each feature to the predicted values of the development samples. And step S32 is the same as the designated order in step S31.
S33, obtaining the NDPS value of the ith characteristic through the following formula:
wherein: and N is the total number of the characteristics of the model.
And S4, adjusting the model according to the NDPS value.
Specifically, the NDPS values of the features may be input into a target model adjustment policy to obtain the features to be adjusted of the target model adjustment policy; and adjusting the characteristics to be adjusted. In the invention, the NDPS value represents the NDPS value of the normalized weight offset of the features in different samples, so as to eliminate the influence of the distribution difference of the samples on the feature weight, the features which cause the instability of the model can be locked through the NDPS value, and the stability of the model can be effectively improved through the adjustment of the features (such as removing the features, processing the features from a new way and the like).
Fig. 3 is a schematic structural diagram of an apparatus for adjusting model stability according to the present invention, which is used to improve model stability when there is a distribution difference between samples, and as shown in fig. 3, the apparatus includes:
the receiving module 31 is configured to receive alarm information that the samples have distribution differences;
an obtaining module 32, configured to obtain a SHAP value of each feature in the current sample of the model;
a conversion module 33, configured to convert the SHAP value into an NDPS value; the NDPS values represent normalized weight offsets of features in different samples;
an adjusting module 34, configured to adjust the model according to the NDPS value.
In one embodiment, the conversion module 33 includes:
a first sorting module, configured to sort the SHAP values of the features in the current sample according to a specified order to obtain an actual sorting value rank of the ith featureact,i；
A second sorting module, configured to obtain a reference SHAP value of each feature in the model, and sort the reference SHAP values of each feature according to a specified order to obtain a reference sorted value rank of the ith featureBZ,i；
A scaling module for obtaining the NDPS value of the ith feature by the following formula:
wherein: and N is the total number of the characteristics of the model.
Further, the apparatus further comprises:
the determining module 35 is configured to determine a target alarm type corresponding to the alarm information according to a preset alarm rule;
a sub-obtaining module 36, configured to obtain a target model adjustment policy corresponding to the target alarm type according to a correspondence between the alarm type and the model adjustment policy; correspondingly, the adjusting module 34 is specifically configured to adjust the model according to the NDPS value and the target model adjusting strategy.
In a specific embodiment, the preset alarm rule is used for classifying the alarm information into different types according to different fluctuation indexes. The fluctuation index comprises at least one of a stability index of the model and a throughput index of the model.
Further, the adjusting module 34 includes:
the input module is used for inputting the NDPS value into a target model adjustment strategy;
the output module is used for outputting the characteristics to be adjusted of the target model adjustment strategy;
and the sub-adjusting module is used for adjusting the characteristics to be adjusted.
Those skilled in the art will appreciate that the modules in the above-described embodiments of the apparatus may be distributed as described in the apparatus, and may be correspondingly modified and distributed in one or more apparatuses other than the above-described embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
In the following, embodiments of the electronic device of the present invention are described, which may be regarded as an implementation in physical form for the above-described embodiments of the method and apparatus of the present invention. Details described in the embodiments of the electronic device of the invention should be considered supplementary to the embodiments of the method or apparatus described above; for details which are not disclosed in embodiments of the electronic device of the invention, reference may be made to the above-described embodiments of the method or the apparatus.
Fig. 4 is a block diagram of an exemplary embodiment of an electronic device according to the present invention. The electronic device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 4, the electronic device 400 of the exemplary embodiment is represented in the form of a general-purpose data processing device. The components of electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one memory unit 420, a bus 430 connecting different electronic device components (including the memory unit 420 and the processing unit 410), a display unit 440, and the like.
The storage unit 420 stores a computer-readable program, which may be a code of a source program or a read-only program. The program may be executed by the processing unit 410 such that the processing unit 410 performs the steps of various embodiments of the present invention. For example, the processing unit 410 may perform the steps as shown in fig. 1.
The storage unit 420 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM) 4201 and/or a cache memory unit 4202, and may further include a read only memory unit (ROM) 4203. The storage unit 420 may also include a program/utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including, but not limited to: operating the electronic device, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 430 may be any bus representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 400 may also communicate with one or more external devices 300 (e.g., keyboard, display, network device, bluetooth device, etc.), enable a user to interact with the electronic device 400 via the external devices 300, and/or enable the electronic device 400 to communicate with one or more other data processing devices (e.g., router, modem, etc.). Such communication may occur via input/output (I/O) interfaces 450, and may also occur via a network adapter 460 with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network such as the Internet). The network adapter 460 may communicate with other modules of the electronic device 400 via the bus 430. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in the electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID electronics, tape drives, and data backup storage electronics, among others.
FIG. 5 is a schematic diagram of one computer-readable medium embodiment of the present invention. As shown in fig. 5, the computer program may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic device, apparatus, or device that is electronic, magnetic, optical, electromagnetic, infrared, or semiconductor, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The computer program, when executed by one or more data processing devices, enables the computer-readable medium to implement the above-described method of the invention, namely: receiving alarm information that the samples have distribution difference; acquiring SHAP values of all characteristics in the current model sample; converting the SHAP value to an NDPS value; the NDPS values represent normalized weight offsets of features in different samples; and adjusting the model according to the NDPS value.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments of the present invention described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a data processing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the present invention.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution electronic device, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including object oriented programming languages such as Java, C + + or the like and conventional procedural programming languages, such as "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the present invention can be implemented as a method, an apparatus, an electronic device, or a computer-readable medium executing a computer program. Some or all of the functions of the present invention may be implemented in practice using a general purpose data processing device such as a microprocessor or a Digital Signal Processor (DSP).
While the foregoing embodiments have described the objects, aspects and advantages of the present invention in further detail, it should be understood that the present invention is not inherently related to any particular computer, virtual machine or electronic device, and various general-purpose machines may be used to implement the present invention. The invention is not to be considered as limited to the specific embodiments thereof, but is to be understood as being modified in all respects, all changes and equivalents that come within the spirit and scope of the invention.
Claims (8)
1. A method for adjusting the stability of a model of a population sample, which is used for improving the stability of the model when the distribution of the population sample is different, the method comprises the following steps:
receiving alarm information of the existence of distribution difference of the crowd samples, wherein the existence of the distribution difference of the crowd samples refers to the existence of the distribution difference between the developed crowd samples and the tested crowd samples of the model;
loading the predicted value of the current population sample of the model into an SHAP packet, and outputting the SHAP value of each characteristic; the SHAP value is the weight assigned to each feature pair prediction value in the population sample;
converting the SHAP value to an NDPS value; the NDPS value represents normalized weight offset of the features in different population samples and is used for eliminating the influence of population sample distribution difference on the feature weight;
adjusting the characteristics to be adjusted of the model according to the NDPS value;
the converting the SHAP value to an NDPS value comprises:
sorting SHAP values of all characteristics in the current crowd sample according to a specified sequence to obtain an actual sorting value rank of the ith characteristicact,i；
Loading the predicted value of the model to the development crowd sample into a SHAP packet, outputting the reference SHAP value of each characteristic, and sequencing the reference SHAP values of each characteristic according to a specified sequence to obtain the reference sequencing value rank of the ith characteristicBZ,i(ii) a The reference SHAP value of each feature refers to the SHAP value of each feature in the model development crowd sample;
the NDPS value of the ith feature is obtained by the following formula:
wherein: and N is the total number of the characteristics of the model.
2. The method of claim 1, wherein after receiving the alarm information of the people sample distribution difference, the method further comprises:
determining a target alarm type corresponding to the alarm information according to a preset alarm rule;
acquiring a target model adjustment strategy corresponding to the target alarm type according to the corresponding relation between the alarm type and the model adjustment strategy;
the adjusting the characteristics to be adjusted of the model according to the NDPS value specifically comprises the following steps:
and adjusting the model according to the NDPS value and the characteristic to be adjusted of the target model adjusting strategy.
3. The method according to claim 2, wherein the preset alarm rules are used for classifying alarm information into different types according to different fluctuation indexes.
4. The method of claim 3, wherein the fluctuation indicator comprises at least one of a stability indicator of the model and a throughput indicator of the model.
5. The method of claim 2, wherein adjusting the model according to the NDPS value and the feature to be adjusted of the target model adjustment strategy comprises:
inputting the NDPS value into a target model adjustment strategy;
outputting the characteristics to be adjusted of the target model adjustment strategy;
and adjusting the characteristic to be adjusted.
6. An apparatus for adjusting the stability of a population sample model, which is used for improving the stability of the model when there is a distribution difference in the population sample, the apparatus comprising:
the receiving module is used for receiving alarm information that the crowd samples have distribution difference, wherein the crowd samples have distribution difference, which means that the developed crowd samples and the tested crowd samples of the model have distribution difference;
the acquisition module is used for loading the predicted value of the current crowd sample of the model into the SHAP packet and outputting the SHAP value of each characteristic; the SHAP value is the weight assigned to each feature pair prediction value in the population sample;
a conversion module, configured to convert the SHAP value into an NDPS value; the NDPS value represents normalized weight offset of the features in different population samples and is used for eliminating the influence of population sample distribution difference on the feature weight;
the adjusting module is used for adjusting the characteristics to be adjusted of the model according to the NDPS value;
wherein the conversion module comprises:
a first sorting module, configured to sort the SHAP values of the features in the current crowd sample according to a specified order to obtain an actual sorting value rank of the ith featureact,i；
A second sorting module, configured to load the predicted value of the model for the development crowd sample into the SHAP packet, output the reference SHAP value of each feature, and sort the reference SHAP values of each feature according to a specified order to obtain a reference sorting value rank of the ith featureBZ,i(ii) a Wherein the reference SHAP value of each feature refers to each feature in the model development crowd sampleThe SHAP value of (1);
a scaling module for obtaining the NDPS value of the ith feature by the following formula:
wherein: and N is the total number of the characteristics of the model.
7. An electronic device, comprising:
a processor; and
a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-5.
8. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110168787.2A CN112508202B (en) | 2021-02-07 | 2021-02-07 | Method and device for adjusting model stability and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110168787.2A CN112508202B (en) | 2021-02-07 | 2021-02-07 | Method and device for adjusting model stability and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112508202A CN112508202A (en) | 2021-03-16 |
| CN112508202B true CN112508202B (en) | 2021-07-30 |
Family
ID=74952733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110168787.2A Active CN112508202B (en) | 2021-02-07 | 2021-02-07 | Method and device for adjusting model stability and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112508202B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346448A (en) * | 2014-10-28 | 2015-02-11 | 浪潮电子信息产业股份有限公司 | Incremental data mining method based on genetic programming algorithm |
| CN110263934A (en) * | 2019-05-31 | 2019-09-20 | 中国信息通信研究院 | A kind of artificial intelligence data mask method and device |
| CN110489314A (en) * | 2019-07-05 | 2019-11-22 | 中国平安人寿保险股份有限公司 | Model method for detecting abnormality, device, computer equipment and storage medium |
| CN111190967A (en) * | 2019-12-16 | 2020-05-22 | 北京淇瑀信息科技有限公司 | User multi-dimensional data processing method and device and electronic equipment |
| CN111311030A (en) * | 2020-03-27 | 2020-06-19 | 中国工商银行股份有限公司 | User credit risk prediction method and device based on influence factor detection |
| US10706323B1 (en) * | 2019-09-04 | 2020-07-07 | SparkCognition, Inc. | Processor and method of weighted feature importance estimation |
| CN111401788A (en) * | 2020-04-10 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Attribution method and device of service timing sequence index |
| CN111414945A (en) * | 2020-03-12 | 2020-07-14 | 支付宝(杭州)信息技术有限公司 | Method and device for identifying fluctuation of model efficiency |
| CN111860698A (en) * | 2020-08-05 | 2020-10-30 | 中国工商银行股份有限公司 | Method and device for determining stability of learning model |
| CN112015711A (en) * | 2020-10-26 | 2020-12-01 | 北京淇瑀信息科技有限公司 | Model monitoring method and device and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046713B (en) * | 2019-04-19 | 2023-05-12 | 井冈山大学 | Robustness ordering learning method based on multi-target particle swarm optimization and application thereof |
| CN111340231A (en) * | 2020-02-11 | 2020-06-26 | 深圳前海微众银行股份有限公司 | SHAP feature attribution method, device, equipment and readable storage medium |
| CN112101572A (en) * | 2020-09-25 | 2020-12-18 | 北京百度网讯科技有限公司 | Model optimization method, device, equipment and medium |
| CN112101574B (en) * | 2020-11-20 | 2021-03-02 | 成都数联铭品科技有限公司 | Machine learning supervised model interpretation method, system and equipment |
-
2021
- 2021-02-07 CN CN202110168787.2A patent/CN112508202B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346448A (en) * | 2014-10-28 | 2015-02-11 | 浪潮电子信息产业股份有限公司 | Incremental data mining method based on genetic programming algorithm |
| CN110263934A (en) * | 2019-05-31 | 2019-09-20 | 中国信息通信研究院 | A kind of artificial intelligence data mask method and device |
| CN110489314A (en) * | 2019-07-05 | 2019-11-22 | 中国平安人寿保险股份有限公司 | Model method for detecting abnormality, device, computer equipment and storage medium |
| US10706323B1 (en) * | 2019-09-04 | 2020-07-07 | SparkCognition, Inc. | Processor and method of weighted feature importance estimation |
| CN111190967A (en) * | 2019-12-16 | 2020-05-22 | 北京淇瑀信息科技有限公司 | User multi-dimensional data processing method and device and electronic equipment |
| CN111414945A (en) * | 2020-03-12 | 2020-07-14 | 支付宝(杭州)信息技术有限公司 | Method and device for identifying fluctuation of model efficiency |
| CN111311030A (en) * | 2020-03-27 | 2020-06-19 | 中国工商银行股份有限公司 | User credit risk prediction method and device based on influence factor detection |
| CN111401788A (en) * | 2020-04-10 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Attribution method and device of service timing sequence index |
| CN111860698A (en) * | 2020-08-05 | 2020-10-30 | 中国工商银行股份有限公司 | Method and device for determining stability of learning model |
| CN112015711A (en) * | 2020-10-26 | 2020-12-01 | 北京淇瑀信息科技有限公司 | Model monitoring method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112508202A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111241287A (en) | Training method and device for generating generation model of confrontation text | |
| US11640551B2 (en) | Method and apparatus for recommending sample data | |
| EP3893169A2 (en) | Method, apparatus and device for generating model and storage medium | |
| CN110245232B (en) | Text classification method, device, medium and computing equipment | |
| CN112395487B (en) | Information recommendation method and device, computer readable storage medium and electronic equipment | |
| WO2020090413A1 (en) | Classification device, classification method, and classification program | |
| CN110708285B (en) | Flow monitoring method, device, medium and electronic equipment | |
| CN115063875A (en) | Model training method, image processing method, device and electronic equipment | |
| JP2022169743A (en) | Information extraction method and device, electronic equipment, and storage medium | |
| CN111199469A (en) | User payment model generation method and device and electronic equipment | |
| CN113298221A (en) | User risk prediction method and device based on logistic regression and graph neural network | |
| CN115357719A (en) | Power audit text classification method and device based on improved BERT model | |
| CN111626381A (en) | User identification method and device based on anti-migration and electronic equipment | |
| CN111191825A (en) | User default prediction method and device and electronic equipment | |
| CN115221516A (en) | Malicious application program identification method and device, storage medium and electronic equipment | |
| CN112508202B (en) | Method and device for adjusting model stability and electronic equipment | |
| CN113420295A (en) | Malicious software detection method and device | |
| CN110929499B (en) | Text similarity obtaining method, device, medium and electronic equipment | |
| CN110348581B (en) | User feature optimizing method, device, medium and electronic equipment in user feature group | |
| CN113568820A (en) | Method, apparatus, electronic device and medium for monitoring model | |
| CN113448821A (en) | Method and device for identifying engineering defects | |
| CN108733702B (en) | Method, device, electronic equipment and medium for extracting upper and lower relation of user query | |
| JPWO2020144736A1 (en) | Semantic relationship learning device, semantic relationship learning method, and semantic relationship learning program | |
| CN114091570A (en) | Service processing system method, device and electronic equipment | |
| CN115860147A (en) | Customs sheet pre-judging model training method and device based on unbalanced ensemble learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |