CN117150254A - Analysis method for AI model fault influence factors - Google Patents

Analysis method for AI model fault influence factors Download PDF

Info

Publication number
CN117150254A
CN117150254A CN202311193985.XA CN202311193985A CN117150254A CN 117150254 A CN117150254 A CN 117150254A CN 202311193985 A CN202311193985 A CN 202311193985A CN 117150254 A CN117150254 A CN 117150254A
Authority
CN
China
Prior art keywords
less
preset
fault
model
setting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311193985.XA
Other languages
Chinese (zh)
Inventor
吴玉美
曲宇航
李智博
任晨锴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202311193985.XA priority Critical patent/CN117150254A/en
Publication of CN117150254A publication Critical patent/CN117150254A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/27—Regression, e.g. linear or logistic regression

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Complex Calculations (AREA)

Abstract

The invention relates to the technical field of AI model target recognition, and discloses an AI model fault influence factor analysis method, which comprises the following steps: determination of fault influencing factors: determining fault influence factors based on structural features of the AI model, training sample features and model use environment features and combining with an AI model fault criterion model; quantification of AI model fault influencing factors: based on the determined fault influence factors, obtaining a fault influence factor quantification result by adopting a quantification method for the fault influence factors; correlation analysis: and carrying out correlation analysis on the model performance based on multiple linear regression according to the quantification result of the fault influence factors to obtain the magnitude order of the correlation of each fault influence factor on the model performance, thereby obtaining the key fault influence factors. The invention reduces the adverse effect of model faults on application programs and systems and improves the performance and maintainability of the AI system.

Description

Analysis method for AI model fault influence factors
Technical Field
The invention relates to the technical field of AI model target recognition, in particular to an AI model fault influence factor analysis method.
Background
Artificial intelligence (Artificial Intelligence, AI) technology has become an important driving force in the scientific field today, deeply changing our lives, works and society. The AI model is a core component of AI technology, and its performance and stability are critical to practical applications. However, AI models often face various failure problems in different scenarios, such as performance degradation, inaccurate predictions, etc., which may have a serious impact on the application. Therefore, in-depth analysis of fault influencing factors of the AI model, and how to quantify and manage these factors, is one of important research directions in the AI field.
Disclosure of Invention
In order to achieve the above objective, the present invention provides an analysis method for AI model fault influencing factors, so as to solve the technical problem that the conventional AI model cannot quantify and manage various fault factors.
The invention provides an analysis method of AI model fault influence factors, comprising the following steps:
determination of fault influencing factors: determining fault influence factors based on structural features of the AI model, training sample features and model use environment features and combining with an AI model fault criterion model; the fault influencing factors of the AI model include: the system comprises a model structure, a training sample and an environment deployment, wherein the model structure is used for reflecting the structural characteristic aspect fault, the training sample is used for reflecting the training sample characteristic aspect fault, and the environment deployment is used for reflecting the model using environment characteristic aspect fault; the training samples comprise data volume and data balance rate; the model structure comprises a learning rate, regularization parameters and an activation function; the environmental deployment includes a model usage frequency;
Quantification of AI model fault influencing factors: based on the determined fault influence factors, obtaining a fault influence factor quantification result by adopting a quantification method for the fault influence factors;
correlation analysis: and carrying out correlation analysis on the model performance based on multiple linear regression according to the quantification result of the fault influence factors to obtain the magnitude order of the correlation of each fault influence factor on the model performance, thereby obtaining the key fault influence factors.
Further, the determining of the fault influencing factor includes: preset minimum gradient value S min And preset the highest gradient value S max And detecting real-time gradient values S0, and respectively and preset minimum gradient values S based on the real-time gradient values S0 min Preset maximum gradient value S max If S min ≤S0≤S max The learning rate is at a normal value and the fault influencing factor is determined to be a fault in the aspect of non-learning rate; if 0 is less than S min Or S0 > S max When the learning rate is at an abnormal value, and the fault influencing factor is judged to be a fault in the learning rate;
if s0=0, the activation function is at an abnormal value and the fault influencing factor is determined to be a fault in the aspect of the activation function;
presetting minimum model complexity T min And a preset maximum model complexity T max And detecting the real-time model complexity T0, and based on the real-time model complexity T0 and the preset minimum model complexity T respectively min Preset maximum model complexity T max If T is determined by relation of min ≤T0≤T max The regularization parameters are in normal values, and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T min ≤T0≤T max The regularization parameters are in normal values and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T0 is less than T min Or T0 > T max And when the regularization parameter is at an abnormal value, and the fault influencing factor is judged to be a fault in the aspect of the regularization parameter.
Further, the determining of the fault influencing factor further includes: presetting a minimum data amount P min And preset the highest data value P max And detecting the real-time data amount P0, and based on the real-time data amount P0 and the preset minimum data amount P respectively min Presetting the highest data amount P max If P min ≤P0≤P max The data volume is at a normal value and the fault influencing factor is judged to be fault in the aspect of non-data volume; if 0 < P min Or P0 > P max When the data volume is in an abnormal value, and the fault influencing factors are judged to be faulty in terms of the data volume;
counting the sample number ratio of each sample class, counting the sample number ratio of each sample class again after the same time interval, calculating and obtaining the sample number change rate Q0 of each sample class, and presetting the sample number change rate Q of the lowest sample class min Rate of change Q of sample number from preset highest sample class max And based on the sample number change rate Q0 of each sample class and the preset minimum data quantity Q min Preset the highest data quantity Q max If Q is determined by relation of min ≤Q0≤Q max The data quantity balance rate is at a normal value and the fault influencing factor is judged to be a fault in the aspect of non-data quantity balance rate; if 0 is less than Q min Or Q0 > Q max And if the data quantity balance rate is at an abnormal value, the fault influencing factor is judged to be a fault in the aspect of the data quantity balance rate.
Further, the determining of the fault influencing factor further includes: acquiring a using frequency value R0 in a fixed period according to a using log of the model, and presetting a lowest using frequency value R min And preset the highest using frequency value R max And detecting a real-time frequency value R0, and based on the real-time frequency value R0 and a preset minimum frequency value R min Presetting a maximum frequency of use value R max If R is determined by relation of min ≤R0≤R max The frequency of use is at a normal value and the fault influencing factor is determined to be a fault in terms of non-frequency of use; if 0 < R min Or R0 > R max And if the frequency of use is at an abnormal value, determining that the fault influencing factor is fault in the frequency of use.
Further, the quantification of the AI model fault influencing factors includes: presetting a gradient value matrix S, setting S (S1, S2, S3, S4, S5 and S6), wherein S1 is a first preset gradient value, S2 is a second preset gradient value, S3 is a third preset gradient value, S4 is a fourth preset gradient value, S5 is a fifth preset gradient value, S6 is a sixth preset gradient value, and S1 is less than S2 is less than S3 is less than S min ＜S max S4 is more than S5 and less than S6; presetting a gradient value quantization coefficient matrix a, and setting a (a 1, a2, a3, a4, a5 and a 6), wherein a1 is a first preset gradient value quantization coefficient, a2 is a second preset gradient value quantization coefficient, a3 is a third preset gradient value quantization coefficient, a4 is a fourth preset gradient value quantization coefficient, a5 is a fifth preset gradient value quantization coefficient, a6 is a sixth preset gradient value quantization coefficient, a1 is more than a2 and less than a3 and less than a4 and less than a5 and less than a6;
when S0 < S1, setting S0 a1 as a quantized result of the learning rate;
when S1 is less than or equal to S0 and less than S2, setting S0 a2 as a quantized result of the learning rate;
when S2 is less than or equal to S0 and less than S3, setting S0 a3 as a quantized result of the learning rate;
when S3 is less than or equal to S0 and less than S4, setting S0 a4 as a quantized result of the learning rate;
when S4 is less than or equal to S0 and less than S5, setting S0 a5 as a quantized result of the learning rate;
when S5 is less than or equal to S0 and less than S6, setting S0 a6 as a quantized result of the learning rate;
the preset b is the quantization result of the activation function, b=b1 when s0=0, b=0 when b=b1 when s0+.0, and 0 when s0+.0.
Further, the quantification of the AI model fault influencing factors further includes: presetting a regularization parameter matrix T, setting T (T1, T2, T3, T4, T5 and T6), wherein T1 is a first preset regularization parameter, T2 is a second preset regularization parameter, T3 is a third preset regularization parameter, T4 is a fourth preset regularization parameter, T5 is a fifth preset regularization parameter, T6 is a sixth preset regularization parameter, and T1 is more than T2 and less than T3 is less than T min ＜T max T4 is more than T5 and less than T6; a preset regularization parameter quantization coefficient matrix c is set, c (c 1, c2, c3, c4, c5, c 6) is set, wherein c1 is a first preset regularization parameter quantization coefficient, c2 is a second preset regularization parameter quantization coefficient, c3 is a third preset regularization parameter quantization coefficient, c4 is a fourth preset regularization parameter quantization coefficient, c5 is a fifth preset regularization parameter quantization coefficient, c6 is a sixth preset regularization parameter quantization coefficient, and c1 < c2 < c3 < c4＜c5＜c6；
When T0 is less than T1, setting T0 c1 as a quantization result of the regularization parameter;
when T1 is less than or equal to T0 and less than T2, setting T0 c2 as a quantization result of the regularization parameter;
when T2 is less than or equal to T0 and less than T3, setting T0 c3 as a quantization result of the regularization parameter;
when T3 is less than or equal to T0 and less than T4, setting T0 c4 as a quantization result of the regularization parameter;
when T4 is less than or equal to T0 and less than T5, setting T0 c5 as a quantization result of the regularization parameter;
when T5 is less than or equal to T0 and less than T6, setting T0 c6 as the quantization result of the regularization parameter.
Further, the quantification of the AI model fault influencing factors further includes: presetting a data quantity matrix P, setting P (P1, P2, P3, P4, P5 and P6), wherein P1 is a first preset data quantity, P2 is a second preset data quantity, P3 is a third preset data quantity, P4 is a fourth preset data quantity, P5 is a fifth preset data quantity, P6 is a sixth preset data quantity, and P1 is less than P2 and less than P3 is less than P min ＜P max P4 is more than P5 and P6; presetting a data quantity quantization coefficient matrix d, and setting d (d 1, d2, d3, d4, d5 and d 6), wherein d1 is a first preset data quantity quantization coefficient, d2 is a second preset data quantity quantization coefficient, d3 is a third preset data quantity quantization coefficient, d4 is a fourth preset data quantity quantization coefficient, d5 is a fifth preset data quantity quantization coefficient, d6 is a sixth preset data quantity quantization coefficient, d1 is more than d2 and less than d3 and less than d4 and less than d5 and less than d6;
setting P0 d1 as a quantization result of the data amount when P0 < P1;
when P1 is less than or equal to P0 and less than P2, setting P0 d2 as a quantization result of the data quantity;
when P2 is less than or equal to P0 and less than P3, setting P0 d3 as a quantization result of the data quantity;
when P3 is less than or equal to P0 and less than P4, setting P0 d4 as a quantization result of the data quantity;
when P4 is less than or equal to P0 and less than P5, setting P0 d5 as a quantization result of the data quantity;
when P5 is less than or equal to P0 and less than P6, setting P0 d6 as the quantization result of the data quantity.
Further, the quantification of the AI model fault influencing factors further includes: a preset data amount balance rate matrix Q, Q (Q1, Q2, Q3, Q4, Q5, Q6) is set, wherein Q1 is a first preset data amount balance rate, Q2 is a second preset data amount balance rate, Q3 is a third preset data amount balance rate, Q4 is a fourth preset data amount balance rate, Q5 is a fifth preset data amount balance rate, Q6 is a sixth preset data amount balance rate, and Q1 < Q2 < Q3 < Q min ＜Q max Q4 is more than Q5 and less than Q6; presetting a data quantity balance rate quantization coefficient matrix e, setting e (e 1, e2, e3, e4, e5, e 6), wherein e1 is a first preset data quantity balance rate quantization coefficient, e2 is a second preset data quantity balance rate quantization coefficient, e3 is a third preset data quantity balance rate quantization coefficient, e4 is a fourth preset data quantity balance rate quantization coefficient, e5 is a fifth preset data quantity balance rate quantization coefficient, e6 is a sixth preset data quantity balance rate quantization coefficient, and e1 is more than e2 is less than e3 is less than e4 is less than e5 is less than e6;
when Q0 is less than Q1, setting Q0 e1 as a quantization result of the data quantity balance rate;
when Q1 is less than or equal to Q0 and less than Q2, setting Q0 e2 as a quantization result of the data quantity balance rate;
when Q2 is less than or equal to Q0 and less than Q3, setting Q0 e3 as a quantization result of the data quantity balance rate;
when Q3 is less than or equal to Q0 and less than Q4, setting Q0 e4 as a quantization result of the data quantity balance rate;
when Q4 is less than or equal to Q0 and less than Q5, setting Q0 e5 as a quantization result of the data quantity balance rate;
when Q5 is less than or equal to Q0 and less than Q6, setting Q0 and e6 as the quantization result of the data quantity balance rate.
Further, the quantification of the AI model fault influencing factors further includes: setting a preset use frequency matrix R, and setting R (R1, R2, R3, R4, R5 and R6), wherein R1 is a first preset use frequency, R2 is a second preset use frequency, R3 is a third preset use frequency, R4 is a fourth preset use frequency, R5 is a fifth preset use frequency, R6 is a sixth preset use frequency, and R1 is more than R2 and less than R3 and less than R min ＜R max R4 is more than R5 and less than R6; presetting a frequency quantization coefficient matrix f, setting f (f 1, f2, f3, f4, f5, f 6), wherein f1 is a first preset frequency quantization coefficient, f2 is a second preset frequency quantization coefficient, f3 is a third preset frequency quantization coefficient, f4 is a fourth preset frequency quantization coefficient, f5 is a fifth preset frequency quantization coefficient, f6 is a sixth preset frequency quantization coefficient, and f1 < f2 < f3 < f4 < f5 < f6;
when R0 < R1, setting R0 f1 as the quantization result of the use frequency;
when R1 is less than or equal to R0 and less than R2, setting R0 f2 as a quantization result of the use frequency;
when R2 is less than or equal to R0 and less than R3, setting R0 f3 as a quantization result of the use frequency;
when R3 is less than or equal to R0 and less than R4, setting R0 f4 as a quantization result of the use frequency;
when R4 is less than or equal to R0 and less than R5, setting R0 f5 as a quantization result of the use frequency;
when R5 is less than or equal to R0 and less than R6, setting R0 f6 as the quantization result of the use frequency.
Further, the key fault influencing factors are the fault influencing factors with the strongest correlation with the model performance, and the model performance comprises: accuracy, precision, recall, and F1 score.
Further, the AI model fault criteria model is specifically: the principles of AI model fault criteria models are based on monitoring and evaluating AI model performance and behavioral characteristics to identify signs of model faults or performance degradation, involving the following principles:
Defining a fault index: a set of fault indexes are definitely defined, wherein the fault indexes are used for measuring the performance of the model, and the fault indexes can comprise accuracy, loss function values, classification error rates and the like; establishing a reference: a baseline or normal state is established for comparing the current performance of the model. The reference may be a performance level of the model under normal operation; monitoring the behavior of the model, and continuously monitoring the behavior characteristics of the model, including the output of the model, the loss function, the change of the activation function and the like, wherein the behavior characteristics should be kept stable under normal conditions; detecting anomalies, the model fault criteria model may detect anomalies that are different from the baseline, possibly including a drop in performance indicators, changes in behavioral characteristics, or other anomalies;
the steps are as follows: the establishment and application of the AI model fault criterion model comprises the following steps:
data acquisition and preparation: data related to AI model performance is collected, including model inputs, outputs, performance metrics, and the like. The data needs to be preprocessed and cleaned to ensure the data quality; defining fault indicators, explicitly defining indicators for monitoring performance and faults, such as accuracy, loss function values, classification error rates, etc.; benchmarks are established, establishing a performance benchmark, typically the performance level of the model under normal operation. The benchmark is determined based on historical data or domain knowledge; model monitoring, which is to use an AI model fault criterion model to monitor the performance and behavior of the model, including real-time monitoring or periodic inspection; abnormality detection, after which a decrease in model performance or signs of abnormal behavior are detected, the model fault criteria model may trigger an alarm or take automated action. This may be achieved by threshold detection, statistical methods, machine learning algorithms, etc.; iteration and improvement, the model fault criterion model is continuously monitored and improved to adapt to the change of the model performance and the new fault mode.
Compared with the prior art, the analysis method of the AI model fault influence factors has the beneficial effects that:
the method can help developers or maintenance personnel of the AI model to locate the reasons of the model faults more quickly. By identifying key fault influencing factors, they can more specifically address the problem, reducing maintenance and troubleshooting costs and time. By monitoring and quantifying the different factors, this approach helps identify potential problems before a fault occurs. For example, if the learning rate is abnormal, measures may be taken to avoid model failure. Optimizing the model performance, knowing the influence of different factors on the model performance can help to optimize the design and training strategy of the model. For example, by adjusting the learning rate or regularization parameters, the generalization performance of the model can be improved. And the data management and balance, and the quantization result of the data quantity and the data balance rate can guide the data management strategy to ensure that the model performs well on samples of different types. The quantization result of the usage frequency can help optimize the resource allocation of the model, ensure that the model for high frequency usage is supported by sufficient resources, and the model for low frequency usage can more effectively utilize the resources.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method of analyzing AI model fault influencing factors in accordance with an embodiment of the application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be clear and complete, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the description of the present application, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present application.
The terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
In the description of the present application, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present application will be understood in specific cases by those of ordinary skill in the art.
As shown in fig. 1, a method for analyzing AI model fault influencing factors according to a preferred embodiment of the present application includes:
s100: determining fault influence factors, namely determining the fault influence factors based on structural features of an AI model, training sample features and model use environment features and combining an AI model fault criterion model; the fault influencing factors of the AI model include: the system comprises a model structure, a training sample and an environment deployment, wherein the model structure is used for reflecting the structural characteristic aspect fault, the training sample is used for reflecting the training sample characteristic aspect fault, and the environment deployment is used for reflecting the model using environment characteristic aspect fault; the training samples comprise data volume and data balance rate; the model structure comprises a learning rate, regularization parameters and an activation function; the environmental deployment includes a model usage frequency;
S200: quantification of fault influence factors of an AI model, and obtaining quantification results of the fault influence factors by adopting a quantification method on the fault influence factors based on the determined fault influence factors;
s300: and performing correlation analysis on the model performance based on multiple linear regression according to the quantitative result of the fault influence factors by the correlation analysis, and obtaining the magnitude order of the correlation of each fault influence factor to the model performance to obtain the key fault influence factors.
It will be appreciated that the above scheme has the following effects: the method can more accurately locate the root cause of the AI model fault by considering a plurality of factors such as model structure, training samples, environmental deployment and the like. This helps engineers to quickly identify and resolve problems, shortening troubleshooting time. Quantification of problem severity: by quantifying the fault factors, the method can assign a numerical value to each factor, thereby quantifying its extent of impact on the model fault. This makes the severity of the problem easier to understand and compare, helping to prioritize the most serious problem. Automated fault detection: the method provides an automated way to detect a plurality of potential failure factors including learning rate, regularization parameters, data volume, data balance rate, frequency of use, and the like. This helps to reduce the need for manual intervention, especially in the case of large scale deployments. And (3) key factor identification: through correlation analysis, the method can determine which factors have the greatest effect on the performance and stability of the model. This helps to decide which factors should be prioritized when improving the model or application, thereby improving overall efficiency. Improving model maintainability: by identifying and addressing potential sources of failure, the method helps to improve maintainability of the AI model. This means that the model is more stable in long-term operation, reducing unnecessary maintenance and repair work.
In some of these embodiments, the determining of the fault influencing factor comprises: preset minimum gradient value S min And preset the highest gradient value S max And detecting real-time gradient values S0, and respectively and preset minimum gradient values S based on the real-time gradient values S0 min Preset maximum gradient value S max If S min ≤S0≤S max The learning rate is at a normal value and the fault influencing factor is determined to be a fault in the aspect of non-learning rate; if 0 is less than S min Or S0 > S max When the learning rate is at an abnormal value, and the fault influencing factor is judged to be a fault in the learning rate;
if s0=0, the activation function is at an abnormal value and the fault influencing factor is determined to be a fault in the aspect of the activation function;
presetting minimum model complexity T min And a preset maximum model complexity T max And detecting the real-time model complexity T0, and based on the real-time model complexity T0 and the preset minimum model complexity T respectively min Preset maximum model complexity T max If T is determined by relation of min ≤T0≤T max The regularization parameters are in normal values, and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T min ≤T0≤T max The regularization parameters are in normal values and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T0 is less than T min Or T0 > T max And when the regularization parameter is at an abnormal value, and the fault influencing factor is judged to be a fault in the aspect of the regularization parameter.
In some of these embodiments, the determining of the fault influencing factor further comprises: presetting a minimum data amount P min And preset the highest data value P max And detecting the real-time data amount P0, and based on the real-time data amount P0 and the preset minimum data amount P respectively min Presetting the highest data amount P max If P min ≤P0≤P max The data volume is at a normal value and the fault influencing factor is judged to be fault in the aspect of non-data volume; if 0 < P min Or P0 > P max When the data volume is in an abnormal value, and the fault influencing factors are judged to be faulty in terms of the data volume;
counting the sample number ratio of each sample class, counting the sample number ratio of each sample class again after the same time interval, calculating and obtaining the sample number change rate Q0 of each sample class, and presetting the sample number change rate Q of the lowest sample class min Rate of change Q of sample number from preset highest sample class max And based on the sample number change rate Q0 of each sample class and the preset minimum data quantity Q min Preset the highest data quantity Q max If Q is determined by relation of min ≤Q0≤Q max The data quantity balance rate is at a normal value and the fault influencing factor is judged to be a fault in the aspect of non-data quantity balance rate; if 0 is less than Q min Or Q0 > Q max And if the data quantity balance rate is at an abnormal value, the fault influencing factor is judged to be a fault in the aspect of the data quantity balance rate.
In some of these embodiments, the determining of the fault influencing factor further comprises: acquiring a using frequency value R0 in a fixed period according to a using log of the model, and presetting a lowest using frequency value R min And preset the highest using frequency value R max And detecting a real-time frequency value R0, and based on the real-time frequency value R0 and a preset minimum frequency value R min Presetting a maximum frequency of use value R max If R is determined by relation of min ≤R0≤R max The frequency of use is at a normal value and the fault influencing factor is determined to be a fault in terms of non-frequency of use; if 0 < R min Or R0 > R max And if the frequency of use is at an abnormal value, determining that the fault influencing factor is fault in the frequency of use.
It will be appreciated that quantification of AI model fault influencing factors may improve model reliability by monitoring and determining anomalies in factors such as learning rate, regularization parameters, data volume, data balance rate, and frequency of use, and may help identify potential fault factors, thereby improving reliability of the model. For example, when learning rate or regularization parameters are abnormal, the model may exhibit unstable behavior, possibly resulting in performance degradation or failure. Maintenance costs are reduced, and by automating fault detection and diagnosis, maintenance costs of the model can be reduced. Engineers can identify problems more quickly without having to manually analyze large amounts of data and logs. This helps to improve maintainability of the system and reduces the time and effort costs of maintenance work. The performance of the model is improved by monitoring and adjusting key model parameters (such as learning rate, regularization parameters and the like). For example, if the learning rate is too high or too low, the model may converge slowly or diverge, and training efficiency may be improved by dynamically adjusting the learning rate. Optimizing resource utilization, by monitoring data volume and data balance rate, can help to rationally plan computing and storage resources. For example, when the data amount balance rate is abnormal, it is considered to adopt a method such as data resampling or data enhancement to improve the performance of the model, and at the same time, reduce the resource waste. The user experience is improved by monitoring the frequency of use of the model and identifying anomalies. For example, if an abnormality in the model is found at a high load, measures may be taken to ensure the stability of the system to avoid inconvenience to the user. The risk is reduced, and the identification and quantification of key factors can better understand which factors may constitute the greatest risk to model performance and system stability. This helps decision makers take steps in a targeted way to reduce potential risks.
In some of these embodiments, the quantification of AI model fault influencing factors includes: presetting a gradient value matrix S, setting S (S1, S2, S3, S4, S5 and S6), wherein S1 is a first preset gradient value, S2 is a second preset gradient value, S3 is a third preset gradient value, S4 is a fourth preset gradient value, S5 is a fifth preset gradient value, S6 is a sixth preset gradient value, and S1 is less than S2 is less than S3 is less than S min ＜S max S4 is more than S5 and less than S6; presetting a gradient value quantization coefficient matrix a, and setting a (a 1, a2, a3, a4, a5 and a 6), wherein a1 is a first preset gradient value quantization coefficient, a2 is a second preset gradient value quantization coefficient, a3 is a third preset gradient value quantization coefficient, a4 is a fourth preset gradient value quantization coefficient, a5 is a fifth preset gradient value quantization coefficient, a6 is a sixth preset gradient value quantization coefficient, a1 is more than a2 and less than a3 and less than a4 and less than a5 and less than a6;
when S0 < S1, setting S0 a1 as a quantized result of the learning rate;
when S1 is less than or equal to S0 and less than S2, setting S0 a2 as a quantized result of the learning rate;
when S2 is less than or equal to S0 and less than S3, setting S0 a3 as a quantized result of the learning rate;
when S3 is less than or equal to S0 and less than S4, setting S0 a4 as a quantized result of the learning rate;
when S4 is less than or equal to S0 and less than S5, setting S0 a5 as a quantized result of the learning rate;
When S5 is less than or equal to S0 and less than S6, setting S0 a6 as a quantized result of the learning rate;
the preset b is the quantization result of the activation function, b=b1 when s0=0, b=0 when b=b1 when s0+.0, and 0 when s0+.0.
In some of these embodiments, the quantifying of AI model fault influencing factors further comprises: presetting a regularization parameter matrix T, setting T (T1, T2, T3, T4, T5 and T6), wherein T1 is a first preset regularization parameter, T2 is a second preset regularization parameter, T3 is a third preset regularization parameter, T4 is a fourth preset regularization parameter, T5 is a fifth preset regularization parameter, T6 is a sixth preset regularization parameter, and T1 is more than T2 and less than T3 is less than T min ＜T max T4 is more than T5 and less than T6; presetting a regularization parameter quantization coefficient matrix c, and setting c (c 1, c2, c3, c4, c5 and c 6), wherein c1 is a first preset regularization parameter quantization coefficient, c2 is a second preset regularization parameter quantization coefficient, c3 is a third preset regularization parameter quantization coefficient, c4 is a fourth preset regularization parameter quantization coefficient, c5 is a fifth preset regularization parameter quantization coefficient, c6 is a sixth preset regularization parameter quantization coefficient, and c1 is more than c2 and less than c3 and less than c4 and less than c5 and less than c6;
When T0 is less than T1, setting T0 c1 as a quantization result of the regularization parameter;
when T1 is less than or equal to T0 and less than T2, setting T0 c2 as a quantization result of the regularization parameter;
when T2 is less than or equal to T0 and less than T3, setting T0 c3 as a quantization result of the regularization parameter;
when T3 is less than or equal to T0 and less than T4, setting T0 c4 as a quantization result of the regularization parameter;
when T4 is less than or equal to T0 and less than T5, setting T0 c5 as a quantization result of the regularization parameter;
when T5 is less than or equal to T0 and less than T6, setting T0 c6 as the quantization result of the regularization parameter.
In some of these embodiments, the quantifying of AI model fault influencing factors further comprises: presetting a data quantity matrix P, setting P (P1, P2, P3, P4, P5 and P6), wherein P1 is a first preset data quantity, P2 is a second preset data quantity, P3 is a third preset data quantity, P4 is a fourth preset data quantity, P5 is a fifth preset data quantity, P6 is a sixth preset data quantity, and P1 is less than P2 and less than P3 is less than P min ＜P max P4 is more than P5 and P6; presetting a data quantity quantization coefficient matrix d, and setting d (d 1, d2, d3, d4, d5 and d 6), wherein d1 is a first preset data quantity quantization coefficient, d2 is a second preset data quantity quantization coefficient, d3 is a third preset data quantity quantization coefficient, d4 is a fourth preset data quantity quantization coefficient, d5 is a fifth preset data quantity quantization coefficient, d6 is a sixth preset data quantity quantization coefficient, d1 is more than d2 and less than d3 and less than d4 and less than d5 and less than d6;
Setting P0 d1 as a quantization result of the data amount when P0 < P1;
when P1 is less than or equal to P0 and less than P2, setting P0 d2 as a quantization result of the data quantity;
when P2 is less than or equal to P0 and less than P3, setting P0 d3 as a quantization result of the data quantity;
when P3 is less than or equal to P0 and less than P4, setting P0 d4 as a quantization result of the data quantity;
when P4 is less than or equal to P0 and less than P5, setting P0 d5 as a quantization result of the data quantity;
when P5 is less than or equal to P0 and less than P6, setting P0 d6 as the quantization result of the data quantity.
In some of these embodiments, the quantifying of AI model fault influencing factors further comprises: a preset data amount balance rate matrix Q, Q (Q1, Q2, Q3, Q4, Q5, Q6) is set, wherein Q1 is a first preset data amount balance rate, Q2 is a second preset data amount balance rate, Q3 is a third preset data amount balance rate, Q4 is a fourth preset data amount balance rate, Q5 is a fifth preset data amount balance rate, Q6 is a sixth preset data amount balance rate, and Q1 < Q2 < Q3 < Q min ＜Q max Q4 is more than Q5 and less than Q6; presetting a data quantity balance rate quantization coefficient matrix e, setting e (e 1, e2, e3, e4, e5, e 6), wherein e1 is a first preset data quantity balance rate quantization coefficient, e2 is a second preset data quantity balance rate quantization coefficient, e3 is a third preset data quantity balance rate quantization coefficient, e4 is a fourth preset data quantity balance rate quantization coefficient, e5 is a fifth preset data quantity balance rate quantization coefficient, e6 is a sixth preset data quantity balance rate quantization coefficient, and e1 is more than e2 is less than e3 is less than e4 is less than e5 is less than e6;
When Q0 is less than Q1, setting Q0 e1 as a quantization result of the data quantity balance rate;
when Q1 is less than or equal to Q0 and less than Q2, setting Q0 e2 as a quantization result of the data quantity balance rate;
when Q2 is less than or equal to Q0 and less than Q3, setting Q0 e3 as a quantization result of the data quantity balance rate;
when Q3 is less than or equal to Q0 and less than Q4, setting Q0 e4 as a quantization result of the data quantity balance rate;
when Q4 is less than or equal to Q0 and less than Q5, setting Q0 e5 as a quantization result of the data quantity balance rate;
when Q5 is less than or equal to Q0 and less than Q6, setting Q0 and e6 as the quantization result of the data quantity balance rate.
In some of these embodiments, the quantifying of AI model fault influencing factors further comprises: setting a preset use frequency matrix R, and setting R (R1, R2, R3, R4, R5 and R6), wherein R1 is a first preset use frequency, R2 is a second preset use frequency, R3 is a third preset use frequency, R4 is a fourth preset use frequency, R5 is a fifth preset use frequency, R6 is a sixth preset use frequency, and R1 is more than R2 and less than R3 and less than R min ＜R max R4 is more than R5 and less than R6; setting f (f 1, f2, f3, f4, f5 and f 6) by presetting a frequency quantization coefficient matrix f, wherein f1 is a first preset frequency quantization coefficient, f2 is a second preset frequency quantization coefficient, f3 is a third preset frequency quantization coefficient, f4 is a fourth preset frequency quantization coefficient, f5 is a fifth preset frequency quantization coefficient, f6 is a sixth preset frequency quantization coefficient, and f1 is more than f2 and less than f3 and less than f4 and less than f5 and less than f6;
When R0 < R1, setting R0 f1 as the quantization result of the use frequency;
when R1 is less than or equal to R0 and less than R2, setting R0 f2 as a quantization result of the use frequency;
when R2 is less than or equal to R0 and less than R3, setting R0 f3 as a quantization result of the use frequency;
when R3 is less than or equal to R0 and less than R4, setting R0 f4 as a quantization result of the use frequency;
when R4 is less than or equal to R0 and less than R5, setting R0 f5 as a quantization result of the use frequency;
when R5 is less than or equal to R0 and less than R6, setting R0 f6 as the quantization result of the use frequency.
In some of these embodiments, the key fault influencing factors are the fault influencing factors that have the strongest correlation with the model performance, which includes: accuracy, precision, recall, and F1 score.
It will be appreciated that by comparing and quantifying fault factors (e.g., learning rate, regularization parameters, data volume, data balance rate, frequency of use) with preset gradient values, regularization parameters, data volume balance rate, frequency of use, and corresponding quantization coefficient matrices, these factors can be converted into specific numerical results. This helps to translate the fault factor from an abstract concept to a measurable metric. By introducing a plurality of preset values and quantization coefficients, the diversity of different fault factors can be better captured. For example, the learning rate quantization coefficients are different in different gradient value ranges, so that different situations of learning rate abnormality are considered, which helps to more accurately identify learning rate-related faults. According to the real-time fault factor value and the preset value, the proper quantization coefficient can be adaptively selected, so that the fault factor can be quantized more accurately. This can avoid the use of the same quantization method for all cases, improving the applicability of the method. By ordering the quantization results in order of magnitude, it can be determined which fault factor has the greatest impact on model performance. The fault factor corresponding to the greatest quantified result is considered a critical fault influencing factor, which helps to prioritize the most important issues. The method is integrated into an AI system, and can monitor the quantitative result of the fault factor in real time and trigger an alarm or an automatic maintenance flow when an abnormal situation is found. This helps to improve the usability and stability of the system. By quantifying and analyzing the fault factors, the cause of the degradation of the model can be identified, thereby taking steps to optimize the model. For example, appropriate learning rates or regularization parameters are automatically selected to improve the performance of the model.
In summary, an embodiment of the present invention provides a method for analyzing an AI model fault influencing factor, including: determining fault factors: the method determines factors that may lead to model failure by combining different features of the AI model, including structure, training samples, and environment. This helps identify potential sources of problems.
Quantification of fault factors: once the fault factors are determined, the method provides a way to quantify these factors so that they can be represented numerically. This helps to quantify the severity of the problem.
Correlation analysis: by performing correlation analysis on the quantified fault factors and the actual fault conditions, it can be determined which factors have a significant impact on the model fault. This helps to determine critical fault factors to prioritize the problem.
Automated fault detection: the method provides an automated way to detect failures of different factors, including learning rate, regularization parameters, data volume, data balance rate, and frequency of use. This helps to reduce the effort of manual troubleshooting.
The foregoing is merely an example of the present invention and is not intended to limit the scope of the present invention, and all changes made in the structure according to the present invention should be considered as falling within the scope of the present invention without departing from the gist of the present invention.
It should be noted that, in the system provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, in practical application, the foregoing functional allocation may be performed by different functional modules, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention.

Claims (10)

1. An analysis method of AI model fault influencing factors is characterized by comprising the following steps:
determination of fault influencing factors: determining fault influence factors based on structural features of the AI model, training sample features and model use environment features and combining with an AI model fault criterion model; the fault influencing factors of the AI model include: the system comprises a model structure, a training sample and an environment deployment, wherein the model structure is used for reflecting the structural characteristic aspect fault, the training sample is used for reflecting the training sample characteristic aspect fault, and the environment deployment is used for reflecting the model using environment characteristic aspect fault; the training samples comprise data volume and data balance rate; the model structure comprises a learning rate, regularization parameters and an activation function; the environmental deployment includes a model usage frequency;
quantification of AI model fault influencing factors: based on the determined fault influence factors, obtaining a fault influence factor quantification result by adopting a quantification method for the fault influence factors;
correlation analysis: and carrying out correlation analysis on the model performance based on multiple linear regression according to the quantification result of the fault influence factors to obtain the magnitude order of the correlation of each fault influence factor on the model performance, thereby obtaining the key fault influence factors.
2. The method for analyzing AI model fault influencing factors as set forth in claim 1, whereinWherein the determining of the fault influencing factor comprises: preset minimum gradient value S min And preset the highest gradient value S max And detecting real-time gradient values S0, and respectively and preset minimum gradient values S based on the real-time gradient values S0 min Preset maximum gradient value S max If S min ≤S0≤S max The learning rate is at a normal value and the fault influencing factor is determined to be a fault in the aspect of non-learning rate; if 0 is less than S min Or S0 > S max When the learning rate is at an abnormal value, and the fault influencing factor is judged to be a fault in the learning rate;
if s0=0, the activation function is at an abnormal value and the fault influencing factor is determined to be a fault in the aspect of the activation function;
presetting minimum model complexity T min And a preset maximum model complexity T max And detecting the real-time model complexity T0, and based on the real-time model complexity T0 and the preset minimum model complexity T respectively min Preset maximum model complexity T max If T is determined by relation of min ≤T0≤T max The regularization parameters are in normal values, and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T min ≤T0≤T max The regularization parameters are in normal values and the fault influencing factors are judged to be faults in the aspect of the non-regularization parameters; if T0 is less than T min Or T0 > T max And when the regularization parameter is at an abnormal value, and the fault influencing factor is judged to be a fault in the aspect of the regularization parameter.
3. The method of analyzing AI model fault influencing factors of claim 2, wherein the determining of the fault influencing factors further comprises: presetting a minimum data amount P min And preset the highest data value P max And detecting the real-time data amount P0, and based on the real-time data amount P0 and the preset minimum data amount P respectively min Presetting the highest data amount P max If P min ≤P0≤P max The data volume is at a normal value and the fault influencing factor is judged to be fault in the aspect of non-data volume;if 0 < P min Or P0 > P max When the data volume is in an abnormal value, and the fault influencing factors are judged to be faulty in terms of the data volume;
counting the sample number ratio of each sample class, counting the sample number ratio of each sample class again after the same time interval, calculating and obtaining the sample number change rate Q0 of each sample class, and presetting the sample number change rate Q of the lowest sample class min Rate of change Q of sample number from preset highest sample class max And based on the sample number change rate Q0 of each sample class and the preset minimum data quantity Q min Preset the highest data quantity Q max If Q is determined by relation of min ≤Q0≤Q max The data quantity balance rate is at a normal value and the fault influencing factor is judged to be a fault in the aspect of non-data quantity balance rate; if 0 is less than Q min Or Q0 > Q max And if the data quantity balance rate is at an abnormal value, the fault influencing factor is judged to be a fault in the aspect of the data quantity balance rate.
4. The AI model fault influence factor analysis method of claim 3, wherein the fault influence factor determination further comprises: acquiring a using frequency value R0 in a fixed period according to a using log of the model, and presetting a lowest using frequency value R min And preset the highest using frequency value R max And detecting a real-time frequency value R0, and based on the real-time frequency value R0 and a preset minimum frequency value R min Presetting a maximum frequency of use value R max If R is determined by relation of min ≤R0≤R max The frequency of use is at a normal value and the fault influencing factor is determined to be a fault in terms of non-frequency of use; if 0 < R min Or R0 > R max And if the frequency of use is at an abnormal value, determining that the fault influencing factor is fault in the frequency of use.
5. The method of analyzing AI model fault influencing factors of claim 4, wherein quantifying the AI model fault influencing factors comprises: preset gradient moment An array S, S (S1, S2, S3, S4, S5, S6) is set, wherein S1 is a first preset gradient value, S2 is a second preset gradient value, S3 is a third preset gradient value, S4 is a fourth preset gradient value, S5 is a fifth preset gradient value, S6 is a sixth preset gradient value, and S1 < S2 < S3 < S min ＜S max S4 is more than S5 and less than S6; presetting a gradient value quantization coefficient matrix a, and setting a (a 1, a2, a3, a4, a5 and a 6), wherein a1 is a first preset gradient value quantization coefficient, a2 is a second preset gradient value quantization coefficient, a3 is a third preset gradient value quantization coefficient, a4 is a fourth preset gradient value quantization coefficient, a5 is a fifth preset gradient value quantization coefficient, a6 is a sixth preset gradient value quantization coefficient, a1 is more than a2 and less than a3 and less than a4 and less than a5 and less than a6;
when S0 < S1, setting S0 a1 as a quantized result of the learning rate;
when S1 is less than or equal to S0 and less than S2, setting S0 a2 as a quantized result of the learning rate;
when S2 is less than or equal to S0 and less than S3, setting S0 a3 as a quantized result of the learning rate;
when S3 is less than or equal to S0 and less than S4, setting S0 a4 as a quantized result of the learning rate;
when S4 is less than or equal to S0 and less than S5, setting S0 a5 as a quantized result of the learning rate;
when S5 is less than or equal to S0 and less than S6, setting S0 a6 as a quantized result of the learning rate;
the preset b is the quantization result of the activation function, b=b1 when s0=0, b=0 when b=b1 when s0+.0, and 0 when s0+.0.
6. The method of analyzing AI model fault influencing factors of claim 5, wherein quantifying the AI model fault influencing factors further comprises: presetting a regularization parameter matrix T, setting T (T1, T2, T3, T4, T5 and T6), wherein T1 is a first preset regularization parameter, T2 is a second preset regularization parameter, T3 is a third preset regularization parameter, T4 is a fourth preset regularization parameter, T5 is a fifth preset regularization parameter, T6 is a sixth preset regularization parameter, and T1 is more than T2 and less than T3 is less than T min ＜T max T4 is more than T5 and less than T6; pre-preparationSetting a regularization parameter quantization coefficient matrix c, and setting c (c 1, c2, c3, c4, c5 and c 6), wherein c1 is a first preset regularization parameter quantization coefficient, c2 is a second preset regularization parameter quantization coefficient, c3 is a third preset regularization parameter quantization coefficient, c4 is a fourth preset regularization parameter quantization coefficient, c5 is a fifth preset regularization parameter quantization coefficient, c6 is a sixth preset regularization parameter quantization coefficient, and c1 is more than c2 and less than c3 and less than c4 and less than c5 and less than c6;
when T0 is less than T1, setting T0 c1 as a quantization result of the regularization parameter;
when T1 is less than or equal to T0 and less than T2, setting T0 c2 as a quantization result of the regularization parameter;
when T2 is less than or equal to T0 and less than T3, setting T0 c3 as a quantization result of the regularization parameter;
When T3 is less than or equal to T0 and less than T4, setting T0 c4 as a quantization result of the regularization parameter;
when T4 is less than or equal to T0 and less than T5, setting T0 c5 as a quantization result of the regularization parameter;
when T5 is less than or equal to T0 and less than T6, setting T0 c6 as the quantization result of the regularization parameter.
7. The method of analyzing AI model fault influencing factors of claim 6, wherein quantifying the AI model fault influencing factors further comprises: presetting a data quantity matrix P, setting P (P1, P2, P3, P4, P5 and P6), wherein P1 is a first preset data quantity, P2 is a second preset data quantity, P3 is a third preset data quantity, P4 is a fourth preset data quantity, P5 is a fifth preset data quantity, P6 is a sixth preset data quantity, and P1 is less than P2 and less than P3 is less than P min ＜P max P4 is more than P5 and P6; presetting a data quantity quantization coefficient matrix d, and setting d (d 1, d2, d3, d4, d5 and d 6), wherein d1 is a first preset data quantity quantization coefficient, d2 is a second preset data quantity quantization coefficient, d3 is a third preset data quantity quantization coefficient, d4 is a fourth preset data quantity quantization coefficient, d5 is a fifth preset data quantity quantization coefficient, d6 is a sixth preset data quantity quantization coefficient, d1 is more than d2 and less than d3 and less than d4 and less than d5 and less than d6;
setting P0 d1 as a quantization result of the data amount when P0 < P1;
When P1 is less than or equal to P0 and less than P2, setting P0 d2 as a quantization result of the data quantity;
when P2 is less than or equal to P0 and less than P3, setting P0 d3 as a quantization result of the data quantity;
when P3 is less than or equal to P0 and less than P4, setting P0 d4 as a quantization result of the data quantity;
when P4 is less than or equal to P0 and less than P5, setting P0 d5 as a quantization result of the data quantity;
when P5 is less than or equal to P0 and less than P6, setting P0 d6 as the quantization result of the data quantity.
8. The method of analyzing AI model fault influencing factors of claim 7, wherein quantifying the AI model fault influencing factors further comprises: a preset data amount balance rate matrix Q, Q (Q1, Q2, Q3, Q4, Q5, Q6) is set, wherein Q1 is a first preset data amount balance rate, Q2 is a second preset data amount balance rate, Q3 is a third preset data amount balance rate, Q4 is a fourth preset data amount balance rate, Q5 is a fifth preset data amount balance rate, Q6 is a sixth preset data amount balance rate, and Q1 < Q2 < Q3 < Q min ＜Q max Q4 is more than Q5 and less than Q6; presetting a data quantity balance rate quantization coefficient matrix e, setting e (e 1, e2, e3, e4, e5, e 6), wherein e1 is a first preset data quantity balance rate quantization coefficient, e2 is a second preset data quantity balance rate quantization coefficient, e3 is a third preset data quantity balance rate quantization coefficient, e4 is a fourth preset data quantity balance rate quantization coefficient, e5 is a fifth preset data quantity balance rate quantization coefficient, e6 is a sixth preset data quantity balance rate quantization coefficient, and e1 is more than e2 is less than e3 is less than e4 is less than e5 is less than e6;
When Q0 is less than Q1, setting Q0 e1 as a quantization result of the data quantity balance rate;
when Q1 is less than or equal to Q0 and less than Q2, setting Q0 e2 as a quantization result of the data quantity balance rate;
when Q2 is less than or equal to Q0 and less than Q3, setting Q0 e3 as a quantization result of the data quantity balance rate;
when Q3 is less than or equal to Q0 and less than Q4, setting Q0 e4 as a quantization result of the data quantity balance rate;
when Q4 is less than or equal to Q0 and less than Q5, setting Q0 e5 as a quantization result of the data quantity balance rate;
when Q5 is less than or equal to Q0 and less than Q6, setting Q0 and e6 as the quantization result of the data quantity balance rate.
9. The method of analyzing AI model fault influencing factors of claim 8, wherein quantifying the AI model fault influencing factors further comprises: setting a preset use frequency matrix R, and setting R (R1, R2, R3, R4, R5 and R6), wherein R1 is a first preset use frequency, R2 is a second preset use frequency, R3 is a third preset use frequency, R4 is a fourth preset use frequency, R5 is a fifth preset use frequency, R6 is a sixth preset use frequency, and R1 is more than R2 and less than R3 and less than R min ＜R max R4 is more than R5 and less than R6; setting f (f 1, f2, f3, f4, f5 and f 6) by presetting a frequency quantization coefficient matrix f, wherein f1 is a first preset frequency quantization coefficient, f2 is a second preset frequency quantization coefficient, f3 is a third preset frequency quantization coefficient, f4 is a fourth preset frequency quantization coefficient, f5 is a fifth preset frequency quantization coefficient, f6 is a sixth preset frequency quantization coefficient, and f1 is more than f2 and less than f3 and less than f4 and less than f5 and less than f6;
When R0 < R1, setting R0 f1 as the quantization result of the use frequency;
when R1 is less than or equal to R0 and less than R2, setting R0 f2 as a quantization result of the use frequency;
when R2 is less than or equal to R0 and less than R3, setting R0 f3 as a quantization result of the use frequency;
when R3 is less than or equal to R0 and less than R4, setting R0 f4 as a quantization result of the use frequency;
when R4 is less than or equal to R0 and less than R5, setting R0 f5 as a quantization result of the use frequency;
when R5 is less than or equal to R0 and less than R6, setting R0 f6 as the quantization result of the use frequency.
10. The method of analyzing AI model fault influencing factors of claim 9, wherein the key fault influencing factors are the fault influencing factors that have the strongest correlation with the model performance, the model performance comprising: accuracy, precision, recall, and F1 score.
CN202311193985.XA 2023-09-15 2023-09-15 Analysis method for AI model fault influence factors Pending CN117150254A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311193985.XA CN117150254A (en) 2023-09-15 2023-09-15 Analysis method for AI model fault influence factors

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311193985.XA CN117150254A (en) 2023-09-15 2023-09-15 Analysis method for AI model fault influence factors

Publications (1)

Publication Number Publication Date
CN117150254A true CN117150254A (en) 2023-12-01

Family

ID=88907951

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311193985.XA Pending CN117150254A (en) 2023-09-15 2023-09-15 Analysis method for AI model fault influence factors

Country Status (1)

Country Link
CN (1) CN117150254A (en)

Similar Documents

Publication Publication Date Title
CN108491305B (en) Method and system for detecting server fault
CN105247379B (en) The system and method analyzed for uninterruptible power supply battery detection and data
CN117060409B (en) Automatic detection and analysis method and system for power line running state
CN111915192A (en) Method and system for establishing power equipment fault management system and computer equipment
CN113739837A (en) State prediction processing system of standard instrument
KR20220073314A (en) A System and Method for Monitoring Manufacturing Process
CN116882804A (en) Intelligent power monitoring method and system
CN117574292A (en) Data fault detection method and system
CN116954624B (en) Compiling method based on software development kit, software development system and server
CN117150254A (en) Analysis method for AI model fault influence factors
CN115114124A (en) Host risk assessment method and device
CN111030753A (en) Power communication network deep inspection method and system based on performance and service channel
CN111614504A (en) Power grid regulation and control data center service characteristic fault positioning method and system based on time sequence and fault tree analysis
CN115080342A (en) Method and system for automatically processing test data of power board card
CN110619413A (en) Method and device for evaluating health degree of wind generating set
CN115809805A (en) Power grid multi-source data processing method based on edge calculation
CN112398706B (en) Data evaluation standard determining method and device, storage medium and electronic equipment
CN114091618A (en) Industrial equipment health state diagnosis management method and device and server
CN117591507B (en) Automatic construction method and system for data quality characteristics of measurement and control equipment
CN112445632A (en) HPC reliability evaluation method based on fault data modeling
CN117014784B (en) Headset head band fault self-checking method and system
CN117272844B (en) Method and system for predicting service life of distribution board
CN117194188A (en) Method, system, electronic equipment and storage medium for server health diagnosis
CN117271247B (en) SSD solid state disk testing method
CN116051262A (en) Training method of risk prediction model, risk prediction method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination