CN108038049A - Real-time logs control system and control method, cloud computing system and server - Google Patents

Abstract

The invention belongs to the technical field of cloud computing, and discloses a real-time log control system and control method, a cloud computing system and a server. Through the analysis of log recording events, error information is classified, filtered, aggregated, extracted into sequences, and trained Fault model and calculate the probability of the sequence belonging to the fault sequence and the probability of the non-fault sequence, and use Bayesian classification theory to obtain the results and make predictions. The present invention classifies, filters, and aggregates all error information through the analysis of log recording events, extracts them into sequences, trains the fault model and calculates the probability that the sequence belongs to a fault sequence and the probability of a non-fault sequence, using Bayeux Compared with a large number of rule matching, the results and predictions of Adams classification theory have improved the speed of judgment; fault prediction research is of great significance for reducing the burden of network management and maintenance and reducing the loss caused by network faults.

Description

Real-time log control system and control method, cloud computing system and server

technical field

The invention belongs to the technical field of cloud computing, and in particular relates to a real-time log control system and control method, a cloud computing system and a server.

Background technique

With the rapid development of computer technology, cloud computing has become one of the most important computer fields, and cloud computing services have penetrated into everyone's life and work. Through the calculation of real-time data, based on the machine learning algorithm, it can predict the possible faults in the cloud computing system in advance, reserve the fault response time, and also support the elastic horizontal expansion of the processing capacity of the cluster to adapt to the growing data volume and user needs. Real-time calculation and processing of massive log data, mining and analysis of system status and fault prediction from the data has a good development direction and application prospect.

To sum up, the problems existing in the existing technology are: in the original fault prediction model, on the one hand, the state duration distribution is mostly exponential distribution by default, but the state probability change of the fault does not satisfy the exponential type in reality; On the one hand, the detection value probability of the fault state has been discretized, which will have an unexpected impact on the experimental analysis of the big data environment. Therefore, this content uses the state duration distribution and the state observation value probability distribution to carry out continuous distribution. That is, assuming Weibull distribution, the probability value of diagnosis and prediction can be improved by adopting an improved prediction model.

Contents of the invention

Aiming at the problems existing in the prior art, the present invention provides a real-time log control system and control method, a cloud computing system and a server.

The present invention is achieved in this way, a real-time log control method, the real-time log control method classifies, filters, and aggregates error information through the analysis of log record events, extracts them into sequences, trains fault models and calculates the sequence belongs to The probability of a fault sequence and the probability of a non-fault sequence are derived using Bayesian classification theory to make predictions.

Further, the real-time log control method specifically includes:

Step 1: Collect log file data on each node in the distributed system, and send newly generated log data to the collector in real time through incremental checks;

Step 2, delete the same type of events reported at the same location within a certain period of time, delete redundant events, by setting the time threshold Represents the time window used to perform event filtering; by removing similar events reported by multiple different locations within a certain period of time, delete redundant events in the log, and save the data stream into the time series database; use the similarity Sim( D ₁ , D ₂ ) to judge:

Among them, D ₁ and D ₂ represent two sequences, W _1K and W _2K represent the vector items of D1 and D2 sequences, and the similarity is represented by the cosine value of the angle between the two vectors. The greater Sim(D ₁ , D ₂ ), Indicates the higher the similarity between the two;

Step 3, when each piece of data is stored in the data table, use the SQL statement to divide the records according to the timestamp, process number, record level, process module, delimiter, and record information;

Step 4, using SQL statements to persist the processed standard formatted data;

Step 5, extracting the log fault sequence;

Step six, the clustering criteria are based on the likelihood value of the sequence Computed as a metric, a hierarchical clustering algorithm is used to group fault-related events, where:

S=[s _i ] represents a long L state sequence, is the probability matrix of the observed values under the initial state probability vector π=[π _i ] in the state s _i (k);

Step seven, using the combination of improved HSMM and Bayesian network BayesNet to make fault prediction for real-time log data;

The standard HSMM can be converted from state to state by probability matrix G(t)=[g _ij (t)], state s _i (k) under initial state probability vector π=[π _i ], probability matrix B= _bi (k), defined as The state duration probability distribution is continuous; the state duration distribution is treated as a continuous distribution, and it is assumed to obey the Weibull distribution to describe the state duration probability distribution. The state state duration probability distribution f _i (l) is:

f _i (l) = αβ(αl) ^β-1 e ^-(αl)β ;

In the formula: α and β are the scale parameter and shape parameter of Weibull distribution respectively;

The probability distribution of the state monitoring value is continuous; it is also set to obey the Weibull distribution, and the probability distribution function ξ _i (θ) of the state detection value is:

Among them, α _i and β _i are the parameters of Weibull distribution in each state stage; the improved HSMM model can be described as

Step 8, fault and non-fault models are trained, parameters and The goal is to evaluate, given a sequence of observations O = [o ₁ , o ₂ , ..., o _l ], whether it is a fault-related sequence; compute the sequence likelihood for a classification model, subsequently classified as fault-free or fault-Bayes Adams decision theory;

Step 9, predict the failure result:

A sequence is marked as a fault-related event sequence, and the system issues a fault prediction; where Indicates the cost of incorrectly judging a fault-related sequence as a fault-independent sequence, P(F) represents the probability of a fault, Indicates taking the logarithm of the sequence likelihood.

Further, the extraction log failure sequence specifically includes:

The first step is to extract the error event sequence: use the SQL statement to extract the records of the ERROR level according to the log level, and retain the timestamp and text message information;

The second step is to merge similar error events: use the Levenshtein edit distance algorithm for event sequences to merge error events with greater similarity; the minimum edit distance includes sub-minimum edit distance;

Among them, d _{[i-1, j]} + 1 means that the target log inserts a letter, and d _{[i, j-1]} -1 means that the matching log deletes a letter; then when x _i = y _j , no modification is required, so it is the same as above The cost of one step d _{[i-1, j-1]} +1 is the same, otherwise +1, d _[i, j _] represents the smallest item among the above three;

The third step is to classify error events: after merging error events in the previous step, classify similar error events according to the keywords in the text information of error events, assign IDs, and store them in the database;

The fourth step is to extract the sequence: in chronological order, extract a period of time before the fault occurs events within, set to a sequence of fault-related events, is the fault lead time, the current fault event is the related fault event; the non-fault related event sequence is the event sequence in the time interval when the system does not fail.

Another object of the present invention is to provide a real-time log control system according to the real-time log control method. The real-time log control system includes: a log information processing module and a log failure analysis module.

Further, the log failure analysis module includes:

The log information collection unit is used to collect the log file data on each node in the distributed system. The log collection function should allow customizing the log files to be monitored, and send the newly generated log data to the collection in real time through the incremental check method. end;

The log information filtering unit is used for de-redundancy and filtering of data;

The log information standard formatting unit is used for data standard formatting of the processed log information;

The log storage unit is used for persistent storage of the processed standard formatted data.

Further, the log failure analysis module includes:

extract log event sequence unit;

The fault-related event clustering unit is used to use events to train a small hidden semi-Markov model in advance to calculate the sequence likelihood value;

The fault prediction unit uses the hidden semi-Markov model and Bayesian decibel theory to determine whether the sequence is a fault-related sequence;

Fault result judgment output unit: When it is judged to be a fault-related sequence, the system sends out a fault warning flow and outputs a status fault warning.

The extraction log event sequence unit further includes:

Extract the error event record unit, extract the records of the ERROR level according to the log level, and retain the time stamp, process module and text message information;

Merge similar error event units, and use the Levenshtein edit distance algorithm to merge error event sequences with greater similarity;

The error event classification unit uses the Levenshtein edit distance algorithm for the event sequence to classify similar error events and assign IDs;

Extract fault-related sequence units, and extract events in a period of time before the fault according to the chronological order, and set them as fault pre-events.

Another object of the present invention is to provide a cloud computing system utilizing the real-time log control method.

There are three main types of fault prediction research work today, including fault detection models based on log frequency, fault detection models based on message frequency, and fault detection models based on state transition.

The present invention collects log information in real time during system operation time and performs clustering processing, and uses machine learning algorithms and models by analyzing event logs to realize the prediction of possible future failures of the system, and to predict system failures in advance during system operation Troubleshooting and positioning are used to improve system operation and maintenance efficiency and prevent emergency failures. The present invention classifies, filters, and aggregates all error information through the analysis of log recording events, extracts them into sequences, trains the fault model and calculates the probability that the sequence belongs to a fault sequence and the probability of a non-fault sequence, using Bayeux The theory of Adams classification draws results and makes predictions.

The effective judgment standard of this method is mainly determined by three parameters, namely the accuracy rate, recall rate and F-measure parameter. The accuracy rate reflects the correct ratio of all predictions, and the recall rate reflects the correct prediction of all faults. The ratio of F.measure is a comprehensive measure combining precision and recall;

The predictions are as follows in Table 1:

Predicted Results\Actual Results system error The system is normal system error TruePositive (TP) False Positive (FP) The system is normal False Negative (FN) True Negative (TN)

Table 1 Forecast

The predictive validity parameters are shown in Table 2:

Table 2 Validity parameter expression

After the system experiment, the following data conclusions can be drawn. It can be seen that the accuracy of this system is better than that before no improvement.

Description of drawings

Fig. 1 is a schematic structural diagram of a real-time log control system provided by an embodiment of the present invention;

In the figure: 1. Log information processing module; 1-1. Collecting log information unit; 1-2. Log information filtering unit; 1-3. Log information standard formatting unit; 1-4. Log storage unit; 2. Log Fault analysis module; 2-1, extraction log event sequence unit; 2-1-1, extraction error event recording unit; 2-1-2, merging similar error event unit; 2-1-3, error event classification unit; 2 -1-4. Extracting fault-related sequence unit; 2-2. Fault-related event clustering unit; 2-3. Fault prediction unit; 2-4. Fault result judgment output unit.

Fig. 2 is a flowchart of a real-time log control method provided by an embodiment of the present invention.

Fig. 3 is a flow chart of realizing the real-time log control method provided by the embodiment of the present invention.

Fig. 4 is a schematic diagram of fault sequence extraction provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The application principle of the present invention will be described in detail below in conjunction with the accompanying drawings.

As shown in FIG. 1 , the real-time log control system provided by the embodiment of the present invention includes: a log information processing module 1 and a log fault analysis module 2 .

Log failure analysis module 1 includes:

Collecting log information unit 1-1: used to collect log file data on each node in the distributed system. The log collection function should allow customizing the log files to be monitored. Through incremental checking, the newly generated log data will be collected in real time. sent to the collector.

Log information filtering unit 1-2: used for de-redundancy and filtering of data.

Log information standard formatting unit 1-3: used for data standard formatting of processed log information, for example, according to: timestamp, process number, record level, process module, separator, record information, where the record level is divided into Several categories, including: ERROR, WARING, TRACE, INFO, DUBUG, CRITICAL, AUDIT, the higher the level, the higher the level, and the higher the level, the higher the importance of the event.

Log storage unit 1-4: used for persistent storage of processed standard formatted data, which is convenient for later data extraction and analysis.

Log failure analysis module 2 includes:

Extract log event sequence unit 2-1:

The fault-related event clustering unit 2-2 is used to train a small hidden semi-Markov (HSMM) model in advance by using the event, and calculate the sequence likelihood value, that is, the observation sequence generated by the training model for a given sequence;

Fault prediction unit 2-3: use the hidden semi-Markov model and Bayesian decibel theory to determine whether the sequence is a fault-related sequence;

Fault result judgment output unit 2-4: When it is judged to be a fault-related sequence, the system sends a fault warning flow and outputs a status fault warning.

Extract log event sequence unit 2-1 further includes:

Extract error event record unit 2-1-1: Extract the records of ERROR level according to the log level, and retain information such as time stamp, process module and text message;

Merge similar error events unit 2-1-2: Use the Levenshtein edit distance algorithm to merge error event sequences with greater similarity;

Error event classification unit 2-1-3: Use the Levenshtein edit distance algorithm for event sequences to classify similar error events and assign IDs;

Extract fault-related sequence unit 2-1-4: According to the chronological order, extract the events within a period of time before the fault, and set it as the pre-fault event.

As shown in Figure 2, the real-time log control method provided by the embodiment of the present invention includes the following steps:

S201: through the analysis of the log record event, classify, filter, aggregate and other operations are performed on all the error information, and extract them into a sequence;

S202: Train the fault model and calculate the probability that the sequence belongs to the fault sequence and the probability of the non-fault sequence, and use Bayesian classification theory to obtain the result and make a prediction.

The application principle of the present invention will be further described below in conjunction with the accompanying drawings.

Compared with utilizing fault keywords to carry out a large amount of rule matching, in the present invention, adopt improved HSMM (hidden Markov model) and Bayesdecision theory (Bayesian classification theory), directly calculate the probability that an error sequence belongs to fault sequence , improve the speed of judgment.

As shown in Figure 3, the specific steps of the real-time log control method provided by the embodiment of the present invention are as follows:

1. Log information processing process

Step 1, log information collection

The system should be able to collect log file data on each node in the distributed system. The log collection function should allow customizing the log files to be monitored, and send newly generated log data to the collector in real time through incremental checking.

Step 2, log information filtering

There are two methods: one is temporal filtering and the other is spatial filtering. When the system detects an anomaly, the system will continuously output a stream of warning information until the system fails. Likewise, once a system fails, the failure message may appear repeatedly in the log many times until the failure problem is resolved.

The time filtering method removes redundant events by removing events of the same type reported at the same location within a certain period of time, by setting a time threshold Indicates the time window used to perform event filtering. The spatial filtering method saves space and improves efficiency by removing similar events reported by multiple different locations within a certain period of time, deleting redundant events in logs, and saving data streams to time series databases. Usually use the similarity Sim(D ₁ , D ₂ ) to judge:

Among them, D ₁ and D ₂ represent two sequences, W _1K and W _2K represent the vector items of D1 and D2 sequences, and the similarity is represented by the cosine value of the angle between the two vectors. The greater Sim(D ₁ , D ₂ ), Indicates the higher the similarity between the two.

Step 3, log format standardization.

When storing each piece of data in the data table, use SQL statements to divide records according to timestamp, process number, record level, process module, delimiter, record information, etc.

Step 4, log storage.

Use SQL statements to store the processed standard formatted data persistently, which is convenient for later data extraction and analysis.

2. Log failure analysis:

Establish a probability-based causal relationship between the fault performance and the system state, train the hidden semi-Markov model and Bayesian network through the prior probability of fault occurrence, and solve various systems under the fault performance according to the prior probability when diagnosing The posterior probability of the state, intuitively expresses the joint probability distribution of the variables, and calculates the probability of failure caused by each feature at the same time.

Step 1, extract the log fault sequence.

The first step is to extract the error event sequence: use the SQL statement to extract the records of the ERROR level according to the log level, and retain information such as time stamps and text messages;

The second step is to merge similar error events: use the Levenshtein edit distance algorithm for the event sequence in the previous step to merge the error events with greater similarity;

The algorithm uses the algorithm strategy of dynamic programming, the problem has an optimal substructure, and the minimum edit distance includes sub-minimum edit distance;

Among them, d _{[i-1, j]} + 1 means that the target log inserts a letter, and d _{[i, j-1]} + 1 means that the matching log deletes a letter; then when x _i = y _j , no modification is required, so it is the same as above The cost of one step d _{[i-1, j-1]} +1 is the same, otherwise +1, d _{[i, j]} represents the smallest item among the above three;

The third step is to classify error events: after merging error events in the previous step, classify similar error events according to the keywords in the text information of error events, assign IDs, and store them in the database;

The fourth step is to extract the sequence: in chronological order, extract a period of time before the fault occurs For the event, set as the sequence of fault-related events, is the fault lead time, and the current fault event is a related fault event; the non-fault related event sequence is the event sequence in the time interval when the system does not fail, as shown in Figure 4:

Step 2, clustering of fault-related events.

In practice, there will be a variety of fault-related event sequences that may lead to the same system fault, and the characteristics of these various fault-related event sequences are different, so clustering is required.

The clustering criteria can be based on the likelihood value of the sequence Calculated as a metric, and finally a hierarchical clustering algorithm is used to group fault-related events, where:

S=[s _i ] represents a state sequence of length L, and b _si (o _i ) is the probability matrix of observed values in state s _i (k) under the initial state probability vector π=[π _i ].

Step 3, training and building a prediction model.

The prediction model is the key to network fault prediction, and the constructed features directly affect the performance of the prediction model. This time, a combination of Hidden Semi-Markov Model (HSMM) and Bayesian Network (Bayes Net) is used to make fault prediction for real-time log data.

The standard HSMM can be converted from state to state by probability matrix G(t)=[g _ij (t)], state s _i (k) under initial state probability vector π=[π _i ], probability matrix B= _bi (k), defined as

The improvements to HSMM this time include: continuous state duration probability distribution. Treat the distribution of the state duration as a continuous distribution, and assume that it obeys the Weibull distribution to describe the probability distribution of the state duration, that is, the state duration probability distribution f _i (l) of the state is:

f _i (l) = αβ(αl) ^β-1 e ^-(αl)β ;

In the formula: d and β are the scale parameter and shape parameter of Weibull distribution respectively;

The probability distribution of state monitoring values is continuous. It is also assumed that it obeys the Weibull distribution, and the probability distribution function ξ _i (θ) of the state detection value is:

Among them, α _i and β _i are the parameters of Weibull distribution in each state stage; therefore, the improved HSMM model can be described as

Step 4, fault prediction.

Assumed faulty and non-faulty models to train, i.e. parameters and The goal is to evaluate, given an observation sequence (error sequence) O=[o ₁ , o ₂ , . . . , o _l ], whether it is a fault-related sequence. Sequence likelihoods are first computed for classification models, which are subsequently classified as failure-free or failure-based Bayesian decision theory.

Step 5, fault result prediction:

When the above formula is established, a sequence is marked as a fault-related event sequence, and the system issues a fault prediction. in Indicates the cost of incorrectly judging a fault-related sequence as a fault-independent sequence, P(F) represents the probability of a fault, Indicates that the logarithm of the sequence likelihood value is taken, which can prevent the overflow problem from occurring when the sequence likelihood value is too small. Through such a method, each sequence can be judged and a fault prediction can be made.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (9)

1. A real-time log control method, characterized in that, the real-time log control method classifies, filters, and aggregates error information through the analysis of the log record event, extracts it as a sequence, trains the fault model and calculates that the sequence belongs to the fault The probabilities of sequences and the probabilities of non-failure sequences are derived using Bayesian classification theory to make predictions. 2. The real-time log control method according to claim 1, wherein the real-time log control method specifically comprises: Step 1: Collect log file data on each node in the distributed system, and send newly generated log data to the collector in real time through incremental checks; Step 2, delete the same type of events reported at the same location within a certain period of time, delete redundant events, by setting the time threshold Represents the time window used to perform event filtering; by removing similar events reported by multiple different locations within a certain period of time, delete redundant events in the log, and save the data stream into the time series database; use the similarity Sim( D ₁ , D ₂ ) to judge: <mrow><mi>S</mi><mi>i</mi><mi>m</mi><mrow><mo>(</mo><msub><mi>D</mi><mn>1</mn></msub><mo>,</mo><msub><mi>D</mi><mn>2</mn></msub><mo>)</mo></mrow><mo>=</mo><mi>c</mi><mi>o</mi><mi>s</mi><mi>&amp;theta;</mi><mo>=</mo><mfrac><mrow><msubsup><mo>&amp;Sigma;</mo><mrow><mi>k</mi><mo>-</mo><mn>1</mo>mn></mrow><mi>n</mi></msubsup><msub><mi>W</mi><mrow><mn>1</mn><mi>K</mi></mrow></msub><mo>&amp;times;</mo><msub><mi>W</mi><mrow><mn>2</mn><mi>K</mi></mrow></msub></mrow><msqrt><mrow><msup><mrow><mo>(</mo><msubsup><mo>&amp;Sigma;</mo><mrow><mi>k</mi><mo>-</mo><mn>1</mn></mrow><mi>n</mi></msubsup><msub><mi>W</mi><mrow><mn>1</mn><mi>K</mi></mrow></msub><mo>)</mo></mrow><mn>2</mn></msup><mo>&amp;times;</mo><msup><mrow><mo>(</mo><msubsup><mo>&amp;Sigma;</mo><mrow><mi>k</mi><mo>-</mo><mn>1</mn></mrow><mi>n</mi></msubsup><msub><mi>W</mi><mrow><mn>2</mn><mi>K</mi></mrow></msub><mo>)</mo></mrow><mn>2</mn></msup></mrow></msqrt></mfrac><mo>;</mo></mrow> Among them, D ₁ and D ₂ represent two sequences, W _1K and W _2K represent the vector items of D1 and D2 sequences, and the similarity is represented by the cosine value of the angle between the two vectors. The greater Sim(D ₁ , D ₂ ), Indicates the higher the similarity between the two; Step 3, when each piece of data is stored in the data table, use the SQL statement to divide the records according to the timestamp, process number, record level, process module, delimiter, and record information; Step 4, using SQL statements to persist the processed standard formatted data; Step 5, extracting the log fault sequence; Step six, the clustering criteria are based on the likelihood value of the sequence Computed as a metric, a hierarchical clustering algorithm is used to group fault-related events, where: S=[s _i ] represents a long L state sequence, is the probability matrix of the observed values under the initial state probability vector π=[π _i ] in the state s _i (k); Step 7, using the combination of hidden semi-Markov model HSMM and Bayesian network Bayes Net to make fault prediction for real-time log data; The standard HSMM can be converted from state to state by probability matrix G(t)=[g _ij (t)], state s _i (k) under initial state probability vector π=[π _i ], probability matrix B= _bi (k), defined as λ=(π, G(t), B); the state duration probability distribution is continuous; the state duration distribution is treated as a continuous distribution, and it is assumed to obey the Weibull distribution to describe the state Duration probability distribution, the state duration probability distribution f _i (l) of the state is: f _i (l) = αβ(αl) ^β-1 e ^-(αl)β ; In the formula: α and β are the scale parameter and shape parameter of Weibull distribution respectively; The probability distribution of the state monitoring value is continuous; it is also set to obey the Weibull distribution, and the probability distribution function ξ _i (θ) of the state detection value is: <mrow><msub><mi>&amp;xi;</mi><mi>i</mi></msub><mrow><mo>(</mo><mi>&amp;theta;</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>y</mi><mi>t</mi></msub><mo>=</mo><mi>&amp;theta;</mi><mo>|</mo><msub><mi>q</mi><mi>t</mi></msub><mo>=</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><msub><mi>&amp;alpha;</mi><mi>i</mi></msub><msub><mi>&amp;beta;</mi><mi>i</mi></msub><msup><mrow><mo>(</mo><msub><mi>&amp;alpha;</mi><mi>i</mi></msub><mi>&amp;theta;</mi><mo>)</mo></mrow><mrow><msub><mi>&amp;beta;</mi><mi>i</mi></msub><mo>-</mo><mn>1</mn></mrow></msup><msup><mi>e</mi><mrow><mo>(</mo><msub><mi>&amp;alpha;</mi><mi>i</mi></msub><mi>&amp;theta;</mi><mo>)</mo><msub><mi>&amp;beta;</mi><mi>i</mi></msub></mrow></msup><mo>;</mo></mrow> Among them, α _i and β _i are the parameters of Weibull distribution in each state stage; the improved HSMM model can be described as Step 8, fault and non-fault models are trained, parameters and The goal is to evaluate, given a sequence of observations O = [o ₁ , o ₂ , ..., o _l ], whether it is a fault-related sequence; compute the sequence likelihood for a classification model, subsequently classified as fault-free or fault-Bayes Adams decision theory; Step 9, predict the failure result: A sequence is marked as a fault-related event sequence, and the system issues a fault prediction; where Indicates the cost of incorrectly judging a fault-related sequence as a fault-independent sequence, P(F) represents the probability of a fault, Indicates taking the logarithm of the sequence likelihood. 3. the real-time log control method as claimed in claim 2, is characterized in that, described extraction log failure sequence specifically comprises: The first step is to extract the error event sequence: use the SQL statement to extract the records of the ERROR level according to the log level, and retain the timestamp and text message information; The second step is to merge similar error events: use the Levenshtein edit distance algorithm for event sequences to merge error events with greater similarity; the minimum edit distance includes sub-minimum edit distance; Among them, d _{[i-1, j]} + 1 means that the target log inserts a letter, and d _{[i, j-1]} + 1 means that the matching log deletes a letter; then when x _i = y _j , no modification is required, so it is the same as above The cost of one step d _{[i-1, j-1]} +1 is the same, otherwise +1, d _{[i, j]} represents the smallest item among the above three; The third step is to classify error events: after merging error events in the previous step, classify similar error events according to the keywords in the text information of error events, assign IDs, and store them in the database; The fourth step is to extract the sequence: in chronological order, extract a period of time before the fault occurs events within, set to a sequence of fault-related events, is the fault lead time, the current fault event is the related fault event; the non-fault related event sequence is the event sequence in the time interval when the system does not fail. 4. A real-time log control system according to the real-time log control method of claim 1, wherein the real-time log control system comprises: a log information processing module and a log failure analysis module. 5. the real-time log control system as claimed in claim 4, is characterized in that, described log failure analysis module comprises: The log information collection unit is used to collect the log file data on each node in the distributed system. The log collection function should allow customizing the log files to be monitored, and send the newly generated log data to the collection in real time through the incremental check method. end; The log information filtering unit is used for de-redundancy and filtering of data; The log information standard formatting unit is used for data standard formatting of the processed log information; The log storage unit is used for persistent storage of the processed standard formatted data. 6. the real-time log control system as claimed in claim 4, is characterized in that, described log failure analysis module comprises: extract log event sequence unit; The fault-related event clustering unit is used to use events to train a small hidden semi-Markov model in advance to calculate the sequence likelihood value; The fault prediction unit uses the hidden semi-Markov model and Bayesian decibel theory to determine whether the sequence is a fault-related sequence; Fault result judgment output unit: When it is judged to be a fault-related sequence, the system sends out a fault warning flow and outputs a status fault warning. 7. The real-time log control system according to claim 6, wherein the extracting log event sequence unit further comprises: Extract the error event record unit, extract the records of the ERROR level according to the log level, and retain the time stamp, process module and text message information; Merge similar error event units, and use the Levenshtein edit distance algorithm to merge error event sequences with greater similarity; The error event classification unit uses the Levenshtein edit distance algorithm for the event sequence to classify similar error events and assign IDs; Extract fault-related sequence units, and extract events in a period of time before the fault according to the chronological order, and set them as fault pre-events. 8. A cloud computing system using the real-time log control method according to any one of claims 1-3. 9. A cloud computing server using the real-time log control method according to any one of claims 1-3.