CN116541238A - Log file acquisition method and device, electronic equipment and readable storage medium - Google Patents

Log file acquisition method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN116541238A
CN116541238A CN202310484049.8A CN202310484049A CN116541238A CN 116541238 A CN116541238 A CN 116541238A CN 202310484049 A CN202310484049 A CN 202310484049A CN 116541238 A CN116541238 A CN 116541238A
Authority
CN
China
Prior art keywords
server
weight value
target
determining
index data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310484049.8A
Other languages
Chinese (zh)
Inventor
王崇娇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202310484049.8A priority Critical patent/CN116541238A/en
Publication of CN116541238A publication Critical patent/CN116541238A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a log file acquisition method, a log file acquisition device, electronic equipment and a readable storage medium. After the target server is determined, the target fault class with a larger first weight value is determined according to the first weight value of the fault class corresponding to the target server, the screening rule corresponding to the target fault class and the weight value of index data are further determined according to the target fault class, and when the log file is acquired for the server, the weight value of the log file can be calculated according to the index data included in the server, so that the log file associated with diagnosing the fault of the server can be accurately acquired when the log file is acquired, the time for acquiring the log is saved, and the storage resource of the server is also saved.

Description

Log file acquisition method and device, electronic equipment and readable storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for collecting log files, an electronic device, and a readable storage medium.
Background
The server log data is an important basis for fault diagnosis or analysis of the server, and the accurate server log data can improve the efficiency of fault diagnosis of the server.
In the related art, a server log collecting tool collects log files of each model of a server, and a server log diagnosis platform analyzes the log files.
When the method collects server logs, the collected server logs are full log files, and possible fault reasons of different servers are different due to the difference between the servers, and the method for collecting the full log files occupies more storage resources, does not distinguish the difference between the servers, causes log file redundancy and reduces the efficiency of fault diagnosis.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention are provided to provide a log file collection method, apparatus, electronic device, and readable storage medium that overcome or at least partially solve the foregoing problems.
In a first aspect, an embodiment of the present application discloses a log file collection method, where the method includes:
acquiring first characteristic information of a server to be acquired;
comparing the first characteristic information with second characteristic information of a server contained in a preset database, and determining a target server belonging to the same class as the server to be acquired in the preset database;
determining a target fault class of which the first weight value is in a first preset range according to a first weight value of the fault class corresponding to the target server;
determining a third weight value of index data according to a second weight value of a screening rule corresponding to the target fault class and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule or not;
and determining a fourth weight value corresponding to each log file contained in the server to be acquired according to the third weight value of the index data, and acquiring the log files of which the fourth weight value belongs to a second preset range.
In a second aspect, an embodiment of the present application discloses a log file collecting device, where the device includes:
The acquisition module is used for acquiring first characteristic information of the server to be acquired;
the first determining module is used for comparing the first characteristic information with second characteristic information of a server contained in a preset database and determining that the target server belonging to the same class as the server to be acquired in the preset database;
the second determining module is used for determining a target fault class of which the first weight value is in a first preset range according to the first weight value of the fault class corresponding to the target server;
the third determining module is used for determining a third weight value of the index data according to a second weight value of the screening rule corresponding to the target fault category and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule or not;
and the acquisition module is used for determining fourth weight values corresponding to the log files contained in the server to be acquired according to the third weight values of the index data, and acquiring the log files of which the fourth weight values belong to a second preset range.
In a third aspect, an embodiment of the present application further discloses an electronic device, including a processor and a memory, where the memory stores a program or instructions executable on the processor, the program or instructions implementing the steps of the method according to the first aspect when executed by the processor.
In a fourth aspect, embodiments of the present application also disclose a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps of the method as described in the first aspect.
In the embodiment of the application, the first characteristic information of the server to be collected is compared with the second characteristic information in the preset database, and the fact that the same type of target server belongs to the same type of server as the server to be collected in the preset database is determined, so that the probability of occurrence of the same type of faults of the same type of server is high. After the target server is determined, the target fault class with a larger first weight value is determined according to the first weight value of the fault class corresponding to the target server, the screening rule corresponding to the target fault class and the weight value of index data are further determined according to the target fault class, when the log file is collected for the server, the weight value of the log file can be calculated according to the index data included in the server, and the log file associated with diagnosing the fault of the server can be accurately collected when the log file is collected by sequencing the weight values of the log file, so that the time for collecting the log is saved, the efficiency of diagnosing the server is improved, and meanwhile, the storage resources of the server are saved for a small number of log files.
Drawings
FIG. 1 is a flow chart of steps of a log file collection method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of a log file collection method according to another embodiment of the present invention;
FIG. 3 is a block diagram of a log file collection device according to an embodiment of the present invention;
FIG. 4 is a block diagram of a terminal according to another embodiment of the present invention;
fig. 5 is a schematic structural view of a terminal according to another embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Referring to fig. 1, a flowchart illustrating steps of a log file collection method provided in an embodiment of the present application is shown, where the method includes:
step 101, obtaining first characteristic information of a server to be acquired.
In the embodiment of the inventionThe server to be collected can be a server which needs to collect log files so as to analyze fault information according to the log files. The first characteristic information corresponding to the server to be collected may be a server asset information index corresponding to the server to be collected, for example: and the machine height, the power supply quantity, the server platform and other characteristic information of the server to be acquired. The acquired k server asset information indexes of the server to be acquired form first characteristic information to be acquired, and the first characteristic information is expressed as { r } 1 ,r 2 ,…r k }. The corresponding first characteristic information is different for different servers.
Step 102, comparing the first characteristic information with second characteristic information of a server contained in a preset database, and determining that the target server belonging to the same class as the server to be acquired in the preset database.
In the embodiment of the present invention, the preset database may be a database established according to the history information of the existing server. The server in the preset database and the log file corresponding to the server can be used as sample data, and the difference of the fault types of different types of servers can be determined through analysis of the sample data. For example, if a server exists in the preset database, the database also stores second characteristic information corresponding to each of the a servers. The second characteristic information may be server asset information indexes corresponding to the servers in the database, and may be represented as { A } 1 ,A 2 ,…A k }。
Further, the k server asset information indexes in the first feature information of the server to be collected are respectively compared with the server asset information indexes corresponding to each server in the preset database, for example, a can be compared 1 And r 1 、A 2 And r 2 And the like, by comparing the characteristics contained in the first characteristic information and the second characteristic information, the target servers which belong to the same class with the servers to be acquired in the preset database can be determined. The hardware or other configurations of the servers in the same class are similar, and the target server can be used as reference data when diagnosing the faults of the servers.
Step 103, determining a target fault class with a first weight value in a first preset range according to a first weight value of the fault class corresponding to the target server.
In the embodiment of the present invention, the target servers determined from the preset database may be represented by a set T, and if there are c determined target servers, the set t= { T 1 ,T 2 ,…T c Each target server in the set T has a corresponding failure category for classifying the different failure categories to which the server corresponds. The first weight value corresponding to each fault class in the fault class corresponding to the target server can be determined by counting the fault class corresponding to each target server and counting the fault class corresponding to each server in a preset database, and the frequency of occurrence of the fault class can be represented by the first weight value. The greater the first weight value is, the more times the fault class appears are indicated, so by sorting the first weight values of all the fault classes corresponding to the target server, the fault class with the first weight value in the first preset range is selected as the target fault class, the first preset range may be the fault class with the first weight value in the first 10% or the first 20% after sorting, and the first preset range may be set according to the actual situation, and the embodiment of the present invention is not limited herein. The determined target fault class is the fault class with high possibility of the target server.
Step 104, determining a third weight value of the index data according to a second weight value of the screening rule corresponding to the target fault category and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule.
In the embodiment of the invention, each fault class corresponds to a plurality of screening rules, for example, the screening rules corresponding to the fault class a include rule 1, rule 2 and rule 3, and then the fault corresponding to the server can be determined to be the fault class a according to the rule 1, rule 2 and rule 3.
Further, a rule may also correspond to multiple fault categories, such as: rule 1 may diagnose that the failure class of the server is failure class a, rule 1 may also diagnose that the failure class of the server is failure class B, that is, the screening rule corresponding to each failure class may be repeated. And counting the occurrence times of the screening rules corresponding to the target fault categories, so as to determine the second weight value corresponding to each screening rule.
Further, each filtering rule may correspond to a plurality of index data, where the index data is used to numerically measure whether each index of the server meets the corresponding filtering rule. For example: rule 1 may correspond to index 1, index 2, index 3, rule 2 may correspond to index 2, index 3, index 4, and rule 3 may correspond to index 3, index 4, and index 5, and a third weight value corresponding to each index data may be determined by counting the number of occurrences of index data corresponding to all the screening rules, and a second weight value of the screening rule including the index. The third weight value of the index data may reflect a degree of importance of the index data to diagnosing the fault category.
Step 105, determining fourth weight values corresponding to the log files included in the server to be collected according to the third weight values of the index data, and collecting the log files of which the fourth weight values belong to a second preset range.
In the embodiment of the invention, after the third weight value of the index data is determined, the weight values respectively corresponding to all log files corresponding to the server to be acquired can be further determined. And analyzing the number of index data contained in each log file, calculating a fourth weight value of each log file according to the third weight value of each index data, sequencing all log files according to the size of the fourth weight value, and determining the log files with the fourth weight value in a second preset range as the log files to be acquired. The second preset range may be a fault category in the range of the first 10% or the first 20% after the fourth weight value is sequenced, and the second preset range may be set according to the actual situation, which is not limited in the embodiment of the present invention.
In summary, in the embodiment of the present application, the first feature information of the server to be collected is compared with the second feature information in the preset database, so that it is determined that the same type of target server belongs to the same type of target server as the server to be collected in the preset database, and the probability of occurrence of the same type of faults is high. After the target server is determined, the target fault class with a larger first weight value is determined according to the first weight value of the fault class corresponding to the target server, the screening rule corresponding to the target fault class and the weight value of index data are further determined according to the target fault class, when the log file is collected for the server, the weight value of the log file can be calculated according to the index data included in the server, and the log file associated with diagnosing the fault of the server can be accurately collected when the log file is collected by sequencing the weight values of the log file, so that the time for collecting the log is saved, the efficiency of diagnosing the server is improved, and meanwhile, the storage resources of the server are saved for a small number of log files.
Referring to fig. 2, a flowchart of steps of another log file collecting method according to an embodiment of the present application is shown, where the method includes:
step 201, obtaining first feature information of a server to be acquired.
This step can refer to step 101, and will not be described here.
Step 202, determining a euclidean distance between the first feature information and second feature information of a server included in the preset database.
In the embodiment of the present invention, the method for judging whether the server in the preset database and the server to be acquired are the same server may be: the Euclidean distance between the first characteristic information of the server to be acquired and the second characteristic information corresponding to each server in a preset database is calculated. Euclidean distance is a discriminating and classifying method based on distance. The distinguishing and classifying refers to distinguishing and classifying according to different points of things, and determining the category of the things, so that the things with more similar points are classified into one category, and the category of each thing can be rapidly distinguished according to a certain rule in a large number of things. The basic idea of distance discrimination is to classify samples closer to each other into one class, and samples farther from each other into different classes. The distance here may be the similarity between samples, the smaller the inter-sample distance the more similar, and vice versa. At the time of classification, the distance from each new sample point (class unknown) to the history sample point (class known) may be calculated, and then the class of the new sample is predicted to be the class of the history sample point most similar thereto. Or the category of the new sample is predicted to be the most similar category among k (k=1, 2, …) history sample points.
Further, in the present application, taking a method of using euclidean distance discriminant analysis as an example, a similarity between each asset information index in the first feature information and the second feature information is calculated, so as to determine a target server in the same class as the server to be collected. The first characteristic information { r may be calculated by the following expression 1 ,r 2 ,…r k And the Euclidean distance between the second characteristic information of the ith server in the preset database.
Wherein k represents k asset information indexes included in the first characteristic information or the second characteristic information, j represents a j-th asset information index in the first characteristic information or the second characteristic information, and D i A value representing the calculated euclidean distance. According to the order of the calculated Euclidean distance values, the servers with smaller Euclidean distances are determined to be more similar to the servers to be collected, the servers with larger Euclidean distances are not similar to the servers to be collected, and the target servers belonging to the same class with the servers to be collected are determined according to the Euclidean distance values.
And 203, determining a server, of which the Euclidean distance between the server and the first characteristic information in the preset database is smaller than a preset threshold value, as a target server belonging to the same class as the server to be acquired.
In the embodiment of the invention, according to the calculated euclidean distance, the server with the euclidean distance smaller than the preset threshold value with the server to be collected is determined as the target server belonging to the same class as the server to be collected by comparing the euclidean distance with the preset threshold value, and the size of the preset threshold value can be set according to actual conditions, so that the embodiment of the invention is not limited.
Step 204, determining a target fault class of which the first weight value is in a first preset range according to a first weight value of the fault class corresponding to the target server.
This step can refer to step 103, and will not be described here.
Optionally, step 204 specifically includes:
in a sub-step 2041, for each failure category in the preset database, a first ratio of the number of servers included in the failure category to the total number of servers in the preset database is determined.
In the embodiment of the invention, the first ratio may represent the prior probability and is recorded as the ratio of the number of servers contained in each fault class to the total number of servers. If the total number of servers included in the preset database is a, and the total number of included fault categories is c, the prior probability of each fault category can be expressed by the following expression.
Wherein P is a set of prior probabilities corresponding to each of all fault categories, P 1 、P 2 …P C Representing the prior probability corresponding to each fault class. N (N) 1 Representing the total number of servers included in the first failure category, and so on, N C Is the total number of servers included in the c-th failure category.
In the sub-step 2042, for each failure category corresponding to the target server, a second ratio of the number of target servers included in the failure category to the number of servers included in the failure category in a preset database is determined.
And step 2043, determining a first weight value of a fault class corresponding to the target server according to the first ratio and the second ratio, and determining the fault class with the first weight value in a first preset range as a target fault class.
In the embodiment of the invention, after the prior probability corresponding to each fault class calculated based on the preset database is determined, the second ratio of each fault class corresponding to the target server is further determined based on the target server set. And comprehensively determining a first weight value corresponding to each fault category through the first ratio and the second ratio.
Further, the larger the first weight value is, the more times the fault class appears are indicated, so that the fault class with the first weight value in the first preset range is selected as the target fault class by sequencing the first weight values of all the fault classes corresponding to the target server. The fault category determined in this way is the fault category with the highest occurrence probability of the server to be collected.
Optionally, the substep 2043 specifically includes:
sub-step 2044, using the product of the first ratio corresponding to the fault class corresponding to the target server and the second ratio corresponding to the fault class corresponding to the target server as the first weight value of the fault class corresponding to the target server.
In the embodiment of the present invention, if there are c determined target servers, the set t= { T1, T2, … Tc }, where Tj represents that Tj servers in the classification set have occurred in the jth failure class, the first weight value of the jth failure class is recorded as
Step 205, determining a third weight value of the index data according to a second weight value of the screening rule corresponding to the target fault class and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule.
This step can refer to step 104, and will not be described here.
Optionally, step 205 specifically includes:
sub-step 2051, determining a second weight value of the screening rule corresponding to the target fault class according to the first weight value corresponding to the target fault class.
In the embodiment of the invention, the c fault categories are ranked according to the calculated first weight value, if the first preset range is ten percent before, the target fault category with the first weight value greater than ten percent before can be selected, and each target fault category can correspond to a plurality of screening rules, and the screening rules capable of diagnosing the same fault category are used as one category. One screening rule may diagnose multiple fault categories at the same time, so one screening rule may exist in multiple fault categories. For example: the first fault class contains { rule 1, rule 2, rule 3}, the second fault class contains { rule 1, rule 3, rule 4}, and the third fault class contains { rule 2, rule 5}. After the target fault class is determined, it may be further determined which screening rules are specifically included in the target fault class.
Further, by combining the occurrence number of each screening rule with the first weight value of the target fault class which can be diagnosed, the second weight value corresponding to each screening rule can be determined.
Optionally, the substep 2051 specifically includes:
sub-step 20511, if the screening rule only appears in one target fault class, using the first weight value corresponding to the target fault class as the second weight value corresponding to the screening rule.
Sub-step 20512, if the screening rule appears in a plurality of target fault categories, adding the first weight values corresponding to the plurality of target fault categories to be the second weight values corresponding to the screening rule.
In the embodiment of the invention, for the screening rule only appearing in one target fault category, the second weight value of the screening rule is equal to the first weight value of the corresponding target fault category; for a screening rule that occurs in multiple target fault categories, the second weight value of the screening rule is equal to the sum of the first weight values of the target fault categories in which it occurs.
Referring to sub-step 2051, and table 1, table 1 is an example table of second weight values corresponding to each screening rule, for example: rule 1 occurs in the first fault class and the second fault class, and then the second weight value WR corresponding to rule 1 is the first weight value W of the first fault class 1 And a first weight value W of a second fault class 2 And, rule 1 occurs a number CR of 2; rule 2 occurs in the first and third fault categories, and then the second weight value WR corresponding to rule 2 is the first weight value W of the first fault category 1 And a first weight value W of a third fault class 3 The sum of which, rule 2 occurs a number CR of 2; rule 3 occurs in the first fault class and the second fault class, then the second weight value WR corresponding to rule 3 is the first weight value W of the first fault class 1 And a first weight value W of a second fault class 2 The sum of which, rule 3 occurs a number CR of 2; rule 4 occurs in the second failure category, then the second weight WR corresponding to rule 4 is the first weight W of the second failure category 2 The number CR of occurrences of rule 4 is 1; rule 5 appears in the third fault category, then the second weight value WR corresponding to rule 5 is the first weight value W of the third fault category 3 Rule 5 occurs a number CR of 1. And when more screening rules exist, similarly, calculating a second weight value corresponding to each screening rule.
TABLE 1
Rules of 1 2 3 4 5 ……
Weight WR W 1 +W 2 W 1 +W 3 W 1 +W 2 W 2 W 3 ……
Number CR of times C 1 ＝2 C 2 ＝2 C 3 ＝2 C 4 ＝1 C 5 ＝1 ……
In step 2052, for each index data, a third weight value corresponding to the index data is determined according to the number of occurrences of the index data in different filtering rules and the second weight value corresponding to the filtering rule in which the index data occurs.
In the embodiment of the invention, each screening rule may correspond to a plurality of index data, and each index data may also appear in a plurality of screening rules, so for each index data, the third weight value corresponding to the index data may be determined by the number of occurrences of the index data in different screening rules and the second weight value corresponding to the screening rule in which the index data occurs.
Referring to table 2, table 2 is a rule index data statistics table, wherein,indicating that the index data is included in the current screening rule. It can be seen that the number of index data contained in each filtering rule may be different, and each index data may appear in a plurality of filtering rules, and the index data may be used to determine whether the log file conforms to the filtering rule.
Further, in table 2, rule 1 includes index 1, index 2, index 3, and index 5, rule 2 includes index 2, index 3, and index 7, rule 3 includes index 4, and index 5, rule 4 includes index 1, index 6, index 7, and index 8, and rule 5 includes index 1, index 4, index 7, and index 8. The third weight value corresponding to each index data may be a sum of products of the occurrence times of the index data in different filtering rules and the second weight value corresponding to the filtering rule of the occurrence index data. The number of occurrences of the index data is counted as the number of screening rules including the index data. For example, index 1 appears in rule 1 and rule 4, the number of occurrences is 2, and the third weight value corresponding to index 1 may be: WT (WT) 1 ＝CR 1 ×WR 1 +CR 4 ×WR 4 . Similarly, a third weight value corresponding to each index data may be determined.
TABLE 2 statistical representation of rule index data
And 206, determining a fourth weight value corresponding to each log file contained in the server to be acquired according to the third weight value of the index data, and acquiring the log files of which the fourth weight value belongs to a second preset range.
This step is referred to step 105 and will not be described here.
Optionally, step 206 specifically includes:
sub-step 2061, determining a third ratio between the total number of index data contained in the log file and the total number of index data corresponding to the screening rule.
In sub-step 2062, a fourth weight value corresponding to each log file is determined according to the sum of the third weight values corresponding to all the index data contained in the log file and the third ratio.
In the embodiment of the present invention, after determining the third weight value corresponding to each index data, the fourth weight value corresponding to each log file of the server to be collected may be determined according to the number of index data included in each log file and the third weight value corresponding to each index data in the log file corresponding to the server to be collected.
For example, if the content of the index data is referred to in the log file information, the log file is considered to contain the index data, the number of index data contained in the log file is recorded as CF, the fourth weight value corresponding to the log file is the product of the proportion of the contained index data and the sum of the third weight values of the contained index data, and the weight WF of the i-th log file can be calculated by the following expression:
the number of index data contained in the log file is CF, and N is the total number of index data. According to the expression, a sorting result of the log files according to the fourth weight value can be output, and the log files with the fourth weight value in the second preset range are used as log files to be collected. The second preset range may be the first hundredth, the first twenty percent, etc. The second preset range may be set according to practical situations, and embodiments of the present invention are not limited herein. The log files in the second preset range are determined to be the log files which need to be acquired by the server to be acquired, so that the number of the log files which need to be acquired by the server to be acquired can be reduced, the log files associated with the fault of the diagnostic server can be accurately acquired, the time for acquiring the log is saved, the efficiency of diagnosing the server is improved, and meanwhile, the storage resources of the server are saved due to the small number of the log files.
In summary, in the embodiment of the present application, the first feature information of the server to be collected is compared with the second feature information in the preset database, so that it is determined that the same type of target server belongs to the same type of target server as the server to be collected in the preset database, and the probability of occurrence of the same type of faults is high. After the target server is determined, the target fault class with a larger first weight value is determined according to the first weight value of the fault class corresponding to the target server, the screening rule corresponding to the target fault class and the weight value of index data are further determined according to the target fault class, when the log file is collected for the server, the weight value of the log file can be calculated according to the index data included in the server, and the log file associated with diagnosing the fault of the server can be accurately collected when the log file is collected by sequencing the weight values of the log file, so that the time for collecting the log is saved, the efficiency of diagnosing the server is improved, and meanwhile, the storage resources of the server are saved for a small number of log files.
Referring to fig. 3, a log file collecting apparatus 30 provided in an embodiment of the present application is shown, where the apparatus includes:
An obtaining module 301, configured to obtain first feature information of a server to be collected;
a first determining module 302, configured to compare the first feature information with second feature information of a server included in a preset database, and determine that a target server in the preset database and the server to be collected belong to the same class;
a second determining module 303, configured to determine, according to a first weight value of a fault class corresponding to the target server, a target fault class where the first weight value is in a first preset range;
a third determining module 304, configured to determine a third weight value of the index data according to a second weight value of a screening rule corresponding to the target fault class and the number of occurrences of the screening rule including the index data; the index data is used for judging whether the log file accords with the screening rule or not;
and the acquisition module 305 is configured to determine fourth weight values corresponding to the log files included in the server to be acquired according to the third weight values of the index data, and acquire the log files whose fourth weight values belong to a second preset range.
Optionally, the first determining module includes:
The first computing sub-module is used for determining the Euclidean distance between the first characteristic information and the second characteristic information of the server contained in the preset database;
and the first determining submodule is used for determining the server, of which the Euclidean distance between the server and the first characteristic information in the preset database is smaller than a preset threshold value, as the target server belonging to the same class as the server to be acquired.
Optionally, the second determining module includes:
a second calculation sub-module, configured to determine, for each failure category in the preset database, a first ratio of the number of servers included in the failure category to the total number of servers in the preset database;
a third calculation sub-module, configured to determine, for each failure category corresponding to the target server, a second ratio of the number of target servers included in the failure category to the number of servers included in a preset database in the failure category;
and the second determining submodule is used for determining a first weight value of the fault category corresponding to the target server according to the first ratio and the second ratio, and determining the fault category of which the first weight value is in a first preset range as the target fault category.
Optionally, the second determining sub-module is further configured to
And taking the product of the first ratio corresponding to the fault class corresponding to the target server and the second ratio corresponding to the fault class corresponding to the target server as a first weight value of the fault class corresponding to the target server.
Optionally, the third determining module includes:
a third determining submodule, configured to determine a second weight value of a screening rule corresponding to the target fault class according to the first weight value corresponding to the target fault class;
and a fourth determining sub-module, configured to determine, for each index data, a third weight value corresponding to the index data according to the number of occurrences of the index data in different filtering rules and a second weight value corresponding to the filtering rule in which the index data occurs.
Optionally, the third determining submodule is further configured to:
if the screening rule only appears in one target fault category, taking a first weight value corresponding to the target fault category as a second weight value corresponding to the screening rule;
and if the screening rule appears in the multiple target fault categories, adding the first weight values corresponding to the multiple target fault categories to serve as the second weight values corresponding to the screening rule.
Optionally, the acquisition module comprises;
a fourth calculation sub-module, configured to determine a third ratio between a total number of index data included in the log file and a total number of index data corresponding to the filtering rule;
and a fifth determining submodule, configured to determine a fourth weight value corresponding to each log file according to a sum of third weight values corresponding to all index data included in the log file and the third ratio.
In summary, in the embodiment of the present application, the first feature information of the server to be collected is compared with the second feature information in the preset database, so that it is determined that the same type of target server belongs to the same type of target server as the server to be collected in the preset database, and the probability of occurrence of the same type of faults is high. After the target server is determined, the target fault class with a larger first weight value is determined according to the first weight value of the fault class corresponding to the target server, the screening rule corresponding to the target fault class and the weight value of index data are further determined according to the target fault class, when the log file is collected for the server, the weight value of the log file can be calculated according to the index data included in the server, and the log file associated with diagnosing the fault of the server can be accurately collected when the log file is collected by sequencing the weight values of the log file, so that the time for collecting the log is saved, the efficiency of diagnosing the server is improved, and meanwhile, the storage resources of the server are saved for a small number of log files.
Fig. 4 illustrates a block diagram of an electronic device 600, according to an example embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 4, the electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input/output (I/O) interface 612, a sensor component 614, and a communication component 616.
The processing component 602 generally controls overall operation of the electronic device 600, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 602 can include one or more modules that facilitate interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate interaction between the multimedia component 608 and the processing component 602.
The memory 604 is used to store various types of data to support operations at the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, multimedia, and so forth. The memory 604 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 606 provides power to the various components of the electronic device 600. The power supply components 606 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.
The multimedia component 608 includes a screen between the electronic device 600 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense demarcations of touch or sliding actions, but also detect durations and pressures associated with the touch or sliding operations. In some embodiments, the multimedia component 608 includes a front camera and/or a rear camera. When the electronic device 600 is in an operational mode, such as a shooting mode or a multimedia mode, the front-facing camera and/or the rear-facing camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 610 is for outputting and/or inputting audio signals. For example, the audio component 610 includes a Microphone (MIC) for receiving external audio signals when the electronic device 600 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, audio component 610 further includes a speaker for outputting audio signals.
The I/O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 614 includes one or more sensors for providing status assessment of various aspects of the electronic device 600. For example, the sensor assembly 614 may detect an on/off state of the electronic device 600, a relative positioning of the components, such as a display and keypad of the electronic device 600, the sensor assembly 614 may also detect a change in position of the electronic device 600 or a component of the electronic device 600, the presence or absence of a user's contact with the electronic device 600, an orientation or acceleration/deceleration of the electronic device 600, and a change in temperature of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 616 is utilized to facilitate communication between the electronic device 600 and other devices, either in a wired or wireless manner. The electronic device 600 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 600 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for implementing a log file collection method as provided by embodiments of the present application.
In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as memory 604, including instructions executable by processor 620 of electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
Fig. 5 illustrates a block diagram of an electronic device 700, according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. Referring to fig. 5, electronic device 700 includes a processing component 722 that further includes one or more processors and memory resources represented by memory 732 for storing instructions, such as application programs, executable by processing component 722. The application programs stored in memory 732 may include one or more modules that each correspond to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform a log file collection method provided in the embodiments of the present application.
The electronic device 700 may also include a power supply component 726 configured to perform power management of the electronic device 700, a wired or wireless network interface 750 configured to connect the electronic device 700 to a network, and an input output (I/O) interface 758. The electronic device 700 may operate based on an operating system stored in memory 732, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A log file collection method, the method comprising:
acquiring first characteristic information of a server to be acquired;
comparing the first characteristic information with second characteristic information of a server contained in a preset database, and determining a target server belonging to the same class as the server to be acquired in the preset database;
determining a target fault class of which the first weight value is in a first preset range according to a first weight value of the fault class corresponding to the target server;
determining a third weight value of index data according to a second weight value of a screening rule corresponding to the target fault class and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule or not;
and determining a fourth weight value corresponding to each log file contained in the server to be acquired according to the third weight value of the index data, and acquiring the log files of which the fourth weight value belongs to a second preset range.
2. The method according to claim 1, wherein the comparing the first characteristic information with second characteristic information of a server included in a preset database, and determining a target server in the preset database that belongs to the same class as the server to be collected, includes:
determining the Euclidean distance between the first characteristic information and second characteristic information of a server contained in the preset database;
and determining the server, of which the Euclidean distance between the server and the first characteristic information in the preset database is smaller than a preset threshold value, as the target server belonging to the same class as the server to be acquired.
3. The method of claim 1, wherein determining, according to a first weight value of the failure category corresponding to the target server, the target failure category having the first weight value in a first preset range comprises:
determining a first ratio of the number of servers contained in the fault class to the total number of servers in the preset database for each fault class in the preset database;
determining a second ratio of the number of target servers contained in the fault class to the number of servers contained in a preset database for each fault class corresponding to the target server;
And determining a first weight value of a fault class corresponding to the target server according to the first ratio and the second ratio, and determining the fault class of which the first weight value is in a first preset range as a target fault class.
4. A method according to claim 3, wherein said determining a first weight value for a failure class corresponding to the target server according to the first ratio and the second ratio comprises:
and taking the product of the first ratio corresponding to the fault class corresponding to the target server and the second ratio corresponding to the fault class corresponding to the target server as a first weight value of the fault class corresponding to the target server.
5. The method according to claim 1, wherein determining the third weight value of the index data according to the second weight value of the screening rule corresponding to the target fault class and the number of occurrences of the screening rule including the index data comprises:
determining a second weight value of a screening rule corresponding to the target fault class according to the first weight value corresponding to the target fault class;
and determining a third weight value corresponding to the index data according to the occurrence times of the index data in different screening rules and the second weight value corresponding to the screening rule of the occurrence of the index data aiming at each index data.
6. The method of claim 5, wherein determining the second weight value of the screening rule corresponding to the target fault class according to the first weight value corresponding to the target fault class comprises:
if the screening rule only appears in one target fault category, taking a first weight value corresponding to the target fault category as a second weight value corresponding to the screening rule;
and if the screening rule appears in the multiple target fault categories, adding the first weight values corresponding to the multiple target fault categories to serve as the second weight values corresponding to the screening rule.
7. The method according to claim 1, wherein the determining, according to the third weight value of the index data, a fourth weight value corresponding to each log file included in the server to be collected includes;
determining a third ratio between the total number of index data contained in the log file and the total number of index data corresponding to the screening rule;
and determining a fourth weight value corresponding to each log file according to the sum of third weight values corresponding to all index data contained in the log files and the third ratio.
8. A log file collection device, the device comprising:
the acquisition module is used for acquiring first characteristic information of the server to be acquired;
the first determining module is used for comparing the first characteristic information with second characteristic information of a server contained in a preset database and determining that the target server belonging to the same class as the server to be acquired in the preset database;
the second determining module is used for determining a target fault class of which the first weight value is in a first preset range according to the first weight value of the fault class corresponding to the target server;
the third determining module is used for determining a third weight value of the index data according to a second weight value of the screening rule corresponding to the target fault category and the occurrence times of the screening rule containing the index data; the index data is used for judging whether the log file accords with the screening rule or not;
and the acquisition module is used for determining fourth weight values corresponding to the log files contained in the server to be acquired according to the third weight values of the index data, and acquiring the log files of which the fourth weight values belong to a second preset range.
9. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method of any one of claims 1 to 7.
10. A readable storage medium, characterized in that it has stored thereon a program or instructions which, when executed by a processor, implement the steps of the method according to any of claims 1 to 7.
CN202310484049.8A 2023-04-28 2023-04-28 Log file acquisition method and device, electronic equipment and readable storage medium Pending CN116541238A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310484049.8A CN116541238A (en) 2023-04-28 2023-04-28 Log file acquisition method and device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310484049.8A CN116541238A (en) 2023-04-28 2023-04-28 Log file acquisition method and device, electronic equipment and readable storage medium

Publications (1)

Publication Number Publication Date
CN116541238A true CN116541238A (en) 2023-08-04

Family

ID=87453651

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310484049.8A Pending CN116541238A (en) 2023-04-28 2023-04-28 Log file acquisition method and device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN116541238A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724937A (en) * 2024-02-07 2024-03-19 荣耀终端有限公司 Log resource management method and related device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724937A (en) * 2024-02-07 2024-03-19 荣耀终端有限公司 Log resource management method and related device

Similar Documents

Publication Publication Date Title
US10061762B2 (en) Method and device for identifying information, and computer-readable storage medium
CN111539443B (en) Image recognition model training method and device and storage medium
CN111859020B (en) Recommendation method, recommendation device, electronic equipment and computer readable storage medium
US11475879B2 (en) Method and device for evaluating quality of content, electronic equipment, and storage medium
CN110717328B (en) Text recognition method and device, electronic equipment and storage medium
EP3767488A1 (en) Method and device for processing untagged data, and storage medium
CN116541238A (en) Log file acquisition method and device, electronic equipment and readable storage medium
CN111813932B (en) Text data processing method, text data classifying device and readable storage medium
CN113779257A (en) Method, device, equipment, medium and product for analyzing text classification model
CN112884040A (en) Training sample data optimization method and system, storage medium and electronic equipment
CN113590605B (en) Data processing method, device, electronic equipment and storage medium
CN113590944B (en) Content searching method and device
CN114416500A (en) Alarm information identification method, device and equipment based on information system
CN114338587B (en) Multimedia data processing method and device, electronic equipment and storage medium
CN112269730A (en) Abnormal log detection method, abnormal log detection device, and storage medium
CN111489260A (en) Item classification method and device, electronic equipment and storage medium
CN112036507B (en) Training method and device of image recognition model, storage medium and electronic equipment
CN115225702B (en) Information pushing method and device, electronic equipment and storage medium
CN117745121A (en) Model input index management method and device, electronic equipment and readable storage medium
CN112307353B (en) Data processing method and device, electronic equipment and storage medium
CN111428806B (en) Image tag determining method and device, electronic equipment and storage medium
CN114676251A (en) Classification model determining method, device, equipment and storage medium
CN115909363A (en) Bill type determining method, device, equipment and medium based on bill image
CN115033784A (en) Information recommendation method based on search term autonomous learning and electronic device
CN110147472B (en) Detection method and device for cheating sites and detection device for cheating sites

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination