CN110990218A - Visualization and alarm method and device based on mass logs and computer equipment - Google Patents
Visualization and alarm method and device based on mass logs and computer equipment Download PDFInfo
- Publication number
- CN110990218A CN110990218A CN201911155971.2A CN201911155971A CN110990218A CN 110990218 A CN110990218 A CN 110990218A CN 201911155971 A CN201911155971 A CN 201911155971A CN 110990218 A CN110990218 A CN 110990218A
- Authority
- CN
- China
- Prior art keywords
- information
- log
- index
- preset
- alarm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012800 visualization Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000000007 visual effect Effects 0.000 claims abstract description 23
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 4
- 230000002401 inhibitory effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000013079 data visualisation Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000001629 suppression Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3086—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/323—Visualisation of programs or trace data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/80—Database-specific techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Library & Information Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a visualization and alarm method and device based on a mass log, computer equipment and a storage medium. The method comprises the steps of receiving collected log data information, caching the log data information to a target cache region to obtain cache information; filtering, assembling and splitting the cache information to obtain information to be stored; sending and storing information to be stored to a target storage area to obtain database information; reading database information according to a reading mode period; judging whether the log index of the database information has keywords which are the same as the keywords included in the keyword list; if the log index has the keywords which are the same as the keywords included in the keyword list, generating alarm information according to an alarm template and sending the alarm information to a receiving end; and reading the log index of the database information, and correspondingly generating an index display graph for the log index according to the query statement. The method enriches the display effect of log visual graphics and the alarm receiving mode, and realizes the alarm inhibiting function.
Description
Technical Field
The invention relates to the field of log visualization and alarm, in particular to a visualization and alarm method and device based on massive logs, computer equipment and a storage medium.
Background
The current method for log management in the industry is usually a management mode of logstack + Elasticsearch + Kibana (elk), where logstack collects logs, Elasticsearch stores and indexes log data, and Kibana visualizes the data. The ELK method makes log management very simple, but it also has technical shortcomings and drawbacks.
The log management in the ELK mode is to realize the visualization of the log through the display function of the Kibana, but the original Kibana has few graphic plug-ins and general effect, and the display content is single and lacks expansibility; the log alarm is generally customized by script, and the alarm is based on the customized script to filter the keywords, so the log alarm has poor expansibility and inflexibility, and has no alarm suppression function.
Disclosure of Invention
The embodiment of the invention provides a visualization and alarm method, a visualization and alarm device and computer equipment based on massive logs, and aims to solve the problems that in the prior art, the visualization display content of log management is single, the alarm cannot be inhibited, and the visualization and alarm expansibility of logs is poor.
In a first aspect, an embodiment of the present invention provides a visualization and alarm method based on a mass log, including:
receiving the collected log data information;
caching the log data information to a preset target cache region to obtain cache information;
carrying out data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored;
sending and storing the information to be stored to a preset target storage area to obtain database information;
reading the database information according to a preset reading mode period;
judging whether a keyword which is the same as a keyword included in a preset keyword list exists in a log index of the database information;
if the log index has the same key words as the key words included in the key word list, generating alarm information according to a preset alarm template, and sending the alarm information to a preset receiving end;
and reading the log index of the database information, and correspondingly generating an index display graph for the log index according to a preset query statement.
In a second aspect, an embodiment of the present invention provides a visualization and alarm device based on a mass log, including:
the receiving unit is used for receiving the collected log data information;
the cache unit is used for caching the log data information to a preset target cache area to obtain cache information;
the processing unit is used for carrying out data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored;
the storage unit is used for sending and storing the information to be stored to a preset target storage area to obtain database information;
the reading unit is used for reading the database information according to a preset reading mode period;
the judging unit is used for judging whether the log index of the database information has keywords which are the same as the keywords included in a preset keyword list;
the warning unit is used for generating warning information according to a preset warning template and sending the warning information to a preset receiving end;
and the graph display unit is used for correspondingly generating an index display graph by the log index according to a preset query statement.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor, when executing the computer program, implements the method for visualization and alarm based on mass logs according to the first aspect.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to perform the method for visualization and alarm based on mass logs according to the first aspect.
The embodiment of the invention provides a visualization and alarm method, a visualization and alarm device, computer equipment and a storage medium based on massive logs, wherein the method comprises the steps of receiving collected log data information; caching the log data information to a preset target cache region to obtain cache information; carrying out data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored; sending and storing the information to be stored to a preset target storage area to obtain database information; reading the database information according to a preset reading mode period; judging whether a keyword which is the same as a keyword included in a preset keyword list exists in a log index of the database information; if the log index has the same key words as the key words included in the key word list, generating alarm information according to a preset alarm template, and sending the alarm information to a preset receiving end; and reading the log index of the database information, and correspondingly generating an index display graph for the log index according to a preset query statement.
The method provides customized display with rich graphs and an alarm inhibiting function, realizes clear, beautiful and readable visual display effect in log management, and can approach real-time alarm, avoid alarm flooding and enrich alarm receiving modes.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a visualization and alarm method based on a mass log according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a visualization and alarm method based on a mass log according to an embodiment of the present invention;
fig. 3 is a sub-flow diagram of a visualization and alarm method based on a mass log according to an embodiment of the present invention;
fig. 4 is another sub-flow diagram of a visualization and alarm method based on a mass log according to an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a visualization and alarm apparatus based on massive logs according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of sub-units of a visualization and alarm device based on massive logs according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of another subunit of the apparatus for visualization and alarm based on massive logs according to the embodiment of the present invention;
FIG. 8 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a visualization and alarm method based on a mass log according to an embodiment of the present invention; fig. 2 is a schematic flow diagram of a visualization and alarm method based on mass logs according to an embodiment of the present invention, where the visualization and alarm method based on mass logs is applied to a server, and the method is executed by application software installed in the server.
As shown in fig. 2, the method includes steps S110 to S180.
And S110, receiving the collected log data information.
In this embodiment, a specific application scenario is based on management work of a large amount of logs, and it is necessary to manage log data information of a daily server, and implement timely warning and graphical display of log data information visualization for error information therein. The file is a log data collector of a local file, monitors a log directory or a specific log file, collects log data information at a server side, and starts one or more finders to check a local path specified by a log administrator for the log file when the file is started. The survey component Prospector and the harvest component Harvester work cooperatively to monitor and read event data change of the log catalog or a specific log file, and send the event data change to an output system preset by a log manager for Filebeat. In specific application, a log data collector is called to collect log data information at a corresponding target server, and the collected log data information is received according to preset settings.
And S120, caching the collected log data information.
In this embodiment, in order to avoid a performance bottleneck of the log collection management tool Logstash caused by receiving a large amount of log file data information collected by the log data collector Filebeat at the server side, after receiving the log file information collected by the log data collector Filebeat, the log file information is sent to the message type Topic of the distributed publish-subscribe message system Kafka according to a target cache region preset by the log administrator, so as to cache the collected log data information to obtain cache information, and the log collection management tool Logstash is gradually made to read the cache information.
Caching the log data information to Kafka is to write the log data information into the Topic so that the log collection management tool Logstash can read the log data information from the Topic step by step. The distributed publish-subscribe messaging system Kafka is a high-throughput distributed publish-subscribe messaging system and has the advantages of high performance, persistence, multi-copy backup and horizontal expansion capability. One Topic is actually composed of a plurality of partitions, the ordering of messages is guaranteed in a single Partition, and when the log data information cached by the distributed publish-subscribe message system Kafka meets a bottleneck, the number of partitions can be increased in the system to perform horizontal expansion.
S130, performing data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored.
After the log data information is cached to a preset target cache region and the cache information is obtained, the cache information is gradually read by the log collection management tool Logstash so as to perform data information filtering, data information assembling and data information splitting on the cache information and obtain information to be stored.
The log collection management tool Logstash is a tool for collecting logs, and the cache information is collected, analyzed and filtered through an Agent deployed on an application program running node. In the data processing process of the log collection management tool logstack, plug-ins Filters, plug-ins Outputs and plug-ins Codecs which need to be used are set by defining a Pipeline configuration file so as to realize specific data collection, data processing and data output of log data information; further, the cache information is obtained by inputting plug-ins, and the plug-ins are used for obtaining the cache information; and then sending the cache information to a filter plug-in, screening the cache information through a screening condition preset in the filter plug-in by the plug-in Filters to obtain screened information, assembling the screened information through a preset data assembly format to obtain assembled information, and cutting and splitting the assembled information through a preset regular expression to obtain split information. A regular expression is a text pattern that describes one or more strings to be matched when searching for text, and further expresses a filtering logic for the strings. Forming information to be stored by all information included in the split information; and the information to be stored is sent to a system target storage area Elasticissearch through the plug-in Outputs according to a preset path.
And S140, sending and storing the information to be stored to a preset target storage area to obtain database information.
In this embodiment, the log collection management tool Logstash performs data information filtering, data information assembling and data information splitting on the cache information to form the information to be stored, and the information to be stored is sent to the target storage area Elasticsearch according to a preset setting so as to store the information to be stored to obtain the database information.
The Elasticissearch in the target storage area is a search server based on Lucene, provides a full-text search engine with distributed multi-user capability based on RESTful web interfaces, can realize distributed, high-expansion and high-real-time search and data analysis, and generates log indexes of the database information for the database information stored in the Elasticissearch through HTTP.
In one embodiment, as shown in fig. 3, step S140 further includes:
and S141, generating a corresponding index name of the database information according to the writing event time of the database information.
In this embodiment, after the information to be stored is sent to a preset target storage area and stored to obtain database information, the Elasticsearch as a search server may automatically create an index according to the database information, and after the database information is sent to the target storage area, the server may create and generate a corresponding index name of the database information according to the write event time of the database information.
And S142, mapping the database information to a corresponding target index according to the index name.
In this embodiment, after generating the corresponding index name of the database information according to the write event time of the database information, the Elasticsearch has an index template, and maps the database information to the corresponding target index according to the index name. The object storage area Elasticissearch itself is used as a search server, and a database of the object storage area Elasticissearch itself can be used as a data source to perform real-time search and analysis on the database information to create a log index for generating the database information.
And S150, reading the database information according to a preset reading mode period.
In this embodiment, after the information to be stored is sent and stored to the Elasticsearch of the target storage area to obtain the database information, Elastaler is used as an alarm background, and the database information is read by the Elastaler every 10 seconds according to a preset reading mode. Among them, Elastaler also provides an alarm suppression function to avoid alarm flooding. According to the method, the Elastaler is used as the alarm background, so that the alarm is more timely, the rule configuration is flexible, the alarm receiving mode is enriched, the alarm suppression is provided, and the working efficiency of log managers and operation and maintenance developers is improved.
S160, judging whether the log index of the database information has the same keywords as the keywords included in the preset keyword list.
In this embodiment, the alarm background Elastaler reads the database information according to a preset reading mode in a cycle of 10s, the Elasticsearch creates a log index for generating the database information by using the database information as a data source, compares the log index of the database information with a keyword list preset by a log administrator to determine whether a keyword identical to a keyword included in the preset keyword list exists in the log index of the database information.
S170, if the log index has the same keywords as the keywords included in the keyword list, generating alarm information according to a preset alarm template, and sending the alarm information to a preset receiving end.
Judging whether a log index of the database information has keywords which are the same as keywords included in a preset keyword list, if so, triggering an alarm by an alarm background Elastaler, generating alarm information according to a preset alarm template, and sending the alarm information to a preset receiving terminal, namely, the alarm background Elastaler generates alarm information according to an alarm type corresponding to the alarm template preset by a log manager and sends the alarm information to the log manager or an operation and maintenance developer. Among them, there are also a plurality of alarm types in Elastaler that can implement customized alarm and receiving mode of alarm, such as e-mail alarm, customized wechat push alarm, etc.
If the log index of the database information does not have the keyword which is the same as the keyword included in the preset keyword list, finishing the comparison and judgment between the log index of the database information and the keyword list preset by a log administrator.
And S180, reading the log index of the database information, and correspondingly generating an index display graph according to the log index according to a preset query statement.
In order to realize rich visualization effect and strong customization of the graphic display of the log data information, an open source data visualization tool Grafana is called in the method to realize the visualization graphic display of the log data information, after the information to be stored is sent to a preset target storage area and the database information is obtained, the open source data visualization tool Grafana reads the log index of the database information, and corresponding graphic display is generated according to a customized query statement preset by a log administrator.
The open source data visualization tool Grafana is an open source application written in the go language, is used for the visual display of large-scale index data and simultaneously supports a plurality of different data sources, including the target storage area Elasticissearch in the method, so that the open source data visualization tool Grafana can be used for providing the database information with graphical rich and customized display.
In one embodiment, as shown in fig. 4, step S180 further includes:
and S181, calling a visual query statement corresponding to the visual graph generation.
In the embodiment, the database information stored in the elastic search needs to be visually displayed, and the dashboard of the Grafana needs to be displayed based on the data source, so that a visualization tool Grafana should be operated to configure the database information of the elastic search as the data source, and a visualization query statement corresponding to the visualization graph generation is called to query the database information.
S182, reading the log index of the database information through the visual query statement to obtain target index information in the log index.
In this embodiment, after configuring the database information of the elastic search for the visualization tool Grafana as a data source, reading a log index of the database information through the visualization query statement, and setting a template variable according to the target index and an actual requirement of a log administrator by the visualization tool Grafana to perform custom field screening to obtain target index information in the log index.
And S183, calling a visual graph generating statement, and correspondingly generating an index display graph by the target index information.
In this embodiment, a visual graph generating statement is called, and the target index information is correspondingly generated into an index display graph. Configuring a data source of an open source data visualization tool, presetting a Grafana visualization mode and adjusting corresponding variable data according to the requirements of a log administrator on log graphical display, establishing a corresponding instrument panel, and realizing the visualization of customized mass log data information. The visualization tool Grafana has various visualization modes, such as Graph, Table, Pie chart and the like.
The embodiment of the invention also provides a visualization and alarm device based on the mass logs, which is used for executing any embodiment of the visualization and alarm method based on the mass logs. Specifically, referring to fig. 5, fig. 5 is a schematic block diagram of a visualization and alarm device based on a mass log according to an embodiment of the present invention. The apparatus 100 for visualization and alarm based on massive logs can be configured in a server.
As shown in fig. 5, the apparatus 100 for visualization and alarm based on mass logs includes a receiving unit 110, a caching unit 120, a processing unit 130, a storage unit 140, a reading unit 150, a judging unit 160, an alarm unit 170, and a graphic presentation unit 180.
The receiving unit 110 is configured to receive the collected log data information.
In this embodiment, a specific application scenario is based on management work of a large amount of logs, and it is necessary to manage log data information of a daily server, and implement timely warning and graphical display of log data information visualization for error information therein. The method comprises the steps of acquiring log data information at a corresponding target server by calling a log data acquisition unit, and receiving the log data information acquired by a filebed of the log data acquisition unit. The journal data collector Filebeat collects daily massive journal data information at a server side and monitors event data changes of a read journal directory or a specific journal file.
The caching unit 120 is configured to cache the log data information to a preset target caching area, so as to obtain caching information.
In this embodiment, after receiving the log data information collected by the log data collector Filebeat, the log data information is sent to and stored in a pre-configured target cache area Kafka, and the log data information is written into the message type Topic of the distributed publish-subscribe message system Kafka to obtain the cache information, so that the log collection management tool logstack can read the cache information step by step, thereby avoiding the performance bottleneck of the log collection management tool logstack caused by the mass log data information collected by the log data collector Filebeat at the server side.
And the processing unit 130 is configured to perform data information filtering, data information assembling, and data information splitting on the cache information to obtain information to be stored.
In this embodiment, after the log data information is cached in a preset target cache region to obtain the cache information, the log collection management tool Logstash gradually reads the cache information from the Topic of the distributed publish-subscribe message system Kafka, so as to perform data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored. And the information to be stored is sent to a target storage area Elasticissearch through an output plug-in according to the preset setting of a log administrator.
The storage unit 140 is configured to send and store the information to be stored to a preset target storage area, so as to obtain database information.
In this embodiment, the log collection management tool Logstash performs data information filtering, data information assembling and data information splitting on the cache information to form the information to be stored, and sends the information to be stored to the target storage area Elasticsearch according to the preset setting of a log administrator through an output plug-in so as to store the information to be stored to obtain the database information.
In one embodiment, as shown in fig. 6, the storage unit 140 includes:
the index name creating unit 141 is configured to generate a corresponding index name of the database information according to the write event time of the database information.
In this embodiment, after the information to be stored is sent to a preset target storage area and stored to obtain database information, the Elasticsearch as a search server may automatically create an index according to the database information, and after the database information is sent to the target storage area, the server may create and generate a corresponding index name of the database information according to the write event time of the database information.
A mapping unit 142, configured to map the database information to a corresponding target index according to the index name.
In this embodiment, after generating the corresponding index name of the database information according to the write event time of the database information, the Elasticsearch has an index template, and maps the database information to the corresponding target index according to the index name.
And a reading unit 150, configured to read the database information according to a preset reading mode cycle.
In this embodiment, after the information to be stored is sent and stored to the Elasticsearch of the target storage area to obtain the database information, Elastaler is used as an alarm background, and the database information is read by the Elastaler every 10 seconds according to a preset reading mode. Among them, Elastaler also provides an alarm suppression function to avoid alarm flooding. According to the method, the Elastaler is used as the alarm background, so that the alarm is more timely, the rule configuration is flexible, the alarm receiving mode is enriched, the alarm suppression is provided, and the working efficiency of log managers and operation and maintenance developers is improved.
A determining unit 160, configured to determine whether a keyword identical to a keyword included in a preset keyword list exists in the log index of the database information.
In this embodiment, the alarm background Elastaler reads the database information according to a preset reading mode in a cycle of 10s, the Elasticsearch creates a log index for generating the database information by using the database information as a data source, compares the log index of the database information with a keyword list preset by a log administrator to determine whether a keyword identical to a keyword included in the preset keyword list exists in the log index of the database information.
And the alarm unit 170 is configured to generate alarm information according to a preset alarm template, and send the alarm information to a preset receiving end.
In this embodiment, the database information is read by an alarm background Elastaler in a cycle of 10s, a log index of the database information is compared with a keyword list preset by a log administrator to judge whether a keyword identical to the keyword included in the preset keyword list exists in the log index of the database information, if the keyword identical to the keyword included in the keyword list exists in the log index, the alarm background Elastaler triggers an alarm, alarm information is generated according to a preset alarm template, and alarm information is sent to a preset receiving end;
if the log index of the database information does not have the keyword which is the same as the keyword included in the preset keyword list, finishing the comparison and judgment between the log index of the database information and the keyword list preset by a log administrator.
And the graph display unit 180 is configured to correspondingly generate an index display graph from the log index according to a preset query statement.
In this embodiment, in order to implement rich visual graphic display on massive log data information, an open source data visualization tool Grafana is called in the method to implement visual graphic display on the log data information. After the information to be stored is sent and stored to a preset target storage area to obtain database information, the Grafana reading the log index of the database information of the elastic search and generating a corresponding graphic display according to a customized query statement preset by a log administrator.
In one embodiment, as shown in fig. 7, the graphic display unit 180 includes:
181. and the first calling unit is used for calling the visual query statement corresponding to the visual graph generation.
In the embodiment, the database information stored in the elastic search needs to be visually displayed, and the dashboard of the Grafana needs to be displayed based on the data source, so that a visualization tool Grafana should be operated to configure the database information of the elastic search as the data source, and a visualization query statement corresponding to the visualization graph generation is called to query the database information.
182. And the query unit is used for reading the log index of the database information through the visual query statement to obtain target index information in the log index.
In this embodiment, after configuring the database information of the elastic search for the visualization tool Grafana as a data source, reading a log index of the database information through the visualization query statement, and setting a template variable according to the target index and an actual requirement of a log administrator by the visualization tool Grafana to perform custom field screening to obtain target index information in the log index.
183. And the second calling unit is used for calling the visual graph generating statement and correspondingly generating the index display graph from the target index information.
In this embodiment, a visual graph generating statement is called, and the target index information is correspondingly generated into an index display graph. Configuring a data source of an open source data visualization tool Grafana, presetting a Grafana visualization mode and adjusting corresponding variable data according to the requirements of a log administrator on log graphical display, establishing a corresponding instrument panel, and realizing customized graphical display of mass log data information. The visualization tool Grafana has various visualization modes, such as Graph, Table, Pie chart and the like.
The above-mentioned visualization and alarm device based on mass logs can be implemented in the form of a computer program, which can be run on a computer device as shown in fig. 8.
Referring to fig. 8, fig. 8 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device 500 is a server, and the server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 8, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer programs 5032, when executed, cause the processor 502 to perform a method for mass log based visualization and alerting.
The processor 502 is used to provide computing and control capabilities that support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be enabled to perform a method for visualization and warning based on mass logs.
The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 8 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 500 to which aspects of the present invention may be applied, and that a particular computing device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The processor 502 is configured to run the computer program 5032 stored in the memory to implement the visualization and alarm method based on mass logs disclosed in the embodiment of the present invention.
Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 8 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 8, and are not described herein again.
It should be understood that, in the embodiment of the present invention, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method for visualization and alarm based on mass logs disclosed by the embodiments of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A visualization and alarm method based on massive logs is characterized by comprising the following steps:
receiving the collected log data information;
caching the log data information to a preset target cache region to obtain cache information;
carrying out data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored;
sending and storing the information to be stored to a preset target storage area to obtain database information;
reading the database information according to a preset reading mode period;
judging whether a keyword which is the same as a keyword included in a preset keyword list exists in a log index of the database information;
if the log index has the same key words as the key words included in the key word list, generating alarm information according to a preset alarm template, and sending the alarm information to a preset receiving end;
and reading the log index of the database information, and correspondingly generating an index display graph for the log index according to a preset query statement.
2. The visualization and alarm method based on massive logs according to claim 1, wherein after the information to be stored is sent to a preset target storage area and stored, and database information is obtained, the method further comprises:
generating a corresponding index name of the database information according to the writing event time of the database information;
and mapping the database information to a corresponding target index according to the index name.
3. The visualization and alarm method based on massive logs according to claim 1, wherein the reading of the log index of the database information and the corresponding generation of the log index into an index display graph according to a preset query statement comprises:
calling a visual query statement corresponding to the visual graph generation;
reading the log index of the database information through the visual query statement to obtain target index information in the log index;
and calling a visual graph generating statement, and correspondingly generating an index display graph from the target index information.
4. The visualization and alarm method based on massive logs according to claim 1, wherein before receiving the collected log data information, the method further comprises:
and calling a plurality of log data collectors, and respectively sending the plurality of log data collectors to corresponding target servers to collect log data information.
5. The visualization and alarm method based on massive logs according to claim 1, wherein the step of performing data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored comprises:
obtaining the cache information through an input plug-in;
the input plug-in sends the cache information to a filter plug-in, and the cache information is screened through screening conditions preset in the filter plug-in to obtain screened information;
assembling the screened information through a preset data assembly format to obtain assembled information;
and cutting and splitting the assembled information through a preset regular expression to obtain split information, and forming information to be stored by each piece of information included in the split information.
6. A visualization and alarm device based on massive logs is characterized by comprising:
the receiving unit is used for receiving the collected log data information;
the cache unit is used for caching the log data information to a preset target cache area to obtain cache information;
the processing unit is used for carrying out data information filtering, data information assembling and data information splitting on the cache information to obtain information to be stored;
the storage unit is used for sending and storing the information to be stored to a preset target storage area to obtain database information;
the reading unit is used for reading the database information according to a preset reading mode period;
the judging unit is used for judging whether the log index of the database information has keywords which are the same as the keywords included in a preset keyword list;
the warning unit is used for generating warning information according to a preset warning template and sending the warning information to a preset receiving end;
and the graph display unit is used for correspondingly generating an index display graph by the log index according to a preset query statement.
7. The mass log based visualization and alarm device according to claim 6, wherein the storage unit comprises:
the index name creating unit is used for generating a corresponding index name of the database information according to the writing event time of the database information;
and the mapping unit is used for mapping the database information to the corresponding target index according to the index name.
8. The mass log based visualization and alarm device according to claim 6, wherein the graphic display unit comprises:
the first calling unit is used for calling a visual query statement corresponding to the visual graph generation;
the query unit is used for reading the log index of the database information through the visual query statement to obtain target index information in the log index;
and the second calling unit is used for calling the visual graph generating statement and correspondingly generating the index display graph from the target index information.
9. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method for mass log based visualization and alerting according to any of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to perform the method of mass log based visualization and alerting of the user according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911155971.2A CN110990218B (en) | 2019-11-22 | 2019-11-22 | Visualization and alarm method and device based on massive logs and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911155971.2A CN110990218B (en) | 2019-11-22 | 2019-11-22 | Visualization and alarm method and device based on massive logs and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110990218A true CN110990218A (en) | 2020-04-10 |
| CN110990218B CN110990218B (en) | 2023-12-26 |
Family
ID=70085919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911155971.2A Active CN110990218B (en) | 2019-11-22 | 2019-11-22 | Visualization and alarm method and device based on massive logs and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990218B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581054A (en) * | 2020-04-30 | 2020-08-25 | 重庆富民银行股份有限公司 | ELK-based log point-burying service analysis and alarm system and method |
| CN111737091A (en) * | 2020-08-27 | 2020-10-02 | 北京安帝科技有限公司 | Log processing method and device and readable medium |
| CN111881011A (en) * | 2020-07-31 | 2020-11-03 | 网易（杭州）网络有限公司 | Log management method, platform, server and storage medium |
| CN112632070A (en) * | 2020-12-23 | 2021-04-09 | 中国人民解放军63921部队 | Method and device for storing and copying massive diversified complex spacecraft simulation data |
| CN112631885A (en) * | 2020-12-18 | 2021-04-09 | 武汉市烽视威科技有限公司 | Method and system for pre-judging fault in advance and automatically repairing fault |
| CN112667574A (en) * | 2020-12-23 | 2021-04-16 | 国网宁夏电力有限公司信息通信公司 | Method and system for screening mass log data |
| CN112685376A (en) * | 2020-12-23 | 2021-04-20 | 国网宁夏电力有限公司信息通信公司 | Massive log data analysis method and system |
| CN112732663A (en) * | 2020-12-30 | 2021-04-30 | 浙江大华技术股份有限公司 | Log information processing method and device |
| CN112767636A (en) * | 2021-01-14 | 2021-05-07 | 广州穗能通能源科技有限责任公司 | Fire alarm method, fire alarm device, computer equipment and storage medium |
| CN113138896A (en) * | 2021-04-25 | 2021-07-20 | 中国工商银行股份有限公司 | Application running condition monitoring method, device and equipment |
| CN113904913A (en) * | 2021-08-19 | 2022-01-07 | 济南浪潮数据技术有限公司 | Alarm processing method, device, equipment and storage medium based on pipeline |
| CN114143178A (en) * | 2021-12-03 | 2022-03-04 | 中国电信集团系统集成有限责任公司 | TR069 protocol-combined alarm root positioning visualization method and device |
| CN114500255A (en) * | 2022-03-01 | 2022-05-13 | 北京百度网讯科技有限公司 | Log data reporting method, device, equipment and storage medium |
| CN114625595A (en) * | 2020-12-14 | 2022-06-14 | 网联清算有限公司 | Method, device and system for rechecking dynamic configuration information of service system |
| CN115348161A (en) * | 2022-08-16 | 2022-11-15 | 中国电信股份有限公司 | Log alarm information generation method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107800592A (en) * | 2017-11-09 | 2018-03-13 | 郑州云海信息技术有限公司 | A kind of server test results acquisition method |
| CN108733538A (en) * | 2018-06-26 | 2018-11-02 | 郑州云海信息技术有限公司 | A kind of visualization daily record warning system and method based on ElastAlert |
| CN109542733A (en) * | 2018-12-05 | 2019-03-29 | 焦点科技股份有限公司 | A kind of highly reliable real-time logs collection and visual m odeling technique method |
| CN110191005A (en) * | 2019-06-25 | 2019-08-30 | 北京九章云极科技有限公司 | A kind of alarm log processing method and system |
| CN110362544A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log processing system, log processing method, terminal and storage medium |
| CN110457178A (en) * | 2019-07-29 | 2019-11-15 | 江苏艾佳家居用品有限公司 | A kind of full link monitoring alarm method based on log collection analysis |
-
2019
- 2019-11-22 CN CN201911155971.2A patent/CN110990218B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107800592A (en) * | 2017-11-09 | 2018-03-13 | 郑州云海信息技术有限公司 | A kind of server test results acquisition method |
| CN108733538A (en) * | 2018-06-26 | 2018-11-02 | 郑州云海信息技术有限公司 | A kind of visualization daily record warning system and method based on ElastAlert |
| CN109542733A (en) * | 2018-12-05 | 2019-03-29 | 焦点科技股份有限公司 | A kind of highly reliable real-time logs collection and visual m odeling technique method |
| CN110362544A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log processing system, log processing method, terminal and storage medium |
| CN110191005A (en) * | 2019-06-25 | 2019-08-30 | 北京九章云极科技有限公司 | A kind of alarm log processing method and system |
| CN110457178A (en) * | 2019-07-29 | 2019-11-15 | 江苏艾佳家居用品有限公司 | A kind of full link monitoring alarm method based on log collection analysis |
Non-Patent Citations (4)
| Title |
|---|
| BADUDD: "elk引入redis kafka", 《CSDN博客》 * |
| BADUDD: "elk引入redis kafka", 《CSDN博客》, 6 October 2019 (2019-10-06), pages 1 - 3 * |
| YEWEIOUYANG: "ELK实现日志监控告警", 《CSDN博客》 * |
| YEWEIOUYANG: "ELK实现日志监控告警", 《CSDN博客》, 9 February 2017 (2017-02-09), pages 1 - 8 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581054A (en) * | 2020-04-30 | 2020-08-25 | 重庆富民银行股份有限公司 | ELK-based log point-burying service analysis and alarm system and method |
| CN111581054B (en) * | 2020-04-30 | 2024-04-09 | 重庆富民银行股份有限公司 | Log embedded point service analysis alarm system and method based on ELK |
| CN111881011A (en) * | 2020-07-31 | 2020-11-03 | 网易（杭州）网络有限公司 | Log management method, platform, server and storage medium |
| CN111737091A (en) * | 2020-08-27 | 2020-10-02 | 北京安帝科技有限公司 | Log processing method and device and readable medium |
| CN111737091B (en) * | 2020-08-27 | 2020-12-08 | 北京安帝科技有限公司 | Log processing method and device and readable medium |
| CN114625595A (en) * | 2020-12-14 | 2022-06-14 | 网联清算有限公司 | Method, device and system for rechecking dynamic configuration information of service system |
| CN112631885A (en) * | 2020-12-18 | 2021-04-09 | 武汉市烽视威科技有限公司 | Method and system for pre-judging fault in advance and automatically repairing fault |
| CN112667574A (en) * | 2020-12-23 | 2021-04-16 | 国网宁夏电力有限公司信息通信公司 | Method and system for screening mass log data |
| CN112685376A (en) * | 2020-12-23 | 2021-04-20 | 国网宁夏电力有限公司信息通信公司 | Massive log data analysis method and system |
| CN112632070A (en) * | 2020-12-23 | 2021-04-09 | 中国人民解放军63921部队 | Method and device for storing and copying massive diversified complex spacecraft simulation data |
| CN112632070B (en) * | 2020-12-23 | 2022-11-25 | 中国人民解放军63921部队 | Method and device for storing and copying massive diversified complex spacecraft simulation data |
| CN112732663A (en) * | 2020-12-30 | 2021-04-30 | 浙江大华技术股份有限公司 | Log information processing method and device |
| CN112767636A (en) * | 2021-01-14 | 2021-05-07 | 广州穗能通能源科技有限责任公司 | Fire alarm method, fire alarm device, computer equipment and storage medium |
| CN113138896A (en) * | 2021-04-25 | 2021-07-20 | 中国工商银行股份有限公司 | Application running condition monitoring method, device and equipment |
| CN113904913A (en) * | 2021-08-19 | 2022-01-07 | 济南浪潮数据技术有限公司 | Alarm processing method, device, equipment and storage medium based on pipeline |
| CN114143178A (en) * | 2021-12-03 | 2022-03-04 | 中国电信集团系统集成有限责任公司 | TR069 protocol-combined alarm root positioning visualization method and device |
| CN114143178B (en) * | 2021-12-03 | 2024-06-04 | 中电信数智科技有限公司 | Alarm root-cause positioning visualization method and device combined with TR069 protocol |
| CN114500255A (en) * | 2022-03-01 | 2022-05-13 | 北京百度网讯科技有限公司 | Log data reporting method, device, equipment and storage medium |
| CN114500255B (en) * | 2022-03-01 | 2024-03-15 | 北京百度网讯科技有限公司 | Log data reporting method, device, equipment and storage medium |
| CN115348161A (en) * | 2022-08-16 | 2022-11-15 | 中国电信股份有限公司 | Log alarm information generation method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110990218B (en) | 2023-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110990218A (en) | Visualization and alarm method and device based on mass logs and computer equipment | |
| CN111241078B (en) | Data analysis system, data analysis method and device | |
| US11880399B2 (en) | Data categorization using inverted indexes | |
| US8533193B2 (en) | Managing log entries | |
| US11409645B1 (en) | Intermittent failure metrics in technological processes | |
| CN111651751B (en) | Security event analysis report generation method and device, storage medium and equipment | |
| US20060048101A1 (en) | Program and system performance data correlation | |
| JP4983604B2 (en) | Performance abnormality analyzer, method and program, and analysis result display method of performance abnormality analyzer | |
| CN110955578A (en) | Log collection method and device based on host machine, computer equipment and storage medium | |
| TW200816008A (en) | Adaptive dissemination of personalized and contextually relevant information | |
| Johnson | Requirement and design trade-offs in Hackystat: An in-process software engineering measurement and analysis system | |
| CN106453320B (en) | The recognition methods of malice sample and device | |
| EP3107045A1 (en) | Deep filtering of health and usage management (hums) data | |
| CN112529528B (en) | Workflow monitoring and warning method, device and system based on big data flow calculation | |
| JP2022058555A (en) | Key-based logging for processing of structured data item with executable logic | |
| CN108768790A (en) | Distributed search cluster monitoring method and device, computing device, storage medium | |
| CN111460333B (en) | Real-time search data analysis system | |
| CN115328894A (en) | Data processing method based on data blood margin | |
| US8103660B2 (en) | Computer method and system for contextual management and awareness of persistent queries and results | |
| CN113326237A (en) | Log data processing method and device, terminal device and storage medium | |
| CN113535038A (en) | Front-end menu tree generation method and device, computer equipment and storage medium | |
| CN110347577B (en) | Page testing method, device and equipment thereof | |
| CN117555541B (en) | Data page display method, device, equipment and storage medium | |
| Alwadi et al. | Toward a performance requirements model for the early design phase of IT systems | |
| RU2316043C2 (en) | Mechanism and method for representing event information in access system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |