CN111143160A - System full link monitoring method and device - Google Patents
System full link monitoring method and device Download PDFInfo
- Publication number
- CN111143160A CN111143160A CN201911242804.1A CN201911242804A CN111143160A CN 111143160 A CN111143160 A CN 111143160A CN 201911242804 A CN201911242804 A CN 201911242804A CN 111143160 A CN111143160 A CN 111143160A
- Authority
- CN
- China
- Prior art keywords
- link
- data
- amount
- log
- alarm information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 76
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 238000012806 monitoring device Methods 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 abstract description 11
- 238000009530 blood pressure measurement Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 5
- 238000013480 data collection Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0677—Localisation of faults
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a system full link monitoring method and device. The method comprises the following steps: regularly calculating and storing the total log amount of each system; determining the starting point data volume and the end point data volume of each link according to the total log volume of each system; acquiring processing condition data of each link according to the starting point data volume and the end point data volume; comparing the processing condition data with a processing reference condition, and if the processing condition data does not meet the processing reference condition, generating alarm information; and regularly receiving and storing the log detail data of each system, and inquiring the log detail data corresponding to the alarm information according to the alarm information. The method and the device can help the user to quickly find problems, improve monitoring efficiency, locate abnormal nodes in abnormal links, acquire detailed log data, help the user to analyze link performance, monitor the running condition of each link in real time and plan the link capacity according to the running condition of each link.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a system full link monitoring method and device.
Background
The link is composed of a plurality of systems and comprises a transmission route for data information interaction among the systems. In order to ensure the accuracy and high efficiency of data information transmission, the link needs to be monitored. At present, the monitoring of links with the system adopts manual monitoring, and the abnormal condition is found and reported in time and then is solved by responsible personnel of each system. However, as the traffic volume gradually increases, the arrangement of system links becomes more and more complex, one link includes multiple scenes, the upstream and downstream interaction is frequent, the all-weather monitoring of the whole link cannot be achieved only through manual monitoring and reporting, and the monitoring efficiency and quality are difficult to improve. At present, more evaluations on the performance and capacity of the system are carried out in a pressure measurement environment, the traditional manual pressure measurement monitoring is adopted, the pressure measurement of a full link is difficult to achieve, the difference evaluation of environmental mechanical hardware and a production environment of the pressure measurement cannot be accurate, and the real-time observation of the traffic volume and the link performance condition of the production environment cannot be achieved.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a system full link monitoring method and apparatus. The technical scheme is as follows:
in one aspect, a method for monitoring a system full link is provided, where the method includes:
regularly calculating and storing the total log amount of each system;
determining the starting point data volume and the end point data volume of each link according to the total log volume of each system;
acquiring processing condition data of each link according to the starting point data volume and the end point data volume;
and comparing the processing condition data with a processing reference condition, and if the processing condition data does not meet the processing reference condition, generating alarm information.
Further, periodically calculating the total amount of logs for each system includes:
and regularly receiving the logs of each system, and carrying out aggregation operation on the logs to obtain the total amount of the logs.
Further, determining the start point data volume and the end point data volume of each link according to the total log amount of each system includes:
matching the system with a link structure to obtain a system corresponding to a link starting point and a system corresponding to a link end point of each link;
and determining the data volume of the starting point of the link according to the total log volume of the system corresponding to the starting point of the link, and determining the data volume of the ending point of the link according to the total log volume of the system corresponding to the ending point of the link.
Further, obtaining the processing condition data of each link according to the starting point data amount and the ending point data amount includes:
acquiring the handling capacity of the link according to the starting point data volume;
acquiring the finished quantity of the link according to the end point data quantity;
and acquiring the unfinished amount, the finished rate and the finished rate of the link according to the amount to be processed and the finished amount.
Further, comparing the processing condition data with a processing reference condition, and if the processing condition data does not satisfy the processing reference condition, generating alarm information includes:
if the unprocessed amount does not meet the unprocessed reference condition, generating unprocessed warning information;
if the completion rate does not meet the completion rate reference condition, generating completion rate alarm information;
and if the completion rate does not meet the rate reference condition, generating rate alarm information.
Further, the method further comprises:
and regularly receiving and storing the log detail data of each system, and inquiring the log detail data corresponding to the alarm information according to the alarm information.
Further, the total log amount is stored in a relational database, the log detail data is respectively stored in a HIVE and a search engine, and the log detail data stored in the search engine is used for inquiring the log detail data corresponding to the alarm information according to the alarm information.
In another aspect, a system full link monitoring apparatus is provided, the apparatus includes: a first server, the first server comprising:
the total amount calculation module is used for periodically calculating the total amount of the logs of each system;
a node data volume determining module, configured to determine a starting point data volume and an ending point data volume of each link according to the total log volume of each system;
a processing condition obtaining module, configured to obtain processing condition data of each link according to the starting point data amount and the ending point data amount;
and the alarm information generation module is used for comparing the processing condition data with a processing reference condition and generating the alarm information when the processing condition data does not meet the processing reference condition.
Further, the total amount calculating module is specifically configured to receive logs of each system periodically, perform aggregation operation on the logs, and obtain the total amount of the logs.
Further, the node data amount determination module includes:
the system matching module is used for matching the system with the link structure to obtain a system corresponding to the link starting point and a system corresponding to the link end point of each link;
and the data volume corresponding module is used for determining the starting point data volume of the link according to the total log volume of the system corresponding to the starting point of the link and determining the end point data volume of the link according to the total log volume of the system corresponding to the end point of the link.
Further, the processing situation acquiring module includes:
a first condition obtaining module, configured to obtain a required throughput of the link according to the starting point data amount, and obtain a completed throughput of the link according to the ending point data amount;
and the second condition acquisition module is used for acquiring the unfinished amount, the finished rate and the finished rate of the link according to the amount to be processed and the finished amount.
Further, the alarm information generating module comprises:
the unprocessed warning information generating module is used for generating unprocessed warning information when the unprocessed amount does not meet the unprocessed reference condition;
the completion rate warning information generating module is used for generating completion rate warning information when the completion rate does not meet the completion rate reference condition;
and the rate alarm information generating module is used for generating rate alarm information when the completion rate does not meet the rate reference condition.
Further, the apparatus further comprises: a second server and a third server,
the second server is configured to store the log detail data,
the third server is used for storing the log detail data so as to inquire the log detail data corresponding to the alarm information according to the alarm information.
Further, the first server is a relational database server, the second server is a HIVE tool server, and the third server is a search engine server.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
1. the invention can help the user to quickly find the problem through the alarm information of abnormal operation of each link, thereby improving the monitoring efficiency;
2. according to the method and the device, the abnormal node in the abnormal link can be quickly positioned in the search engine according to the alarm information, and the detailed log data can be obtained;
3. the invention can help the user to analyze the link performance by calculating the link processing condition data, monitor the running condition of each link in real time and plan the link capacity according to the running condition of each link.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a system link monitoring full-architecture diagram provided in an embodiment of the present invention;
FIG. 2 is a flowchart of a method for monitoring a link of a relational database according to an embodiment of the present invention;
fig. 3 is a schematic block diagram of a system link monitoring apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the development of data information technology, the interaction of mass data among multiple systems becomes more and more complex, for example, in the field of logistics, multiple processes such as 'receiving', 'sending', 'getting', 'dispatching', 'signing', and the like in the packaging, sorting, and transportation of goods all involve the data interaction of multiple different modules, each module also contains multiple systems, and with the continuous expansion of services, the types of services are continuously increased, links generated by the data interaction become more and more complex, and the data information amount is larger and larger. Therefore, the requirement of comprehensive monitoring and real-time monitoring cannot be met only by adopting the traditional manual link monitoring during pressure measurement and normal operation, and in order to solve the problems of low accuracy, low efficiency and incapability of comprehensive real-time monitoring of the link monitoring in the prior art, the embodiment of the invention provides a system full-link monitoring method and a system full-link monitoring device, and the specific technical scheme is as follows:
as shown in fig. 1, the data collection platform collects logs of each system, and the data collection platform may adopt any one of KAFKA, flash, and standard API, and in this embodiment, KAFKA is taken as an example. The acquired logs are respectively issued to the HIVE, the search engine and the relational database through the data processing module, the data processing module can be SPARK specifically, and the relational database can be MYSQL, POSTFRESQL and other relational databases.
As shown in fig. 2, after receiving the log, the relational database periodically calculates and stores the total amount of the log according to the log, and specifically includes: and carrying out aggregation operation on the logs to obtain the total number of the logs.
Then, according to the total log amount of each system, determining the starting point data volume and the end point data volume of each link, specifically: matching each system with each link structure to obtain a system corresponding to a link starting point and a system corresponding to a link end point of each link; the total log amount of the system corresponding to the link starting point is used as the starting point data amount of the link, and the total log amount of the system corresponding to the link end point is used as the end point data amount of the link. It should be noted that: the link structure may be stored in a relational database in advance, and the specific link structure may be stored in the relational database in the form of a mapping relationship of the link and the system.
After the starting point data volume and the end point data volume are obtained, the relational database obtains processing condition data of each link according to the starting point data volume and the end point data volume, and specifically, the processing condition data includes: amount of outstanding processing, amount completed, amount not completed, completion rate, and completion rate. Wherein,
acquiring the amount of the required processing according to the amount of the starting point data;
the finished quantity is obtained according to the end point data quantity;
the unfinished amount is the amount of handling-finished amount;
the completion rate is the completed amount/the handling amount;
the completion rate is expressed in transactions per second, which is 5 seconds worth of data processed/5 seconds.
After the processing condition data is obtained, comparing the processing condition data with the processing reference condition, generating alarm information for the processing condition data which does not meet the processing reference condition, wherein the processing reference condition comprises the following steps corresponding to the processing condition data: an unprocessed reference condition, a completed rate reference condition, and a rate reference condition. Specifically, comparing the processing condition data with the processing reference condition, and generating the alarm information includes:
if the unprocessed amount does not meet the unprocessed reference condition, generating unprocessed warning information;
if the completion rate does not meet the completion rate reference condition, generating completion rate alarm information;
and if the finishing rate does not meet the rate reference condition, generating rate alarm information.
The above method illustrates that the alarm information is generated when any one of the processing condition data does not meet the processing reference condition. The advantage of generating the alarm information for each processing condition data is that the user can intuitively know which processing condition data is abnormal to know the link problem. The processing reference data is preset in the relational database and is set according to the load capacity of the system.
It should be noted that, the total log data and alarm information of each system generated by the relational database are stored in the relational database, the relational database sends the total log data and alarm information to the front end for being displayed by the user, each total log is displayed at the front end in a "link detail" manner, and in the "link detail", the total log data of each node of the links at different periods can be displayed in a broken line diagram manner, so that the user can know the operating conditions of each system of each link, and the capacity planning of the link provides data support.
In the method, the relational database analyzes the log quantity and matches the data quantity of the starting point and the end point of the link to acquire the overall condition of the link processing data. Further, when the system is matched with the link structure, the system corresponding to each intermediate node of the link can be obtained besides the system corresponding to the starting point of the link and the system corresponding to the end point of the link, the data volume of each intermediate node of the link can be determined according to the total log amount of the system corresponding to each intermediate node of the link, when the link is analyzed, the data volume of which node is abnormal can be found by combing the data volume of each intermediate node on the link structure, so that the problem can be positioned to the node, when the alarm information is sent out, the problem node can be further positioned, and the problem can be found by a user.
Different from the relational database, the HIVE and the search engine store the detail data in the log after acquiring the log. The HIVE is mainly used for storing the log detail data, the search engine is mainly used for providing a search function of the log detail data, and the storage of the log detail data can adopt a temporary storage mode of coverage updating, namely when new log detail data are issued, old log detail data are covered, and the storage of the log detail data by the HIVE is long-term. The log detail data includes specific data in each log in each system.
After the relational database generates the alarm information, the alarm information is sent to the front end to be displayed, and after the user obtains the alarm information at the front end, the user can search the specific data information in the problem link problem node through the search engine, so that the problem is further found and solved.
The system link monitoring method disclosed by the invention respectively issues the collected logs to the HIVE, the search engine and the relational database, and the HIVE, the search engine and the relational database respectively perform independent functions, so that overlarge running load cannot be caused to each system, and the smooth operation of link monitoring is ensured.
As shown in fig. 3, an embodiment of the present invention corresponding to the method further provides a system full link monitoring apparatus, where the apparatus includes: the system comprises a first server, a second server and a third server, wherein the first server is a relational database server, the second server is a HIVE tool server, and the third server is a search engine server.
The first server includes: the system comprises a total amount calculation module, a node data amount determination module, a processing condition acquisition module and an alarm information generation module.
The total amount calculating module is used for periodically calculating the total amount of the logs of each system, and specifically comprises the following steps: and regularly receiving the logs of each system to perform aggregation operation on the logs to obtain the total amount of the logs.
And the node data volume determining module is used for determining the starting point data volume and the end point data volume of each link according to the total log volume of each system, and is further used for determining the data volume of each node in each link. The node data volume determination module comprises:
the system matching module is used for matching the system with the link structure, acquiring the system corresponding to the link starting point and the system corresponding to the link end point of each link, and further acquiring the system corresponding to each intermediate node of each link;
and the data volume corresponding module is used for determining the starting point data volume of the link according to the total log volume of the system corresponding to the starting point of the link, determining the end point data volume of the link according to the total log volume of the system corresponding to the end point of the link, and further determining the data volume of each intermediate node of the link according to the total log volume of the system corresponding to each intermediate node of the link.
And the processing condition acquisition module is used for acquiring the processing condition data of each link according to the starting point data volume and the end point data volume. The processing situation acquisition module comprises:
the first condition acquisition module is used for determining the starting point data volume as the amount of the link to be processed and determining the end point data volume as the finished amount of the link;
and a second condition obtaining module, configured to obtain the unfinished amount, the finished rate, and the finished rate of the link according to the amount to be processed and the finished amount, where a calculation method of the unfinished amount, the finished rate, and the finished rate is the same as that in the method, and is not described herein again.
And the alarm information generation module is used for comparing the processing condition data with the processing reference condition and generating alarm information when the processing condition data does not meet the processing reference condition. The alarm information generation module comprises:
the unprocessed warning information generating module is used for generating unprocessed warning information when the unprocessed amount does not meet the unprocessed reference condition;
the completion rate warning information generating module is used for generating completion rate warning information when the completion rate does not meet the completion rate reference condition;
and the rate alarm information generating module is used for generating rate alarm information when the finishing rate does not meet the rate reference condition.
In the device, the logs received by the first server and the second server are collected by the data collection platform to each system and are issued by the data processing module. In the first server, a total amount calculation module calculates the total log amount of each system, and a node data amount determination module calls and queries the data amount corresponding to each node to the total amount calculation module after each system corresponds to the node of each link. The node data volume determining module sends the data volume corresponding to each node of each link to the processing condition acquiring module, and the processing condition acquiring module acquires the processing condition data of each link according to the data volume corresponding to each node and sends the processing condition data to the alarm information generating module. And the alarm information generation module compares the processing condition data with a preset processing reference condition, generates alarm information according to a comparison result and sends the alarm information to the front end for displaying. After the front end acquires the alarm information, the search engine is used for inquiring concrete log detail data of a link corresponding to the alarm information according to the alarm information, so that log data of problems can be acquired, and the problems can be quickly found.
In the process of pressure measurement, the system link monitoring device disclosed by the embodiment of the invention can be used for monitoring the running condition of the whole system link in real time, rapidly finding out faults, and judging which links the system with the problems exist in according to the corresponding relation between each system and the link node, thereby determining the influence range of the faults, adjusting the link structure and well planning the capacity of the links.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
1. the invention can help the user to quickly find the problem through the alarm information of abnormal operation of each link, thereby improving the monitoring efficiency;
2. according to the method and the device, the abnormal node in the abnormal link can be quickly positioned in the search engine according to the alarm information, and the detailed log data can be obtained;
3. the invention can help the user to analyze the link performance by calculating the link processing condition data, monitor the running condition of each link in real time and plan the link capacity according to the running condition of each link.
All the above-mentioned optional technical solutions can be combined arbitrarily to form the optional embodiments of the present invention, and are not described herein again.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A system full link monitoring method is characterized by comprising the following steps:
regularly calculating and storing the total log amount of each system;
determining the starting point data volume and the end point data volume of each link according to the total log volume of each system;
acquiring processing condition data of each link according to the starting point data volume and the end point data volume;
and comparing the processing condition data with a processing reference condition, and if the processing condition data does not meet the processing reference condition, generating alarm information.
2. The method as claimed in claim 1, wherein periodically calculating the total log amount of each system comprises:
and regularly receiving the logs of each system, and carrying out aggregation operation on the logs to obtain the total amount of the logs.
3. The method as claimed in claim 1, wherein determining the start point data amount and the end point data amount of each link according to the total log amount of each system comprises:
matching the system with a link structure to obtain a system corresponding to a link starting point and a system corresponding to a link end point of each link;
and determining the data volume of the starting point of the link according to the total log volume of the system corresponding to the starting point of the link, and determining the data volume of the ending point of the link according to the total log volume of the system corresponding to the ending point of the link.
4. The method as claimed in claim 1, wherein obtaining the processing condition data of each link according to the starting point data amount and the ending point data amount comprises:
acquiring the handling capacity of the link according to the starting point data volume;
acquiring the finished quantity of the link according to the end point data quantity;
and acquiring the unfinished amount, the finished rate and the finished rate of the link according to the amount to be processed and the finished amount.
5. The method as claimed in claim 4, wherein the step of comparing the processing condition data with a processing reference condition, and if the processing condition data does not satisfy the processing reference condition, the step of generating the alarm information includes:
if the unprocessed amount does not meet the unprocessed reference condition, generating unprocessed warning information;
if the completion rate does not meet the completion rate reference condition, generating completion rate alarm information;
and if the completion rate does not meet the rate reference condition, generating rate alarm information.
6. The system full link monitoring method according to any one of claims 1 to 5, wherein the method further comprises:
and regularly receiving and storing the log detail data of each system, and inquiring the log detail data corresponding to the alarm information according to the alarm information.
7. The system link-wide monitoring method according to claim 6, wherein the total log amount is stored in a relational database, and the log detail data is stored in a HIVE and a search engine respectively, wherein the log detail data stored in the search engine is used for querying the log detail data corresponding to the alarm information according to the alarm information.
8. A system full link monitoring apparatus, comprising a first server, the first server comprising:
the total amount calculation module is used for periodically calculating the total amount of the logs of each system;
a node data volume determining module, configured to determine a starting point data volume and an ending point data volume of each link according to the total log volume of each system;
a processing condition obtaining module, configured to obtain processing condition data of each link according to the starting point data amount and the ending point data amount;
and the alarm information generation module is used for comparing the processing condition data with the processing reference condition and generating alarm information when the processing condition data does not meet the processing reference condition.
9. The system link-wide monitoring apparatus of claim 8, wherein the apparatus further comprises: a second server and a third server,
the second server is configured to store the log detail data,
the third server is used for storing the log detail data so as to inquire the log detail data corresponding to the alarm information according to the alarm information.
10. The system link-wide monitoring device of claim 9, wherein the first server is a relational database server, the second server is a HIVE tool server, and the third server is a search engine server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911242804.1A CN111143160B (en) | 2019-12-06 | 2019-12-06 | System full link monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911242804.1A CN111143160B (en) | 2019-12-06 | 2019-12-06 | System full link monitoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111143160A true CN111143160A (en) | 2020-05-12 |
| CN111143160B CN111143160B (en) | 2022-09-09 |
Family
ID=70517758
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911242804.1A Active CN111143160B (en) | 2019-12-06 | 2019-12-06 | System full link monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111143160B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111770022A (en) * | 2020-06-28 | 2020-10-13 | 中国平安财产保险股份有限公司 | Link monitoring-based capacity expansion method, system, equipment and computer storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649670A (en) * | 2016-12-14 | 2017-05-10 | 北京五八信息技术有限公司 | Streaming computing-based data monitoring method and apparatus |
| CN110191094A (en) * | 2019-04-26 | 2019-08-30 | 北京奇安信科技有限公司 | Monitoring method and device, storage medium, the terminal of abnormal data |
-
2019
- 2019-12-06 CN CN201911242804.1A patent/CN111143160B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649670A (en) * | 2016-12-14 | 2017-05-10 | 北京五八信息技术有限公司 | Streaming computing-based data monitoring method and apparatus |
| CN110191094A (en) * | 2019-04-26 | 2019-08-30 | 北京奇安信科技有限公司 | Monitoring method and device, storage medium, the terminal of abnormal data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111770022A (en) * | 2020-06-28 | 2020-10-13 | 中国平安财产保险股份有限公司 | Link monitoring-based capacity expansion method, system, equipment and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111143160B (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111158977B (en) | Abnormal event root cause positioning method and device | |
| CN112199421B (en) | Multi-source heterogeneous data fusion and measurement data multi-source mutual verification method and system | |
| CN103607463B (en) | Location data-storage system and storage method | |
| CN111143160B (en) | System full link monitoring method and device | |
| CN114500543A (en) | Distributed elastic edge acquisition system and application method thereof | |
| CN114866396B (en) | Method for realizing network fault location under inaccurate resources based on text similarity | |
| CN110555619A (en) | Power supply capacity evaluation method based on intelligent power distribution network | |
| CN115221218A (en) | Quality evaluation method and device for vehicle data, computer equipment and storage medium | |
| CN111897771A (en) | Gas meter message testing method and system based on intelligent analysis model | |
| CN117172721B (en) | Data flow supervision early warning method and system for financing service | |
| CN114138601A (en) | Service alarm method, device, equipment and storage medium | |
| US20160358387A1 (en) | Method and system for predicting the operation of an aircraft by similarity analysis using distributed storage and calculation capacities | |
| CN113469507A (en) | Human resource management system based on network platform | |
| CN112765553B (en) | Engineering project management system based on big data | |
| CN112288103B (en) | Starting method for equipment detection in petroleum drilling system | |
| CN117135037A (en) | Method and device for defining network traffic performance abnormality | |
| CN111814001B (en) | Method and device for feeding back information | |
| CN114531338A (en) | Monitoring alarm and tracing method and system based on call chain data | |
| CN114331688A (en) | Method and device for detecting batch operation state of bank counter system business | |
| CN113568804A (en) | Web application-oriented performance bottleneck accurate positioning system | |
| CN117743093B (en) | Data quality evaluation method, device, equipment and medium of call chain | |
| CN117692302B (en) | Method and system for data collection, storage and intelligent monitoring and alarming | |
| CN118260167B (en) | Meteorological data product processing flow monitoring method, system, equipment and storage medium | |
| CN117376089A (en) | Multidimensional monitoring method based on intelligent cloud platform | |
| CN113032227B (en) | Abnormal network element detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |