CN115733762A - Monitoring system with big data analysis capability - Google Patents
Monitoring system with big data analysis capability Download PDFInfo
- Publication number
- CN115733762A CN115733762A CN202211421202.4A CN202211421202A CN115733762A CN 115733762 A CN115733762 A CN 115733762A CN 202211421202 A CN202211421202 A CN 202211421202A CN 115733762 A CN115733762 A CN 115733762A
- Authority
- CN
- China
- Prior art keywords
- module
- data
- alarm
- signal
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a monitoring system with big data analysis capability, and relates to the technical field related to big data intelligent monitoring. The monitoring system comprises a signal acquisition module, a signal analysis module, a signal storage module, a database comparison and identification module, a trend prediction module, a statistical analysis module, a control terminal module, an early warning module, a display module, a signal transmission module, a signal management module, a cloud signal storage module, a cloud server module, a security monitoring module, a data processing module, an abnormal data detection module, a wireless communication module, a network operation and maintenance service system, a collected data storage module and a threshold value storage module. According to the invention, the preprocessing of data analysis is performed through big data analysis to ensure the scientificity of the database, the basic value contained in the database is mined, meanwhile, various planning prejudgment analyses are performed according to the field complexity of network planning, so that overfitting can be effectively avoided, and the best result is obtained by utilizing joint decision.
Description
Technical Field
The invention relates to the technical field of big data intelligent monitoring, in particular to a monitoring system with big data analysis capability.
Background
The Internet of things is an important component of a new generation of information technology and is also an important development stage of the 'informatization' era. As the name implies, the internet of things is the internet with connected objects, and has two layers: firstly, the core and the foundation of the internet of things are still the internet, and the internet is an extended and expanded network on the basis of the internet; and secondly, the user side extends and expands to any article to perform information exchange and communication, namely, the article information.
At present, with the increasing popularity of computers, data communication is becoming the main way for people-to-people communication. Therefore, it is of great practical significance to strengthen the security of data communication. In the modern age of internet technology in China, which is continuously and deeply promoted, china provides various protective measures on the network security level, wherein the most convenient and effective method is maintenance and management. The maintenance management has a great significance for data communication, and under the management of professional technicians, the network can normally operate under the guarantee of a professional system, so that the information and property of customers can not be maliciously stolen, the market communication cost is further reduced, and the authenticity of information communication is ensured. However, our country still has a big problem in the data communication field, and must make intensive research.
Accordingly, those skilled in the art have provided monitoring systems with large data analysis capabilities to solve the problems set forth in the background art described above.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides the monitoring system with big data analysis capability, the data analysis can be preprocessed through big data analysis to ensure the scientificity of the database, the basic value contained in the database is mined, meanwhile, various planning and prejudging analyses are carried out according to the field complexity of network planning, over-fitting can be effectively avoided, and the best result is obtained by utilizing combined decision.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
a monitoring system with big data analysis capability comprises a signal acquisition module, a signal analysis module, a signal storage module, a database comparison and identification module, a trend prediction module, a statistical analysis module, a control terminal module, an early warning module, a display module, a signal transmission module, a signal management module, a cloud signal storage module, a cloud server module, a security monitoring module, a data processing module, an abnormal data detection module, a wireless communication module, a network operation and maintenance service system, a collected data storage module and a threshold value storage module, wherein the output end of the signal acquisition module is connected with the input end of the signal analysis module, the output end of the signal analysis module is connected to the input end of the signal storage module, the output end of the signal storage module transmits signals to the output end of the control terminal module through the database comparison and identification module, the control terminal module transmits signals to the output end of the cloud server module through the signal transmission module, the signal transmission module also transmits signals to the signal management module, the output end of the signal management module is connected to the cloud signal storage module, the output end of the data processing module is connected to the input end of the monitoring module, the output end of the cloud server module is connected to the input end of the early warning terminal module, the wireless communication module, the output end of the security monitoring terminal module is connected to the trend prediction module, the output end of the wireless communication module, the wireless communication module is connected to the input end of the control terminal module, the trend prediction module, the network operation and the display module, the control terminal module is connected with the abnormal data monitoring module;
the signal acquisition module is used for acquiring operation and maintenance monitoring data resources of the network;
the signal analysis module is used for carrying out statistical analysis on the acquired network data resources;
the signal storage module is used for temporarily storing the data after statistical analysis;
the database comparison and identification module is used for comparing the closely matched network operation and maintenance service system working state model with the network operation and maintenance service system working threshold parameters stored in the threshold storage module;
the trend prediction module is used for performing trend prediction on the working state model of the network operation and maintenance service system;
the statistical analysis module is used for constructing a working state model of the network operation and maintenance service system matched in the future according to the prediction result of the trend prediction module;
the early warning module is used for early warning fault alarms generated in data transmission;
the display module is used for displaying the comparison result of the database comparison and identification module by a histogram and displaying the working state model of the network operation and maintenance service system which is matched in the future and is constructed by the statistical analysis module;
the signal transmission module is used for transmitting the signal of the control terminal module to the cloud server module;
the signal management module is used for transmitting the signal of the signal transmission module to the cloud signal storage module;
the abnormal data detection module is used for detecting whether the network operation and maintenance service system has an abnormal state;
the network operation and maintenance service system is used for acquiring operation and maintenance monitoring data;
the acquisition data storage module is used for storing operation and maintenance monitoring data in the operation and maintenance monitoring process;
and the threshold value storage module is used for storing the parameters of the working threshold value of the network operation and maintenance service system.
Preferably, the signal acquisition module acquires data synchronously in multiple links in the whole network range, the data includes network data, information data, region data, network management data, environment data, user data and the like, the information data and the like belong to important big data sources, the network management data and the user data belong to the category of traditional data, and the semi-structured data and the structured data are acquired together in order to improve the data acquisition capacity.
Preferably, the signal analysis module and the signal storage module perform preliminary processing on data, the original file and data information are stored in a data caching time under the condition that the data format, the structure and the like are not changed, then, the resource data in the basic data are uniformly stored and processed according to the minimum granularity, the large data are obtained through a multidimensional analysis technology, the hot spot data with high value and large demand are collected, the coverage data are collected in a classified mode on the basis of 4G/5G frequency scanning, the comprehensiveness of the collected data is ensured through comprehensive analysis, and various types of data are stored in a classified mode.
Preferably, the early warning module performs real-time monitoring and statistical processing on the received warning information, performs multi-thread and multi-process parallel processing on the warning risk data by using a cloud computing data processing technology, and simultaneously, communication network supervision personnel also need to perform preferential analysis and processing on the fault warning, perform standardized analysis on non-important fault warning data, mine and correlate analysis on the warning data, and cache the processed fault warning.
Preferably, the abnormal data detection unit comprises an invalid alarm analysis module, a repeated alarm analysis module and an abnormal alarm analysis module;
the invalid alarm analysis module is used for detecting and analyzing an invalid alarm signal of the network operation and maintenance service system;
the repeated alarm analysis module is used for detecting and analyzing repeated alarm signals of the network operation and maintenance service system;
and the abnormal alarm analysis module is used for detecting and analyzing the abnormal alarm signal of the network operation and maintenance service system.
Preferably, the detection method of the invalid alarm analysis module is as follows: and judging whether the received alarm signal occurs in the same time period every day, and if the received alarm signal occurs in the same time period every day, judging that the alarm signal is an invalid alarm signal.
Preferably, the detection method of the repeated alarm analysis module is as follows: and judging whether the alarm signal traced back by a time period from the current moment has repeated alarm in a short time or not every other time period, and if so, judging the alarm signal to be a repeated alarm signal.
Preferably, the method for determining that the network operation and maintenance service system is in the abnormal state by the abnormal data detection unit is as follows: and counting the number of invalid alarm signals, repeated alarm signals and abnormal alarm signals in each week, adding the invalid alarm signals, repeated alarm signals and abnormal alarm signals to obtain the total alarm number of the network operation and maintenance service system, and judging that the network operation and maintenance service system is in an abnormal state when the total alarm number is not lower than a threshold value.
Preferably, a data processing method of a monitoring system with big data analysis capability includes the following steps:
s1, synchronously acquiring data in multiple links in the whole network range, wherein the data comprises network data, information data, region data, network management data, environment data, user data and the like;
s2, under the conditions of not changing data formats, structures and the like, storing original files and data information in data caching time, then uniformly storing resource data in basic data according to minimum granularity, acquiring big data by using a multidimensional analysis technology, acquiring hot spot data with higher value and larger demand, classifying and acquiring coverage data on the basis of 4G/5G frequency sweep, ensuring the comprehensiveness of the acquired data by comprehensive analysis, and classifying and storing various data;
s3, in the management and storage of various performance alarm indexes and engineering alarm indexes, the performance alarms generated by the network are monitored, cached and managed in real time usually by means of a cloud computing management platform, a background database and the like;
s4, according to the massive alarm storm data in the network, the cloud computing data processing technology can perform multi-thread and multi-process parallel processing on the alarm data, divide the alarm data into a plurality of different alarm levels, perform priority processing on major faults and major alarm message sources, and perform delay processing on conventional level alarms so as to ensure normal monitoring and management of all alarm data;
s5, screening engineering alarms in aspects of communication equipment, network communication software and the like, carrying out clever processing and analysis on various alarm data, and finishing timely discovery and timely fault processing of engineering alarm information as far as possible;
and S6, association rule mining and quality evaluation. Mining and analyzing the existing alarm data by using an association rule mining and analyzing tool and a multi-mode character string matching algorithm, generating a fault work order after the alarm messages are associated and matched, and distributing mechanical energy to network operation maintenance personnel, thereby being beneficial to the centralized management of communication network faults;
and S7, distributing and processing the fault work order. After the fault management system finishes processing the alarm message, the palm APP software is used for dispatching the MQ queue of the fault work order in a network instruction transmission mode, the real-time states of fault equipment and network communication software are inquired, finally the fault work order and the fault processing supervision items are sent to mobile phones of operation and maintenance engineers, direct fault communication is conducted with the operation and maintenance personnel through the phones, and the fact that communication network monitoring and management responsibilities are carried out to each operation and maintenance engineer is guaranteed.
(III) advantageous effects
The invention provides a monitoring system with big data analysis capability. The method has the following beneficial effects:
1. the invention provides a monitoring system with big data analysis capability, which detects whether a network operation and maintenance service system has an abnormal state through an abnormal data detection unit, and controls a terminal module to delete the operation and maintenance data of the network operation and maintenance service system in the abnormal state from a collected data storage module, so that the abnormal data can be deleted, the quality of the operation and maintenance data is ensured, and the accuracy of later-stage adjustment of the network operation and maintenance service system can be ensured.
2. The invention provides a monitoring system with big data analysis capability, which compares a closely matched network operation and maintenance service system working state model with a network operation and maintenance service system working threshold parameter stored in a threshold storage module through a comparison identification module, a trend prediction module performs trend prediction on the network working state model, and a statistical analysis module constructs a network operation and maintenance service system working state model matched in the future according to a prediction result of the trend prediction module, so that the response speed of an intelligent operation and maintenance monitoring system and the degree of fit with the network operation condition can be effectively improved.
3. The invention provides a monitoring system with big data analysis capability, which carries out data analysis preprocessing through big data analysis to ensure the scientificity of a database, digs out the basic value contained in the database, takes the requirements of data size and data analysis efficiency into consideration, and utilizes machine learning as the key point of big data analysis.
Drawings
FIG. 1 is a schematic diagram of the system architecture of the present invention;
FIG. 2 is a schematic diagram of an abnormal data detection module according to the present invention;
FIG. 3 is a flow chart of a data processing method according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example (b):
as shown in fig. 1 to 3, a monitoring system with big data analysis capability according to an embodiment of the present invention includes a signal acquisition module, a signal analysis module, a signal storage module, a database comparison and identification module, a trend prediction module, a statistical analysis module, a control terminal module, an early warning module, a display module, a signal transmission module, a signal management module, a cloud signal storage module, a cloud server module, a security monitoring module, a data processing module, an abnormal data detection module, a wireless communication module, a network operation and maintenance service system, a collected data storage module, and a threshold storage module, wherein an output end of the signal acquisition module is connected to an input end of the signal analysis module, an output end of the signal analysis module is connected to an input end of the signal storage module, and an output end of the signal storage module transmits a signal to an output end of the control terminal module through the database comparison and identification module, the control terminal module transmits signals to the output end of the cloud server module through a signal transmission module, the signal transmission module also transmits the signals to the signal management module, the output end of the signal management module is connected to the cloud signal storage module, the output end of the data processing module is connected to the input end of the security monitoring module, the output end of the security monitoring module is connected to the input end of the cloud server module, the output end of the control terminal module is connected to the input end of the early warning module, the output end of the early warning module is connected to the input end of the display module, the output end of the control terminal module is further connected to the input ends of the trend prediction module and the statistical analysis module, and the control terminal module is connected with the network operation and maintenance service system through a wireless communication module in a remote wireless communication manner, the control terminal module is connected with the abnormal data monitoring module;
the signal acquisition module is used for acquiring operation and maintenance monitoring data resources of the network;
the signal analysis module is used for carrying out statistical analysis on the acquired network data resources;
the signal storage module is used for temporarily storing the data after statistical analysis;
the database comparison and identification module is used for comparing the closely matched network operation and maintenance service system working state model with the network operation and maintenance service system working threshold parameters stored in the threshold storage module;
the trend prediction module is used for performing trend prediction on the working state model of the network operation and maintenance service system;
the statistical analysis module is used for constructing a working state model of the network operation and maintenance service system matched in the future according to the prediction result of the trend prediction module;
the early warning module is used for early warning fault alarms generated in data transmission;
the display module is used for displaying the comparison result of the database comparison and identification module by using a histogram and displaying a working state model of the network operation and maintenance service system which is matched in the future and is constructed by the statistical analysis module;
the signal transmission module is used for transmitting the signal of the control terminal module to the cloud server module;
the signal management module is used for transmitting the signal of the signal transmission module to the cloud signal storage module;
the abnormal data detection module is used for detecting whether the network operation and maintenance service system has an abnormal state;
the network operation and maintenance service system is used for acquiring operation and maintenance monitoring data;
the acquisition data storage module is used for storing operation and maintenance monitoring data in the operation and maintenance monitoring process;
and the threshold value storage module is used for storing the parameters of the working threshold value of the network operation and maintenance service system.
The signal acquisition module synchronously acquires data in multiple links in the whole network range, the data comprises network data, information data, region data, network management data, environment data, user data and the like, the information data and the like belong to important big data sources, the network management data and the user data belong to the category of traditional data, and the semi-structured data and the structured data are acquired together in order to improve the data acquisition capacity;
the method comprises the steps that a signal analysis module and a signal storage module carry out primary processing on data, original files and data information are stored in data caching time under the condition that data formats, structures and the like are not changed, then resource data in basic data are uniformly stored and processed according to minimum granularity, big data are obtained through a multi-dimensional analysis technology, hot point data with high value and large demand are collected, 4G/5G frequency sweeping is used as a technical basis, coverage data are collected in a classified mode, comprehensiveness of the collected data is guaranteed through comprehensive analysis, and various types of data are stored in a classified mode;
the early warning module can carry out real-time monitoring and statistical processing on the received warning information, and by utilizing a cloud computing data processing technology, multi-thread and multi-process parallel processing of warning risk data is developed, meanwhile, communication network supervision personnel also need to carry out preferential analysis and processing on fault warning, carry out standardized analysis on non-important fault warning data, and mining and correlation analysis on the warning data, and cache the processed fault warning;
the abnormal data detection unit comprises an invalid alarm analysis module, a repeated alarm analysis module and an abnormal alarm analysis module;
the invalid alarm analysis module is used for detecting and analyzing an invalid alarm signal of the network operation and maintenance service system;
the repeated alarm analysis module is used for detecting and analyzing repeated alarm signals of the network operation and maintenance service system;
and the abnormal alarm analysis module is used for detecting and analyzing an abnormal alarm signal of the network operation and maintenance service system.
The detection method of the invalid alarm analysis module comprises the following steps: judging whether the received alarm signal occurs in the same time period every day, and if the received alarm signal occurs in the same time period every day, judging that the alarm signal is an invalid alarm signal;
the detection method of the repeated alarm analysis module comprises the following steps: judging whether the alarm signal traced back forward from the current moment in a time period has repeated alarm in a short time or not every other time period, and if yes, judging the alarm signal to be a repeated alarm signal;
the method for judging that the network operation and maintenance service system is in the abnormal state by the abnormal data detection unit comprises the following steps: and counting the number of invalid alarm signals, repeated alarm signals and abnormal alarm signals in each week, adding the invalid alarm signals, repeated alarm signals and abnormal alarm signals to obtain the total alarm number of the network operation and maintenance service system, and judging that the network operation and maintenance service system is in an abnormal state when the total alarm number is not lower than a threshold value.
The data processing method of the monitoring system with the big data analysis capability comprises the following steps:
s1, synchronously acquiring data in multiple links in the whole network range, wherein the data comprises network data, information data, region data, network management data, environment data, user data and the like;
s2, under the conditions of not changing data formats, structures and the like, storing original files and data information in data caching time, uniformly storing resource data in basic data according to minimum granularity, acquiring big data by using a multi-dimensional analysis technology, acquiring hot-point data with high value and large demand, classifying and acquiring coverage data on the basis of 4G/5G frequency sweep, ensuring the comprehensiveness of the acquired data by comprehensive analysis, and classifying and storing various data;
s3, in the management and storage of various performance alarm indexes and engineering alarm indexes, the cloud computing management platform, the background database and the like are usually used for carrying out real-time monitoring, caching and management on the network generated performance alarm;
s4, according to the massive alarm storm data in the network, the cloud computing data processing technology can perform multi-thread and multi-process parallel processing on the alarm data, divide the alarm data into a plurality of different alarm levels, perform priority processing on major faults and major alarm message sources, and perform delay processing on conventional level alarms so as to ensure normal monitoring and management of all alarm data;
s5, screening engineering alarms in aspects of communication equipment, network communication software and the like, carrying out intelligent processing and analysis on various alarm data, and completing timely discovery and timely fault processing of engineering alarm information as much as possible;
and S6, association rule mining and quality evaluation. The existing alarm data are mined and analyzed by using an association rule mining analysis tool and a multi-mode character string matching algorithm, and a fault work order is generated after alarm messages are associated and matched and is distributed to network operation maintenance personnel, so that the centralized management of communication network faults is facilitated;
and S7, distributing and processing the fault work order. After the fault management system finishes processing the alarm message, the palm APP software is used for dispatching the MQ queue of the fault work order in a network instruction transmission mode, the real-time states of fault equipment and network communication software are inquired, finally the fault work order and the fault processing supervision items are sent to mobile phones of operation and maintenance engineers, direct fault communication is conducted with the operation and maintenance personnel through the phones, and the fact that communication network monitoring and management responsibilities are carried out to each operation and maintenance engineer is guaranteed.
Through the scheme of the invention, the application of the communication network operation maintenance monitoring system with the big data analysis capability provided by the invention is explained in detail.
The construction of the communication network operation maintenance monitoring system mainly depends on a big data technology, a cloud computing technology, an Internet of things intelligent technology and the like to form connection of network communication equipment and user terminal equipment, and fault alarms generated in data transmission are monitored. The whole operation and maintenance system monitored by the current communication network comprises a network visual monitoring platform, modules for implementing fault monitoring, electronic operation and maintenance, fault management and the like, wherein different modules are respectively responsible for acquisition, mining, statistical analysis and storage of network data resources.
A user can know fault alarms, engineering alarms and performance alarms in the fault management system through the palm client APP, execute data analysis links such as memory calculation, load balancing, incremental processing and the like, and divide different alarms into multiple levels. Under the conditions of serious alarm and serious network communication fault, the fault early warning module can receive a large amount of alarm storm data in a short time. At the moment, the fault early warning module can perform real-time monitoring and statistical processing on the received warning information, and multi-thread and multi-process parallel processing of the warning storm data is performed by utilizing a cloud computing data processing technology.
Meanwhile, communication network supervision personnel also need to perform prior analysis and processing on fault alarms, perform standardized analysis on non-important fault alarm data, perform mining and correlation analysis on alarm data, and cache the processed fault alarms. The concrete expression is as follows:
on one hand, for alarms formed by aging of communication equipment and communication lines and untimely update of communication network application software, the alarms need to be marked as engineering alarms, then network monitoring management personnel are informed to perform troubleshooting on software and hardware facilities in the communication network, and corresponding measures are taken to remove fault alarms.
On the other hand, for the fault alarm caused by non-engineering reasons, after alarm fault positioning, fault verification and correlation analysis are carried out, form dispatching data covering alarm IDs, alarm titles and alarm contents are generated, and the EOMS system dispatches fault work forms. The monitoring department dispatches the work order to the communication network supervisory personnel aiming at the severity of the fault work order, the supervisory personnel repairs the network signal quality, data transmission, service function service and the like, and sends the control work order back to the visual management platform, so that a closed loop flow of fault alarm is formed.
After the staff finds out the factors threatening the network security, a perfect treatment measure should be made at the first time, and the security performance of the whole network is effectively improved.
Practical investigation shows that various measures are taken in the process of processing network security problems, and when the security problems of virus attack are solved, the most common method is that a user downloads relevant antivirus software or establishes a firewall, so that the data communication network has the highest security, and the following means can be utilized.
Firstly, reasonably analyzing software, files and the like influencing safety in the use process of a user network, doing relevant processing work, and limiting the access of a webpage possibly containing viruses;
secondly, hardware equipment of the network is continuously upgraded, so that the whole network environment has higher stability;
thirdly, for some enterprises, the security problem that viruses attack servers by virtue of hosts can be avoided by virtue of an outsourcing form that a host system is separated from the servers, and a secure network environment is constructed.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. Monitoring system with big data analysis ability, including the signal acquisition module, the signal analysis module, the signal storage module, database contrast identification module, trend prediction module, statistics analysis module, control terminal module, the early warning module, the display module, the signal transmission module, the signal management module, cloud signal storage module, cloud server module, security protection monitoring module, data processing module, unusual data detection module, wireless communication module, network operation and maintenance service system, the data storage module gathers, threshold value storage module, its characterized in that: the output end of the signal acquisition module is connected with the input end of the signal analysis module, the output end of the signal analysis module is connected to the input end of a signal storage module, the output end of the signal storage module transmits signals to the output end of the control terminal module through a database comparison and identification module, the control terminal module transmits signals to the output end of the cloud server module through a signal transmission module, the signal transmission module also transmits signals to the signal management module, the output end of the signal management module is connected to the cloud signal storage module, the output end of the data processing module is connected to the input end of the security monitoring module, the output end of the security monitoring module is connected to the input end of the cloud server module, the output end of the control terminal module is connected to the input end of an early warning module, the output end of the early warning module is connected to the input end of a display module, the output end of the control terminal module is further connected to the input ends of a trend prediction module and a statistical analysis module, the control terminal module establishes remote wireless communication connection with a network operation and maintenance service system finger through a wireless communication module, and the control terminal module is connected with an abnormal data monitoring module;
the signal acquisition module is used for acquiring operation and maintenance monitoring data resources of the network;
the signal analysis module is used for carrying out statistical analysis on the acquired network data resources;
the signal storage module is used for temporarily storing the data after statistical analysis;
the database comparison and identification module is used for comparing the closely matched network operation and maintenance service system working state model with the network operation and maintenance service system working threshold parameters stored in the threshold storage module;
the trend prediction module is used for performing trend prediction on the working state model of the network operation and maintenance service system;
the statistical analysis module is used for constructing a working state model of the network operation and maintenance service system matched in the future according to the prediction result of the trend prediction module;
the early warning module is used for early warning fault alarms generated in data transmission;
the display module is used for displaying the comparison result of the database comparison and identification module by using a histogram and displaying a working state model of the network operation and maintenance service system which is matched in the future and is constructed by the statistical analysis module;
the signal transmission module is used for transmitting the signal of the control terminal module to the cloud server module;
the signal management module is used for transmitting the signal of the signal transmission module to the cloud signal storage module;
the abnormal data detection module is used for detecting whether the network operation and maintenance service system has an abnormal state;
the network operation and maintenance service system is used for collecting operation and maintenance monitoring data;
the acquisition data storage module is used for storing operation and maintenance monitoring data in the operation and maintenance monitoring process;
and the threshold value storage module is used for storing the parameters of the working threshold value of the network operation and maintenance service system.
2. The monitoring system with big data analysis capability of claim 1, wherein: the signal acquisition module synchronously acquires data in multiple links in the whole network range, the data comprises network data, information data, region data, network management data, environment data, user data and the like, the information data and the like belong to important big data sources, the network management data and the user data belong to the traditional data category, and the semi-structured data and the structured data are acquired together in order to improve the data acquisition capacity.
3. The monitoring system with big data analysis capability according to claim 1, wherein: the signal analysis module and the signal storage module perform primary processing on data, original files and data information are stored in data caching time under the condition that data formats, structures and the like are not changed, then resource data in basic data are uniformly stored and processed according to minimum granularity, big data are obtained through a multi-dimensional analysis technology, hot point data with high value and large demand are collected, 4G/5G frequency sweeping is used as a technical basis, coverage class data are collected in a classified mode, comprehensiveness of the collected data is guaranteed through comprehensive analysis, and various types of data are stored in a classified mode.
4. The monitoring system with big data analysis capability of claim 1, wherein: the early warning module can carry out real-time monitoring and statistical processing on received warning information, multi-thread and multi-process parallel processing of warning risk data is developed by utilizing a cloud computing data processing technology, meanwhile, communication network supervision personnel also need to carry out preferential analysis and processing on fault warning, carry out standardized analysis on non-important fault warning data, carry out mining and correlation analysis on the warning data, and cache the processed fault warning.
5. The monitoring system with big data analysis capability of claim 1, wherein: the abnormal data detection unit comprises an invalid alarm analysis module, a repeated alarm analysis module and an abnormal alarm analysis module;
the invalid alarm analysis module is used for detecting and analyzing an invalid alarm signal of the network operation and maintenance service system;
the repeated alarm analysis module is used for detecting and analyzing repeated alarm signals of the network operation and maintenance service system;
and the abnormal alarm analysis module is used for detecting and analyzing an abnormal alarm signal of the network operation and maintenance service system.
6. The monitoring system with big data analysis capability of claim 5, wherein: the detection method of the invalid alarm analysis module comprises the following steps: and judging whether the received alarm signal occurs in the same time period every day, and if the received alarm signal occurs in the same time period every day, judging that the alarm signal is an invalid alarm signal.
7. The monitoring system with big data analysis capability of claim 5, wherein: the detection method of the repeated alarm analysis module comprises the following steps: and judging whether the alarm signal traced back forward from the current moment in a time period has repeated alarm in a short time every other time period, and if so, judging the alarm signal to be the repeated alarm signal.
8. The monitoring system with big data analysis capability of claim 5, wherein: the method for judging that the network operation and maintenance service system is in the abnormal state by the abnormal data detection unit comprises the following steps: and counting the number of invalid alarm signals, repeated alarm signals and abnormal alarm signals in each week, adding the invalid alarm signals, repeated alarm signals and abnormal alarm signals to obtain the total alarm number of the network operation and maintenance service system, and judging that the network operation and maintenance service system is in an abnormal state when the total alarm number is not lower than a threshold value.
9. A data processing method of a monitoring system with big data analysis capability is characterized in that: the method comprises the following steps:
s1, synchronously acquiring data in multiple links in the whole network range, wherein the data comprises network data, information data, region data, network management data, environment data, user data and the like;
s2, under the conditions of not changing data formats, structures and the like, storing original files and data information in data caching time, uniformly storing resource data in basic data according to minimum granularity, acquiring big data by using a multi-dimensional analysis technology, acquiring hot-point data with high value and large demand, classifying and acquiring coverage data on the basis of 4G/5G frequency sweep, ensuring the comprehensiveness of the acquired data by comprehensive analysis, and classifying and storing various data;
s3, in the management and storage of various performance alarm indexes and engineering alarm indexes, the performance alarms generated by the network are monitored, cached and managed in real time usually by means of a cloud computing management platform, a background database and the like;
s4, according to the massive alarm storm data in the network, the cloud computing data processing technology can perform multi-thread and multi-process parallel processing on the alarm data, divide the alarm data into a plurality of different alarm levels, perform priority processing on major faults and major alarm message sources, and perform delay processing on conventional level alarms so as to ensure normal monitoring and management of all alarm data;
s5, screening engineering alarms in aspects of communication equipment, network communication software and the like, carrying out clever processing and analysis on various alarm data, and finishing timely discovery and timely fault processing of engineering alarm information as far as possible;
and S6, association rule mining and quality evaluation. Mining and analyzing the existing alarm data by using an association rule mining and analyzing tool and a multi-mode character string matching algorithm, generating a fault work order after the alarm messages are associated and matched, and distributing mechanical energy to network operation maintenance personnel, thereby being beneficial to the centralized management of communication network faults;
and S7, distributing and processing the fault work order. After the fault management system finishes processing the alarm message, the palm APP software is used for dispatching the MQ queue of the fault work order in a network instruction transmission mode, the real-time states of fault equipment and network communication software are inquired, finally the fault work order and the fault processing supervision items are sent to mobile phones of operation and maintenance engineers, direct fault communication is conducted with the operation and maintenance personnel through the phones, and the fact that communication network monitoring and management responsibilities are carried out to each operation and maintenance engineer is guaranteed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211421202.4A CN115733762A (en) | 2022-11-14 | 2022-11-14 | Monitoring system with big data analysis capability |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211421202.4A CN115733762A (en) | 2022-11-14 | 2022-11-14 | Monitoring system with big data analysis capability |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115733762A true CN115733762A (en) | 2023-03-03 |
Family
ID=85295577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211421202.4A Pending CN115733762A (en) | 2022-11-14 | 2022-11-14 | Monitoring system with big data analysis capability |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115733762A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116404755A (en) * | 2023-04-18 | 2023-07-07 | 内蒙古铖品科技有限公司 | Big data processing system and method based on Internet of things |
| CN116483290A (en) * | 2023-06-26 | 2023-07-25 | 深圳市亲邻科技有限公司 | Remote monitoring system and method for data storage device |
| CN117221151A (en) * | 2023-09-12 | 2023-12-12 | 北京城建智控科技股份有限公司 | Visual management device and method for cloud computing storage |
-
2022
- 2022-11-14 CN CN202211421202.4A patent/CN115733762A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116404755A (en) * | 2023-04-18 | 2023-07-07 | 内蒙古铖品科技有限公司 | Big data processing system and method based on Internet of things |
| CN116483290A (en) * | 2023-06-26 | 2023-07-25 | 深圳市亲邻科技有限公司 | Remote monitoring system and method for data storage device |
| CN116483290B (en) * | 2023-06-26 | 2024-02-09 | 深圳市亲邻科技有限公司 | Remote monitoring system and method for data storage device |
| CN117221151A (en) * | 2023-09-12 | 2023-12-12 | 北京城建智控科技股份有限公司 | Visual management device and method for cloud computing storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110620759B (en) | Multi-dimensional association-based network security event hazard index evaluation method and system | |
| CN115733762A (en) | Monitoring system with big data analysis capability | |
| CN101562537B (en) | Distributed self-optimized intrusion detection alarm associated system | |
| CN108197261A (en) | A kind of wisdom traffic operating system | |
| CN114757590B (en) | Enterprise operation risk early warning and management and control system based on big data analysis | |
| CN101808351B (en) | Method and system for business impact analysis | |
| CN114267178B (en) | Intelligent operation maintenance method and device for station | |
| CN115809183A (en) | Method for discovering and disposing information-creating terminal fault based on knowledge graph | |
| CN105184886A (en) | Cloud data center intelligence inspection system and cloud data center intelligence inspection method | |
| CN104574219A (en) | System and method for monitoring and early warning of operation conditions of power grid service information system | |
| CN115225536B (en) | Virtual machine abnormality detection method and system based on unsupervised learning | |
| CN106570567A (en) | Main network maintenance multi-constraint multi-target evaluation expert system and optimization method | |
| CN114465874A (en) | Fault prediction method, device, electronic equipment and storage medium | |
| CN114358106A (en) | System anomaly detection method and device, computer program product and electronic equipment | |
| CN115544519A (en) | Method for carrying out security association analysis on threat information of metering automation system | |
| CN105930255A (en) | Method and apparatus for predicting health degree of system | |
| US20120054324A1 (en) | Device, method, and storage medium for detecting multiplexed relation of applications | |
| CN106649034B (en) | Visual intelligent operation and maintenance method and platform | |
| CN117408561A (en) | Residential engineering construction supervision method and device and electronic equipment | |
| CN110149303B (en) | Party-school network security early warning method and early warning system | |
| CN117034149A (en) | Fault processing strategy determining method and device, electronic equipment and storage medium | |
| CN115037559B (en) | Data safety monitoring system based on flow, electronic equipment and storage medium | |
| CN116881958A (en) | Power grid big data safety protection method, system, electronic equipment and storage medium | |
| CN116128197A (en) | Intelligent airport management system and method | |
| CN115408236A (en) | Log data auditing system, method, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |