CN114154035A - Data processing system for dynamic loop monitoring - Google Patents

Data processing system for dynamic loop monitoring Download PDF

Info

Publication number
CN114154035A
CN114154035A CN202111304949.7A CN202111304949A CN114154035A CN 114154035 A CN114154035 A CN 114154035A CN 202111304949 A CN202111304949 A CN 202111304949A CN 114154035 A CN114154035 A CN 114154035A
Authority
CN
China
Prior art keywords
data
alarm
configuration
database
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111304949.7A
Other languages
Chinese (zh)
Inventor
刘坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Yindun Tai'an Network Technology Co ltd
Original Assignee
Beijing Yindun Tai'an Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Yindun Tai'an Network Technology Co ltd filed Critical Beijing Yindun Tai'an Network Technology Co ltd
Priority to CN202111304949.7A priority Critical patent/CN114154035A/en
Publication of CN114154035A publication Critical patent/CN114154035A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/906—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G06F11/1469—Backup restoration techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/219—Managing data history or versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/90335—Query processing
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/182—Level alarms, e.g. alarms responsive to variables exceeding a threshold
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/185—Electrical failure alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/80—Database-specific techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a data processing system for dynamic loop monitoring, which comprises a data acquisition module: the data processing system is used for acquiring data of the moving ring equipment, determining the acquired data, screening and classifying the data according to the data attribute of the acquired data, and determining classified data; a data storage module: storing the classified data into a corresponding database according to the data types; wherein the database comprises a real-time database, a historical database and a configuration management database; a data processing module: for evaluating an abnormal value of the data stored in the database, and performing an alarm when the abnormal value reaches a predetermined threshold value. The invention has the beneficial effects that: the invention is more convenient, and the data processing is less; in the aspect of abnormal judgment, the problem of parameter setting or actually which equipment has problems can be judged, incremental data and historical data are divided, and data comparison is more convenient to carry out.

Description

Data processing system for dynamic loop monitoring
Technical Field
The invention relates to the technical field of computers, in particular to a data processing system for monitoring a moving ring.
Background
Before the construction of the dynamic ring monitoring system, when a machine room has a fault, maintenance personnel can only be arranged to go to the field for processing after the device network management system gives an alarm or a user reports an error, and the maintenance personnel can only gradually troubleshoot the fault and identify the fault reason after the maintenance personnel arrive at the machine room, so that the next step of measures is taken to troubleshoot the fault. Obviously, the timeliness of the method can not meet the high-standard operation and maintenance requirements, thereby seriously affecting the service quality.
The core link of a general moving loop monitoring system is to process and analyze data of a mains supply monitoring system, a split air conditioner monitoring system, a temperature and humidity monitoring system, a smoke monitoring system, a water leakage monitoring system, an access control monitoring system and a video monitoring system;
as shown in fig. 4 and 5, the existing dong-ring system has complex steps to process in the software level, numerous data to be managed and controlled in the hardware system, and huge data processing amount, so that the data cannot be efficiently and quickly divided, and the data quality cannot be evaluated, so that data abnormality cannot be timely found, and the failure processing efficiency is slow, which is also a reason that a large amount of data is difficult to timely manage in the conventional data database, and the system is easy to fail.
Disclosure of Invention
The invention provides a data processing system for monitoring a moving ring, which is used for solving the technical scheme adopted under the conditions of low speed and inaccurate operation result when the data processing system for monitoring the moving ring processes data, and comprises the following steps:
a data processing system for dynamic loop monitoring, comprising:
a data acquisition module: the data processing system is used for acquiring data of the moving ring equipment, determining the acquired data, screening and classifying the data according to the data attribute of the acquired data, and determining classified data;
a data storage module: storing the classified data into a corresponding database according to the data types; wherein the content of the first and second substances,
the database comprises a real-time database, a historical database and a configuration management database;
a data processing module: for evaluating an abnormal value of the data stored in the database, and performing an alarm when the abnormal value reaches a predetermined threshold value.
As an embodiment of the present invention: the data acquisition module comprises:
a communication interface unit: the system comprises a communication port, a dynamic ring device, a data acquisition device and a data stream transmission device, wherein the communication port is used for being butted with the dynamic ring device, and performing data acquisition and data stream transmission to generate acquired data;
a data screening unit: the data processing device is used for dividing the acquired data through a preset data attribute list to generate classified data; wherein the content of the first and second substances,
the classification data includes real-time data, historical data, and configuration data.
As an embodiment of the present invention: the data storage module includes:
a real-time storage unit: the log is used for determining real-time measuring point access and real-time data access records in the classified data, determining real-time data and storing the real-time data into a real-time database;
a history storage unit: the system is used for carrying out time marking on the classification data and storing historical data into a historical database according to the time marking;
configuration management data unit: the system is used for storing the attribute data of software and hardware of the dynamic ring monitoring system, determining the configuration item and configuration parameter information of the dynamic ring monitoring system, and storing a value configuration management database.
As an embodiment of the present invention: the real-time storage unit includes:
a configuration management subunit: the device is used for carrying out detailed storage on the real-time data stream and determining a storage position;
real-time measurement point access subunit: the system comprises a database, a data acquisition unit, a data processing unit and a data processing unit, wherein the database is used for storing basic information and configuration information of a measuring point;
real-time transaction access subunit: and the access data of the real-time transaction is determined through the transaction access port.
As an embodiment of the present invention: the history storage unit includes:
a history data storage subunit: the device is used for detecting the historical data stream and transferring the detected historical data stream to a corresponding storage unit;
the historical data query subunit: the method comprises the steps of continuously inquiring data and periodically inquiring data; wherein the content of the first and second substances,
the historical data continuous query subunit: outputting a continuously updated and changed data set;
periodic data query subunit: for outputting a set of data for one cycle interval.
As an embodiment of the present invention: the configuration management data unit includes:
configuration item basic information subunit: the system is used for storing hardware configuration items, software configuration items, network configuration items, document configuration items, association configuration items and parameter configuration items;
configuration item tree structure subunit: a relationship for associating configuration items with configuration parameters;
the configuration management database is constructed by the following steps:
s1: the scope of the configuration management is determined,
determining the width and depth and division of configuration items and defining the life cycle of the configuration items;
s2: defining a configuration item object, including determining the characteristics of the configuration item and determining the common attribute, the private attribute, the associated item and the relationship of the configuration item;
s3: and constructing a configuration model among the configuration items.
As an embodiment of the present invention: the data processing module comprises:
a data scheduling unit: the data queue management system is used for scheduling data flow of a database, generating a data queue, searching similar data queues in a historical database, determining an abnormal data queue, performing abnormal value estimation on data of the abnormal data queue, and determining an abnormal value;
an alarm and confirmation unit: the system comprises a data flow monitoring module, a data processing module and a data processing module, wherein the data flow monitoring module is used for monitoring abnormal values of data of an abnormal queue in the data flow and executing alarm information when a threshold value is reached;
a concurrency control unit: the system is used for locking data items of data and executing concurrent data processing;
a data recovery unit: for recovering aborted data for deadlock handling.
As an embodiment of the present invention: the system further comprises:
and the deadlock processing unit is used for checking the activity time of the data, rolling back the data when the activity time of the data exceeds the set time of the system, adding the data into the data queue again, and waiting for the next scheduling execution.
As an embodiment of the present invention: the data processing module further comprises:
an alarm and confirmation module: the alarm system is used for monitoring the data of the moving ring equipment, and when the data of the moving ring equipment reaches the threshold value, alarm information is sent out, wherein the alarm levels are 4 types and respectively:
if the alarm is not given, the alarm condition is not met, the alarm is not triggered, and the alarm information is not executed;
and low-level alarming is performed, an alarming condition is met, alarming is triggered, alarming information is recorded, and a user is prompted to check whether hidden dangers exist in a corresponding area.
Advanced alarm, alarm condition is achieved, alarm is triggered, alarm information is recorded, and a user is reminded to check that an error occurs in a corresponding area.
And interlocking alarm is carried out, alarm conditions are met, alarm is triggered, alarm information is recorded, a user is reminded of checking that errors occur in a corresponding area, and preset safety operation is automatically executed to prevent system errors from being enlarged.
As an embodiment of the present invention: the system further comprises:
a log management module: the system is used for recording the state information of each module operated by the data processing system, executing system optimization analysis and executing data recovery when the system crashes.
The invention has the beneficial effects that: compared with the prior art, the invention can not accurately classify the original monitoring data, and the invention controls different data to be stored through an independent database mechanism, so that the data processing can be carried out by analyzing abnormal values of different data, and whether a fault occurs is judged, thereby being more convenient and less in data processing compared with the prior art; in the aspect of abnormal judgment, the problem of parameter setting or actually which equipment has problems can be judged, incremental data and historical data are divided, and data comparison is more convenient to carry out.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a block diagram of a data processing system with dynamic ring monitoring according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a periodic query of a data processing system with dynamic ring monitoring according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an alarm level mapping of a data processing system with dynamic ring monitoring according to an embodiment of the present invention
FIG. 4 is a block diagram of a data processing system of a conventional dynamic ring monitoring system according to an embodiment of the present invention;
fig. 5 is a diagram of an apparatus of a moving loop monitoring system according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Example 1:
an embodiment of the present invention provides a data processing system for monitoring a dynamic ring, as shown in fig. 1, including:
a data acquisition module: the system comprises a data acquisition module, a data sorting module and a data processing module, wherein the data acquisition module is used for acquiring environmental data of the moving ring equipment, determining the acquired data, and performing data screening and data sorting according to the data attribute of the acquired data to determine classified data;
a data storage module: storing the classified data into a corresponding database according to the data types; wherein the content of the first and second substances,
the database comprises a real-time database, a historical database and a configuration management database;
a data processing module: for evaluating an abnormal value of the data stored in the database, and performing an alarm when the abnormal value reaches a predetermined threshold value.
The working principle of the technical scheme is as follows: in the prior art, data classification of a dynamic loop monitoring system is generally performed according to texts, pictures and videos during data processing. The classification leads to analysis and fusion of different types of data for comprehensive judgment if the data are judged to be abnormal. This method is a conventional method in the prior art, but is extremely responsible for not only requiring a large amount of data processing and data operation, but also easily causing computational processing burden, and further causing the failure cause to be unable to be discovered.
In the present invention, the collection of real-time data is the first step of managing data by the data processing system. In the process of acquisition, data acquisition is realized, and data screening and classification are carried out according to data attributes. In the process, the data are classified into real-time data, historical data and configuration data, incremental data and historical data can be classified through the classification in the form, and configuration parameter data of equipment can also be determined; by means of the data comparison method, real-time data and historical data can be directly compared, and whether the data are abnormal or not can be judged quickly. In the step of estimating the abnormal value, the data in the real-time database and the data in the historical database can be estimated together, so that whether the data reaches the preset threshold value of the abnormal value or not is judged, and then an alarm is given.
The beneficial effects of the above technical scheme are: compared with the prior art, the invention can not accurately classify the original monitoring data, and the invention controls different data to be stored through an independent database mechanism, so that the data processing can be carried out by analyzing abnormal values of different data, and whether a fault occurs is judged, thereby being more convenient and less in data processing compared with the prior art; in the aspect of abnormal judgment, the problem of parameter setting or actually which equipment has problems can be judged, incremental data and historical data are divided, and data comparison is more convenient to carry out.
Example 2:
in one embodiment, as shown in fig. 2, the data storage module includes:
the data acquisition module comprises:
a communication interface unit: the system comprises a communication port, a dynamic ring device, a data acquisition device and a data stream transmission device, wherein the communication port is used for being butted with the dynamic ring device, and performing data acquisition and data stream transmission to generate acquired data;
a data screening unit: the data processing device is used for dividing the acquired data through a preset data attribute list to generate classified data; wherein the content of the first and second substances,
the classification data includes real-time data, historical data, and configuration data.
The working principle of the technical scheme is as follows: in the prior art, the data is divided mainly into types of text, pictures, video and the like. This increases the amount of data processing and calculation, although the data division is more clear. Therefore, the invention improves the prior art when processing data, data stream transmission is realized after butt joint, and the invention is based on the preset data attribute list in the process of dividing and screening the collected data, so that only effective data is processed during data processing, and irrelevant data can be filtered.
The beneficial effects of the above technical scheme are that: the invention can realize that the data is filtered in the process of screening the data, namely the data is filtered, and the interference data is filtered.
In the present invention:
the method also comprises the safety processing of real-time data, and the quality of the data is ensured to be favorable for the stable operation of the system. For high-speed data access requirements, a historical database is established to store real-time data, and good support is provided for data updating and querying. The historical data management part mainly uses a traditional database to store the historical data permanently so as to meet the requirements of later query and optimization analysis. The configuration management data associates the configuration items with their relationships through a configuration item tree structure.
The beneficial effects of the above technical scheme are: the data is stored through data screening and data classification, data query and access are facilitated, and data processing can be carried out in a subsequent data processing module more quickly.
Example 3:
in one embodiment of the invention:
the data storage module includes:
a real-time storage unit: the log is used for determining real-time measuring point access and real-time data access records in the classified data, determining real-time data and storing the real-time data into a real-time database;
a history storage unit: the system is used for carrying out time marking on the classification data and storing historical data into a historical database according to the time marking;
configuring a management database: the system is used for storing the attribute data of software and hardware of the dynamic ring monitoring system, determining the configuration item and configuration parameter information of the dynamic ring monitoring system, and storing a value configuration database.
The principle of the technical scheme is as follows: the invention relates to a method for detecting a point in real time, which comprises the following steps that real-time measuring point access is to implement access information of a detecting point, namely real-time detection information, and real-time data access is to implement real-time data access, namely real-time data access information, namely real-time data call information; the historical data is stored according to the time mark, so that the time point of the data can be determined, and the timeliness of the data can be judged conveniently. The configuration management database is used for storing configuration parameters of the equipment during monitoring, so that the equipment information and the equipment parameters are configured during storage.
The beneficial effects of the above technical scheme are that: the data division of the invention can make the part of data needed to be used for processing the part of data, and for other data, the data are stored in the historical database, so that the system has no interference when processing the data.
Example 4:
the real-time storage unit includes:
a configuration management subunit: the device is used for carrying out detailed storage on the real-time data stream and determining a storage position;
real-time measurement point access subunit: the system comprises a database, a data acquisition unit, a data processing unit and a data processing unit, wherein the database is used for storing basic information and configuration information of a measuring point;
real-time transaction access subunit: and the access data of the real-time transaction is determined through the transaction access port.
The principle and the beneficial effects of the technical scheme are as follows: when real-time data storage is carried out, the real-time database is equivalent to a cache database and only stores real-time data needing to be processed, the configuration management subunit refines the data columns and stores the real-time data according to the processing sequence, when the real-time measuring points access the subunit, the real-time data values are used for retrieving the measuring points, namely configuration information and basic information of the real-time monitoring equipment, and the real-time data values are parameters, output parameters and time parameters of configuration in the real-time data, and can be used for carrying out data calculation, judging whether the data is abnormal or not and carrying out other processing. The real-time transaction access subunit is used for determining the data being processed according to the access information of the real-time data, and the purpose of accessing the data is to determine what needs to be done.
Example 5:
in a specific embodiment, the historical database of the present invention comprises:
the history storage unit includes:
a history data storage subunit: the device is used for detecting the historical data stream and transferring the detected historical data stream to a corresponding storage unit;
the historical data query subunit: including data continuous queries and periodic data queries, as shown in fig. 2;
the historical data continuous query subunit: outputting a continuously updated and changed data set;
periodic data query subunit: for outputting a set of data for one cycle interval.
The working principle of the technical scheme is as follows: the history database is responsible for permanently storing data. The historical database is in a disk, and has I/O operation and low speed. In order to avoid the consumption of system resources due to the too frequent disk writing operation, the writing of small data is generally avoided, the real-time data is stored in a real-time database firstly and then written into a large amount of data together after being summarized, and the system efficiency is improved. The real-time database is a historical data buffer. When the data in the buffer area is accumulated to a certain condition, the data is permanently stored in the unified write-once disk file. There is a write condition for the buffer, which is generally determined by 2 parameters: one is the buffer data size and the other is the buffer data storage time. When the storage data in the buffer area is accumulated to a certain amount, or after a certain period of time during the process of buffering the data, the storage data should be written into the disk, so as to avoid the problem of excessive memory overhead.
The historical data is stored by using a data compression method, and the compression is realized in a memory database. In particular, two compression algorithms are used. One is a revolving door algorithm, and the other is discrete point saving according to the set effective precision. When a user accesses data, the revolving door generally needs to perform data interpolation and wait for data of sampling points other than the storage points, the discrete point storage rule directly uses the data of the previous storage point, and database query mainly helps the user to obtain the data for data analysis and optimization analysis.
The beneficial effects of the above technical scheme are: the compression algorithm of the historical database can save storage space, and the analysis and processing of data can be more conveniently carried out through continuous query and periodic query.
Example 6:
in a particular embodiment of the present invention,
the configuration management data unit includes:
configuration item basic information subunit: the system is used for storing hardware configuration items, software configuration items, network configuration items, document configuration items, association configuration items and parameter configuration items;
configuration item tree structure subunit: a relationship for associating configuration items with configuration parameters;
the configuration management database is constructed by the following steps:
s1: the scope of the configuration management is determined,
determining the width and depth and division of configuration items and defining the life cycle of the configuration items;
s2: defining a configuration item object, including determining the characteristics of the configuration item and determining the common attribute, the private attribute, the associated item and the relationship of the configuration item;
s3: and constructing a configuration model among the configuration items.
The working principle of the technical scheme is as follows: all components of the data processing system are called configuration items, a hardware configuration item of the operating environment of the system is provided with a dynamic ring storage server which comprises a case, a mainboard, a CPU, a memory, a hard disk, a RAID card, an SSD and a network card, and a software configuration item is provided with a computer room device for monitoring: the system provides various equipment communication access ports by itself, and is connected with various equipment, such as infrared, smoke, water immersion, door control, air conditioner, electric quantity instrument and UPS configuration items, and the diesel engine power monitoring support: UPS, electric quantity of commercial power, distribution switch, storage battery, precise power distribution cabinet, ATS/STS, power branch current, PDU cabinet power supply, lightning protection device, and generator equipment monitoring configuration items; and (3) environment monitoring support: monitoring air conditioning, water leakage, temperature and humidity, hydrogen and the like; microenvironment monitoring support: monitoring configuration items of temperature and humidity states in the cabinet; the finally constructed configuration model contains all the information of the configuration items, so that the management of the configuration information is facilitated.
The beneficial effects of the above technical scheme are: the configuration management database provides accurate system infrastructure information, stores and manages various configuration information of each device in the dynamic monitoring system, is tightly connected with a data processing module of the dynamic monitoring system, supports the operation of a process, exerts the value of the configuration information, and simultaneously provides the configuration information quickly and accurately in the auditing process.
Example 7:
in a specific embodiment, the data processing module includes:
a data scheduling unit: the data queue management system is used for scheduling data flow of a database, generating a data queue, searching similar data queues in a historical database, determining an abnormal data queue, performing abnormal value estimation on data of the abnormal data queue, and determining an abnormal value;
an alarm and confirmation unit: the data flow monitoring device is used for monitoring the data flow and executing alarm information when a threshold value is reached;
a concurrency control unit: the system is used for locking data items of data and executing concurrent data processing;
a data recovery unit: for recovering aborted data for deadlock handling.
The principle and the beneficial effects of the technical scheme are as follows: when continuing to process, the data processing module carries out data arrangement according to the data flow, judges whether abnormal data are determined according to the condition that the data arrangement is the same as the history or not, extracts the abnormal data, and then carries out abnormal value calculation on the abnormal data, thereby judging whether the abnormal value reaches a threshold value or not and realizing alarm. The data item locking is used for preventing the data from being deleted, or the data is aborted due to insufficient storage space, and then data recovery is realized, and the data cannot be deleted and compressed after locking.
In one particular embodiment:
when a data queue is generated for a data column, the queue is divided according to a time-limited scheduling strategy and time, so that priority allocation needs to be performed on each queue when the queues are arranged, more complete priority scheduling is realized, meanwhile, a plurality of uncertain parameters are added to realize the self-adaptive capacity of real-time scheduling, and a dynamic time priority algorithm is adopted:
s(T)＝r×(ω1×d-ω2×ta+ω3×c-ω4×n)
where T is data and r is a coefficient, indicating the urgency of the data T. Omega1，ω2，ω3，ω4All weights are used for adjusting the rationality of the algorithm, the value is near 1, 4 weights are determined according to the running state of the system, and therefore the priority scheduling of data is corrected, d is deadline, c is accumulated service volume, n is running time, ta is arrival time, and s is slack. Urgency r is related to the parameters of real-time data that needs to be processed in time if arriving real-time data is bursty, r is a small value, and r is a relatively large value to buffer the priority of a transaction if arriving real-time data is a periodic value.
n is an uncertain value and is predicted by characteristic parameters of data when the data of the database arrives at a data processing module, wherein the characteristic parameters of the data comprise the operation type (insertion, query) of the data and the quantity and type of the data. N is predicted by the following algorithm:
n＝αTt+α(1-α)Tt-1+α(1-α)2Tt-2+…
ω1，ω2，ω3，ω4the 4 coefficient parameters can be determined by recording system state information through a log device.
The beneficial effects of the above technical scheme are: the invention adopts the data queue to manage the priority, adopts the multithreading technology for data scheduling, can greatly simplify the problem of optimizing system resources by the multithreading technology, better executes concurrent data, can more accurately distribute the priority by using a time-first algorithm, and simultaneously realizes more perfect priority scheduling.
Example 8:
the system further comprises:
and the deadlock processing unit is used for checking the activity time of the data, rolling back the data when the activity time of the data exceeds the set time of the system, adding the data into the data queue again, and waiting for the next scheduling execution.
The technical scheme is that the data stream is always in an active state when being arranged, so the invention can set a rollback function and execute scheduling.
Example 9:
in one embodiment, as shown in fig. 3, the system further comprises:
an alarm confirmation unit: the alarm system is used for monitoring the data of the moving ring equipment, and when the data of the moving ring equipment reaches the threshold value, alarm information is sent out, wherein the alarm levels are 4 types and respectively:
if the alarm is not given, the alarm condition is not met, the alarm is not triggered, and the alarm information is not executed;
and low-level alarming is performed, an alarming condition is met, alarming is triggered, alarming information is recorded, and a user is prompted to check whether hidden dangers exist in a corresponding area. Neglectable, automatically confirm when the alarm condition is not satisfied;
advanced alarm, alarm condition is achieved, alarm is triggered, alarm information is recorded, and a user is reminded to check that an error occurs in a corresponding area. Non-negligible, requiring confirmation;
and interlocking alarm is carried out, alarm conditions are met, alarm is triggered, alarm information is recorded, a user is reminded of checking that errors occur in a corresponding area, and preset safety operation is automatically executed to prevent system errors from being enlarged. Not negligible, requiring confirmation.
The working principle of the technical scheme is as follows: the alarm and confirmation module in the system is responsible for monitoring whether the limit value of the data reaches the alarm value for realizing the setting, if so, the alarm information is sent out, and after a user checks the alarm information, the alarm can be confirmed, wherein the alarm types comprise input bad value alarm, data abnormity alarm, system error alarm and deviation alarm. The deviation alarm means that the alarm is given when the difference between the set value and the limit value is larger than the specified deviation alarm limit value. The deviation alarm comprises a positive deviation alarm, a negative deviation alarm and an absolute value deviation alarm.
The alarm information records are often required to be output, printed and stored, the alarm output information only concerns the time information, the position information and the alarm event information of the alarm event, and partial information can be omitted to improve the output efficiency and save the storage space by referring to the alarm information. The alarm level is replaced by integer number, and the alarm function can automatically trigger user-defined actions, including informing an operator, sending out a control signal and generating a report.
After the user views the alarm, an alarm confirmation may be performed. The alarm confirmation means that the alarm information is notified to the machine room personnel, and the machine room personnel know that a problem occurs in a certain link and feed back the problem to the system to confirm that the notification is effective. The alarm confirmation can confirm a single alarm message, can confirm all alarm messages in a unit or in an area, and can also confirm all alarm messages in a system.
The beneficial effects of the above technical scheme are: the alarm and confirmation module is used for ensuring the safety of system operation, automatically triggering user-defined convenience for the user to respond to the alarm signal, and referring to the alarm information, partial information can be omitted to improve the output efficiency and save the storage space.
Example 10:
in one embodiment, the system comprises:
and the log management device is used for recording the state information of each module operated by the data processing system and performing optimization analysis on the system and recovering data when the system crashes.
The working principle of the technical scheme is as follows: the log management equipment is responsible for monitoring the running condition of the whole system and the running state and other information of each module and is responsible for uniformly storing the information into a system log. The system log can be used for recovering the system when the system crashes, and can also be used for carrying out system optimization analysis. During the operation of the system, system failure may occur to cause system crash. The system fault mainly comprises: (1) system crash (hardware, software, network) (2) transactional system errors (3) deadlock in transaction detection (4) local abnormal errors (4) concurrent control (5) disk failures (6) catastrophic failures (caused by external environmental disasters) to enable the system to cope with a variety of possible system failures, the system needs a recovery mechanism.
In order to be able to recover from a failure affecting the execution of transactions by the system, the system maintains a log via a log management device. And this log records the status of all systems and the operations that affect the system data items. When the system fails, the previous state can be restored by the system log. The system operation log information contains information source (which module), time, type (normal/error), content. And the log management module stores the log information into a disk. Therefore, for frequent logging and storage, optimization processing is required to reduce the system load. The log management module maintains a log buffer queue. This queue is responsible for temporarily storing real-time log information of the system. And when a certain time period is reached, writing the log information in the buffer queue into the disk once. Through a buffer mechanism, the read-write times of the disk are effectively reduced, and the system efficiency is greatly improved.
The beneficial effects of the above technical scheme are: the log management equipment has good stability and reliability, supports backup and recovery systems, and performs periodic backup disk storage on log data and backup data to prevent data loss.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (10)

1. A data processing system for dynamic loop monitoring, comprising:
a data acquisition module: the data processing system is used for acquiring data of the moving ring equipment, determining the acquired data, screening and classifying the data according to the data attribute of the acquired data, and determining classified data;
a data storage module: storing the classified data into a corresponding database according to the data types; wherein the content of the first and second substances,
the database comprises a real-time database, a historical database and a configuration management database;
a data processing module: for evaluating an abnormal value of the data stored in the database, and performing an alarm when the abnormal value reaches a predetermined threshold value.
2. A data processing system for dynamic loop monitoring as claimed in claim 1, wherein said data acquisition module comprises:
a communication interface unit: the system comprises a communication port, a dynamic ring device, a data acquisition device and a data stream transmission device, wherein the communication port is used for being butted with the dynamic ring device, and performing data acquisition and data stream transmission to generate acquired data;
a data screening unit: the data processing device is used for dividing the acquired data through a preset data attribute list to generate classified data; wherein the content of the first and second substances,
the classification data includes real-time data, historical data, and configuration data.
3. A data processing system for dynamic loop monitoring as claimed in claim 1, wherein said data storage module comprises:
a real-time storage unit: the log is used for determining real-time measuring point access and real-time data access records in the classified data, determining real-time data and storing the real-time data into a real-time database;
a history storage unit: the system is used for carrying out time marking on the classification data and storing historical data into a historical database according to the time marking;
configuration management data unit: the system is used for storing the attribute data of software and hardware of the dynamic ring monitoring system, determining the configuration item and configuration parameter information of the dynamic ring monitoring system, and storing a value configuration management database.
4. A data processing system for dynamic loop monitoring as claimed in claim 3, wherein said real-time storage unit comprises:
a configuration management subunit: the device is used for carrying out detailed storage on the real-time data stream and determining a storage position;
real-time measurement point access subunit: the system comprises a database, a data acquisition unit, a data processing unit and a data processing unit, wherein the database is used for storing basic information and configuration information of a measuring point;
real-time transaction access subunit: and the access data of the real-time transaction is determined through the transaction access port.
5. A data processing system for dynamic loop monitoring as claimed in claim 3, wherein said history storage unit comprises:
a history data storage subunit: the device is used for detecting the historical data stream and transferring the detected historical data stream to a corresponding storage unit;
the historical data query subunit: the method comprises the steps of continuously inquiring data and periodically inquiring data; wherein the content of the first and second substances,
the historical data continuous query subunit: outputting a continuously updated and changed data set;
periodic data query subunit: for outputting a set of data for one cycle interval.
6. A data processing system for dynamic ring monitoring as claimed in claim 3, wherein said configuration management data unit comprises:
configuration item basic information subunit: the system is used for storing hardware configuration items, software configuration items, network configuration items, document configuration items, association configuration items and parameter configuration items;
configuration item tree structure subunit: a relationship for associating configuration items with configuration parameters;
the configuration management database is constructed by the following steps:
s1: the scope of the configuration management is determined,
determining the width and depth and division of configuration items and defining the life cycle of the configuration items;
s2: defining a configuration item object, including determining the characteristics of the configuration item and determining the common attribute, the private attribute, the associated item and the relationship of the configuration item;
s3: and constructing a configuration model among the configuration items.
7. A data processing system for dynamic loop monitoring as claimed in claim 1, wherein said data processing module comprises:
a data scheduling unit: the data queue management system is used for scheduling data flow of a database, generating a data queue, searching similar data queues in a historical database, determining an abnormal data queue, performing abnormal value estimation on data of the abnormal data queue, and determining an abnormal value;
an alarm and confirmation unit: the system comprises a data flow monitoring module, a data processing module and a data processing module, wherein the data flow monitoring module is used for monitoring abnormal values of data of an abnormal queue in the data flow and executing alarm information when a threshold value is reached;
a concurrency control unit: the system is used for locking data items of data and executing concurrent data processing;
a data recovery unit: for recovering aborted data for deadlock handling.
8. A data processing system for dynamic loop monitoring as claimed in claim 1, wherein said system further comprises:
and the deadlock processing unit is used for checking the activity time of the data, rolling back the data when the activity time of the data exceeds the set time of the system, adding the data into the data queue again, and waiting for the next scheduling execution.
9. A data processing system for dynamic loop monitoring as claimed in claim 7, wherein said data processing module further comprises:
an alarm and confirmation module: the method is used for monitoring the data of the moving ring equipment, when the data of the moving ring equipment reaches the threshold value, alarm information is sent out, wherein the alarm information comprises:
if the alarm is not given, the alarm condition is not met, the alarm is not triggered, and the alarm information is not executed;
low-level alarming, wherein alarming conditions are met, alarming is triggered, alarming information is recorded, and a user is prompted to check whether hidden danger exists in a corresponding area;
advanced alarm, namely triggering alarm when the alarm condition is met, recording alarm information and reminding a user of checking that an error occurs in a corresponding area;
and interlocking alarm is carried out, alarm conditions are met, alarm is triggered, alarm information is recorded, a user is reminded of checking that errors occur in a corresponding area, and preset safety operation is automatically executed to prevent system errors from being enlarged.
10. A data processing system for dynamic loop monitoring as claimed in claim 1, wherein said system further comprises:
a log management module: the system is used for recording the state information of each module operated by the data processing system, executing system optimization analysis and executing data recovery when the system crashes.
CN202111304949.7A 2021-11-05 2021-11-05 Data processing system for dynamic loop monitoring Pending CN114154035A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111304949.7A CN114154035A (en) 2021-11-05 2021-11-05 Data processing system for dynamic loop monitoring

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111304949.7A CN114154035A (en) 2021-11-05 2021-11-05 Data processing system for dynamic loop monitoring

Publications (1)

Publication Number Publication Date
CN114154035A true CN114154035A (en) 2022-03-08

Family

ID=80459259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111304949.7A Pending CN114154035A (en) 2021-11-05 2021-11-05 Data processing system for dynamic loop monitoring

Country Status (1)

Country Link
CN (1) CN114154035A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114579408A (en) * 2022-05-05 2022-06-03 西安热工研究院有限公司 System and method for analyzing real-time equation of real-time database
CN116627959A (en) * 2023-07-26 2023-08-22 合肥思迈科技有限公司 Method for clearing operation history data of movable ring equipment of machine room

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114579408A (en) * 2022-05-05 2022-06-03 西安热工研究院有限公司 System and method for analyzing real-time equation of real-time database
CN116627959A (en) * 2023-07-26 2023-08-22 合肥思迈科技有限公司 Method for clearing operation history data of movable ring equipment of machine room
CN116627959B (en) * 2023-07-26 2023-10-13 合肥思迈科技有限公司 Method for clearing operation history data of movable ring equipment of machine room

Similar Documents

Publication Publication Date Title
CN107015872B (en) The processing method and processing device of monitoring data
CN101876938B (en) Message queue-based application software response time measuring method
CN106487574A (en) Automatic operating safeguards monitoring system
CN110287081A (en) A kind of service monitoring system and method
CN104011719B (en) The method and system that message is tracked and checked
US11030038B2 (en) Fault prediction and detection using time-based distributed data
CN111339175B (en) Data processing method, device, electronic equipment and readable storage medium
CN114154035A (en) Data processing system for dynamic loop monitoring
CN105224888B (en) A kind of data of magnetic disk array protection system based on safe early warning technology
CN109165138A (en) A kind of method and apparatus of monitoring equipment fault
CN109144789A (en) A kind of method, apparatus and system for restarting OSD
JP2012208646A (en) Monitoring system
CN112865311B (en) Method and device for monitoring message bus of power system
CN106911519A (en) A kind of data acquisition monitoring method and device
CN112118127B (en) Service reliability guarantee method based on fault similarity
CN110717130A (en) Dotting method, dotting device, dotting terminal and storage medium
CN111240936A (en) Data integrity checking method and equipment
CN111414355A (en) Offshore wind farm data monitoring and storing system, method and device
US11782873B2 (en) System and method for managing timeseries data
CN113407374A (en) Fault processing method and device, fault processing equipment and storage medium
CN113434366A (en) Event processing method and system
CN111917599B (en) Management system and method for cloud platform host state
CN117743473B (en) Data management synchronization performance monitoring system
CN116389584B (en) Cloud edge data transmission method for electrochemical energy storage station
CN115269623A (en) Data double-writing monitoring method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination