CN112579571B - Monitoring data configuration, data monitoring method, device, equipment and storage medium - Google Patents
Monitoring data configuration, data monitoring method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112579571B CN112579571B CN202011496403.1A CN202011496403A CN112579571B CN 112579571 B CN112579571 B CN 112579571B CN 202011496403 A CN202011496403 A CN 202011496403A CN 112579571 B CN112579571 B CN 112579571B
- Authority
- CN
- China
- Prior art keywords
- monitoring
- data
- configuration
- monitored
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 356
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000004590 computer program Methods 0.000 claims description 19
- 230000003068 static effect Effects 0.000 claims description 14
- 238000012806 monitoring device Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 6
- 238000012795 verification Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of data monitoring, and discloses a method, a device, equipment and a storage medium for monitoring data configuration and data monitoring. The method for configuring the monitoring data comprises the steps of performing monitoring configuration on the data to be monitored according to configuration rules to obtain monitoring configuration information corresponding to the data to be monitored; and generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to the monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script. The method reduces the complexity of the computer system. The data monitoring method comprises the steps of reading a monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script; executing a data monitoring program to monitor the data to be monitored, and generating a monitoring result corresponding to the data to be monitored; and displaying the monitoring result in a preset display page. The method improves the accuracy and the efficiency of data monitoring.
Description
Technical Field
The present invention relates to the field of data monitoring technologies, and in particular, to a method, an apparatus, a device, and a storage medium for monitoring data configuration and data monitoring.
Background
With the development of science and technology, important data are mostly stored in various databases without being stored through paper, and with the increase of the storage capacity of the data, the storage capacity of the databases is limited, so that quality monitoring management is required for the data before the data are stored.
In the prior art, the traditional database monitoring method cannot find potential problems in the system operation process in time, database data loss is easy to occur, and the problems of increased computer complexity, incomplete monitoring, low efficiency and the like are caused by complex data and multiple elements of operation and maintenance personnel.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for monitoring data configuration and data monitoring, which are used for solving the problem that the potential problem in the running process of a system cannot be found in time in the traditional database monitoring, so that the data loss of the database is easy to cause.
A method of monitoring data configuration, comprising:
receiving a data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
monitoring configuration is carried out on the data to be monitored according to the configuration rule, and monitoring configuration information corresponding to the data to be monitored is obtained;
and generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script.
A data monitoring method, comprising:
receiving a data monitoring instruction containing a monitoring execution script; the monitoring execution script is obtained by configuring the data to be monitored according to the monitoring data configuration method;
reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script;
executing the data monitoring program, monitoring the data to be monitored, and generating a monitoring result corresponding to the data to be monitored;
and displaying the monitoring result in a preset display page.
A monitoring data configuration device, comprising:
the configuration instruction receiving module is used for receiving the data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
the monitoring configuration module is used for carrying out monitoring configuration on the data to be monitored according to the configuration rule to obtain monitoring configuration information corresponding to the data to be monitored;
and the monitoring execution script generation module is used for generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script.
A data monitoring device, comprising:
the monitoring instruction receiving module is used for receiving a data monitoring instruction containing a monitoring execution script; the monitoring execution script is obtained by configuring the data to be monitored according to the monitoring data configuration method;
the monitoring program generating module is used for reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script;
the monitoring program execution module is used for executing the data monitoring program, monitoring the data to be monitored and generating a monitoring result corresponding to the data to be monitored;
and the monitoring result display module is used for displaying the monitoring result in a preset display page.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the above-mentioned monitoring data configuration method when executing the computer program or the above-mentioned data monitoring method when executing the computer program.
A computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described monitoring data configuration method or which, when executed by a processor, implements the above-described data monitoring method.
The monitoring data configuration method, the data monitoring method, the device, the equipment and the storage medium are characterized in that the monitoring data configuration method receives a data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored; monitoring configuration is carried out on the data to be monitored according to the configuration rule, and monitoring configuration information corresponding to the data to be monitored is obtained; and generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script. According to the method, the monitoring execution script which can be identified by the monitoring system, namely the executable program which can be identified by the monitoring system and accords with logic is generated according to the configuration rule corresponding to the data to be monitored, so that the monitoring system can directly identify the executable program and execute monitoring, and the complexity of the computer system is reduced.
The data monitoring method of the invention receives the data monitoring instruction containing the monitoring execution script; the monitoring execution script is obtained by configuring the data to be monitored according to the monitoring data configuration method; reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script; executing the data monitoring program, monitoring the data to be monitored, and generating a monitoring result corresponding to the data to be monitored; and displaying the monitoring result in a preset display page. The data monitoring method can realize the monitoring operation of all the data to be monitored through one monitoring execution script, greatly reduces the computational complexity of a computer, improves the data monitoring efficiency, can timely and accurately determine whether the data to be monitored is abnormal or not, and displays the monitoring result in the preset display page, so that a user can clearly know the specific condition of the abnormal data.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an application environment of a method for configuring or monitoring data according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for configuring monitoring data according to an embodiment of the invention;
FIG. 3 is a flowchart of step S20 in a method for configuring monitoring data according to an embodiment of the present invention;
FIG. 4 is another flow chart of a method of monitoring data configuration in an embodiment of the invention;
FIG. 5 is a flow chart of a method of data monitoring in an embodiment of the invention;
FIG. 6 is a schematic block diagram of a monitoring data configuration device according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of a monitoring configuration module in a monitoring data configuration device according to an embodiment of the present invention;
FIG. 8 is a schematic block diagram of a data monitoring device in accordance with an embodiment of the present invention;
FIG. 9 is a schematic diagram of a computer device in accordance with an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method for configuring monitoring data (or the method for monitoring data) provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1. Specifically, the monitoring data configuration method (or data monitoring method) is applied to a monitoring data configuration system (or data monitoring system), the monitoring data configuration system (or data monitoring system) comprises a client and a server as shown in fig. 1, and the client and the server communicate through a network, so that the problem that potential problems in the running process of the system cannot be found in time during monitoring of a traditional database, and the problem that the data of the database is easy to lose is solved. The client is also called a client, and refers to a program corresponding to the server for providing local service for the client. The client may be installed on, but is not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a method for configuring monitoring data is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
s10: receiving a data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
it can be understood that the data configuration instruction may be an instruction sent by other users, or may be an instruction triggered by the client after the server inputs the data to be monitored and the corresponding configuration rule. The data to be monitored can be a single data table in the database, or can be the result of the association of a plurality of data tables. The configuration rule refers to a monitoring configuration rule set for different data to be monitored, and the configuration rule can be null detection, repeated field detection, standard detection (such as identity card number, mobile phone number and mailbox number), regular expression detection, logic detection and the like, and can be adopted for different data to be monitored, so that the efficiency and accuracy of monitoring each type of data to be monitored are improved, and the computational complexity of a computer system is further reduced.
S20: monitoring configuration is carried out on the data to be monitored according to the configuration rule, and monitoring configuration information corresponding to the data to be monitored is obtained;
after receiving a data monitoring configuration instruction, acquiring data to be monitored and configuration rules corresponding to the data to be monitored in the monitoring configuration instruction, performing monitoring configuration on all the data to be monitored one by one according to the configuration rules, and exemplarily, if the data to be monitored contains an identity card number, and the corresponding configuration rules are identity card format verification and null value verification, performing monitoring configuration on the identity card number according to the configuration rules, when the position of the identity card number is monitored, determining whether the position of the identity card number is null, if not null, acquiring the identity card number at the position and verifying the identity card number format, such as the number of digits of the identity card, and whether meanings represented at each position of the identity card number meet requirements, such as whether counterfeits exist at the birth year, month and day positions (such as future days of birth, namely counterfeits); if the position of the identification card number is empty, confirming that the identification card number in the data to be monitored is abnormal.
It can be understood that in the whole monitoring flow of the above example, it is determined whether the position of the identification card number is empty, and if the number of digits of the identification card is equal to the requirement, and the meaning represented on each position of the identification card number is the monitoring configuration information corresponding to the identification card number.
In an embodiment, as shown in fig. 3, the configuration rule includes a static rule, and in step S20, that is, the monitoring configuration is performed on the data to be monitored according to the configuration rule, to obtain monitoring configuration information corresponding to the data to be monitored, including:
s201: acquiring all monitoring fields in the data to be monitored; one of the monitoring fields is associated with one of the configuration rules;
s202: recording a monitoring field with the associated configuration rule as the static rule as a conventional field;
illustratively, the monitoring field in the data to be monitored may include, for example, an identification card number field, a mobile phone number field, a mailbox number field, etc.; the fields can be checked through common static rules, so that the identification card number field, the mobile phone number field and the mailbox number field are conventional fields. The static rules may be null value monitoring, format monitoring (conventional format verification may be adopted because the identification card number, the mailbox number, the mobile phone number, etc. have a relatively fixed format), etc.
S203: and performing conventional monitoring configuration on the conventional fields according to the static rules to obtain conventional configuration information corresponding to the conventional fields.
Specifically, a static rule or several static rule association checks may be set for different conventional fields, for example, if the id number field is monitored, id format monitoring (to detect whether the format of the id number field conforms to the existing id number rule), null value monitoring (to detect whether the id number field is null) may be set.
In another embodiment, as shown in fig. 3, the configuration rule further includes a dynamic rule, and in step S20, that is, the monitoring configuration is performed on the data to be monitored according to the configuration rule, so as to obtain monitoring configuration information corresponding to the data to be monitored, further includes:
s204: recording a monitoring field with the associated configuration rule being a dynamic rule as a special field;
it will be appreciated that since some fields to be monitored cannot be monitored conventionally according to existing rules or regulations (e.g. the number of downloads of an application in a region needs to be monitored), dynamic rule-based monitoring verification of these specific fields is required. Dynamic rules refer to rules that perform logical relational transformations on special fields so that logical monitoring checks can be performed on the special fields.
S205: performing dynamic expression configuration on the special field to obtain a logic expression corresponding to the special field; and recording the logic expression as special configuration information corresponding to the special field.
Specifically, after recording a monitoring field with an associated configuration rule being a dynamic rule as a special field, carrying out dynamic expression configuration on the special field to obtain a logic expression corresponding to the special field; and recording the logic expression as special configuration information corresponding to the special field.
For example, assuming that it is necessary to investigate the download rate of one or both applications in a certain region, such as "Shenzhen" region, and "Wang" application, after acquiring the download data of all applications, dynamic rule configuration is performed on "Wang" download data, and a logical expression such as if (area= 'Shen, n') and (app_name= 'Wang') True else Fase pattern is obtained.
S30: and generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script.
It can be understood that after all the data to be monitored are configured for monitoring, that is, after the monitoring configuration information corresponding to each data to be monitored is generated, a corresponding monitoring execution script is generated according to the monitoring configuration information, so that the monitoring system can identify the monitoring execution script and perform a monitoring process based on the corresponding configuration rule on the data to be monitored. The monitoring execution script is an executable program which can be identified by the monitoring system and accords with logic.
In this embodiment, according to the configuration rule corresponding to the data to be monitored, a monitoring execution script that can be identified by the monitoring system, that is, an executable program that can be identified by the monitoring system and conforms to logic is generated, so that the monitoring system can directly identify the executable program and execute monitoring, and the complexity of the computer system is reduced.
In an embodiment, as shown in fig. 4, before step S10, that is, before the receiving the data configuration instruction, the method further includes:
s01: performing data configuration on a data source to access initial monitoring data in the data source;
the data configuration refers to a configuration method for acquiring initial monitoring data, the data configuration process determines which databases, data tables or data information can be acquired, and the data configuration can include database configuration, ftp file server configuration or local offline file configuration. Alternatively, the initial monitoring data may be stored in tabular form.
S02: identifying a data acquisition rule containing user monitoring requirements; the data acquisition rule is used for acquiring the initial monitoring data; the user monitoring requirement comprises at least one field to be monitored;
the data acquisition rules include multiple table association rules (i.e., acquiring multiple associated data tables) and a single acquisition rule (i.e., acquiring separate tables). The field to be monitored refers to a data field that the user needs to monitor. It will be appreciated that in step S01, the data source is configured so that the server can obtain the right to access the data in the data source, and after the data acquisition rule including the user monitoring requirement is identified in step S02, the initial monitoring data associated with the field to be monitored is acquired from the data source through the data acquisition rule.
S03: detecting whether the data acquisition rule is a multi-table association rule;
s04: and when the data acquisition rule is a multi-table association rule, recording all initial monitoring data association related to the field to be monitored as the data to be monitored.
It can be understood that, there may be a plurality of fields to be monitored in the initial monitoring data (for example, the initial monitoring data is stored in each table, and the fields to be monitored are included in the plurality of tables), and when the data acquisition rule is a multi-table association rule, all tables associated with the fields to be monitored need to be used as the data to be monitored, so that when the field to be monitored is abnormal, it can be accurately known which table has the abnormality in the field to be monitored, and further accuracy and efficiency of data monitoring are improved.
In an embodiment, as shown in fig. 4, after step S03, that is, after the detecting whether the data acquisition rule is a multi-table association rule, the method further includes:
s05: when the data acquisition rule is not a multi-table association rule, acquiring a user positioning requirement; the user positioning requirement indicates a storage position of the field to be monitored;
it will be appreciated that each initial monitoring data in the data source corresponds to a storage location, that is, one storage location is associated with one initial monitoring data, and if the user has determined the storage location of the data to be monitored, the system may obtain the data to be monitored from the initial monitoring data according to the storage location and the field to be monitored.
S06: and recording the initial monitoring data matched with the storage position and the field to be monitored as the data to be monitored.
It may be appreciated that when the data acquisition rule is not a multi-table association rule, after the user positioning requirement is acquired, after the storage location is determined from the data source, all initial monitoring data in the storage location is matched with the field to be monitored, such as similarity matching, character matching, and the like, so that initial monitoring data matched with the field to be monitored (such as initial monitoring data with a similarity higher than 95% between the initial monitoring data and the field to be monitored) is recorded as the data to be monitored.
Further, if the user cannot determine the specific storage location of the data to be monitored, the field to be monitored and all initial monitoring data in the data source can be matched, so that the initial monitoring data matched with the field to be monitored can be determined and recorded as the data to be monitored.
In one embodiment, as shown in fig. 5, a data monitoring method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
s40: receiving a data monitoring instruction containing a monitoring execution script; the monitoring execution script is obtained by configuring the data to be monitored according to the monitoring data configuration method;
s50: reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script;
s60: executing the data monitoring program, monitoring the data to be monitored, and generating a monitoring result corresponding to the data to be monitored;
s70: and displaying the monitoring result in a preset display page.
Specifically, after the monitoring execution script is generated according to the monitoring data configuration method in the above embodiment, the system may read and identify the monitoring execution script, and generate a piece of metadata corresponding to the monitoring execution script, so as to read the metadata, and automatically generate a corresponding data monitoring program; the data monitoring program is executed to monitor the data to be monitored, and corresponding monitoring results are generated.
For example, assuming that the data to be monitored is an identification card field in the user table (the identification card field is configured with identification card rule verification and generates a corresponding monitoring execution script), the system generates a piece of metadata to complete identification card rule verification on the identification card field of the user table, and then the system reads the metadata and automatically generates a verification program, and then performs full-table data scanning on all identification cards in the user table to obtain a corresponding monitoring result. Finally, the monitoring result is written into the appointed data table or displayed in a preset display page, and the monitoring result can output data for checking abnormality (for checking and correcting by service personnel).
Further, in the process of monitoring the data to be monitored, the data to be monitored can be monitored according to the monitoring frequency set by the user, for example, the monitoring frequency is ten times a day, or the monitoring is performed once a day; different monitoring frequencies can be set for the data to be monitored with different importance degrees (for example, the data to be monitored with higher importance degree and higher quality requirement can be monitored with high frequency, such as monitoring once every half hour, etc.).
Further, the monitoring result is displayed in a preset display page, namely the monitoring result is displayed on a UI interface, wherein the preset UI interface is a visual interface. And the display can be performed on the UI interface in a chart form, so that the time of data abnormality or the frequency of data abnormality and the like can be more intuitively seen.
In the embodiment, the monitoring operation of all the data to be monitored can be realized through one monitoring execution script, so that the calculation complexity of a computer is greatly reduced, and the data monitoring efficiency is improved; in addition, the data monitoring method can timely and accurately determine whether the data to be monitored is abnormal or not, and display the monitoring result in the preset display page, so that a user can clearly know the specific condition of the abnormal data.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a monitoring data configuration device is provided, where the monitoring data configuration device corresponds to the monitoring data configuration method in the foregoing embodiment one by one. As shown in fig. 6, the monitoring data configuration device includes a configuration instruction receiving module 10, a monitoring configuration module 20, and a monitoring execution script generating module 30. The functional modules are described in detail as follows:
a configuration instruction receiving module 10, configured to receive a data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
the monitoring configuration module 20 is configured to monitor and configure the data to be monitored according to the configuration rule, so as to obtain monitoring configuration information corresponding to the data to be monitored;
and the monitoring execution script generating module 30 is configured to generate a monitoring execution script according to the monitoring configuration information and send the monitoring execution script to a monitoring system, so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script.
Preferably, as shown in fig. 7, the monitoring configuration module 20 includes the following units:
a monitoring field obtaining unit 201, configured to obtain all monitoring fields in the data to be monitored; one of the monitoring fields is associated with one of the configuration rules;
a regular field recording unit 202, configured to record a monitoring field, in which the associated configuration rule is the static rule, as a regular field;
and a conventional monitoring configuration unit 203, configured to perform conventional monitoring configuration on the conventional field according to the static rule, so as to obtain conventional configuration information corresponding to the conventional field.
Preferably, as shown in fig. 7, the monitoring configuration module 20 further includes the following units:
a special field recording unit 204, configured to record the monitoring field, in which the associated configuration rule is a dynamic rule, as a special field;
a special monitoring configuration unit 205, configured to perform dynamic expression configuration on the special field, so as to obtain a logic expression corresponding to the special field; and recording the logic expression as special configuration information corresponding to the special field.
Preferably, the monitoring data configuration device further includes:
the data configuration module is used for carrying out data configuration on the data source so as to access initial monitoring data in the data source;
the data acquisition rule identification module is used for identifying data acquisition rules containing user monitoring requirements; the data acquisition rule is used for acquiring the initial monitoring data; the user monitoring requirement comprises at least one field to be monitored;
the data acquisition rule detection module is used for detecting whether the data acquisition rule is a multi-table association rule or not;
and the first data record module to be monitored is used for recording all initial monitoring data associated with the field to be monitored as the data to be monitored when the data acquisition rule is a multi-table association rule.
Preferably, the monitoring data configuration device further includes:
the user positioning requirement acquisition module is used for acquiring the user positioning requirement when the data acquisition rule is not a multi-table association rule; the user positioning requirement indicates a storage position of the field to be monitored;
and the second data recording module to be monitored is used for recording the initial monitoring data matched with the storage position and the field to be monitored as the data to be monitored.
For specific limitations of the monitoring data configuration device, reference may be made to the above limitation of the monitoring data configuration method, and no further description is given here. The respective modules in the above-described monitoring data configuration apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In an embodiment, a data monitoring device is provided, where the data monitoring device corresponds to the data monitoring method in the foregoing embodiment one by one. As shown in fig. 8, the data monitoring apparatus includes a monitoring instruction receiving module 40, a monitoring program generating module 50, a monitoring program executing module 60, and a monitoring result displaying module 70. The functional modules are described in detail as follows:
a monitoring instruction receiving module 40, configured to receive a data monitoring instruction including a monitoring execution script; the monitoring execution script is obtained by configuring the data to be monitored according to the monitoring data configuration method;
the monitoring program generating module 50 is configured to read the monitoring execution script and generate a data monitoring program corresponding to the monitoring execution script;
the monitoring program executing module 60 is configured to execute the data monitoring program, monitor the data to be monitored, and generate a monitoring result corresponding to the data to be monitored;
and the monitoring result display module 70 is configured to display the monitoring result in a preset display page.
For specific limitations of the data monitoring device, reference may be made to the above limitation of the data monitoring method, and no further description is given here. The respective modules in the data monitoring apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. X. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the configuration method of the monitoring data in the above embodiment or the data used by the data monitoring method in the above embodiment. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by the processor implements a method of monitoring data configuration, or the computer program when executed by the processor implements a method of data monitoring.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the method of monitoring data configuration in the above embodiments when executing the computer program, or the processor implements the method of monitoring data in the above embodiments when executing the computer program.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the method of monitoring data configuration in the above embodiment, or which when executed by a processor implements the method of monitoring data in the above embodiment.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (7)
1. A method of monitoring data configuration, comprising:
performing data configuration on a data source to access initial monitoring data in the data source;
identifying a data acquisition rule containing user monitoring requirements; the data acquisition rule is used for acquiring the initial monitoring data; the user monitoring requirement comprises at least one field to be monitored;
detecting whether the data acquisition rule is a multi-table association rule;
when the data acquisition rule is a multi-table association rule, all initial monitoring data associated with the field to be monitored are recorded as data to be monitored;
receiving a data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
monitoring configuration is carried out on the data to be monitored according to the configuration rule, and monitoring configuration information corresponding to the data to be monitored is obtained;
generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script;
the configuration rules comprise static rules; the monitoring configuration is carried out on the data to be monitored according to the configuration rule to obtain monitoring configuration information corresponding to the data to be monitored, and the method comprises the following steps:
acquiring all monitoring fields in the data to be monitored; one of the monitoring fields is associated with one of the configuration rules;
recording a monitoring field with the associated configuration rule as the static rule as a conventional field; the conventional field is a field which can be checked by a common static rule;
performing conventional monitoring configuration on the conventional fields according to the static rules to obtain conventional configuration information corresponding to the conventional fields;
the configuration rules also include dynamic rules; the monitoring configuration is carried out on the data to be monitored according to the configuration rule to obtain monitoring configuration information corresponding to the data to be monitored, and the method further comprises the following steps:
recording a monitoring field with the associated configuration rule being a dynamic rule as a special field; the special field refers to a field which cannot be monitored conventionally according to the existing rule or rule;
performing dynamic expression configuration on the special field to obtain a logic expression corresponding to the special field; and recording the logic expression as special configuration information corresponding to the special field.
2. The method for configuring monitoring data according to claim 1, wherein after detecting whether the data acquisition rule is a multi-table association rule, further comprising:
when the data acquisition rule is not a multi-table association rule, acquiring a user positioning requirement; the user positioning requirement indicates a storage position of the field to be monitored;
and recording the initial monitoring data matched with the storage position and the field to be monitored as the data to be monitored.
3. A method of data monitoring, comprising:
receiving a data monitoring instruction containing a monitoring execution script; the monitoring execution script is obtained by configuring data to be monitored according to the monitoring data configuration method as set forth in claim 1 or 2;
reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script;
executing the data monitoring program, monitoring the data to be monitored, and generating a monitoring result corresponding to the data to be monitored;
and displaying the monitoring result in a preset display page.
4. A monitoring data configuration device for performing the monitoring data configuration method of claim 1 or 2, the monitoring data configuration device comprising:
the configuration instruction receiving module is used for receiving the data monitoring configuration instruction; the data configuration instruction comprises data to be monitored and configuration rules corresponding to the data to be monitored;
the monitoring configuration module is used for carrying out monitoring configuration on the data to be monitored according to the configuration rule to obtain monitoring configuration information corresponding to the data to be monitored;
and the monitoring execution script generation module is used for generating a monitoring execution script according to the monitoring configuration information and sending the monitoring execution script to a monitoring system so that the monitoring system monitors the data to be monitored after identifying the monitoring execution script.
5. A data monitoring device is characterized in that,
the monitoring instruction receiving module is used for receiving a data monitoring instruction containing a monitoring execution script; the monitoring execution script is obtained by configuring data to be monitored according to the monitoring data configuration method as set forth in claim 1 or 2;
the monitoring program generating module is used for reading the monitoring execution script and generating a data monitoring program corresponding to the monitoring execution script;
the monitoring program execution module is used for executing the data monitoring program, monitoring the data to be monitored and generating a monitoring result corresponding to the data to be monitored;
and the monitoring result display module is used for displaying the monitoring result in a preset display page.
6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method of monitoring data configuration according to claim 1 or 2 when executing the computer program or the data monitoring method according to claim 3 when the processor executes the computer program.
7. A computer-readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method of monitoring data configuration according to claim 1 or 2, or the computer program when executed by a processor implements the method of data monitoring according to claim 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011496403.1A CN112579571B (en) | 2020-12-17 | 2020-12-17 | Monitoring data configuration, data monitoring method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011496403.1A CN112579571B (en) | 2020-12-17 | 2020-12-17 | Monitoring data configuration, data monitoring method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112579571A CN112579571A (en) | 2021-03-30 |
| CN112579571B true CN112579571B (en) | 2024-03-19 |
Family
ID=75135976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011496403.1A Active CN112579571B (en) | 2020-12-17 | 2020-12-17 | Monitoring data configuration, data monitoring method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579571B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114416805B (en) * | 2022-03-10 | 2022-08-02 | 深圳高灯计算机科技有限公司 | Data checking method and device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622289A (en) * | 2011-01-26 | 2012-08-01 | 阿里巴巴集团控股有限公司 | Program setting method, program monitoring method, and devices and systems for program setting and monitoring |
| WO2017107853A1 (en) * | 2015-12-25 | 2017-06-29 | 阿里巴巴集团控股有限公司 | Data monitoring management method, and data monitoring method and system |
| CN110161431A (en) * | 2018-02-12 | 2019-08-23 | 西门子公司 | Processing method, device and the storage medium of frequency converter monitoring data |
-
2020
- 2020-12-17 CN CN202011496403.1A patent/CN112579571B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622289A (en) * | 2011-01-26 | 2012-08-01 | 阿里巴巴集团控股有限公司 | Program setting method, program monitoring method, and devices and systems for program setting and monitoring |
| WO2017107853A1 (en) * | 2015-12-25 | 2017-06-29 | 阿里巴巴集团控股有限公司 | Data monitoring management method, and data monitoring method and system |
| CN110161431A (en) * | 2018-02-12 | 2019-08-23 | 西门子公司 | Processing method, device and the storage medium of frequency converter monitoring data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112579571A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109474578B (en) | Message checking method, device, computer equipment and storage medium | |
| CN109446068B (en) | Interface test method, device, computer equipment and storage medium | |
| CN109189367B (en) | Data processing method, device, server and storage medium | |
| CN110751550B (en) | Account checking method and device, computer equipment and storage medium | |
| CN110597511B (en) | Page automatic generation method, system, terminal equipment and storage medium | |
| CN110321142B (en) | Interface document updating method and device, electronic equipment and storage medium | |
| CN111552506B (en) | Service rule configuration method, device and storage medium | |
| CN112395157B (en) | Audit log acquisition method and device, computer equipment and storage medium | |
| CN109542764B (en) | Webpage automatic testing method and device, computer equipment and storage medium | |
| CN112613288B (en) | Method, device, computer equipment and storage medium for generating bill | |
| CN112035471A (en) | Transaction processing method and computer equipment | |
| CN112579571B (en) | Monitoring data configuration, data monitoring method, device, equipment and storage medium | |
| CN110147354B (en) | Batch data editing method, device, computer equipment and storage medium | |
| CN110275703B (en) | Method and device for assigning key value to data, computer equipment and storage medium | |
| CN111767192A (en) | Service data detection method, device, equipment and medium based on artificial intelligence | |
| CN112783866B (en) | Data reading method, device, computer equipment and storage medium | |
| CN112463783A (en) | Index data monitoring method and device, computer equipment and storage medium | |
| CN110443072B (en) | Data signature method, data verification device and storage medium | |
| CN117331956A (en) | Task processing method, device, computer equipment and storage medium | |
| CN113722225A (en) | Page testing method and device, computer equipment and storage medium | |
| WO2019062087A1 (en) | Attendance check data testing method, terminal and device, and computer readable storage medium | |
| CN113918593A (en) | Method and device for identifying financial data abnormity, storage medium and computing equipment | |
| CN114116761A (en) | Variable processing method, variable processing device, computer equipment and storage medium | |
| CN113806372B (en) | New data information construction method, device, computer equipment and storage medium | |
| CN113052463B (en) | Workflow verification method, workflow verification device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |