CN111767283A - Data system monitoring method and system - Google Patents

Data system monitoring method and system Download PDF

Info

Publication number
CN111767283A
CN111767283A CN202010567262.1A CN202010567262A CN111767283A CN 111767283 A CN111767283 A CN 111767283A CN 202010567262 A CN202010567262 A CN 202010567262A CN 111767283 A CN111767283 A CN 111767283A
Authority
CN
China
Prior art keywords
query
data
audit
field
main body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010567262.1A
Other languages
Chinese (zh)
Other versions
CN111767283B (en
Inventor
郭杨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Si Tech Information Technology Co Ltd
Original Assignee
Beijing Si Tech Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Si Tech Information Technology Co Ltd filed Critical Beijing Si Tech Information Technology Co Ltd
Priority to CN202010567262.1A priority Critical patent/CN111767283B/en
Publication of CN111767283A publication Critical patent/CN111767283A/en
Application granted granted Critical
Publication of CN111767283B publication Critical patent/CN111767283B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462Approximate or statistical queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • G06Q50/40

Abstract

The invention discloses a data system monitoring method and a system, which are used for acquiring a query result at a set time and auditing the query result according to a set auditing algorithm, so that the query result and the auditing result can be automatically acquired conveniently by an automatic tool; the method has the advantages that the main body table fields are summarized and divided through the main body categories, so that the main body table fields can be selected quickly, and system maintenance personnel are prevented from missing part of the main body table fields to cause data errors or recalculation; through summarizing the action fields, the method is beneficial to monitoring or screening the data of the main body in different action links in the production flow, improves the monitoring fineness and ensures the accuracy and the integrity of the statistical main body.

Description

Data system monitoring method and system
Technical Field
The invention relates to the technical field of data monitoring, in particular to a data system monitoring method and system.
Background
With the development of information technology, the information amount is increased explosively, which is reflected in that the service data is various, the service data amount or content is greatly increased, the number of butt-joint systems or butt-joint personnel is increased, the time point of the service data is not fixed, so that the maintenance of the data system is difficult, the problems of data leakage or data maintenance errors and the like are caused, the calculation errors of scores, costs and the like are finally caused, the accounting needs to be repeated, and the statistical data and the statistical period are seriously influenced.
With the increase of service data and types of the system, the full data system does not need to be counted more often, only a part of statistical data with relatively concerned indexes is needed, and the statistical period or time point of each service is also various, which may cause omission and result in incomplete data, thereby providing new requirements for the maintenance and statistical data of the data system.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a database-based vulnerability management method, which is used for monitoring the data of a main body in different action links in a production flow and improving the monitoring fineness.
The invention discloses a data system monitoring method, which comprises the following steps: respectively establishing a body type and an action type according to a body table field and an action field of the data system; establishing a master-slave relationship between the main body category and the main body table field; constructing a data auditing planning table according to the action types, the subject types and the subject table fields, wherein the auditing planning table also comprises an auditing algorithm and data query time; acquiring a query field according to the subject type, the subject table field and the action type of the data audit planning table; acquiring a query result of a data system according to the query field within the data query time; and acquiring the audit result according to the audit algorithm and the query result.
Preferably, the invention further comprises a method for generating the early warning information: generating early warning information according to the audit result and the early warning range; and generating warning information according to the warning information and the warning information template.
Preferably, the present invention further comprises a method for sending an alert message, wherein the alert message is sent to an alert message recipient by one or a combination of the following methods: short message push, instant chat tool sending, mail sending, announcement push, or PC display.
Preferably, the audit planning table comprises a plurality of action types, and the query field is constructed by the main body table field and the action types.
Preferably, the audit plan table further comprises a screening time, and the query result of the screening time is obtained according to the query field.
Preferably, the data query time includes a query time point and a query period.
Preferably, the invention further comprises a monitoring method of the audit plan table: acquiring data query time and audit planning quantity; acquiring the audit planning quantity in unit time according to the data query time and the audit planning quantity; and generating early warning information according to the quantity of the audit plans in unit time and a preset early warning range, and sending the early warning information to an early warning receiver.
Preferably, the data system comprises at least one database.
The invention also provides a data system monitoring system, which comprises a first construction module, a second construction module, an acquisition module, an inquiry module and an audit module, wherein the first construction module is used for respectively establishing a main body type and an action type according to the main body surface field and the action field of the data system and establishing the master-slave relationship between the main body type and the main body table field; the second construction module is used for constructing a data auditing plan table according to the action types, the subject types and the subject table fields, and the auditing plan table also comprises an auditing algorithm and data query time; the acquisition module is used for acquiring a query field according to the subject type, the subject table field and the action type of the data audit planning table; the query module is used for acquiring a query result of the data system according to the query field at the data query time; the auditing module is used for acquiring auditing results according to the auditing algorithm and the query results.
Preferably, the data system monitoring system further comprises an early warning module, wherein the early warning module is used for generating early warning information according to the audit result and the early warning range, and generating warning information according to the early warning information and the warning information template.
Compared with the prior art, the invention has the beneficial effects that:
1. the method has the advantages that the main body table fields are summarized and divided through the main body categories, so that the main body table fields can be selected quickly, and system maintenance personnel are prevented from missing part of the main body table fields to cause data errors or recalculation;
2. the data of the main body on different action links in the production flow can be monitored or screened by summarizing the action fields, and the monitoring fineness is improved.
Drawings
FIG. 1 is a flow chart of a data system monitoring method of the present invention;
FIG. 2 is a flow chart of a method for monitoring an audit plan table;
FIG. 3 is a logical block diagram of a data system monitoring system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
a data system monitoring method, as shown in fig. 1, the method comprising:
step 101: and respectively establishing a body type and an action type according to the body table field and the action field of the data system. The data system has various main body table fields, which are divided into several main body categories such as interface categories, storage categories and host categories according to the contents of the main body table fields, and action categories are divided according to action fields, wherein the action categories reflect actions of the main body in the production process link, such as acquisition, warehousing or inspection of the interface categories.
Step 102: and establishing a master-slave relationship between the main body category and the main body table field. And dividing the main body table field under the corresponding main body category, and establishing a form of a secondary catalogue, such as establishing a user income interface under the interface category.
Step 103: and constructing a data auditing planning table according to the action types, the subject types and the subject table fields, wherein the auditing planning table also comprises an auditing algorithm and data query time. The subject type, the subject table field, the action type, the auditing algorithm and the data query time are stored in the auditing planning table, so that unified planning and maintenance are facilitated.
Step 104: and acquiring a query field according to the subject type, the subject table field and the action type of the data audit planning table. And the main body table field and the action type group form key elements of the query field, and the query field is constructed according to the selected main body type, main body table field and action type.
Step 105: and acquiring a query result of the data system according to the query field in the data query time.
Step 106: and obtaining an auditing result according to the auditing algorithm and the query result.
And acquiring the query result at a set time, auditing the query result according to a set auditing algorithm, and automatically acquiring the query result and the auditing result through an automatic tool.
The method has the advantages that the method can be used for summarizing and dividing the fields of the main body table according to the types of the main bodies, is favorable for quickly selecting the fields of the main body table, prevents system maintenance personnel from missing part of the fields of the main body table to cause data errors or recalculation, and ensures the accuracy and the integrity of the statistical main bodies; the data of the main body on different action links in the production flow can be monitored or screened by summarizing the action fields, and the monitoring fineness is improved.
The invention can also comprise a method for generating early warning information, which comprises the following steps:
step 111: and generating early warning information according to the audit result and the early warning range.
Step 112: and generating warning information according to the warning information and the warning information template. Different warning information templates can be adopted to generate different warning information according to different warning ranges of the audit result.
The invention may also include a method of sending alert information, step 113: the warning information is sent to the warning information receiver through one of the following modes or the combination of the following modes: short message pushing, instant chat tool sending, mail sending, announcement pushing and PC displaying.
The early warning range and the warning information receiver can be stored in the data audit planning table, and unified scheduling and planning are facilitated.
The audit planning table can comprise a plurality of action types, and a query field is constructed through a main body table field and the action types. A plurality of action categories of the subject are monitored.
The audit planning table can also comprise screening time, and the query result in the screening time is obtained according to the query field.
The data query time may include a query time point and a query period.
Example 1
And counting users who do not send short messages, use flow and have calls within three months, and sending the short messages to remind a customer manager. Type of subject: a service index; a main body table field: a mobile user; action type: sending short messages, using flow or talking; screening time: within three months; an auditing algorithm comprises the following steps: the mobile user sending short message, the sum of the traffic and the call volume is equal to zero; inquiring time: 09:00:00, query period: no. 5 per month; warning information receiver: a customer manager; the warning information mode is as follows: and (5) short messages.
That is, No. 5 a month obtains the short credit, the flow and the call volume of the mobile user in three months, audits the short credit, the flow and the call volume, and obtains the audit result: and the mobile user sends the audit result to the customer manager in a short message mode.
Example 2
And counting the number of the broadband centralized tear downs. Type of subject: a service index; a main body table field, a broadband centralized dismantling number; screening time: the previous day; an auditing algorithm comprises the following steps: the number of the broadband centralized dismantling machines is more than 80; inquiring time: 09:00: 00; and (3) query period: every day; warning information receiver: a tear-down worker and a broadband principal; the warning information mode is as follows: and (5) short messages.
Example 3
And monitoring the integrity of the examination data maintenance. Type of subject: storing the process; a main body table field for maintaining data; the action type is as follows: executing; screening time: the previous month; an auditing algorithm comprises the following steps: the execution number of the data maintenance is less than 80% of the average value of the execution numbers of the data maintenance in the past six months; inquiring time: 09:00: 00; and (3) query period: no. 20 per month; warning information receiver: a data maintenance person; the warning information mode is as follows: and E, mail.
Namely, when the execution quantity of the previous month data maintenance is less than 80% of the average value of the data maintenance quantity of the past six months, an alarm mail is sent to the data maintenance personnel. In the specific implementation, two query fields are constructed, the execution number of data maintenance in the last month and the execution number of data maintenance in the last six months are respectively obtained, and the two query results are audited.
Example 4
Revenue situations are monitored. Type of subject: accounting; a host table field, income; the action type is as follows: obtaining or warehousing; screening time: the previous month; an auditing algorithm comprises the following steps: the sum of the income of the previous month is greater than a preset value; inquiring time: 11:00: 00; and (3) query period: no. 7 per month; warning information receiver: finance; the warning information mode is as follows: and E, mail.
The invention may also include a monitoring method of the audit plan table, as shown in fig. 2:
step 201: and acquiring data query time and audit planning quantity.
Step 202: and acquiring the audit planning quantity in unit time according to the data query time and the audit planning quantity. In the specific embodiment, the number of audit plans in each half hour is obtained by taking the half hour as a calculation standard.
Step 203: and generating early warning information according to the number of the audit plans in unit time and a preset early warning range.
Step 204: and sending the early warning information to an early warning receiver.
In order to avoid excessive audit planning data in the same time period, the audit planning quantity in unit time is audited, the peak staggering arrangement audit planning is facilitated, and congestion of a data system and the audit planning is prevented.
The data system can comprise one database or a plurality of databases, and the query results are obtained in different databases according to the query fields, and the query results returned by the different databases are audited.
The invention also provides a data system monitoring system, as shown in fig. 3, which comprises a first construction module 1, a second construction module 2, an acquisition module 3, an inquiry module 4 and an audit module 5, wherein the first construction module 1 is used for respectively establishing a subject type and an action type according to a subject table field and an action field of the data system, and is used for establishing a master-slave relationship between the subject type and the subject table field; the second construction module 2 is used for constructing a data auditing plan table according to the action types, the subject types and the subject table fields, and the auditing plan table also comprises an auditing algorithm and data query time; the acquisition module 3 is used for acquiring query fields according to the subject types of the data audit planning table and the subject surface fields and action types thereof; the query module 4 is used for acquiring a query result of the data system according to the query field at the data query time; the auditing module 5 is used for obtaining the auditing result according to the auditing algorithm and the query result.
The data system monitoring system provided by the invention can also comprise an early warning module 6, wherein the early warning module 6 is used for generating early warning information according to the audit result and the early warning range and generating warning information according to the early warning information and the warning information template.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method for monitoring a data system, the method comprising:
respectively establishing a body type and an action type according to a body table field and an action field of the data system;
establishing a master-slave relationship between the main body category and the main body table field;
constructing a data audit planning table according to the action types, the subject types and the subject table fields, wherein the audit planning table also comprises an audit algorithm and data query time;
acquiring a query field according to the subject type, the subject table field and the action type of the data audit planning table;
acquiring a query result of a data system according to the query field within the data query time;
and obtaining an auditing result according to the auditing algorithm and the query result.
2. The data system monitoring method of claim 1, further comprising a method of generating early warning information:
generating early warning information according to the audit result and the early warning range;
and generating warning information according to the warning information and the warning information template.
3. The data system monitoring method of claim 2, further comprising a method of sending an alert message to an alert message recipient by one or a combination of: short message pushing, instant chat tool sending, mail sending, announcement pushing and PC displaying.
4. The data system monitoring method of claim 1, wherein the audit plan table comprises a plurality of action categories, and the query field is constructed by a body table field and the plurality of action categories.
5. The method as claimed in claim 1, wherein the audit plan table further includes a screening time, and the query result in the screening time is obtained according to the query field.
6. The data system monitoring method of claim 1, wherein the data query time comprises a query time point and a query period.
7. The data system monitoring method according to any one of claims 1 to 6, further comprising a monitoring method of an audit plan table:
acquiring data query time and audit planning quantity;
acquiring the audit planning quantity in unit time according to the data query time and the audit planning quantity;
generating early warning information according to the quantity of the audit planning in unit time and a preset early warning range,
and sending the early warning information to an early warning receiver.
8. A data system monitoring method according to claim 1, wherein the data system comprises at least one database.
9. A data system monitoring system is characterized by comprising a first construction module, a second construction module, an acquisition module, a query module and an audit module,
the first building module is used for respectively building a main body type and an action type according to a main body table field and an action field of the data system and building a master-slave relationship between the main body type and the main body table field;
the second construction module is used for constructing a data audit plan table according to the action types, the subject types and the subject table fields, and the audit plan table also comprises an audit algorithm and data query time;
the acquisition module is used for acquiring a query field according to the subject type of the data audit planning table, the subject table field and the action type;
the query module is used for acquiring a query result of the data system according to the query field at the data query time;
the auditing module is used for acquiring auditing results according to the auditing algorithm and the query results.
10. The data system monitoring system of claim 9, further comprising an early warning module,
the early warning module is used for generating early warning information according to the audit result and the early warning range and generating warning information according to the early warning information and the warning information template.
CN202010567262.1A 2020-06-19 2020-06-19 Data system monitoring method and system Active CN111767283B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010567262.1A CN111767283B (en) 2020-06-19 2020-06-19 Data system monitoring method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010567262.1A CN111767283B (en) 2020-06-19 2020-06-19 Data system monitoring method and system

Publications (2)

Publication Number Publication Date
CN111767283A true CN111767283A (en) 2020-10-13
CN111767283B CN111767283B (en) 2023-08-18

Family

ID=72721394

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010567262.1A Active CN111767283B (en) 2020-06-19 2020-06-19 Data system monitoring method and system

Country Status (1)

Country Link
CN (1) CN111767283B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070168335A1 (en) * 2006-01-17 2007-07-19 Moore Dennis B Deep enterprise search
CN101415168A (en) * 2007-10-17 2009-04-22 华为技术有限公司 Method and apparatus for checking charging result and configuration data
CN101902532A (en) * 2009-05-27 2010-12-01 北京汉铭通信有限公司 Data auditing method and system of telecommunication services
EP2804121A2 (en) * 2013-05-17 2014-11-19 Hitachi Ltd. Analysis system and health business support method
CN104424307A (en) * 2013-09-04 2015-03-18 腾讯科技(深圳)有限公司 Intelligent terminal application classifying method, system and intelligent terminal,
CN105468352A (en) * 2014-09-12 2016-04-06 中兴通讯股份有限公司 Query interface generation method and apparatus
CN105956103A (en) * 2016-05-04 2016-09-21 北京思特奇信息技术股份有限公司 Automatic amount calculation method and system
CN109947746A (en) * 2017-10-26 2019-06-28 亿阳信通股份有限公司 A kind of quality of data management-control method and system based on ETL process
CN110311850A (en) * 2019-07-04 2019-10-08 北京天融信网络安全技术有限公司 A kind of network-based data processing method and electronic equipment
CN111159171A (en) * 2019-12-31 2020-05-15 中国铁塔股份有限公司 Data auditing method and system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070168335A1 (en) * 2006-01-17 2007-07-19 Moore Dennis B Deep enterprise search
CN101415168A (en) * 2007-10-17 2009-04-22 华为技术有限公司 Method and apparatus for checking charging result and configuration data
CN101902532A (en) * 2009-05-27 2010-12-01 北京汉铭通信有限公司 Data auditing method and system of telecommunication services
EP2804121A2 (en) * 2013-05-17 2014-11-19 Hitachi Ltd. Analysis system and health business support method
CN104424307A (en) * 2013-09-04 2015-03-18 腾讯科技(深圳)有限公司 Intelligent terminal application classifying method, system and intelligent terminal,
CN105468352A (en) * 2014-09-12 2016-04-06 中兴通讯股份有限公司 Query interface generation method and apparatus
CN105956103A (en) * 2016-05-04 2016-09-21 北京思特奇信息技术股份有限公司 Automatic amount calculation method and system
CN109947746A (en) * 2017-10-26 2019-06-28 亿阳信通股份有限公司 A kind of quality of data management-control method and system based on ETL process
CN110311850A (en) * 2019-07-04 2019-10-08 北京天融信网络安全技术有限公司 A kind of network-based data processing method and electronic equipment
CN111159171A (en) * 2019-12-31 2020-05-15 中国铁塔股份有限公司 Data auditing method and system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LIU JING 等: "Research and design of security audit system for compliance", 2012 INTERNATIONAL SYMPOSIUM ON INFORMATION TECHNOLOGY IN MEDICINE AND EDUCATION, pages 905 - 909 *
曹德发: "电力监控信息智能提取与分类处理的系统及方法", 电工技术, no. 03, pages 137 - 139 *
李斌 等: "基于溯源数据的业务流程合规性检测", 清华大学学报(自然科学版), vol. 53, no. 12, pages 1768 - 1776 *
谷红勋 等: "基于大数据的移动用户行为分析系统与应用案例", 电信科学, vol. 32, no. 03, pages 139 - 146 *

Also Published As

Publication number Publication date
CN111767283B (en) 2023-08-18

Similar Documents

Publication Publication Date Title
CN105868373B (en) Method and device for processing key data of power business information system
US6687560B2 (en) Processing performance data describing a relationship between a provider and a client
CN100589503C (en) Method for implementing configurational performance measurement in communication system
CN109254901B (en) A kind of Monitoring Indexes method and system
EP1695192A2 (en) Automatic monitoring and statistical analysis of dynamic process metrics to expose meaningful changes
CN109063066A (en) Data query method and device, data management system
CN115080546B (en) Enterprise data diagnosis system based on big data
CN114757639A (en) Data processing method, device, equipment and storage medium
CN113240309A (en) Enterprise dynamic monitoring method and device
CN111767283A (en) Data system monitoring method and system
CN110930121A (en) Industry integration management system based on wisdom travel
CN115423361A (en) Data processing method and device for risk view, storage medium and equipment
CN113411253B (en) Relation topology analysis method and device based on mail, terminal equipment and medium
CN110704454A (en) Report data acquisition system and method
CN115293682A (en) Abnormal logistics order monitoring method and related device
CN112396349A (en) Data quality monitoring method based on business entity
CN114240360A (en) Multi-measurement-in-one geographic information management system
CN115048379A (en) Statistical method, terminal device and computer-readable storage medium
CN113706092A (en) Engineering project supervision method and system
CN111210115A (en) Safety index early warning analysis method for specific area
US20030065553A1 (en) Reporting performance data describing a relationship between a provider and a client
CN116961241B (en) Unified application monitoring platform based on power grid business
CN115082046B (en) Block chain-based enterprise table tracing management system
CN115017875B (en) Enterprise information processing method, device, system, equipment and medium
CN113822679A (en) Intelligent anti-fraud loan management system and method with automatic decision making function

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant