CN105095056A - Method for monitoring data in data warehouse - Google Patents
Method for monitoring data in data warehouse Download PDFInfo
- Publication number
- CN105095056A CN105095056A CN201510502221.3A CN201510502221A CN105095056A CN 105095056 A CN105095056 A CN 105095056A CN 201510502221 A CN201510502221 A CN 201510502221A CN 105095056 A CN105095056 A CN 105095056A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- configuration
- content
- alarm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A method for monitoring data in a data warehouse comprises the steps of determining the monitor table and content of the data warehouse to be monitored and conducting parameter configuration. Monitor table parameter configuration comprises data size monitoring configuration, database data source table structure monitoring configuration and data source table field value monitoring configuration. Data size monitoring configuration is mainly about monitoring whether the data size listed by a monitoring table and the data size newly added on the day are abnormal. Database data source table structure monitoring configuration is about monitoring whether the table structure of a data source changes, including new fields, deleted fields, modified field type and modified field length, some of the changes may cause data synchronization program error, some of the changes may indicate that services are changed, and the changes require attention of data analysis personnel. Data source table field value monitoring configuration is about monitoring the values of key fields of a data source table, usually the value of a dimension table.
Description
Technical field
The present invention relates to a kind of to occurring the abnormal method monitored in the business datum source processing procedure of database.
Background technology
A data warehouse correspond to multiple business datum source.Along with business deepens constantly, the data of Water demand are also on the increase, and accordingly, data warehouse task is various.There are every day a large amount of newly-increased data to be stored in data warehouse, if every day occurs abnormal in task processes, the quality of data can be affected, and may the data in the following time backward be had an impact.Therefore the exception of Timeliness coverage data in processing procedure, processes.To data warehouse is very important.CN103605722A database monitoring method, according to the information of each database, obtains the monitoring configuration file corresponding with the information of each database; According to the monitoring configuration file corresponding with the information of each database obtained, utilize the watchdog routine corresponding with the information of each database write in advance, each self-corresponding database is monitored.
CN103746837A database monitoring system, comprises database monitoring device and monitoring cluster, and wherein, described database monitoring device, for receiving the triggering command from monitoring cluster; Send the first instruction to load balancing apparatus, whether whether whether described first instruction be used to indicate described load balancing apparatus and detect cluster and can exist with the software agent of each node referred in described cluster with, described cluster; Receive the first response message from described load balancing apparatus; Emphasize to monitor load balancing.
But prior art does not have and to change business datum source and will the early warning of output exception and the comprehensive monitoring to database, this is to ensureing that the stable of data warehouse is accurately highly effective.For reaching existing Mission Monitor, the prevention data disappearance more effectively utilized is omitted, the object of monitoring business DSN change simultaneously, now proposes a kind of data monitoring technology of data warehouse, carries out omnibearing monitoring to the quality of data that the every day of data warehouse processes.
Summary of the invention
The present invention proposes the method for a kind of data warehouse data monitoring, it can realize monitoring the data cases of the table of data warehouse renewal every day by performing regularly monitor task and relevant a series of configurations, can also realize because business datum source changes will the early warning of output exception, can Timeliness coverage in the various problems of program emerged in operation.To ensureing that the stable and accurate operation of data warehouse can play considerable effect.
Technical scheme of the present invention is, a kind of method of data warehouse data monitoring, comprising:
1) determine the monitoring form that needs to carry out the data warehouse monitored and content, carry out parameter configuration, the content of monitoring form parameter configuration divides three kinds;
1-1) data volume monitoring configuration
Whether the content of the main monitoring of data volume monitoring is that monitoring form lists the data volume newly-increased with the same day abnormal, judges whether abnormal standard be first data volume is 0, if being 0 is exactly so extremely; Next be with yesterday and last week historical data on the same day contrast, differ by more than certain proportion or threshold value if compared with historical data, such as 50%, be then considered as exception;
The content of monitoring configuration is needed to comprise the contents such as data (project) title, table name, time field, time field type, statistical item, statistical condition, monitoring period; The content that watchdog routine can monitor configuration is as required spliced into corresponding SQL statement, treats scheduler program to perform;
1-2) the data source list structure monitoring configuration of database
The content that data source list structure is mainly monitored is whether the list structure in monitor data source changes, and comprises new field, deletes field, revises field type, revises field length etc.; These change some and data syn-chronization program can be caused to report an error, and some may represent business and change and all need data analyst to pay close attention to.
The content of monitoring configuration is needed mainly to comprise the content such as type (such as Oracle or Mysql), DataBase combining mode, database table name, corresponding data library name of database; Watchdog routine can check list structure information according in these contents to associated databases; And compare with the corresponding service database information preserved before, if find that there is variation, system sends out alarm to designated person;
1-3) the literary name section value monitoring configuration of data source
The content of the main monitoring of data source literary name section value monitoring is the value of monitor data source table emphasis field, the normally value of dimension table, and change appears in these values, and change has appearred in the business of representing; Also be that Water demand personnel carry out paying close attention to.
The content of configuration is needed mainly to comprise the contents such as database table name, field name, data type; Watchdog routine can according to checking relevant field value in these contents to data warehouse; And compare to the corresponding field value information of preserving before, if find that there is variation, system sends out alarm to designated person.
2) deployment server task, timing performs watchdog routine.
Because data warehouse data is all upgrading every day, corresponding watchdog routine also will perform in every day.Will according to every day data warehouse working time roughly determine execution time of monitoring every day.
And because each data service has repugnancy update time, the promptness monitored needs better more in time simultaneously, all like this watchdog routines also can not be unified in a time execution.Take timing monitoring when realizing, if monitor task does not all complete, watchdog routine can continue the mode performed dormancy a period of time.Ensure that monitoring promptness and integrality.
3) monitored results is formed report and send to designated person;
Monitoring performs last, and need the abnormal conditions checked out to inform responsible person concerned, alarm mode comprises form, mail, note etc., selects alarm mode according to the rank of alarm.High severity alarm must notify maintainer in time by short message mode, the employing lettergram mode alarm that the alarm of common grade need not solve at once.
Also comprise alarm personnel depaly in system monitoring configuration in addition, the content of configuration comprises the contents such as personnel's name, mobile phone, mailbox.Different tables is divided according to project, the responsible person that each project configuration is corresponding.The exception found in system cloud gray model can issue related personnel according to the configuration of correspondence.
Also have a set of corresponding reporting system simultaneously, facilitate monitor staff to check history alarm logging.
Beneficial effect of the present invention:
1, can know that whether the implementation status grasping data warehouse scheduler routine every day is normal;
2, abnormal conditions in data processing, the method process combined by system early warning and artificial treatment, what ensure in data handling procedure is reliable and stable;
3, comprehensive system monitoring, improves the availability of system, also improves the efficiency of system maintenance;
4, to database overall monitor, can have been unified by hardware and software platform multiple monitoring demand, avoid overlapping development, reduce costs.
Accompanying drawing explanation
The method processing flow chart of a kind of data warehouse data monitoring of Fig. 1 the present embodiment.
Embodiment
As Fig. 1, the method treatment scheme of a kind of data warehouse data monitoring of the present embodiment, comprising:
Step 11, determine the information such as data warehouse table to be monitored, data source table and data sheet field.
Wherein, the cardinal rule of amount of configuration data monitoring form is, the table of data warehouse monitoring form to be all important table.Should be able to cover data warehouse analyze more than 80% index.Data volume every day of monitoring form is not excessive simultaneously, too not detailed, and should select to monitor the table after adding up.The detailed access log table in such as website, have millions of data every day, and monitored by the table of access log table after statistics that this data warehouse is monitored, derivative table after statistics may only have hundreds of bar, this derivative table of such monitoring can ensure the efficiency of watchdog routine, and can meet the demand of monitoring.
When disposition data source is monitored, each table in synchrodata source is configured, guarantee that the impact by the variation in factor data source produces controls to minimum.
When configuration data table field, to determine which field needs monitoring according to business.It is general that what need monitoring is the dimension field of main business table.The member's type such as registered, member's state these.These are changed and all may represent service discovering adjustment.Very likely can produce certain impact to data statistics.
Step 12, analyze the information of table to be monitored, be kept in monitoring allocation list according to the requirement of allocation list.Because system comprises three monitoring functions, therefore need configuration three allocation lists.Introduce in detailed deploy content step below.
Step 13, according to configuration information, monitoring form data volume and trend whether normal
The every day such as will monitoring registered members's table increases data volume newly, needs the information of configuration as follows
1) project name: data warehouse underlying table (monitoring of data warehouse data amount)
2) table name: USERS
3) time field: TRUNC (ADD_TIME)
4) time field type: DATE.In native system, date type divides 3 kinds: DATE, NUMBER (as 20150101), ALL (representing not fixed time and All Time scope).
5) statistical: COUNT (*).Representing the data number in time range, also can be SUM (USER_COUNT), i.e. the statistics summation of a certain row.
6) statistical condition: nothing.Can add restriction according to project demand, such as require that supervisory user state (STATUS) is the member by auditing (value is 1), configuration information is STATUS=1 here.
7) period is monitored: 6.Here 6 representatives hour, representing monitoring after 6, is generally obtain according to historical experience value.If system time is less than 6 points, the monitoring of this table can not be performed.Monitor data to 6 and have exception, be also likely because data also do not perform, next monitoring period also can proceed monitoring.Until data are normal or maintainer's manual setting abnormality.
By above-mentioned configuration, system can go out query statement (being performed by monitoring degree) by automatic Mosaic when monitoring, as follows:
SELECTCOUNT(*)FROMUSERSWHERETRUNC(ADD_TIME)＝DATE’2015-01-01’
Watchdog routine can perform this query statement in a database, and execution result is saved in result table, and for alarm program, next step is analyzed.If find in analysis that data are 0, so can be judged as exception.And note and the corresponding project leader of mail notification.And if if contrast data yesterday and last week simultaneously data data fluctuations all more than 50%, so also may be problematic, also can mail to project leader.Project leader judges whether exception again in conjunction with business.
Step 14, according to configuration information, whether monitor data source list structure changes.
The table needing the business library that related to by data warehouse synchronous before monitoring is all kept in allocation list.The information of configuration is needed to comprise: the information such as corresponding data warehouse table name, table owning user name, Service Database connected mode, Service Database type of Service Database table name, former or history.
By above-mentioned configuration, system can remove when monitoring the literary name segment information extracting data source in specified database.If this system view of dba_tab_columns can be used to obtain literary name segment information at oracle database, if information_schema.columns can be used in Mysql database to obtain literary name segment information, other databases are also the same.The information obtained and the historical information of preserving before are contrasted, just can recognize which Service Database there occurs and change.If likely had an impact to synchrodata, the structure of the correspondence table of data warehouse will be adjusted in time.
Step 15, according to configuration information, monitoring specifies the value of literary name section whether to have increase.
The information of appointment table is needed all to be kept in allocation list before monitoring.The information of configuration is needed to comprise: the information such as table name, field name, field type, time field, time type.
By above-mentioned configuration, system goes the corresponding field of data warehouse in the newly-increased unduplicated information of extracting data and value when monitoring, preserve, and contrast with historical data before, if have increase to represent business may change, need to carry out alarm.
, there is warning information to designated person in step 16, generation report.
Be found exception and non-conformance description can output in monitoring form by watchdog routine in the process of implementation.Rank according to configuration information and alarm selects alarm mode.High severity alarm needs to notify maintainer in time by short message mode, the employing lettergram mode alarm that the alarm of common grade need not solve at once.
In addition also have a set of corresponding reporting system simultaneously, facilitate monitor staff to check history alarm logging.
The foregoing is only the embodiment of patent of the present invention, not in order to limit patent of the present invention, any amendment made within all spirit in patent of the present invention and principle, equivalently to replace and improvement etc., all with within the protection domain being included in patent of the present invention.
Claims (6)
1. a method for data warehouse data monitoring, is characterized in that comprising the steps:
1) determine the monitoring form that needs to carry out the data warehouse monitored and content, carry out parameter configuration, the content of monitoring form parameter configuration divides three kinds;
1-1) data volume monitoring configuration
Whether the content of data volume monitoring is that monitoring form lists the data volume newly-increased with the same day abnormal, judges whether abnormal standard be first data volume is 0, if being 0 is exactly so extremely; Next be with yesterday and last week historical data on the same day contrast, differ by more than certain proportion or threshold value if compared with historical data, be then considered as exception;
The content of monitoring configuration is needed to comprise data
The content such as title, table name, time field, time field type, statistical item, statistical condition, monitoring period; The content that watchdog routine can monitor configuration is as required spliced into corresponding SQL statement, treats scheduler program to perform;
1-2) the data source list structure monitoring configuration of database
The content that data source list structure is monitored is whether the list structure in monitor data source changes, and comprises new field, deletes field, revises field type, revises field length etc.; These change some and data syn-chronization program can be caused to report an error, and some may represent business and change and all need data analyst to pay close attention to;
The content of monitoring configuration is needed to comprise the type of database, the contents such as DataBase combining mode, database table name, corresponding data library name; Watchdog routine can check list structure information according in these contents to associated databases; And compare with the corresponding service database information preserved before, if find that there is variation, system sends out alarm to designated person;
1-3) the literary name section value monitoring configuration of data source
The content of the main monitoring of data source literary name section value monitoring is the value of monitor data source table emphasis field, the normally value of dimension table, and change appears in these values, and change has appearred in the business of representing; Also be that Water demand personnel carry out paying close attention to;
The content of configuration is needed to comprise database table name, field name, data type content; Watchdog routine can according to checking relevant field value in these contents to data warehouse; And compare to the corresponding field value information of preserving before, if find that there is variation, system sends out alarm to designated person;
2) deployment server task, timing performs watchdog routine;
Because data warehouse data is all upgrading every day, corresponding watchdog routine also will perform in every day; Will according to every day data warehouse working time roughly determine execution time of monitoring every day;
Take timing monitoring when realizing, if monitor task does not all complete, watchdog routine can continue the mode performed dormancy a period of time; Ensure that monitoring promptness and integrality;
3) monitored results is formed report and send to designated person; Monitoring performs last, and need the abnormal conditions checked out to inform responsible person concerned, alarm mode comprises form, mail, note etc.
2. the method for data warehouse data monitoring according to claim 1, it is characterized in that also comprising alarm personnel depaly in monitoring configuration, the content of configuration comprises the contents such as personnel's name, mobile phone, mailbox; Different tables is divided according to project, the responsible person that each project configuration is corresponding; The exception found in system cloud gray model can issue related personnel according to the configuration of correspondence.
3. the method for data warehouse data monitoring according to claim 1 and 2, is characterized in that in step 3), selects alarm mode according to the rank of alarm; High severity alarm must notify maintainer in time by short message mode, the employing lettergram mode alarm that the alarm of common grade need not solve at once.
4. the method for data warehouse data monitoring according to claim 3, is characterized in that the reporting system being simultaneously provided with a set of warning, facilitates monitor staff to check history alarm logging.
5. the method for data warehouse data monitoring according to claim 1 and 2, is characterized in that data warehouse monitoring form is that the table of access log table after statistics is monitored.
6. the method for data warehouse data according to claim 1 and 2 monitoring, is characterized in that step 1-1) in compare with historical data and differ by more than certain proportion or threshold value, get 50%.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510502221.3A CN105095056B (en) | 2015-08-14 | 2015-08-14 | A kind of method of data warehouse data monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510502221.3A CN105095056B (en) | 2015-08-14 | 2015-08-14 | A kind of method of data warehouse data monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105095056A true CN105095056A (en) | 2015-11-25 |
| CN105095056B CN105095056B (en) | 2018-01-12 |
Family
ID=54575552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510502221.3A Active CN105095056B (en) | 2015-08-14 | 2015-08-14 | A kind of method of data warehouse data monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105095056B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677567A (en) * | 2016-01-10 | 2016-06-15 | 上海与德通讯技术有限公司 | Automation testing method and system |
| WO2017107853A1 (en) * | 2015-12-25 | 2017-06-29 | 阿里巴巴集团控股有限公司 | Data monitoring management method, and data monitoring method and system |
| CN107193821A (en) * | 2016-03-14 | 2017-09-22 | 阿里巴巴集团控股有限公司 | Monitoring method and system |
| CN107220301A (en) * | 2017-05-10 | 2017-09-29 | 北京小度信息科技有限公司 | The data monitoring method and device of a kind of configurableization |
| CN108090138A (en) * | 2017-11-29 | 2018-05-29 | 链家网（北京）科技有限公司 | The monitoring method and system of a kind of data warehouse |
| CN108304413A (en) * | 2017-01-13 | 2018-07-20 | 北京京东尚科信息技术有限公司 | distributed data warehouse monitoring method, device, electronic equipment and storage medium |
| CN108415814A (en) * | 2018-01-11 | 2018-08-17 | 平安科技（深圳）有限公司 | Automatically record field changing method, application server and computer readable storage medium |
| CN108874644A (en) * | 2018-06-06 | 2018-11-23 | 平安科技（深圳）有限公司 | Data monitoring method, device, computer equipment and storage medium |
| CN108959564A (en) * | 2018-07-04 | 2018-12-07 | 玖富金科控股集团有限责任公司 | Data warehouse metadata management method, readable storage medium storing program for executing and computer equipment |
| CN109218051A (en) * | 2017-07-03 | 2019-01-15 | 中国移动通信有限公司研究院 | Manage method, management server, internet-of-things terminal and the system of internet-of-things terminal |
| CN109800229A (en) * | 2018-12-29 | 2019-05-24 | 深圳云天励飞技术有限公司 | Data access method and relevant device |
| CN109857720A (en) * | 2018-12-20 | 2019-06-07 | 中国平安人寿保险股份有限公司 | Database table monitoring method, device, computer installation and readable storage medium storing program for executing |
| CN109857619A (en) * | 2019-02-03 | 2019-06-07 | 北京字节跳动网络技术有限公司 | State subscription method, apparatus, storage medium and the electronic equipment of data warehouse table |
| CN109918271A (en) * | 2019-03-28 | 2019-06-21 | 上海中通吉网络技术有限公司 | Data quality monitoring method, system and storage medium |
| CN110134680A (en) * | 2019-04-04 | 2019-08-16 | 平安科技（深圳）有限公司 | Space monitoring method, device, computer equipment and storage medium |
| CN110399376A (en) * | 2019-08-08 | 2019-11-01 | 北京明略软件系统有限公司 | The method and device of the former data variation of automatic identification table |
| CN110647452A (en) * | 2019-08-30 | 2020-01-03 | 深圳壹账通智能科技有限公司 | Test method, test device, computer equipment and storage medium |
| CN110837458A (en) * | 2019-11-08 | 2020-02-25 | 深圳市彬讯科技有限公司 | Data balance verification method, equipment and storage medium |
| CN110851325A (en) * | 2019-11-08 | 2020-02-28 | 深圳市彬讯科技有限公司 | Method, device and equipment for monitoring data warehouse based on Hive table |
| CN113553372A (en) * | 2020-04-26 | 2021-10-26 | 北京沃东天骏信息技术有限公司 | Writing method and device for database, computing equipment and medium |
| CN113836160A (en) * | 2021-09-28 | 2021-12-24 | 上海市大数据股份有限公司 | Data flow state monitoring and warning system based on master-slave synchronization |
| CN115129687A (en) * | 2022-03-16 | 2022-09-30 | 希望知舟技术(深圳)有限公司 | Abnormal condition database management-based method, related device, medium and program |
| CN116149969A (en) * | 2023-04-04 | 2023-05-23 | 湖南中青能科技有限公司 | Database model matching anomaly monitoring and processing method |
| CN116361391A (en) * | 2023-03-30 | 2023-06-30 | 中电云数智科技有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
| CN116820892A (en) * | 2023-07-14 | 2023-09-29 | 佛山众陶联供应链服务有限公司 | Data processing monitoring system for a plurality of bins |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101989278A (en) * | 2009-08-06 | 2011-03-23 | 上海杉达学院 | Database inquiry device with automatic splicing function |
| CN102508833A (en) * | 2011-09-22 | 2012-06-20 | 用友软件股份有限公司 | Data monitoring device and data monitoring method |
| CN102855319A (en) * | 2012-09-05 | 2013-01-02 | 国家电网公司 | ORACLE database operation monitoring system |
| CN103036736A (en) * | 2012-11-30 | 2013-04-10 | 航天恒星科技有限公司 | Configuration equipment monitoring system and monitoring method based on data sources |
| CN104636450A (en) * | 2015-01-26 | 2015-05-20 | 上海新炬网络信息技术有限公司 | Database table space monitoring method |
| CN104636483A (en) * | 2015-02-16 | 2015-05-20 | 广东省公安厅 | Data monitoring method |
-
2015
- 2015-08-14 CN CN201510502221.3A patent/CN105095056B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101989278A (en) * | 2009-08-06 | 2011-03-23 | 上海杉达学院 | Database inquiry device with automatic splicing function |
| CN102508833A (en) * | 2011-09-22 | 2012-06-20 | 用友软件股份有限公司 | Data monitoring device and data monitoring method |
| CN102855319A (en) * | 2012-09-05 | 2013-01-02 | 国家电网公司 | ORACLE database operation monitoring system |
| CN103036736A (en) * | 2012-11-30 | 2013-04-10 | 航天恒星科技有限公司 | Configuration equipment monitoring system and monitoring method based on data sources |
| CN104636450A (en) * | 2015-01-26 | 2015-05-20 | 上海新炬网络信息技术有限公司 | Database table space monitoring method |
| CN104636483A (en) * | 2015-02-16 | 2015-05-20 | 广东省公安厅 | Data monitoring method |
Non-Patent Citations (1)
| Title |
|---|
| 赵清: "数据仓库增量维护的体系结构研究", 《中国优秀硕士论文全文数据库 信息科技辑》 * |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017107853A1 (en) * | 2015-12-25 | 2017-06-29 | 阿里巴巴集团控股有限公司 | Data monitoring management method, and data monitoring method and system |
| CN106919602A (en) * | 2015-12-25 | 2017-07-04 | 阿里巴巴集团控股有限公司 | A kind of data monitoring management method, data monitoring method and system |
| CN105677567A (en) * | 2016-01-10 | 2016-06-15 | 上海与德通讯技术有限公司 | Automation testing method and system |
| CN107193821A (en) * | 2016-03-14 | 2017-09-22 | 阿里巴巴集团控股有限公司 | Monitoring method and system |
| CN108304413A (en) * | 2017-01-13 | 2018-07-20 | 北京京东尚科信息技术有限公司 | distributed data warehouse monitoring method, device, electronic equipment and storage medium |
| CN107220301A (en) * | 2017-05-10 | 2017-09-29 | 北京小度信息科技有限公司 | The data monitoring method and device of a kind of configurableization |
| CN109218051A (en) * | 2017-07-03 | 2019-01-15 | 中国移动通信有限公司研究院 | Manage method, management server, internet-of-things terminal and the system of internet-of-things terminal |
| CN109218051B (en) * | 2017-07-03 | 2022-04-01 | 中国移动通信有限公司研究院 | Method for managing terminal of internet of things, management server, terminal of internet of things and system |
| CN108090138A (en) * | 2017-11-29 | 2018-05-29 | 链家网（北京）科技有限公司 | The monitoring method and system of a kind of data warehouse |
| CN108415814A (en) * | 2018-01-11 | 2018-08-17 | 平安科技（深圳）有限公司 | Automatically record field changing method, application server and computer readable storage medium |
| CN108415814B (en) * | 2018-01-11 | 2021-02-19 | 平安科技（深圳）有限公司 | Method for automatically recording field change, application server and computer readable storage medium |
| CN108874644A (en) * | 2018-06-06 | 2018-11-23 | 平安科技（深圳）有限公司 | Data monitoring method, device, computer equipment and storage medium |
| CN108959564A (en) * | 2018-07-04 | 2018-12-07 | 玖富金科控股集团有限责任公司 | Data warehouse metadata management method, readable storage medium storing program for executing and computer equipment |
| CN108959564B (en) * | 2018-07-04 | 2020-11-27 | 玖富金科控股集团有限责任公司 | Data warehouse metadata management method, readable storage medium and computer device |
| CN109857720A (en) * | 2018-12-20 | 2019-06-07 | 中国平安人寿保险股份有限公司 | Database table monitoring method, device, computer installation and readable storage medium storing program for executing |
| CN109857720B (en) * | 2018-12-20 | 2024-02-02 | 中国平安人寿保险股份有限公司 | Database table monitoring method, device, computer device and readable storage medium |
| CN109800229A (en) * | 2018-12-29 | 2019-05-24 | 深圳云天励飞技术有限公司 | Data access method and relevant device |
| CN109857619A (en) * | 2019-02-03 | 2019-06-07 | 北京字节跳动网络技术有限公司 | State subscription method, apparatus, storage medium and the electronic equipment of data warehouse table |
| CN109918271A (en) * | 2019-03-28 | 2019-06-21 | 上海中通吉网络技术有限公司 | Data quality monitoring method, system and storage medium |
| CN110134680A (en) * | 2019-04-04 | 2019-08-16 | 平安科技（深圳）有限公司 | Space monitoring method, device, computer equipment and storage medium |
| CN110399376A (en) * | 2019-08-08 | 2019-11-01 | 北京明略软件系统有限公司 | The method and device of the former data variation of automatic identification table |
| CN110647452A (en) * | 2019-08-30 | 2020-01-03 | 深圳壹账通智能科技有限公司 | Test method, test device, computer equipment and storage medium |
| CN110647452B (en) * | 2019-08-30 | 2023-02-07 | 深圳壹账通智能科技有限公司 | Test method, test device, computer equipment and storage medium |
| CN110837458A (en) * | 2019-11-08 | 2020-02-25 | 深圳市彬讯科技有限公司 | Data balance verification method, equipment and storage medium |
| CN110851325A (en) * | 2019-11-08 | 2020-02-28 | 深圳市彬讯科技有限公司 | Method, device and equipment for monitoring data warehouse based on Hive table |
| CN110851325B (en) * | 2019-11-08 | 2024-03-15 | 土巴兔集团股份有限公司 | Method, device and equipment for monitoring data warehouse based on Hive table |
| CN110837458B (en) * | 2019-11-08 | 2024-03-29 | 土巴兔集团股份有限公司 | Method, equipment and storage medium for data balance verification |
| CN113553372A (en) * | 2020-04-26 | 2021-10-26 | 北京沃东天骏信息技术有限公司 | Writing method and device for database, computing equipment and medium |
| CN113836160A (en) * | 2021-09-28 | 2021-12-24 | 上海市大数据股份有限公司 | Data flow state monitoring and warning system based on master-slave synchronization |
| CN113836160B (en) * | 2021-09-28 | 2024-01-23 | 上海市大数据股份有限公司 | Data stream state monitoring alarm system based on master-slave synchronization |
| CN115129687A (en) * | 2022-03-16 | 2022-09-30 | 希望知舟技术(深圳)有限公司 | Abnormal condition database management-based method, related device, medium and program |
| CN116361391A (en) * | 2023-03-30 | 2023-06-30 | 中电云数智科技有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
| CN116361391B (en) * | 2023-03-30 | 2024-05-07 | 中电云计算技术有限公司 | Method and device for detecting and repairing structural abnormality of data synchronization table |
| CN116149969A (en) * | 2023-04-04 | 2023-05-23 | 湖南中青能科技有限公司 | Database model matching anomaly monitoring and processing method |
| CN116149969B (en) * | 2023-04-04 | 2023-06-20 | 湖南中青能科技有限公司 | Database model matching anomaly monitoring and processing method |
| CN116820892A (en) * | 2023-07-14 | 2023-09-29 | 佛山众陶联供应链服务有限公司 | Data processing monitoring system for a plurality of bins |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105095056B (en) | 2018-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105095056A (en) | Method for monitoring data in data warehouse | |
| US10592308B2 (en) | Aggregation based event identification | |
| AU2019201687B2 (en) | Network device vulnerability prediction | |
| CN109257200B (en) | Method and device for monitoring big data platform | |
| CN105580032B (en) | For reducing instable method and system when upgrading software | |
| CN102340415B (en) | Server cluster system and monitoring method thereof | |
| CN101997709B (en) | Root alarm data analysis method and system | |
| US10635557B2 (en) | System and method for automated detection of anomalies in the values of configuration item parameters | |
| CN110232006B (en) | Equipment alarm method and related device | |
| CN105183625A (en) | Log data processing method and apparatus | |
| US20190146774A1 (en) | Deployment routing of clients by analytics | |
| US10769641B2 (en) | Service request management in cloud computing systems | |
| US20210366268A1 (en) | Automatic tuning of incident noise | |
| JP2014102661A (en) | Application determination program, fault detection device, and application determination method | |
| JP2018160186A (en) | Monitoring program, monitoring method, and monitoring device | |
| US20220222266A1 (en) | Monitoring and alerting platform for extract, transform, and load jobs | |
| CN112199394A (en) | Alarm information pushing method and system, intelligent terminal and storage medium | |
| CN104954181A (en) | Method for warning faults of distributed cluster devices | |
| US20210286780A1 (en) | Auto reinforced anomaly detection | |
| CN114443437A (en) | Alarm root cause output method, apparatus, device, medium, and program product | |
| US20170213142A1 (en) | System and method for incident root cause analysis | |
| CN107666399A (en) | A kind of method and apparatus of monitoring data | |
| CN115496470A (en) | Full-link configuration data processing method and device and electronic equipment | |
| CN115529219A (en) | Alarm analysis method and device, computer readable storage medium and electronic equipment | |
| CN114138615A (en) | Service alarm processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |