CN110955684A - Data monitoring method and device - Google Patents
Data monitoring method and device Download PDFInfo
- Publication number
- CN110955684A CN110955684A CN201911190323.0A CN201911190323A CN110955684A CN 110955684 A CN110955684 A CN 110955684A CN 201911190323 A CN201911190323 A CN 201911190323A CN 110955684 A CN110955684 A CN 110955684A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- task
- monitored
- data volume
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000004806 packaging method and process Methods 0.000 claims abstract description 6
- 230000000903 blocking effect Effects 0.000 claims description 10
- 238000005538 encapsulation Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000012806 monitoring device Methods 0.000 claims 1
- 238000001914 filtration Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 240000008042 Zea mays Species 0.000 description 1
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 description 1
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 235000005822 corn Nutrition 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Debugging And Monitoring (AREA)
Abstract
A data monitoring method and device comprises the following steps: collecting data using the ETL; packaging data to be monitored in the data into a monitored object; allocating system resources and continuously monitoring the monitored object; and calculating the monitored object, and storing the monitoring result into a buffer area. According to the method and the device, the ETL is used for collecting the tasks, and the data to be monitored in the data of the tasks are packaged into the monitored object; allocating system resources and continuously monitoring the monitored object; the monitoring object is calculated, and the monitoring result is stored in the cache region, so that the monitoring object is monitored in real time by using fewer resources, the data condition is determined, the data condition is recorded, an operator can know the data condition in real time, and the monitoring object is timely repaired when abnormity occurs in the acquisition.
Description
Technical Field
The present application relates to data management, and in particular, to a method and an apparatus for monitoring data.
Background
With the development of digital environmental protection systems, data centers are more and more important and undertake the tasks of data acquisition, cleaning and sharing. Where the task of acquisition is mainly done using ETL. Although the ETL itself has a corresponding log to provide monitoring capabilities. But the function is relatively simple, and it is difficult to do some more detailed monitoring. The operator often is not able to specifically troubleshoot data problems in the event of anomalies in the customer feedback data, and needs to determine whether the source system is not producing data, the ETL is not functioning properly, or the application system itself is a problem. Such an implementation is very costly and inefficient.
Disclosure of Invention
The application provides a data monitoring method and device.
According to a first aspect of the present application, there is provided a data monitoring method, comprising:
using the ETL to collect data in a task;
packaging data to be monitored in the data in the task as a monitoring object;
allocating system resources and continuously monitoring the monitored object;
and calculating the monitored object, and storing the monitoring result into a buffer area.
Further, the allocating system resources and continuously monitoring the monitored object includes:
defining a thread pool according to configuration information, wherein the configuration information comprises the size of the thread pool and the configuration of a blocking queue;
and the calling task generator acquires the monitoring object and adds the monitoring object into a task pool, takes out the task to be executed from the task pool, and calls a thread pool to acquire a thread to execute the task to be executed.
Further, the calculating the monitoring object and storing the monitoring result into a buffer area includes:
calculating the number of the monitoring objects;
storing the data volume of the monitored object in a continuously updated data volume record table;
and storing the data volume record table in a cache.
Further, the method further comprises:
providing a data volume interface;
providing data in the data volume record table through the data volume interface;
and setting an early warning task for the data in the data volume record table.
Further, the setting of an early warning task for the data in the data volume record table includes:
setting an expected quantity and early warning information aiming at the monitored object;
when the data in the data volume record table is monitored to be not in the expected number, alarming is carried out;
and calling back the expected quantity according to the early warning information.
According to a second aspect of the present application, there is provided a data monitoring apparatus comprising:
the acquisition module is used for acquiring data in the task by using the ETL;
the encapsulation module is used for encapsulating data to be monitored in the data in the task into a monitoring object;
the monitoring module is used for allocating system resources and continuously monitoring the monitored object;
and the processing module is used for calculating the monitored object and storing the monitoring result into the cache region.
Further, the monitoring module includes:
the configuration unit is used for defining a thread pool according to configuration information, and the configuration information comprises the size of the thread pool and the configuration of a blocking queue;
and the execution unit is used for calling the task generator to acquire the monitoring object and add the monitoring object into the task pool, taking out the task to be executed from the task pool, and calling the thread pool to acquire the thread to execute the task to be executed.
Further, the processing module includes:
a calculation unit configured to calculate the number of the monitoring objects;
the storage unit is used for storing the data volume of the monitored object in a continuously updated data volume record table; and storing the data volume record table in a cache.
Further, the apparatus further comprises:
the interface module is used for providing a data volume interface; providing data in the data volume record table through the data volume interface; and setting an early warning task for the data in the data volume record table.
Further, the interface module includes:
the setting unit is used for setting the expected quantity and the early warning information aiming at the monitored object;
the call-back unit is used for giving an alarm when the data in the data volume record table is monitored to be not in the expected number; and calling back the expected quantity according to the early warning information.
According to the method and the device, the ETL is used for collecting the tasks, and the data to be monitored in the data of the tasks are packaged into the monitored object; allocating system resources and continuously monitoring the monitored object; the monitoring object is calculated, and the monitoring result is stored in the cache region, so that the monitoring object is monitored in real time by using fewer resources, the data condition is determined, the data condition is recorded, an operator can know the data condition in real time, and the monitoring object is timely repaired when abnormity occurs in the acquisition.
Drawings
FIG. 1 is a flow chart of a method in one embodiment of the present application;
FIG. 2 is a flow chart of a method in another embodiment according to the first embodiment of the present application;
FIG. 3 is a schematic diagram of program modules of an apparatus according to a second embodiment of the present application;
fig. 4 is a schematic diagram of program modules of an apparatus in an embodiment two of the present application in another implementation manner.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. The present application may be embodied in many different forms and is not limited to the embodiments described in the present embodiment. The following detailed description is provided to facilitate a more thorough understanding of the present disclosure, and the words used to indicate orientation, top, bottom, left, right, etc. are used solely to describe the illustrated structure in connection with the accompanying figures.
One skilled in the relevant art will recognize, however, that one or more of the specific details can be omitted, or other methods, components, or materials can be used. In some instances, some embodiments are not described or not described in detail.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning.
Furthermore, the technical features, aspects or characteristics described herein may be combined in any suitable manner in one or more embodiments. It will be readily appreciated by those of skill in the art that the order of the steps or operations of the methods associated with the embodiments provided herein may be varied. Thus, any sequence in the figures and examples is for illustrative purposes only and does not imply a requirement in a certain order unless explicitly stated to require a certain order.
The first embodiment is as follows:
as shown in fig. 1, an embodiment of the data monitoring method of the present application includes the following steps:
step 102: the ETL is used to collect data in the task.
Step 104: and packaging data to be monitored in the data in the task into a monitoring object.
The monitoring object comprises a database, a database table, a database view, a returned number of sql statements and/or a returned number of java statements, etc.
Step 106: and allocating system resources and continuously monitoring the monitored object.
Step 108: and calculating the monitored object, and storing the monitoring result into a buffer area.
The monitoring result is stored in the buffer area, and the time for storing the historical data can be determined according to the configuration. And the generated monitoring result data is used as basic support data of data early warning and an asset map.
In an embodiment, the data monitoring method of the present application may further include:
step 110: and providing a data volume interface, providing data in the data volume record table through the data volume interface, and setting an early warning task for the data in the data volume record table.
After the data to be monitored is packaged into the monitored object, the monitored object can be monitored, and alarm information is set according to the expected quantity and the filtering rule of the monitored object configuration. And the data can be automatically monitored after the early warning task is started, and the data can be recorded and early warned when the data is found to have problems. The early warning can be divided into data abnormity early warning, acquisition interruption early warning and no data abnormity.
As shown in fig. 2, another embodiment of the data processing method of the present application includes the following steps:
step 202: the ETL is used to collect data in the task.
Step 204: and packaging data to be monitored in the data in the task into a monitoring object. The monitoring object comprises a database, a database table, a database view, a returned number of sql statements and/or a returned number of java statements, etc.
Step 206: and defining a thread pool according to configuration information, wherein the configuration information comprises the size of the thread pool and the configuration of a blocking queue.
Step 208: and the calling task generator acquires the monitoring object and adds the monitoring object into a task pool, takes out the task to be executed from the task pool, and calls a thread pool to acquire a thread to execute the task to be executed.
And calling a task generator to obtain objects (tables, data sets and the like) to be monitored and adding the objects into a task pool, wherein each task comprises data source connection information, parameters required by task execution and task execution content, and the tasks are independent of each other. The database operation in the task uses a connection pool, whether the connection pool corresponding to the data source exists is judged according to the ID of the data source, if so, the connection is acquired from the connection pool, and if not, the connection pool is created according to the configuration and then the connection is acquired. The connection pool parameters support configuration.
And the task iterator continuously takes out the tasks from the task pool and calls the thread pool to acquire the thread execution tasks. When the number of threads is less than the number of tasks executed, the tasks enter a blocking queue. When the blocking queue of the thread pool is full, the calling thread executing tasks will be blocked until the queue is free.
Step 210: and calculating the data volume of the monitored object.
Step 212: and storing the data volume of the monitored object in a continuously updated data volume record table.
Step 214: and storing the data volume record table in a cache.
Step 216: a data volume interface is provided.
The general view of the data can be displayed through the data volume interface, the monitoring result can be shared, and the monitoring result can be easily integrated in other systems.
Step 218: and providing the data in the data volume record table through the data volume interface.
Step 220: and setting the expected quantity and the early warning information aiming at the monitored object.
And selecting the monitored object, wherein the monitored object can be selected more. Data filtering can be performed on the monitored object, so that data needing to be monitored can be selected in a finer granularity mode. After the monitoring object is configured, the expected number of data can be configured, the expected expression can be an expression which is larger than, larger than or equal to, smaller than or equal to, and the expected number is an integer. The early warning information has the function of how to perform warning notification when the data are monitored to be not satisfied with the expected quantity, the warning mode adopts two modes of http interface callback and nailing robot, and the template of warning can be customized.
Step 222: and alarming when the data in the data volume record table is monitored to be not enough to meet the expected quantity.
After the monitoring task configuration is completed, task data is constructed in a background of the system, and configuration information is constructed into corresponding sql statements and java codes. And pushing the generated task data to a service early warning platform and generating a corresponding task. And the service early warning platform executes tasks regularly according to the corn expression.
Step 224: and calling back the expected quantity according to the early warning information.
And when the execution task finds that the abnormal data volume exists, calling back is carried out according to the early warning information.
The early warning task is mainly used for monitoring a database table and a view and judging whether data are generated and transferred in time. The cron expression is used to determine when to execute, monitor objects are used, and filter rules are configured. The filtering rules can achieve fine-grained data monitoring. The alarm setting can achieve the purpose of using different notified texts aiming at different early warnings.
The method and the system monitor the data in real time with very few resources, determine the data conditions of a source library, an ODS library, a DW library and a DM library, and record the data. And interface disclosure, monitoring and early warning are carried out on the records. The data acquisition system can enable an operator to know the data condition in real time, timely repair the abnormal data acquisition condition, is simple to use, can complete monitoring and early warning functions only by simple configuration, and can be used only by basic computer knowledge.
Example two:
as shown in fig. 3, the data monitoring apparatus of the present application, in one embodiment, includes an acquisition module 310, a packaging module 320, a monitoring module 330, and a processing module 340.
An acquisition module 310 for acquiring data in a task using ETL. And the encapsulating module 320 is configured to encapsulate data to be monitored in the data in the task as a monitoring object. The monitoring object comprises a database, a database table, a database view, a returned number of sql statements and/or a returned number of java statements, etc. And the monitoring module 330 is configured to allocate system resources and continuously monitor the monitored object. The processing module 340 is configured to calculate the monitored object, and store the monitoring result in the buffer area. The monitoring result is stored in the buffer area, and the time for storing the historical data can be determined according to the configuration. And the generated monitoring result data is used as basic support data of data early warning and an asset map.
As shown in fig. 4, another embodiment of the data monitoring apparatus of the present application includes an acquisition module 410, an encapsulation module 420, a monitoring module 430, a processing module 440, and an interface module 450.
An acquisition module 410 for acquiring data in a task using ETL. And the encapsulating module 420 is configured to encapsulate data to be monitored in the data in the task as a monitoring object. The monitoring object comprises a database, a database table, a database view, a returned number of sql statements and/or a returned number of java statements, etc. And the monitoring module 430 is configured to allocate system resources and continuously monitor the monitored object. The processing module 440 is configured to calculate the monitored object, and store the monitoring result in the buffer area. The monitoring result is stored in the buffer area, and the time for storing the historical data can be determined according to the configuration. And the generated monitoring result data is used as basic support data of data early warning and an asset map. An interface module 450 for providing a data volume interface; providing data in a data volume record table through a data volume interface; and setting an early warning task for the data in the data volume record table.
After the data to be monitored is packaged into the monitored object, the monitored object can be monitored, and alarm information is set according to the expected quantity and the filtering rule of the monitored object configuration. And the data can be automatically monitored after the early warning task is started, and the data can be recorded and early warned when the data is found to have problems. The early warning can be divided into data abnormity early warning, acquisition interruption early warning and no data abnormity.
Further, the monitoring module 430 may include a configuration unit 431 and an execution unit 432.
The configuration unit 431 is configured to define a thread pool according to configuration information, where the configuration information includes a thread pool size and a blocking queue configuration.
And calling a task generator to obtain objects (tables, data sets and the like) to be monitored and adding the objects into a task pool, wherein each task comprises data source connection information, parameters required by task execution and task execution content, and the tasks are independent of each other. The database operation in the task uses a connection pool, whether the connection pool corresponding to the data source exists is judged according to the ID of the data source, if so, the connection is acquired from the connection pool, and if not, the connection pool is created according to the configuration and then the connection is acquired. The connection pool parameters support configuration.
And the execution unit 432 is configured to call the task generator to obtain the monitoring object, add the monitoring object to the task pool, take out the to-be-executed task from the task pool, and call the thread pool to obtain a thread to execute the to-be-executed task.
And the task iterator continuously takes out the tasks from the task pool and calls the thread pool to acquire the thread execution tasks. When the number of threads is less than the number of tasks executed, the tasks enter a blocking queue. When the blocking queue of the thread pool is full, the calling thread executing tasks will be blocked until the queue is free.
Further, the processing module 440 may comprise a computing unit 441 and a storage unit 442.
A calculating unit 441 for calculating the number of the monitored objects.
A storage unit 442 for storing the data amount of the monitoring object in a continuously updated data amount record table; and storing the data volume record table in a cache.
Further, the interface module 450 may include a setting unit 451 and a callback unit 452.
A setting unit 451 for setting the expected number and the warning information for the monitored object.
And selecting the monitored object, wherein the monitored object can be selected more. Data filtering can be performed on the monitored object, so that data needing to be monitored can be selected in a finer granularity mode. After the monitoring object is configured, the expected number of data can be configured, the expected expression can be an expression which is larger than, larger than or equal to, smaller than or equal to, and the expected number is an integer. The early warning information has the function of how to perform warning notification when the data are monitored to be not satisfied with the expected quantity, the warning mode adopts two modes of http interface callback and nailing robot, and the template of warning can be customized.
A callback unit 452, configured to perform an alarm when it is monitored that the data in the data volume record table does not meet the expected volume; and calling back the expected quantity according to the early warning information.
The early warning task is mainly used for monitoring a database table and a view and judging whether data are generated and transferred in time. The cron expression is used to determine when to execute, monitor objects are used, and filter rules are configured. The filtering rules can achieve fine-grained data monitoring. The alarm setting can achieve the purpose of using different notified texts aiming at different early warnings.
The method and the system monitor the data in real time with very few resources, determine the data conditions of a source library, an ODS library, a DW library and a DM library, and record the data. And interface disclosure, monitoring and early warning are carried out on the records. The data acquisition system can enable an operator to know the data condition in real time, timely repair the abnormal data acquisition condition, is simple to use, can complete monitoring and early warning functions only by simple configuration, and can be used only by basic computer knowledge.
Those skilled in the art will appreciate that all or part of the steps of the various methods in the above embodiments may be implemented by instructions associated with hardware via a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read-only memory, random access memory, magnetic or optical disk, and the like.
The foregoing is a more detailed description of the present application in connection with specific embodiments thereof, and it is not intended that the present application be limited to the specific embodiments thereof. It will be apparent to those skilled in the art from this disclosure that many more simple derivations or substitutions can be made without departing from the spirit of the disclosure.
Claims (10)
1. A method for monitoring data, comprising:
using the ETL to collect data in a task;
packaging data to be monitored in the data in the task as a monitoring object;
allocating system resources and continuously monitoring the monitored object;
and calculating the monitored object, and storing the monitoring result into a buffer area.
2. The method of claim 1, wherein said allocating system resources for continuous monitoring of said monitored object comprises:
defining a thread pool according to configuration information, wherein the configuration information comprises the size of the thread pool and the configuration of a blocking queue;
and the calling task generator acquires the monitoring object and adds the monitoring object into a task pool, takes out the task to be executed from the task pool, and calls a thread pool to acquire a thread to execute the task to be executed.
3. The method of claim 2, wherein said computing the monitored object and storing the monitoring results in a buffer, comprises:
calculating the number of the monitoring objects;
storing the data volume of the monitored object in a continuously updated data volume record table;
and storing the data volume record table in a cache.
4. The method of claim 3, further comprising:
providing a data volume interface;
providing data in the data volume record table through the data volume interface;
and setting an early warning task for the data in the data volume record table.
5. The method of claim 4, wherein the setting of the pre-warning task for the data in the data volume record table comprises:
setting an expected quantity and early warning information aiming at the monitored object;
when the data in the data volume record table is monitored to be not in the expected number, alarming is carried out;
and calling back the expected quantity according to the early warning information.
6. A data monitoring device, comprising:
the acquisition module is used for acquiring data in the task by using the ETL;
the encapsulation module is used for encapsulating data to be monitored in the data in the task into a monitoring object;
the monitoring module is used for allocating system resources and continuously monitoring the monitored object;
and the processing module is used for calculating the monitored object and storing the monitoring result into the cache region.
7. The apparatus of claim 6, wherein the monitoring module comprises:
the configuration unit is used for defining a thread pool according to configuration information, and the configuration information comprises the size of the thread pool and the configuration of a blocking queue;
and the execution unit is used for calling the task generator to acquire the monitoring object and add the monitoring object into the task pool, taking out the task to be executed from the task pool, and calling the thread pool to acquire the thread to execute the task to be executed.
8. The apparatus of claim 7, wherein the processing module comprises:
a calculation unit configured to calculate the number of the monitoring objects;
the storage unit is used for storing the data volume of the monitored object in a continuously updated data volume record table; and storing the data volume record table in a cache.
9. The apparatus of claim 8, further comprising:
the interface module is used for providing a data volume interface; providing data in the data volume record table through the data volume interface; and setting an early warning task for the data in the data volume record table.
10. The apparatus of claim 9, wherein the interface module comprises:
the setting unit is used for setting the expected quantity and the early warning information aiming at the monitored object;
the call-back unit is used for giving an alarm when the data in the data volume record table is monitored to be not in the expected number; and calling back the expected quantity according to the early warning information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911190323.0A CN110955684A (en) | 2019-11-28 | 2019-11-28 | Data monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911190323.0A CN110955684A (en) | 2019-11-28 | 2019-11-28 | Data monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110955684A true CN110955684A (en) | 2020-04-03 |
Family
ID=69978738
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911190323.0A Pending CN110955684A (en) | 2019-11-28 | 2019-11-28 | Data monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110955684A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597203A (en) * | 2020-12-28 | 2021-04-02 | 恩亿科(北京)数据科技有限公司 | General data monitoring method and system based on big data platform |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117306A (en) * | 2010-01-04 | 2011-07-06 | 阿里巴巴集团控股有限公司 | Method and system for monitoring ETL (extract-transform-load) data processing process |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
| CN108197199A (en) * | 2017-12-27 | 2018-06-22 | 珠海市君天电子科技有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN110442628A (en) * | 2019-07-09 | 2019-11-12 | 恩亿科(北京)数据科技有限公司 | A kind of data monitoring method, system and computer equipment |
-
2019
- 2019-11-28 CN CN201911190323.0A patent/CN110955684A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117306A (en) * | 2010-01-04 | 2011-07-06 | 阿里巴巴集团控股有限公司 | Method and system for monitoring ETL (extract-transform-load) data processing process |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
| CN108197199A (en) * | 2017-12-27 | 2018-06-22 | 珠海市君天电子科技有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN110442628A (en) * | 2019-07-09 | 2019-11-12 | 恩亿科(北京)数据科技有限公司 | A kind of data monitoring method, system and computer equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597203A (en) * | 2020-12-28 | 2021-04-02 | 恩亿科(北京)数据科技有限公司 | General data monitoring method and system based on big data platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109726072B (en) | WebLogic server monitoring and alarming method, device and system and computer storage medium | |
| CN107678907B (en) | Database service logic monitoring method, system and storage medium | |
| US9117025B2 (en) | Tracking of code base and defect diagnostic coupling with automated triage | |
| US8510603B2 (en) | Systems and methods providing an exception buffer to facilitate processing of event handler errors | |
| CN111047190A (en) | Diversified business modeling framework system based on interactive learning technology | |
| CN106371984B (en) | A kind of data monitoring method, equipment and system | |
| US10911447B2 (en) | Application error fingerprinting | |
| KR20180030521A (en) | Data quality analysis | |
| CN107193714B (en) | Alarm display method and device | |
| CN110597861A (en) | Real-time alarm method, device and equipment and computer readable storage medium | |
| CN107566172B (en) | Active management method and system based on storage system | |
| CN110806730A (en) | Big data operation and maintenance platform, server and storage medium | |
| CN111754123A (en) | Data monitoring method and device, computer equipment and storage medium | |
| JP5973714B2 (en) | Emergency response instruction formulation support apparatus and emergency response command formulation support method | |
| CN110955684A (en) | Data monitoring method and device | |
| CN114064402A (en) | Server system monitoring method | |
| CN114116429A (en) | Abnormal log collection method, device, equipment, medium and product | |
| CN113760491A (en) | Task scheduling system, method, equipment and storage medium | |
| CN111597091A (en) | Data monitoring method and system, electronic equipment and computer storage medium | |
| Wiese et al. | Autonomic tuning expert: a framework for best-practice oriented autonomic database tuning | |
| JPWO2013035266A1 (en) | Monitoring device, monitoring method and program | |
| CN115408271A (en) | One-stop closed loop test method, system, equipment and medium | |
| CN115237728A (en) | Visual monitoring method for real-time operating system running state | |
| CN112860523A (en) | Fault prediction method and device for batch job processing and server | |
| CN110427294B (en) | System integration environment monitoring method, apparatus, readable storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200403 |
|
| RJ01 | Rejection of invention patent application after publication |