CN109558306B - Education system-based data acquisition and report omission monitoring method and electronic equipment - Google Patents
Education system-based data acquisition and report omission monitoring method and electronic equipment Download PDFInfo
- Publication number
- CN109558306B CN109558306B CN201811223534.5A CN201811223534A CN109558306B CN 109558306 B CN109558306 B CN 109558306B CN 201811223534 A CN201811223534 A CN 201811223534A CN 109558306 B CN109558306 B CN 109558306B
- Authority
- CN
- China
- Prior art keywords
- data
- production
- consumption
- time
- column
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
Abstract
The invention discloses a data acquisition and report missing monitoring method based on an education system and electronic equipment, wherein the method comprises the following steps: A. reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data; B. judging whether the number of the production data is equal to the number of the consumption data; C. and if the number of the production data is not equal to that of the consumption data, triggering an alarm notification. The invention can monitor and alarm the incomplete data in time, namely inform the condition of missing report in the data acquisition process, is beneficial to positioning problems of analysts, and is beneficial to quickly positioning the reason of wrong statistical results and correcting the statistical results.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data acquisition and report missing monitoring method based on an education system and electronic equipment.
Background
In the data statistical analysis process, the integrity of the source data is the basis of data calculation and is also a key factor for relying on statistical results. However, in some cases, the source data is incomplete, and there is no perfect monitoring mechanism to alarm the situation, which results in large deviation of statistical results and increases difficulty and time cost for analysts to troubleshoot problems.
Accordingly, the prior art is yet to be improved and developed.
Disclosure of Invention
In view of the above deficiencies of the prior art, an object of the present invention is to provide a data collection and report omission monitoring method and an electronic device based on an educational system, which aim to solve the problem in the prior art that the integrity of source data cannot be accurately monitored.
The technical scheme of the invention is as follows:
a data acquisition report missing monitoring method based on an education system comprises the following steps:
A. reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
B. judging whether the number of the production data is equal to the number of the consumption data;
C. and if the number of the production data is not equal to that of the consumption data, triggering an alarm notification.
In the step C, if the number of the production data is equal to the number of the consumption data, returning to the step B to continuously monitor whether the production data and the consumption data are equal to each other.
The education system-based data acquisition and report missing monitoring method comprises the following steps of:
when the first module is producing new data, storing the production data in a corresponding message queue;
the second module consumes data from the message queue and records the number of pieces of consumed data.
The monitoring method for the missing report of the data acquisition based on the education system is characterized in that in the step A, the production data in the message queue are read regularly and the number of the production data is obtained, and the consumption data is read regularly and the number of the consumption data is obtained.
The data acquisition and report missing monitoring method based on the education system is characterized in that production data and consumption data are read simultaneously, and the number of corresponding data is obtained.
The data acquisition and report missing monitoring method based on the education system is characterized in that production data are stored in corresponding message queues one by one according to production time.
The data acquisition and report missing monitoring method based on the education system is characterized in that the production data and the consumption data are in json format.
An electronic device, comprising:
a processor adapted to implement the instructions, an
A storage device adapted to store a plurality of instructions, the instructions adapted to be loaded and executed by a processor to:
reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
judging whether the number of the production data is equal to the number of the consumption data;
and if the number of the production data is not equal to that of the consumption data, triggering an alarm notification.
A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the educational system-based data collection under-run monitoring method.
A computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a processor, cause the processor to perform the educational system-based data collection under-report monitoring method.
Has the advantages that: the invention can monitor and alarm the incomplete data in time, namely inform the condition of missing report in the data acquisition process, is beneficial to positioning problems of analysts, and is beneficial to quickly positioning the reason of wrong statistical results and correcting the statistical results.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a data collection under-report monitoring method based on an educational system of the present invention.
Fig. 2 is a block diagram of an electronic device according to a preferred embodiment of the invention.
Detailed Description
The invention provides a data acquisition and report missing monitoring method based on an education system and electronic equipment, and the invention is further described in detail below in order to make the purpose, technical scheme and effect of the invention clearer and clearer. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for monitoring data collection and report omission in an educational system according to a preferred embodiment of the present invention, which includes the following steps:
s1, reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
s2, judging whether the number of the production data is equal to the number of the consumption data;
and S3, if the number of the production data is not equal to the number of the consumption data, triggering an alarm notification.
The invention can monitor and alarm the incomplete data in time, namely inform the condition of missing report in the data acquisition process, is beneficial to positioning problems of analysts, and is beneficial to quickly positioning the reason of wrong statistical results and correcting the statistical results.
The invention can be applied to a plurality of scenes, for example, the invention can be applied to analyzing system logs, for example, whether the data produced by the monitoring module A can be completely consumed by the monitoring module B is only needed, and if the module B can not completely consume the data of the monitoring module A, the condition that the module B fails to report is shown.
Further, step S1 is preceded by:
when the first module is producing new data, storing the production data in a corresponding message queue;
the second module consumes data from the message queue and records the number of pieces of consumed data.
That is, the present invention records and stores production data in the form of message queues for subsequent real-time monitoring.
In addition, in the invention, when the second module needs to consume the data, the message queue is updated, the number of the consumption data is recorded, and the produced data and the consumption data are convenient to count.
Further, the production data is stored in the corresponding message queue item by item according to the production time.
The invention stores the production time one by one, and is convenient to manage.
Further, in the step S1, the production data in the message queue is read regularly and the number of pieces of production data is acquired, and the consumption data is read regularly and the number of pieces of consumption data is acquired.
That is, the number of production data and the number of consumption data are acquired at intervals, so that the purpose of timing monitoring can be achieved. And the length of the interval time can be determined according to actual conditions. In addition, the invention reads the production data and the consumption data at the same time and obtains the number of the corresponding data. That is, although the production data is read at intervals and the consumption data is also read at intervals, the reading time of the production data and the reading time of the consumption data are the same, so that whether the number of the production data and the consumption data is the same or not can be accurately judged.
In step S2, it is determined whether the number of production data pieces is equal to the number of consumption data pieces.
In said step S3, if the number of production data pieces is not equal to the number of consumption data pieces, an alarm notification is triggered.
If the number of the production data is equal to that of the consumption data, returning to the step B to continuously monitor whether the production data and the consumption data are equal.
Further, the format of the production data and the consumption data is json format.
One specific example is as follows:
the first step is as follows: module A produces JSON data in N bars, such as { id:1, name: zs, age:15}, which are stored in the message queue.
The second step is that: module B consumes M pieces of data from the message queue.
The third step: and the monitoring system monitors in real time and finds that M is equal to N, an alarm cannot be triggered, and if M' N is found, the monitoring module triggers an alarm notification.
The fourth step: and (6) ending.
In the invention, the message queue comprises a data sequence number column, a data column, a production time column and a consumption time column, so that when new data is produced, the new data sequence number is added in the data sequence number column of the message queue, corresponding data (which can be called production data) is recorded in the data column, and meanwhile, the time for producing the data is recorded, namely, the production time is recorded in the production time column, and when the data is consumed, the consumption time is recorded in the consumption time column.
Thus, each piece of data can be used as production data when it is produced and as consumption data when it is consumed. The number of the production data can be obtained by counting the number of the production time in the production time column, that is, the number of the production time is the number of the production data. The number of consumption data can be obtained by counting the number of consumption time in the consumption time column, namely the number of consumption time is the number of consumption data.
The alarm notification can be judged whether to be triggered or not by comparing the number of the production data with the number of the consumption data and judging whether the two are equal, if so, the alarm notification is not needed to be triggered, and if not, the alarm notification is triggered.
Since the number of production data and the number of consumption data are continuously updated, the reading time interval needs to be properly reduced, and the reading time interval may be the same or different. For example, the production time column and the consumption time column may be read at the same time interval, for example, the production time column and the consumption time column may be read once every 1 day, specifically, the production time column and the consumption time column may be read once at 12 am every day. The number of the production time and the number of the consumption time are obtained, so that the number of the production data and the number of the consumption data can be known, and comparison is convenient.
In addition, because the invention can record the production time every time when producing the data, and can record the consumption time every time consuming the corresponding data, the invention not only can compare the number of the production data and the number of the consumption data, but also can solve the problem of which data occurs when the number of the production data is not equal to the number of the consumption data.
For example, when the number of generated data is not equal to the number of consumed data, the production time and the consumption time of each piece of data in the message queue are obtained, and whether there is only production time or only consumption time for any piece of data is judged, if there is only production time but not consumption time for a piece of data, it is indicated that the piece of data is only produced but not consumed, so the piece of data has a problem that the piece of data is not consumed in time, and if there is only consumption time but not production time for a piece of data, it is indicated that a record of the production time is possibly missed by the piece of data.
In order to analyze the reason of the alarm conveniently, the invention also stores the message queue in real time and updates the message queue in real time, for example, when the number of the production data and the consumption data in the message queue is read once, the message queue is stored once at the same time, and the message queue which is stored before is covered.
In addition, the invention can mark the data only with production time but not with consumption time and the data only with consumption time but not with production time in the message queue, so that a user can directly find out which data has a problem when analyzing the reason of the alarm, and correct or adjust the data and the like according to the problem. The marking mode can be that the piece of data is marked with red or yellow for convenient viewing.
In the invention, the alarm notification can be carried out in various ways, and the alarm notification can be sent by a short message, a mail or other ways.
The present invention also provides an electronic device 10, as shown in fig. 2, comprising:
a processor 110 adapted to implement instructions, an
A storage device 120 adapted to store a plurality of instructions adapted to be loaded and executed by the processor 110:
reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
judging whether the number of the production data is equal to that of the consumption data or not;
and if the number of the production data is not equal to that of the consumption data, triggering an alarm notification.
The processor 110 may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a single chip, an arm (acorn RISC machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Also, the processor may be any conventional processor, microprocessor, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The storage device 120 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions corresponding to the data collection and report omission monitoring method based on the educational system in the embodiment of the present invention. The processor executes various functional applications and data processing of the education system-based data acquisition and false negative monitoring method by running nonvolatile software programs, instructions and units stored in the storage device, namely, the method embodiment is realized.
The details of the electronic device 10 are described in the above method, and thus will not be described in detail.
The present invention also provides a non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the educational system-based data collection under-run monitoring method.
The present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method for monitoring for data collection under-reporting based on an educational system.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (7)
1. A data acquisition and report missing monitoring method based on an education system is characterized by comprising the following steps:
A. reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
B. judging whether the number of the production data is equal to the number of the consumption data;
C. if the number of the production data is not equal to the number of the consumption data, triggering an alarm notification;
the step A is also preceded by:
when the first module is producing new data, storing the production data in a corresponding message queue;
the second module consumes the data in the message queue and records the number of consumed data;
in the step A, regularly reading the production data in the message queue and acquiring the number of the production data, and regularly reading the consumption data and acquiring the number of the consumption data; reading production data and consumption data at the same time and obtaining the number of corresponding data;
the message queue comprises a data sequence number column, a data column, a production time column and a consumption time column; when new data are produced, adding a new data serial number in a data serial number column of the message queue, recording corresponding data in the data column, and simultaneously recording production time of the production data in a production time column, wherein the number of the production time is the number of the production data; when the data is consumed, recording the consumption time of the consumption data in a consumption time column, wherein the number of the consumption time is the number of the consumption data;
and when the number of the production data is not equal to the number of the consumption data, acquiring the production time and the consumption time of each piece of data in the message queue, and judging whether the data only has the production time or only has the consumption time.
2. The educational-system-based data collection false negative monitoring method according to claim 1, wherein in step C, if the number of production data is equal to the number of consumption data, the method returns to step B to continue monitoring whether the number of production data is equal to the number of consumption data.
3. The educational system-based data collection false negative monitoring method of claim 1, wherein the production data is stored in the corresponding message queue item by item according to production time.
4. The educational system-based data collection false negative monitoring method of claim 1, wherein the production data and the consumption data are in json format.
5. An electronic device, comprising:
a processor adapted to implement the instructions, an
A storage device adapted to store a plurality of instructions, the instructions adapted to be loaded and executed by a processor to:
reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data;
judging whether the number of the production data is equal to the number of the consumption data;
if the number of the production data is not equal to the number of the consumption data, triggering an alarm notification;
before reading the production data and obtaining the number of the production data, and reading the consumption data and obtaining the number of the consumption data, the method further comprises the following steps:
when the first module is producing new data, storing the production data in a corresponding message queue;
the second module consumes the data in the message queue and records the number of consumed data:
the method comprises the steps of reading production data and obtaining the number of the production data, reading consumption data and obtaining the number of the consumption data, regularly reading the production data in a message queue and obtaining the number of the production data, and regularly reading the consumption data and obtaining the number of the consumption data; reading production data and consumption data at the same time and obtaining the number of corresponding data;
the message queue comprises a data sequence number column, a data column, a production time column and a consumption time column; when new data are produced, adding a new data serial number in a data serial number column of the message queue, recording corresponding data in the data column, and simultaneously recording production time of the production data in a production time column, wherein the number of the production time is the number of the production data; when the data is consumed, recording the consumption time of the consumption data in a consumption time column, wherein the number of the consumption time is the number of the consumption data;
and when the number of the production data is not equal to the number of the consumption data, acquiring the production time and the consumption time of each piece of data in the message queue, and judging whether the data only has the production time or only has the consumption time.
6. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the educational system-based data collection false-positive monitoring method of any one of claims 1-4.
7. A computer program product, characterized in that the computer program product comprises a computer program stored on a non-volatile computer-readable storage medium, the computer program comprising program instructions which, when executed by a processor, cause the processor to carry out the educational system-based data collection under-run monitoring method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811223534.5A CN109558306B (en) | 2018-10-19 | 2018-10-19 | Education system-based data acquisition and report omission monitoring method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811223534.5A CN109558306B (en) | 2018-10-19 | 2018-10-19 | Education system-based data acquisition and report omission monitoring method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109558306A CN109558306A (en) | 2019-04-02 |
| CN109558306B true CN109558306B (en) | 2022-06-17 |
Family
ID=65865152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811223534.5A Active CN109558306B (en) | 2018-10-19 | 2018-10-19 | Education system-based data acquisition and report omission monitoring method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109558306B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110187980B (en) * | 2019-04-19 | 2021-11-19 | 北京三快在线科技有限公司 | Message queue monitoring method and device, electronic equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101877653A (en) * | 2010-06-29 | 2010-11-03 | 中兴通讯股份有限公司 | Method and system for judging whether alarm variable information is lost or not as well as master control board |
| CN104683175A (en) * | 2013-12-03 | 2015-06-03 | 华为技术有限公司 | Network performance testing method and testing device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10331481B2 (en) * | 2017-03-08 | 2019-06-25 | International Business Machines Corporation | Automatic reconfiguration of high performance computing job schedulers based on user behavior, user feedback, and job performance monitoring |
-
2018
- 2018-10-19 CN CN201811223534.5A patent/CN109558306B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101877653A (en) * | 2010-06-29 | 2010-11-03 | 中兴通讯股份有限公司 | Method and system for judging whether alarm variable information is lost or not as well as master control board |
| CN104683175A (en) * | 2013-12-03 | 2015-06-03 | 华为技术有限公司 | Network performance testing method and testing device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109558306A (en) | 2019-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10585774B2 (en) | Detection of misbehaving components for large scale distributed systems | |
| US9846632B2 (en) | Real-time reporting based on instrumentation of software | |
| CN110928718A (en) | Exception handling method, system, terminal and medium based on correlation analysis | |
| CN109493573B (en) | User-defined event alarm method and system based on time sliding window | |
| US20170213025A1 (en) | Methods, systems, apparatus, and storage media for use in detecting anomalous behavior and/or in preventing data loss | |
| US11221904B2 (en) | Log analysis system, log analysis method, and log analysis program | |
| CN110362455B (en) | Data processing method and data processing device | |
| TW202215243A (en) | Abnormity alarm method, device and equipment and storage medium | |
| CN108664603B (en) | Method and device for repairing abnormal aggregation value of time sequence data | |
| CN112311617A (en) | Configured data monitoring and alarming method and system | |
| US10037504B2 (en) | Methods for determining manufacturing waste to optimize productivity and devices thereof | |
| WO2009110329A1 (en) | Failure analysis device, failure analysis method, and recording medium | |
| CN109818787A (en) | Service exception detection method, device and storage medium | |
| CN107357731A (en) | Process produces monitoring, analysis and the processing method of core dump problems | |
| CN109558306B (en) | Education system-based data acquisition and report omission monitoring method and electronic equipment | |
| CN110662024A (en) | Video quality diagnosis method and device based on multiple frames and electronic equipment | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN108197002B (en) | Mobile device non-buried point data statistical method, system, terminal and medium | |
| CN110597694A (en) | Method and terminal for monitoring front-end page | |
| CN108600690A (en) | Instrument board meter register method and system, storage medium and server based on image recognition | |
| CN113434498A (en) | Method and device for monitoring data abnormity of database of power system and electronic equipment | |
| CN112883765B (en) | Target movement track acquisition method and device, storage medium and electronic equipment | |
| CN110874355A (en) | Method, system, terminal and medium for detecting abnormal behavior of vehicle loitering winding | |
| CN114996080A (en) | Data processing method, device, equipment and storage medium | |
| US10623236B2 (en) | Alert management system for enterprises |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |