CN113297261A - Early warning method and device for data processing, server and storage medium - Google Patents
Early warning method and device for data processing, server and storage medium Download PDFInfo
- Publication number
- CN113297261A CN113297261A CN202110674729.7A CN202110674729A CN113297261A CN 113297261 A CN113297261 A CN 113297261A CN 202110674729 A CN202110674729 A CN 202110674729A CN 113297261 A CN113297261 A CN 113297261A
- Authority
- CN
- China
- Prior art keywords
- data file
- early warning
- time
- information system
- received
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012545 processing Methods 0.000 title claims description 82
- 230000005540 biological transmission Effects 0.000 claims description 79
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013461 design Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 10
- 238000011144 upstream manufacturing Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/24569—Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Abstract
The method comprises the steps of obtaining a list to be processed, determining whether a data file corresponding to each data file identifier is received by a second information system or not based on the data file identifiers in the list to be processed, and pushing first early warning information to a preset contact person if it is determined that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within a preset first early warning time. According to the technical scheme, whether the data files corresponding to the data file identifications in the list to be processed are received or not is judged, early warning that the data files are not received in time is achieved, and a foundation is provided for workers to timely process possible problems.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing early warning method, apparatus, server, and storage medium.
Background
With the continuous development of the information-oriented era, interactive collaboration among different information systems is becoming more common, for example, when a system a needs to complete a certain service, a corresponding data file needs to be obtained from a system B according to a set time, and this process is also called batch processing.
For the situation that the data file of the system B does not arrive in time, generally, after a preset time period, the system a and the system B retry the transmission of the data file.
However, the automatic retry method in the prior art cannot early warn that the data file does not arrive in time in advance, so that the user misses an optimal time for solving the situation of the data file which does not arrive in time.
Disclosure of Invention
The embodiment of the application provides a data processing early warning method, a data processing early warning device, a server and a storage medium, and is used for solving the problem that a data file cannot be early warned not to arrive in time in the prior art.
In a first aspect, an embodiment of the present application provides an early warning method for data processing, including:
acquiring a to-be-processed list, wherein the to-be-processed list comprises a plurality of data file identifications needing to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification;
determining whether the data file corresponding to each data file identifier is received by the second information system or not based on the data file identifiers in the list to be processed;
if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within the preset first early warning time, pushing first early warning information to a preset contact person, wherein the first early warning information is used for reminding that the data file is not received, the first early warning time is determined according to the theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to the historical time of the second information system for receiving the data file.
In a possible design of the first aspect, the method further includes:
and if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within a preset second early warning time, pushing second early warning information to the contact person, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
In this possible design, the method further comprises:
and if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within a preset third early warning time, pushing third early warning information to the contact person, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
In another possible design of the first aspect, the determining, based on the data file identifiers in the to-be-processed manifest, whether the data file corresponding to each data file identifier is received by the second information system includes:
and periodically inquiring files stored in the second information system according to a preset processing period and the data file identifications in the list to be processed, and determining whether the data file corresponding to each data file identification is received by the second information system.
In yet another possible design of the first aspect, the obtaining the to-be-processed list includes:
acquiring all data file identifications which need to be transmitted from the first information system to the second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
and adding a plurality of data file identifications with transmission dates as the current dates and transmission times corresponding to the data file identifications to a pre-generated processing list to obtain the list to be processed.
In a second aspect, an embodiment of the present application provides an early warning apparatus for data processing, including: the device comprises an acquisition module, a determination module and a processing module;
the acquisition module is used for acquiring a to-be-processed list, wherein the to-be-processed list comprises a plurality of data file identifications which need to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification;
the determining module is configured to determine, based on the data file identifiers in the to-be-processed list, whether the data file corresponding to each data file identifier is received by the second information system;
the processing module is used for pushing first early warning information to a preset contact after determining that the data file corresponding to any data file identification is not received by the second information system and the current moment is within a preset first early warning time, wherein the first early warning information is used for reminding that the data file is not received, the first early warning time is determined according to the theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to the historical time of the second information system for receiving the data file.
In a possible design of the second aspect, the processing module is further configured to: and after determining that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within a preset second early warning time, pushing second early warning information to the contact person, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
In this possible design, the processing module is further configured to: and after determining that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within a preset third early warning time, pushing third early warning information to the contact, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
In another possible design of the second aspect, the determining module is specifically configured to periodically query files stored in the second information system according to a preset processing period and data file identifiers in the to-be-processed list, and determine whether the data file corresponding to each data file identifier is received by the second information system.
In yet another possible design of the second aspect, the obtaining module is specifically configured to:
acquiring all data file identifications which need to be transmitted from the first information system to the second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
and adding a plurality of data file identifications with transmission dates as the current dates and transmission times corresponding to the data file identifications to a pre-generated processing list to obtain the list to be processed.
In a third aspect, an embodiment of the present application provides a server, including: a processor, a memory;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions to cause the server to perform the method of pre-warning of data processing as described in the first aspect and various possible designs above.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is used to implement the data processing early warning method as described in the first aspect and various possible designs.
In a fifth aspect, embodiments of the present application provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program is configured to implement the method for early warning of data processing as described in the first aspect and various possible designs.
According to the data processing early warning method, the data processing early warning device, the server and the storage medium, the list to be processed is obtained, whether the data files corresponding to the data file identifications are received by the second information system or not is determined based on the data file identifications in the list to be processed, and then if it is determined that the data files corresponding to any data file identification are not received by the second information system and the current time is within the preset first early warning time, first early warning information is pushed to the preset contact person. According to the technical scheme, whether the data files corresponding to the data file identifications in the list to be processed are received or not is judged, early warning that the data files are not received in time is achieved, and a foundation is provided for workers to timely process possible problems.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a schematic view of an application scenario of an early warning method for data processing according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a first embodiment of an early warning method for data processing according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a second early warning method for data processing according to an embodiment of the present disclosure;
fig. 4 is a schematic functional architecture diagram of a server according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a third embodiment of an early warning method for data processing according to the present application;
fig. 6 is a schematic structural diagram of an early warning apparatus for data processing according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application.
With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Before introducing the embodiments of the present application, the background of the present application is explained first:
with the continuous development of the information-oriented era, most business and management fields establish respective information systems, the information systems do not exist independently, but provide services through mutual cooperation among the information systems, for example, the services provided by other information systems are called in a real-time mode, or data files acquired from an upstream system are processed in a batch processing mode, so that the functional requirements of the system of the user are met.
The upstream system is an information system corresponding to a provider of the data file, and the downstream system is an information system corresponding to a user of the data file.
When the upstream system and the downstream system perform data file interaction, the data file is generally sent from the upstream system to the downstream system according to the appointed sending time of the data file, and when the downstream system does not receive the data file at the appointed time, an error is reported during batch processing operation, and then the process of transmitting the data file is retried.
However, this automatic retry method is a passive solution, that is, there is a situation that a downstream system already needs a data file to implement a corresponding service, but the data file has not yet arrived, or there is not enough time for a technician to quickly solve the existing problem due to determining that the data file has not arrived in time, thereby affecting production and life.
Based on the problems in the prior art, fig. 1 is a schematic view of an application scenario of the data processing early warning method provided in the embodiment of the present application, so as to solve the technical problems. As shown in fig. 1, the application scenario diagram includes: a first information system 11, a second information system 12 and a server 13.
The first information system 11 is an upstream system, and may be a plurality of information systems for transmitting data files; the second information system 12 is a downstream system and may be an information system that receives data files.
It should be understood that: the server 13 is an execution subject of the embodiment of the present application, and the functions of the server 13 may be integrated on the second information system 12, be a part of the second information system 12, or be a third-party server independent from the second information system 12.
The application scenario is exemplarily described by taking the server 13 as an example independent of the second information system 12, and other possible implementations are described in detail by the following embodiments.
In a possible implementation, the first information system 11 transmits a data file to the second information system 12 according to the needs of the second information system 12, the server 13 determines a to-be-processed list including the data file identifier according to the identifier of the data file to be transmitted, then the server 13 determines whether the data file corresponding to the data file identifier on the to-be-processed list is received according to a set query frequency, specifically, by querying a cache of the second information system 12, whether the data file corresponding to the data file identifier is stored, and when the data file identifier is not queried, sends early warning information to a contact (user) corresponding to the data file identifier.
In order to solve the technical problems, the technical conception process of the inventor is as follows: the inventor finds that the data files which need to be received at present can be counted, each data file is monitored according to a preset rule, technicians are reminded, the problems in the prior art can be solved, and therefore early warning that the data files do not arrive in time is achieved.
The technical solution of the present application is described in detail below with reference to an application scenario diagram shown in fig. 1 by specific embodiments. It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a schematic flowchart of a first embodiment of an early warning method for data processing according to an embodiment of the present disclosure. As shown in fig. 2, the data processing early warning method may include the following steps:
and step 21, acquiring a list to be processed.
The list to be processed comprises a plurality of data file identifications which need to be transmitted from the first information system to the second information system and transmission time of the data file corresponding to each data file identification.
In this step, a warning is given for the situation that the data file does not arrive in time, and the data file that may be transmitted between the first information system and the second information system at the current date needs to be acquired first.
The execution main body of the scheme can be a third-party server independent of the first information system and the second information system, or can be the second information system, and the third-party server or the second information system should have the corresponding functions of the early warning method for data processing provided by the embodiment of the application.
Optionally, the data file identifier transmitted between the first information system and the second information system on the current date may be added to a to-be-processed list of the server or the second information system, so as to determine whether the corresponding data file is received according to the data file identifier in the following, and when it is determined whether an early warning is required because any data file is not received, it is also necessary to acquire the transmission time corresponding to the data file.
Specifically, the step can be realized by the following steps:
step 1, acquiring all data file identifications which need to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
optionally, the transmission time of all data files transmitted from the first information system to the second information system (may be at least one transmission time in a week, at least one transmission time in a month, and at least one transmission time in a day), may be negotiated between the two information systems, or may be predetermined according to a service provided by a technician for the second information system.
Further, the transmission date only defines the date when the data file is transmitted, but does not define the specific transmission time, and cannot provide a basis for the accuracy of the warning (for example, the time when the data file provides service for the second information system is assumed to be 14 points, the transmission time of the data file is assumed to be 10 points, and when the data file cannot be at 8 points, the warning is issued because the data file is not queried in the second information system), at this time, the transmission time of the data file also needs to be acquired.
And step 2, adding a plurality of data file identifications with the transmission dates being the current dates and the transmission times corresponding to the plurality of data file identifications to a pre-generated processing list to obtain a list to be processed.
Optionally, in the execution main body, a processing list is preset, where the processing list may be in a form of a table (by way of example only), and as a carrier, each row is used for recording transmission information of a data file, where the processing list includes: data file identification and transmission time.
Further, adding a plurality of data file identifications of which the transmission date is the current date and transmission time corresponding to the plurality of data file identifications into the table to obtain the list to be processed.
And step 22, determining whether the data file corresponding to each data file identifier is received by the second information system or not based on the data file identifiers in the list to be processed.
In this step, the list to be processed is processed, specifically, for each data file identifier, whether the data file corresponding to the data file identifier is already stored is queried in the database.
Optionally, files stored in the second information system may be periodically queried according to a preset processing period and the data file identifiers in the to-be-processed list, and whether the data file corresponding to each data file identifier is received by the second information system is determined.
The preset treatment period may be ten minutes or twenty minutes, which is not limited herein.
Specifically, according to each data file identifier in the list to be processed, the files stored in the second information system may be sequentially queried according to the previous adding sequence, and whether the data file corresponding to the corresponding data file identifier is received is determined; or in a parallel mode, simultaneously inquiring whether the data files corresponding to the data file identifications are received.
Optionally, after the data file corresponding to the data file identifier is received, the corresponding data file identifier and the corresponding other information are deleted from the to-be-processed list, so as to avoid the problem of too low efficiency in repeated execution.
Further, for the case where the data file identification corresponding to the data file is not received, the query process may be re-executed after a preset processing period (e.g., ten minutes).
And 23, if it is determined that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within the preset first early warning time, pushing first early warning information to a preset contact person.
The first early warning information is used for reminding that the data file is not received, the first early warning time is determined according to the theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to the historical time of the second information system for receiving the data file.
In this step, if it is found that the data file corresponding to the data file identifier is not received by the second information system, the data file is compared with a preset first early warning time (for example, the first early warning time may be a time at which the data file is theoretically received) according to the current time, and when the current time is after the first early warning time (for example, the first early warning time is 10 points, and the current time is 10 points and 5 minutes), it is preliminarily determined that the data file is abnormal in transmission, and the first early warning information may be pushed to a preset contact person.
In a possible implementation, the setting of the first early warning time is related to the historical time of the data file, specifically, ten times of receiving time closest to the current time may be obtained first, the ten times of receiving time are sorted from early to late, and the ninth time is taken as the first early warning time.
In another possible implementation, when the data file corresponding to the data file identifier is received by the second information system, the new first warning time may be determined according to the current time and the latest nine historical times by the above method.
Optionally, the first warning information may display a yellow warning notice through the display interface, or may be a mail, a ring, a short message, or the like associated with a low attention degree.
Alternatively, the preset contact may be a technician related to the first information system, a technician related to the second information system, a transmitter of the data file, and the like.
According to the data processing early warning method provided by the embodiment of the application, the list to be processed is obtained, whether the data file corresponding to each data file identification is received by the second information system is determined based on the data file identification in the list to be processed, and then if it is determined that the data file corresponding to any data file identification is not received by the second information system and the current moment is within the preset first early warning time, the first early warning information is pushed to the preset contact person. According to the technical scheme, whether the data files corresponding to the data file identifications in the list to be processed are received or not is judged, early warning that the data files are not received in time is achieved, and a foundation is provided for workers to timely process possible problems.
On the basis of the foregoing embodiments, fig. 3 is a schematic flowchart of a second embodiment of the data processing early warning method provided in the embodiments of the present application. As shown in fig. 3, after the step 23, the data processing method may further include the following steps:
and 31, if it is determined that the data file corresponding to any data file identifier is not received by the second information system and the current time is after a preset second early warning time, pushing second early warning information to the contact, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
In this step, the query task on the to-be-processed list is executed according to the preset processing cycle, after the first early warning information of the data file corresponding to the data file identifier is sent, the data file corresponding to the data file identifier is still not received, at this time, the relationship between the current time and the second early warning time is judged, and whether the second early warning information needs to be sent or not is determined.
Alternatively, the second warning time may be determined based on the time at which the data file is executed by the second information system to provide the corresponding service, and may be several hours (e.g., 3 hours) earlier than the time at which the data file is executed by the second information system, but the second warning time should be later than the first warning time (e.g., 9 o ' clock for the first warning time, 11 o ' clock for the second warning time, and 14 o ' clock for the second information system).
Optionally, the second warning information may display an orange prompt for attention through the display interface, or may be an email, a ring, a short message, or the like having a medium attention degree correlation.
And step 32, if it is determined that the data file corresponding to any data file identifier is not received by the second information system and the current time is after a preset third early warning time, pushing third early warning information to the contact, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
In this step, the query task in the to-be-processed list is continuously executed according to the preset processing period, after the second early warning information of the data file corresponding to the data file identifier is sent, the data file corresponding to the data file identifier is still not received, at this time, the relationship between the current time and the third early warning time is judged, and whether the third early warning information needs to be sent or not is determined.
Optionally, the third warning time may also be determined based on a time at which the data file is executed by the second information system to provide the corresponding service, and may be earlier than an upgrade time interval (e.g., 2 hours) executed by the second information system, and the third warning time should be later than the second warning time (e.g., 11 o ' clock for the second warning time, 12 o ' clock for the third warning time, and 14 o ' clock for the second information system).
Optionally, the third warning information may display a red warning notice through the display interface, or may be a mail, a ring, a short message, or the like associated with a high attention degree.
According to the data processing early warning method provided by the embodiment of the application, after the data file corresponding to any data file identifier is determined not to be received by the second information system and the current time is within the preset second early warning time, the second early warning information is pushed to the contact person, and then if the data file corresponding to any data file identifier is determined not to be received by the second information system and the current time is within the preset third early warning time, the third early warning information is pushed to the contact person. In the technical scheme, along with the grade promotion of the early warning information, the attention degree to the possible problems is increased, so that the technical personnel attach importance to the problems of information receiving error and the like, and unnecessary troubles are avoided.
On the basis of the foregoing embodiments, fig. 4 is a schematic functional architecture diagram of a server provided in the embodiment of the present application, fig. 5 is a schematic flow diagram of a third embodiment of an early warning method for data processing provided in the embodiment of the present application, and fig. 4 is combined with fig. 5 to integrally describe steps of a technical solution related to the present application.
As shown in fig. 4, the architecture diagram includes: a pending list generation service 41, a data file monitoring service 42, an early warning service 43, and a configuration management service 44.
Optionally, the configuration management service 44 provides all data file identifiers and transmission time of the data file corresponding to each data file identifier, the to-be-processed list generation service 41 determines the to-be-processed list according to all data file identifiers and transmission time of the data file corresponding to each data file identifier, the data file monitoring service 42 queries whether the data file corresponding to the data file identifier has been received according to the data file identifier recorded on the to-be-processed list, and if not, the early warning service 43 sends early warning information to the contact corresponding to the data file identifier.
Specifically, as shown in fig. 5 and in conjunction with fig. 4, the data processing early warning method may include the following steps:
step 1, acquiring a first data file identifier and transmission time of a data file corresponding to the data file identifier;
the data file system may include a plurality of data file identifiers, data files corresponding to the data file identifiers, and transmission times corresponding to the data files.
Step 2, judging whether the current date is the transmission date in the transmission time of the data file, if so, executing step 3; if not, executing the step 1;
when the transmission date corresponding to the first data file identifier is not the current date, taking down one data file identifier and the transmission time of the data file corresponding to the data file identifier in the step 1, and repeating the step 2;
step 3, adding the list to be processed;
and repeatedly executing the operation, and adding all data file identifications conforming to the current date to the list to be processed.
Step 4, acquiring data file identifications one by one from the list to be processed at regular time;
step 5, checking whether the data file corresponding to the data file identifier is received or not, if so, executing step 4, deleting the data file identifier from the list to be processed, and acquiring the next data file identifier in the list to be processed; if not, executing the step 6;
step 6, according to the preset early warning time, determining early warning information that the data file corresponding to the data file identification is not received;
and 7, sending early warning information to the contact corresponding to the data file identification.
The early warning method for data processing provided by the embodiment of the application judges whether each data file is received at the current date or not by acquiring all data file identifications and the transmission time of the data file corresponding to each data file identification, if so, the data file identifications and the transmission times of the data files corresponding to the respective data file identifications are added to the pending list, inquiring the cache according to each data file identifier in the list to be processed, determining whether the data file corresponding to the data file identifier is received, if not, according to the preset early warning time, whether the early warning information is sent to the contact corresponding to the data file identification is determined, when the data file corresponding to the data file identification in the list to be processed is not received, and sending early warning information to the technical staff so that the technical staff can timely process the possible problems.
On the basis of the above-mentioned data processing early warning method embodiment, fig. 6 is a schematic structural diagram of the data processing early warning apparatus provided in the embodiment of the present application. As shown in fig. 6, the apparatus includes: an acquisition module 61, a determination module 62 and a processing module 63.
The acquiring module 61 is configured to acquire a to-be-processed list, where the to-be-processed list includes a plurality of data file identifiers that need to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identifier;
a determining module 62, configured to determine, based on the data file identifiers in the to-be-processed list, whether the data file corresponding to each data file identifier is received by the second information system;
and the processing module 63 is configured to, after it is determined that the data file corresponding to any data file identifier is not received by the second information system and the current time is within a preset first early warning time, push first early warning information to a preset contact, where the first early warning information is used to remind that the data file is not received, the first early warning time is determined according to theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to historical time of the second information system for receiving the data file.
In one possible design of the embodiment of the present application, the processing module 63 is further configured to: and after the data file corresponding to any data file identifier is determined not to be received by the second information system and the current moment is within the preset second early warning time, pushing second early warning information to the contact person, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
In this possible design, the processing module 63 is also configured to: and after the data file corresponding to any data file identifier is determined not to be received by the second information system and the current moment is within a preset third early warning time, pushing third early warning information to the contact person, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
In another possible design of the embodiment of the present application, the determining module 62 is specifically configured to periodically query files stored in the second information system according to a preset processing period and data file identifiers in the to-be-processed list, and determine whether the data file corresponding to each data file identifier is received by the second information system.
In yet another possible design of the embodiment of the present application, the obtaining module 61 is specifically configured to:
acquiring all data file identifications which need to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
and adding the plurality of data file identifications with the transmission date as the current date and the transmission time corresponding to the plurality of data file identifications to a pre-generated processing list to obtain a list to be processed.
The data processing early warning device provided in the embodiment of the application can be used for implementing the technical scheme of the data processing early warning method in the embodiment, and the implementation principle and the technical effect are similar, and are not described herein again.
It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the determining module 62 may be a separate processing element, or may be integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the functions of the determining module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element here may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
Fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application. As shown in fig. 7, the server may include: a processor 71, a memory 72.
The server may be the second information system in the above embodiment, or may be a server device independent of the second information system.
The processor 71 executes the computer execution instruction stored in the memory, so that the processor 71 executes the technical solution of the data processing early warning method in the above embodiment. The processor 71 may be a general-purpose processor including a central processing unit CPU, a Network Processor (NP), and the like; but also a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
A memory 72 is coupled to the processor 71 via the system bus and communicates with each other, the memory 72 storing computer program instructions.
Optionally, the server may further comprise a transceiver for communicating with other computers. Optionally, in terms of hardware implementation, the obtaining module 61 in the embodiment shown in fig. 6 corresponds to a transceiver in this embodiment, and the transceiver constitutes a communication interface.
The system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may comprise Random Access Memory (RAM) and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The server provided in the embodiment of the present application may be used in a technical scheme for executing the early warning method for data processing in the foregoing embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
The embodiment of the application further provides a chip for running the instruction, and the chip is used for executing the technical scheme of the early warning method for data processing in the embodiment.
The embodiment of the present application further provides a computer-readable storage medium, where a computer instruction is stored in the computer-readable storage medium, and when the computer instruction runs on a computer, the computer is enabled to execute the technical solution of the early warning method for data processing in the foregoing embodiment.
The embodiment of the present application further provides a computer program product, where the computer program product includes a computer program, the computer program is stored in a computer-readable storage medium, a processor can read the computer program from the computer-readable storage medium, and when the processor executes the computer program, the processor can implement the technical solution of the early warning method for data processing in the foregoing embodiment.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (13)
1. An early warning method for data processing is characterized by comprising the following steps:
acquiring a to-be-processed list, wherein the to-be-processed list comprises a plurality of data file identifications needing to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification;
determining whether the data file corresponding to each data file identifier is received by the second information system or not based on the data file identifiers in the list to be processed;
if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within the preset first early warning time, pushing first early warning information to a preset contact person, wherein the first early warning information is used for reminding that the data file is not received, the first early warning time is determined according to the theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to the historical time of the second information system for receiving the data file.
2. The method of claim 1, further comprising:
and if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within a preset second early warning time, pushing second early warning information to the contact person, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
3. The method of claim 2, further comprising:
and if the data file corresponding to any data file identification is determined not to be received by the second information system and the current moment is within a preset third early warning time, pushing third early warning information to the contact person, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
4. The method according to any one of claims 1 to 3, wherein the determining whether the data file corresponding to each data file identifier is received by the second information system based on the data file identifiers in the pending list comprises:
and periodically inquiring files stored in the second information system according to a preset processing period and the data file identifications in the list to be processed, and determining whether the data file corresponding to each data file identification is received by the second information system.
5. The method according to any one of claims 1 to 3, wherein the obtaining the to-be-processed list comprises:
acquiring all data file identifications which need to be transmitted from the first information system to the second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
and adding a plurality of data file identifications with transmission dates as the current dates and transmission times corresponding to the data file identifications to a pre-generated processing list to obtain the list to be processed.
6. An early warning device for data processing, comprising: the device comprises an acquisition module, a determination module and a processing module;
the acquisition module is used for acquiring a to-be-processed list, wherein the to-be-processed list comprises a plurality of data file identifications which need to be transmitted from a first information system to a second information system and transmission time of a data file corresponding to each data file identification;
the determining module is configured to determine, based on the data file identifiers in the to-be-processed list, whether the data file corresponding to each data file identifier is received by the second information system;
the processing module is used for pushing first early warning information to a preset contact after determining that the data file corresponding to any data file identification is not received by the second information system and the current moment is within a preset first early warning time, wherein the first early warning information is used for reminding that the data file is not received, the first early warning time is determined according to the theoretical transmission time of the data file, and the theoretical transmission time of the data file is determined according to the historical time of the second information system for receiving the data file.
7. The apparatus of claim 6, wherein the processing module is further configured to: and after determining that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within a preset second early warning time, pushing second early warning information to the contact person, wherein the second early warning information is used for reminding that the data file is not received, the second early warning time is determined according to the running time of the data file, and the second early warning time is later than the first early warning time.
8. The apparatus of claim 7, wherein the processing module is further configured to: and after determining that the data file corresponding to any data file identifier is not received by the second information system and the current moment is within a preset third early warning time, pushing third early warning information to the contact, wherein the third early warning information is used for reminding that the data file is not received, the third early warning time is determined according to the running time of the data file, and the third early warning time is later than the second early warning time.
9. The apparatus according to any one of claims 6 to 8, wherein the determining module is specifically configured to periodically query files stored in the second information system according to a preset processing period and data file identifiers in the to-be-processed list, and determine whether a data file corresponding to each data file identifier is received by the second information system.
10. The apparatus according to any one of claims 6 to 8, wherein the obtaining module is specifically configured to:
acquiring all data file identifications which need to be transmitted from the first information system to the second information system and transmission time of a data file corresponding to each data file identification, wherein the transmission time comprises transmission date and transmission time;
and adding a plurality of data file identifications with transmission dates as the current dates and transmission times corresponding to the data file identifications to a pre-generated processing list to obtain the list to be processed.
11. A server, comprising: processor, memory and computer program instructions stored on the memory and executable on the processor, characterized in that the processor implements the method of pre-warning of data processing according to any of the preceding claims 1 to 5 when executing the computer program instructions.
12. A computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and when executed by a processor, the computer-executable instructions are configured to implement the data processing warning method according to any one of claims 1 to 5.
13. A computer program product comprising a computer program for implementing a warning method of data processing according to any one of claims 1 to 5 when the computer program is executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674729.7A CN113297261A (en) | 2021-06-17 | 2021-06-17 | Early warning method and device for data processing, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674729.7A CN113297261A (en) | 2021-06-17 | 2021-06-17 | Early warning method and device for data processing, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113297261A true CN113297261A (en) | 2021-08-24 |
Family
ID=77328679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110674729.7A Pending CN113297261A (en) | 2021-06-17 | 2021-06-17 | Early warning method and device for data processing, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113297261A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114255019A (en) * | 2022-03-01 | 2022-03-29 | 飞狐信息技术（天津）有限公司 | Data pushing method and device, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170318115A1 (en) * | 2016-04-29 | 2017-11-02 | Canon Information And Imaging Solutions, Inc. | Apparatus, system and method for cache management |
| CN107784481A (en) * | 2017-08-30 | 2018-03-09 | 平安科技（深圳）有限公司 | Task timeliness method for early warning and device |
| CN110991154A (en) * | 2019-12-20 | 2020-04-10 | 深圳前海环融联易信息科技服务有限公司 | Report generation method, system, computer equipment and computer readable storage medium |
| CN111245934A (en) * | 2020-01-10 | 2020-06-05 | 深圳壹账通智能科技有限公司 | Feedback method, device and equipment for file transmission and storage medium |
| CN112346931A (en) * | 2020-11-04 | 2021-02-09 | 青岛以萨数据技术有限公司 | Raspberry pie-based private network service cluster monitoring alarm system, method and medium |
-
2021
- 2021-06-17 CN CN202110674729.7A patent/CN113297261A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170318115A1 (en) * | 2016-04-29 | 2017-11-02 | Canon Information And Imaging Solutions, Inc. | Apparatus, system and method for cache management |
| CN107784481A (en) * | 2017-08-30 | 2018-03-09 | 平安科技（深圳）有限公司 | Task timeliness method for early warning and device |
| CN110991154A (en) * | 2019-12-20 | 2020-04-10 | 深圳前海环融联易信息科技服务有限公司 | Report generation method, system, computer equipment and computer readable storage medium |
| CN111245934A (en) * | 2020-01-10 | 2020-06-05 | 深圳壹账通智能科技有限公司 | Feedback method, device and equipment for file transmission and storage medium |
| CN112346931A (en) * | 2020-11-04 | 2021-02-09 | 青岛以萨数据技术有限公司 | Raspberry pie-based private network service cluster monitoring alarm system, method and medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114255019A (en) * | 2022-03-01 | 2022-03-29 | 飞狐信息技术（天津）有限公司 | Data pushing method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108512689B (en) | Micro-service monitoring method and server | |
| CN106548402B (en) | Resource transfer monitoring method and device | |
| CN112507029B (en) | Data processing system and data real-time processing method | |
| CN110362455B (en) | Data processing method and data processing device | |
| CN110503408B (en) | Business process management system based on custom configuration | |
| CN108833199A (en) | Method, apparatus, equipment and the storage medium that data report | |
| US11742075B2 (en) | Network-based systems and methods for providing readmission notifications | |
| CN113297261A (en) | Early warning method and device for data processing, server and storage medium | |
| CN111651595A (en) | Abnormal log processing method and device | |
| CN110958161A (en) | Block link point monitoring method and device | |
| CN114757639A (en) | Data processing method, device, equipment and storage medium | |
| CN113434498A (en) | Method and device for monitoring data abnormity of database of power system and electronic equipment | |
| CN108038127A (en) | A kind of method, apparatus, terminal device and the storage medium of data synchronization | |
| CN111241112A (en) | Micro-service call supervision system and method for guaranteeing idempotent | |
| CN108429636B (en) | Method and device for positioning abnormal system and electronic equipment | |
| JP6574310B2 (en) | Evaluation information matching method, apparatus and server | |
| CN112965793B (en) | Identification analysis data-oriented data warehouse task scheduling method and system | |
| CN106682997B (en) | Method and device for reminding regular inspection of engineering truck | |
| CN114518989A (en) | Abnormal data monitoring method and system | |
| CN110532253B (en) | Service analysis method, system and cluster | |
| CN114969077A (en) | Service data processing method and device | |
| CN113377604A (en) | Data processing method, device, equipment and storage medium | |
| CN117312297B (en) | Data management method and device | |
| CN106295812A (en) | Prerun regular inspection instrument censorship Notification Method and device | |
| CN111767283B (en) | Data system monitoring method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |