CN110866834A - Batch processing program execution method and system - Google Patents
Batch processing program execution method and system Download PDFInfo
- Publication number
- CN110866834A CN110866834A CN201910968970.3A CN201910968970A CN110866834A CN 110866834 A CN110866834 A CN 110866834A CN 201910968970 A CN201910968970 A CN 201910968970A CN 110866834 A CN110866834 A CN 110866834A
- Authority
- CN
- China
- Prior art keywords
- module
- batch
- batch processing
- processing
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a batch program execution method, which comprises the following steps: the method comprises the steps of decoupling a service unit into a plurality of modules with logical relations, executing batch processing operation according to a decoupling result, marking each module in the batch processing operation process with a corresponding processing state label, recording abnormal information with interruption in a log table when interruption occurs in the batch processing execution process, and starting to continuously execute the batch processing operation from the abnormal module with interruption when the abnormality is detected to be eliminated and a batch processing operation continuous execution instruction is received. By the embodiment of the invention, the batch processing efficiency can be improved, the batch processing time can be saved, and the use of downstream system data can be guaranteed.
Description
Technical Field
Embodiments of the present invention relate to the field of communications, and in particular, to a batch program execution method, a batch program execution system, a computer device, and a readable storage medium.
Background
As insurance services increase year by year, more and more data is generated by the core system. However, batch data processing and synchronization are currently implemented in a batch manner. In the daily operation and maintenance process, if the batch processing operation is in a data abnormality or system abnormality, reloading is needed after each abnormality interruption, a large amount of time and labor cost is wasted, and meanwhile, the use of the system is influenced by data loading delay.
Therefore, how to accelerate the data loading time and reduce the data loading delay when the data loading exception is encountered is an urgent technical problem to be solved.
Disclosure of Invention
In view of the above, it is desirable to provide a batch program executing method, a batch program executing system, a computer device and a readable storage medium, which can improve the batch processing efficiency, save the batch processing time and ensure the use of the downstream system data.
In order to achieve the above object, an embodiment of the present invention provides a method for executing a batch program, where the method includes:
when a batch processing instruction input by a user is monitored, batch processing operation is carried out on a service unit to be subjected to batch processing according to the batch processing instruction, wherein the service unit comprises a plurality of modules with logical relations;
monitoring a process of executing batch processing operation on the service unit according to the logical relationship among the modules, and marking each module in the executing process of the batch processing operation with a corresponding processing state label, wherein the processing state label at least comprises: processing a success tag and processing a failure tag;
when the batch processing operation is monitored to be interrupted, detecting a module interrupted in the batch processing operation process, recording the module interrupted in the batch processing operation process as an abnormal module, and recording abnormal information interrupted in a log table, wherein the abnormal information comprises: an abnormal module with abnormality in the service unit, abnormal time for the abnormal module to have abnormality and an abnormal reason for the abnormal module to have abnormality;
and when the exception reason is eliminated and an instruction for continuing to execute the batch processing operation is received, continuing to execute the batch processing operation from the exception module according to the processing failure tag.
Optionally, before the step of performing, according to the batch processing instruction, batch processing operation on the service units to be batch processed when the batch processing instruction input by the user is monitored, the method further includes:
and decoupling the service units into the plurality of modules with the logical relationship so as to split the service units according to a preset rule, wherein the preset rule at least comprises the functions of the service units and the service scenes applied by the service units.
Optionally, the step of monitoring an execution process of performing a batch processing operation on the service unit according to the logical relationship among the modules further includes:
recording the plurality of modules and the processing time of the plurality of modules in a log table;
establishing a batch processing time point table according to the log table so as to record each module in the service unit and the processing time corresponding to each module; and
and storing the batch processing time point table in a database.
Optionally, the step of, when it is detected that the cause of the abnormality is eliminated and an instruction to continue to execute the batch processing operation is received, continuing to execute the batch processing operation from the abnormality module according to the processing failure tag further includes:
analyzing the processing failure label and the time;
positioning the abnormal module according to the analysis result; and
continuing to execute the batch operation from the exception module.
Optionally, after the step of continuing to execute the batch processing operation from the exception module according to the processing failure tag when it is detected that the exception cause is excluded and an instruction to continue to execute the batch processing operation is received, the method further includes the step of:
monitoring the execution process of the abnormal module; and
and when the exception module executes normally, updating the processing state tag and the batch processing time point table.
In order to achieve the above object, an embodiment of the present invention further provides an execution system of a batch program, including:
the system comprises an execution module, a processing module and a processing module, wherein the execution module is used for executing batch processing operation on a service unit to be subjected to batch processing according to a batch processing instruction when the batch processing instruction input by a user is monitored, and the service unit comprises a plurality of modules with logical relations;
the monitoring module is used for monitoring the process of executing batch processing operation on the service units according to the logical relationship among the modules;
a tag module, configured to mark a corresponding processing status tag on each module in an execution process of the batch processing operation, where the processing status tag at least includes: processing a success tag and processing a failure tag;
a recording module, configured to detect, when it is monitored that the batch processing operation is interrupted, a module that is interrupted in the process of the batch processing operation, record the module as an abnormal module, and record abnormal information that is interrupted in a log table, where the abnormal information includes: an abnormal module with abnormality in the service unit, abnormal time for the abnormal module to have abnormality and an abnormal reason for the abnormal module to have abnormality;
the execution module is further configured to, when it is detected that the exception cause is eliminated and an instruction to continue to execute the batch processing operation is received, continue to execute the batch processing operation from the exception module according to the processing failure tag.
Optionally, the batch program execution system further includes a decoupling module, configured to decouple the service unit into the plurality of modules having the logical relationship, so as to split the service unit according to a preset rule, where the preset rule at least includes a function of the service unit and a service scenario applied by the service unit.
Optionally, the monitoring module further comprises:
the recording unit is used for recording the plurality of modules and the processing time of the plurality of modules in a log table;
the establishing unit is used for establishing a batch processing time point table according to the log table so as to record each module in the service unit and the processing time corresponding to each module; and
and the storage unit is used for storing the batch processing time point table in a database.
Optionally, the execution module further includes:
the analysis unit is used for analyzing the processing failure label and the time;
the positioning unit is used for positioning the abnormal module according to the analysis result; and
and the execution unit is also used for continuously executing the batch processing operation from the exception module.
Optionally, the monitoring module is further configured to monitor an execution process of the exception module.
Optionally, the execution system of the batch program further includes an updating module, configured to update the processing status tag and the batch processing time point table when the exception module executes normally.
To achieve the above object, an embodiment of the present invention further provides a computer device, a memory of the computer device, a processor, and a computer program stored on the memory and executable on the processor, where the computer program, when executed by the processor, implements the steps of the execution method of the batch program as described above.
To achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is executable by at least one processor to cause the at least one processor to execute the steps of the batch processing program execution method described above.
According to the batch processing program executing method, the batch processing program executing system, the computer device and the readable storage medium provided by the embodiment of the invention, the service units are decoupled into the modules with the logical relation, the batch processing operation is executed according to the decoupling result, the corresponding processing state labels are marked on the modules in the batch processing operation process, when the batch processing process is executed, the abnormal information with the interruption is recorded in the log table, and when the abnormality is detected to be eliminated and the batch processing operation command is received and continuously executed, the batch processing operation is continuously executed from the abnormal module with the interruption. By the embodiment of the invention, when the batch processing is abnormal, the batch processing can be directly continuously executed from the module at the abnormal time point, the phenomenon that the batch processing needs to run from the beginning after each abnormal interruption is avoided, the batch processing efficiency is greatly improved, the batch processing time is saved, and the use of downstream system data is ensured.
Drawings
FIG. 1 is a flowchart illustrating a method for performing a batch process according to an embodiment of the present invention.
Fig. 2 is a schematic hardware architecture diagram of a computer device according to a second embodiment of the present invention.
FIG. 3 is a block diagram of a third embodiment of a batch process execution system.
Reference numerals:
| |
2 |
| Memory device | 21 |
| Processor with a memory having a plurality of memory cells | 22 |
| Network interface | 23 |
| Batch processing |
20 |
| Execution module | 201 |
| Monitoring module | 202 |
| Label module | 203 |
| Recording module | 204 |
| Decoupling module | 205 |
| Updating module | 206 |
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Example one
Referring to fig. 1, a flowchart illustrating steps of a method for performing a batch process according to a first embodiment of the invention is shown. It is to be understood that the flow charts in the embodiments of the present method are not intended to limit the order in which the steps are performed. It should be noted that the present embodiment is exemplarily described with the computer device 2 as an execution subject. The method comprises the following specific steps:
step S100, when a batch processing instruction input by a user is monitored, batch processing operation is carried out on a service unit to be batch processed according to the batch processing instruction, wherein the service unit comprises a plurality of modules with logical relations.
Illustratively, when the vehicle insurance claim system data needs to be batched, the batch processing operation is performed on the vehicle insurance claim system data to be batched.
Batch processing refers to a procedure unit that combines calls of a plurality of batch processing services.
In another preferred embodiment, when a batch processing instruction input by a user is monitored, before a batch processing operation is performed on a service unit to be batch processed according to the batch processing instruction, the service unit is further decoupled into the plurality of modules having the logical relationship so as to split the service unit according to a preset rule, where the preset rule at least includes a function of the service unit and a service scenario applied by the service unit.
Illustratively, the data of the car insurance claim settlement system is imported to a big data platform in a batch loading mode. The vehicle insurance claim settlement system data is decomposed into a plurality of unit modules with logical relations according to the functions of the business units: the system comprises an underwriting module, a case reporting module, a surveying module, a loss assessment module, a document collection module, a calculation module, a price checking module, a case settlement module and a payment module. And then, further decoupling the unit modules according to the service scenes of the unit modules. For example, the reporting module is decomposed into a reported non-reporting module, a reported logged-off module, etc. Of course, the same service scene of the same unit module can be divided according to different contractors, so as to further divide the unit module to obtain the module with the smallest unit.
Step S102, monitoring the execution process of the batch processing operation executed by the service unit according to the logical relationship among the modules, and marking each module in the execution process of the batch processing operation with a corresponding processing state label, wherein the processing state label at least comprises: a process success label and a process failure label.
Illustratively, during execution of a batch operation, it is monitored whether the batch process is executing normally, e.g., whether an interrupt occurs to the batch process. When the batch processing program is normally executed until a certain module is interrupted, the normally executed modules are marked with processing success labels, and the module is marked with processing failure labels.
In another preferred embodiment, when monitoring the execution process of the batch processing operation executed by the service unit according to the logical relationship among the modules, the method further records the modules and the processing time of the modules in a log table, then establishes a batch processing time point table according to the log table to record each module and the processing time in the service unit, and finally stores the batch processing time point table in a database.
Illustratively, if the processing loss assessment module is executed at time t1, the processing reported case module is executed at time t2, and the processing document collection module is executed at time t3, the processing loss assessment module at time t1, the processing reported case module at time t2, and the processing document collection module at time t3 are recorded in the log table, and then a batch processing time point table is established according to the log table and saved in the database.
It should be noted that, before the batch processing is executed, since the batch processing is not started yet, all modules in the log table are in the to-be-executed state, and thus no time information exists.
In another preferred embodiment, a batch processing time point table may also be directly established according to the service unit and the execution states corresponding to the modules of the service unit, so as to record the processing states of the modules in the service unit.
Step S104, when it is monitored that the batch processing operation is interrupted, detecting a module interrupted in the batch processing operation process, recording the module interrupted in the batch processing operation process as an abnormal module, and recording abnormal information of the interruption in a log table, wherein the abnormal information comprises: the abnormal module with abnormality in the service unit, the abnormal time of the abnormal module with abnormality and the abnormal reason of the abnormal module with abnormality.
Illustratively, when an interruption occurs when the batch processing program is monitored to execute to a reported and filed module, the module which completes processing is recorded as a normal module, the reported and filed module is an abnormal module, and a subsequent module after the abnormal module is processed, and the normal module, the abnormal module and the subsequent module are saved in a log table.
In another preferred embodiment, the normal module with completed processing, the abnormal module with abnormal processing and the subsequent modules after the abnormal module can also be directly recorded in the batch processing time point table.
And step S106, when the exception reason is eliminated and an instruction for continuously executing the batch processing operation is received, continuously executing the batch processing operation from the exception module according to the processing failure tag.
Specifically, when the user returns the abnormal module to normal according to the abnormal reason, detects that the abnormal reason is eliminated, and receives an instruction for continuing to execute the batch processing operation, if the module with the processing failure tag is the reported and filed module, the batch processing operation is continued to be executed from the reported and filed module.
In addition, when the batch job is stopped in an abnormal situation, the batch job is restarted after the problem is eliminated, and the batch job is continuously processed from the position where the previous error occurs.
In another preferred embodiment, when it is detected that the reason for the abnormality is eliminated and an instruction to continue to execute the batch processing operation is received, the processing failure tag and the time are analyzed, the abnormality module is located according to an analysis result, and then the batch processing operation is continued to be executed from the abnormality module.
For example, when the reported case module is marked with the processing failure tag and the exception time T1, the processing failure tag and the time T1 are parsed, the case module reported by the exception module is located, and finally the batch processing operation is continued from the case module reported by the case module.
In another preferred embodiment, when it is detected that the exception cause is excluded and an instruction to continue to execute the batch processing operation is received, after the batch processing operation is continued to be executed from the exception module according to the processing failure tag, the execution process of the exception module is monitored, and when the exception module is normally executed, the processing state tag and the batch processing time point table are updated.
Illustratively, when the exception reason is detected to be eliminated and a continuous execution instruction input by a user is received, the execution is started according to the analyzed exception module reported case module with exception, when the execution of the reported case module is successful, the processing failure tag is updated to the processing success tag, and the time of the reported case module in the batch processing time point table is updated to the current time.
By the embodiment of the invention, when the batch processing is abnormal, the batch processing can be directly continuously executed from the module at the abnormal time point, the phenomenon that the batch processing needs to run from the beginning after each abnormal interruption is avoided, the batch processing efficiency is greatly improved, the batch processing time is saved, and the use of downstream system data is ensured.
Example two
Referring to fig. 2, a hardware architecture diagram of a computer device according to a second embodiment of the invention is shown. The computer device 2 includes, but is not limited to, a memory 21, a processor 22, and a network interface 23 communicatively coupled to each other via a system bus, and fig. 2 illustrates only the computer device 2 having components 21-23, but it is to be understood that not all of the illustrated components are required and that more or fewer components may alternatively be implemented.
The memory 21 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the computer device 2, such as a hard disk or a memory of the computer device 2. In other embodiments, the memory may also be an external storage device of the computer device 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a flash Card (FlashCard), and the like, provided on the computer device 2. Of course, the memory 21 may also comprise both an internal storage unit of the computer device 2 and an external storage device thereof. In this embodiment, the memory 21 is generally used for storing an operating system installed in the computer device 2 and various types of application software, such as program codes of the execution system 20 of the batch program. Further, the memory 21 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 22 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 22 is typically used to control the overall operation of the computer device 2. In this embodiment, the processor 22 is configured to run the program code or the processing data stored in the memory 21, for example, the execution system 20 running the batch program.
The network interface 23 may comprise a wireless network interface or a wired network interface, and the network interface 23 is generally used for establishing communication connection between the computer device 2 and other electronic devices. For example, the network interface 23 is used to connect the computer device 2 to an external terminal through a network, establish a data transmission channel and a communication connection between the computer device 2 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, and the like.
EXAMPLE III
Referring to fig. 3, a schematic diagram of program modules of an execution system of a batch process according to a third embodiment of the invention is shown. In the embodiment, the execution system 20 of the batch program may include or be divided into one or more program modules, and the one or more program modules are stored in the storage medium and executed by one or more processors to implement the present invention and implement the execution method of the batch program. The program modules referred to in the embodiments of the present invention refer to a series of computer program instruction segments capable of performing specific functions, and are more suitable than the programs themselves for describing the execution process of the batch program execution system 20 in the storage medium. The following description will specifically describe the functions of the program modules of the present embodiment:
the execution module 201 is configured to, when a batch processing instruction input by a user is monitored, execute a batch processing operation on a service unit to be batch processed according to the batch processing instruction, where the service unit includes a plurality of modules having a logical relationship.
Illustratively, when the vehicle insurance claim system data needs to be batched, the execution module 201 executes a batch processing operation on the vehicle insurance claim system data to be batched.
Batch processing refers to a procedure unit that combines calls of a plurality of batch processing services.
And the monitoring module 202 is configured to monitor an execution process of performing a batch processing operation on the service unit according to the logical relationship among the modules.
A tag module 203, configured to mark a corresponding processing status tag on each module in the execution process of the batch processing operation, where the processing status tag at least includes: a process success label and a process failure label.
Illustratively, the monitoring module 202 monitors whether a batch process is executing normally, e.g., whether an interrupt occurs during execution of a batch operation. When the batch processing program is normally executed until a certain module is interrupted, the tag module 203 marks all the normally executed modules with processing success tags, and marks the modules with interruption with processing failure tags.
In another preferred embodiment, the monitoring module 202 further includes a recording unit, an establishing unit and a storing unit. The recording unit is used for recording the service units and the processing time in a log table when monitoring the execution process of the batch processing of the service units. The establishing unit is used for establishing a batch processing time point table according to the log table so as to record each module and processing time in the service unit. The storage unit is used for storing the batch processing time point table in a database.
For example, if the execution module 201 executes the processing damage assessment module at time t1, the processing reported case module at time t2, and the processing document collection module at time t3, the recording unit records the processing damage assessment module at time t1, the processing reported case module at time t2, and the processing document collection module at time t3 in the log table, the establishing unit establishes the batch processing time point table according to the log table, and the storing unit stores the batch processing time point table in the database.
It should be noted that, before the batch processing is executed, since the batch processing is not started yet, all modules in the log table are in the to-be-executed state, and thus no time information exists.
In another preferred embodiment, a batch processing time point table may also be directly established according to the service unit and the execution states corresponding to the modules of the service unit, so as to record the processing states of the modules in the service unit.
A recording module 204, configured to detect, when it is monitored that the batch processing operation is interrupted, a module interrupted in the process of the batch processing operation, record the module as an abnormal module, and record abnormal information of the interruption in a log table, where the abnormal information includes: the abnormal module with abnormality in the service unit, the abnormal time of the abnormal module with abnormality and the abnormal reason of the abnormal module with abnormality.
Illustratively, when it is monitored that an interrupt occurs when the batch processing program executes to a reported and filed module, the recording module 204 records the module that has completed processing as a normal module, the reported and filed module as an abnormal module, and a subsequent module after the processed abnormal module, and stores the normal module, the abnormal module, and the subsequent module in a log table.
In another preferred embodiment, the normal module with completed processing, the abnormal module with abnormal processing and the subsequent modules after the abnormal module can also be directly recorded in the batch processing time point table.
The execution module 201 is further configured to, when it is detected that the exception cause is eliminated and an execution instruction is received, continue to execute the batch processing operation from the exception module according to the processing failure tag.
Specifically, when the user returns the abnormal module to normal according to the abnormal reason, detects that the abnormal reason is eliminated, and receives an instruction to continue to execute the batch processing operation, if the module with the processing failure tag is the reported and filed module, the execution module 201 continues to execute the batch processing operation from the reported and filed module.
In addition, when the batch job is stopped in an abnormal situation, the batch job is restarted after the problem is eliminated, and the batch job is continuously processed from the position where the previous error occurs.
In another preferred embodiment, the execution module 201 further includes a parsing unit, a positioning unit and an execution unit. The analysis unit is used for analyzing the processing failure label and the time when receiving an execution instruction. The positioning unit is used for positioning the abnormal module according to the analysis result. The execution unit is used for continuing to execute the batch processing operation from the exception module.
For example, when the reported case module is marked with a processing failure tag and an abnormal time T1, the parsing unit parses the processing failure tag and the time T1, the locating unit locates the abnormal module reported case module, and the execution unit continues to execute the batch processing operation from the reported case module.
A decoupling module 205, configured to decouple the service unit into the plurality of modules having the logical relationship, so as to split the service unit according to a preset rule, where the preset rule at least includes a function of the service unit and a service scenario applied by the service unit.
Illustratively, the data of the car insurance claim system is imported into the big data platform in a batch loading manner, and the decoupling module 205 decomposes the data of the car insurance claim system into a plurality of unit modules with logical relations according to the functions of the business units: the system comprises an underwriting module, a case reporting module, a surveying module, a loss assessment module, a document collection module, a calculation module, a price checking module, a case settlement module and a payment module. And then, further decoupling the unit modules according to the service scenes of the unit modules. For example, the reporting module is decomposed into a reported non-reporting module, a reported logged-off module, etc. Of course, the same service scene of the same unit module can be divided according to different contractors, so as to further divide the unit module to obtain the module with the smallest unit.
The monitoring module 202 is further configured to monitor an execution process of the exception module when it is detected that the exception cause is eliminated and an instruction for continuing to execute the batch processing operation is received.
An updating module 206, configured to update the processing status tag and the batch processing time point table when the exception module executes normally.
Illustratively, when the exception reason is detected to be eliminated and a continuous execution instruction input by a user is received, the execution is started according to the analyzed exception module reported case module with exception, when the execution of the reported case module is successful, the processing failure tag is updated to the processing success tag, and the time of the reported case module in the batch processing time point table is updated to the current time.
By the embodiment of the invention, when the batch processing is abnormal, the batch processing can be directly continuously executed from the module at the abnormal time point, the phenomenon that the batch processing needs to run from the beginning after each abnormal interruption is avoided, the batch processing efficiency is greatly improved, the batch processing time is saved, and the use of downstream system data is ensured.
The present invention also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) capable of executing programs, and the like. The computer device of the embodiment at least includes but is not limited to: memory, processor, etc. communicatively coupled to each other via a system bus.
The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer-readable storage medium of the embodiment is used for storing the batch program execution system 20, and when being executed by the processor, the batch program execution method of the first embodiment is implemented.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for executing a batch process, comprising:
when a batch processing instruction input by a user is monitored, batch processing operation is carried out on a service unit to be subjected to batch processing according to the batch processing instruction, wherein the service unit comprises a plurality of modules with logical relations;
monitoring a process of executing batch processing operation on the service unit according to the logical relationship among the modules, and marking each module in the executing process of the batch processing operation with a corresponding processing state label, wherein the processing state label at least comprises: processing a success tag and processing a failure tag;
when the batch processing operation is monitored to be interrupted, detecting a module interrupted in the batch processing operation process, recording the module interrupted in the batch processing operation process as an abnormal module, and recording abnormal information interrupted in a log table, wherein the abnormal information comprises: an abnormal module with abnormality in the service unit, abnormal time for the abnormal module to have abnormality and an abnormal reason for the abnormal module to have abnormality;
and when the exception reason is eliminated and an instruction for continuing to execute the batch processing operation is received, continuing to execute the batch processing operation from the exception module according to the processing failure tag.
2. The method for executing a batch program according to claim 1, wherein when a batch instruction input by a user is monitored, before the step of executing a batch operation on the service units to be batch-processed according to the batch instruction, the method further comprises:
and decoupling the service units into the plurality of modules with the logical relationship so as to split the service units according to a preset rule, wherein the preset rule at least comprises the functions of the service units and the service scenes applied by the service units.
3. The method of batch process execution according to claim 1, wherein said step of monitoring a process of performing a batch operation on said business unit in a logical relationship between said plurality of modules further comprises:
recording the plurality of modules and the processing time of the plurality of modules in a log table;
establishing a batch processing time point table according to the log table so as to record each module in the service unit and the processing time corresponding to each module; and
and storing the batch processing time point table in a database.
4. The batch program executing method according to claim 1, wherein the step of continuing to execute the batch operation from the exception module according to the processing failure tag when the exception cause is detected to be excluded and an instruction to continue to execute the batch operation is received, further comprises:
analyzing the processing failure label and the time;
positioning the abnormal module according to the analysis result; and
continuing to execute the batch operation from the exception module.
5. The batch program executing method according to claim 1, wherein, when it is detected that the cause of the abnormality is excluded and an instruction to continue executing the batch operation is received, after the step of continuing executing the batch operation from the abnormality module according to the processing failure tag, further comprising the step of:
monitoring the execution process of the abnormal module; and
and when the exception module executes normally, updating the processing state tag and the batch processing time point table.
6. A system for executing a batch process, comprising:
the system comprises an execution module, a processing module and a processing module, wherein the execution module is used for executing batch processing operation on a service unit to be subjected to batch processing according to a batch processing instruction when the batch processing instruction input by a user is monitored, and the service unit comprises a plurality of modules with logical relations;
the monitoring module is used for monitoring the process of executing batch processing operation on the service units according to the logical relationship among the modules;
a tag module, configured to mark a corresponding processing status tag on each module in an execution process of the batch processing operation, where the processing status tag at least includes: processing a success tag and processing a failure tag;
a recording module, configured to detect, when it is monitored that the batch processing operation is interrupted, a module that is interrupted in the process of the batch processing operation, record the module as an abnormal module, and record abnormal information that is interrupted in a log table, where the abnormal information includes: an abnormal module with abnormality in the service unit, abnormal time for the abnormal module to have abnormality and an abnormal reason for the abnormal module to have abnormality;
and the execution module is used for continuously executing the batch processing operation from the exception module according to the processing failure tag when the exception reason is detected to be eliminated and an instruction for continuously executing the batch processing operation is received.
7. The batch process execution system of claim 6, wherein the monitoring module further comprises:
the recording unit is used for recording the plurality of modules and the processing time of the plurality of modules in a log table;
the establishing unit is used for establishing a batch processing time point table according to the log table so as to record each module in the service unit and the processing time corresponding to each module; and
and the storage unit is used for storing the batch processing time point table in a database.
8. The batch program execution system of claim 6, wherein the execution module further comprises:
the analysis unit is used for analyzing the processing failure label and the time;
the positioning unit is used for positioning the abnormal module according to the analysis result; and
and the execution unit is also used for continuously executing the batch processing operation from the exception module.
9. A computer arrangement, characterized by a computer arrangement memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the method of execution of a batch program according to any one of claims 1-5.
10. A computer-readable storage medium, in which a computer program is stored which is executable by at least one processor to cause the at least one processor to perform the steps of the batch program execution method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910968970.3A CN110866834B (en) | 2019-10-12 | 2019-10-12 | Execution method and system of batch processing program |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910968970.3A CN110866834B (en) | 2019-10-12 | 2019-10-12 | Execution method and system of batch processing program |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110866834A true CN110866834A (en) | 2020-03-06 |
| CN110866834B CN110866834B (en) | 2023-08-18 |
Family
ID=69652698
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910968970.3A Active CN110866834B (en) | 2019-10-12 | 2019-10-12 | Execution method and system of batch processing program |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110866834B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767163A (en) * | 2020-06-17 | 2020-10-13 | 北京思特奇信息技术股份有限公司 | Method and system for avoiding downtime of application server caused by excessive transaction concurrency |
| CN112363860A (en) * | 2020-11-11 | 2021-02-12 | 中国建设银行股份有限公司 | Batch processing operation abnormal interruption detection method and device |
| CN112882807A (en) * | 2021-02-04 | 2021-06-01 | 永辉云金科技有限公司 | Breakpoint re-running batch processing system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1884850A1 (en) * | 2006-07-27 | 2008-02-06 | ABB Research Ltd. | System for analyzing batch processes |
| JP2010176303A (en) * | 2009-01-28 | 2010-08-12 | Nippon Yunishisu Kk | Batch processing system, information terminal apparatus for use in the same, and method for recovering batch processing |
| CN104317960A (en) * | 2014-11-14 | 2015-01-28 | 中国建设银行股份有限公司 | Method and device for processing batch job |
| CN104407922A (en) * | 2014-10-29 | 2015-03-11 | 中国建设银行股份有限公司 | Asynchronous batch-processing dispatching method and system |
| CN109669766A (en) * | 2018-09-11 | 2019-04-23 | 深圳平安财富宝投资咨询有限公司 | Processing method, device, equipment and the storage medium of batch processing job |
-
2019
- 2019-10-12 CN CN201910968970.3A patent/CN110866834B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1884850A1 (en) * | 2006-07-27 | 2008-02-06 | ABB Research Ltd. | System for analyzing batch processes |
| JP2010176303A (en) * | 2009-01-28 | 2010-08-12 | Nippon Yunishisu Kk | Batch processing system, information terminal apparatus for use in the same, and method for recovering batch processing |
| CN104407922A (en) * | 2014-10-29 | 2015-03-11 | 中国建设银行股份有限公司 | Asynchronous batch-processing dispatching method and system |
| CN104317960A (en) * | 2014-11-14 | 2015-01-28 | 中国建设银行股份有限公司 | Method and device for processing batch job |
| CN109669766A (en) * | 2018-09-11 | 2019-04-23 | 深圳平安财富宝投资咨询有限公司 | Processing method, device, equipment and the storage medium of batch processing job |
Non-Patent Citations (1)
| Title |
|---|
| 陈亚;: "基于中间件技术的数据批处理系统的设计与实现", 电脑知识与技术, no. 34, pages 45 - 46 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767163A (en) * | 2020-06-17 | 2020-10-13 | 北京思特奇信息技术股份有限公司 | Method and system for avoiding downtime of application server caused by excessive transaction concurrency |
| CN112363860A (en) * | 2020-11-11 | 2021-02-12 | 中国建设银行股份有限公司 | Batch processing operation abnormal interruption detection method and device |
| CN112882807A (en) * | 2021-02-04 | 2021-06-01 | 永辉云金科技有限公司 | Breakpoint re-running batch processing system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110866834B (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107844634B (en) | Modeling method of multivariate general model platform, electronic equipment and computer readable storage medium | |
| CN110866834B (en) | Execution method and system of batch processing program | |
| CN108491301B (en) | Electronic device, abnormality early warning method based on redis and storage medium | |
| CN108416485B (en) | User identity recognition method, electronic device and computer readable storage medium | |
| CN108170551B (en) | Crawler system based front-end and back-end error processing method, server and storage medium | |
| CN110704521A (en) | Interface data access method and system | |
| CN110599354B (en) | Online checking method, online checking system, computer device and computer readable storage medium | |
| CN110222535B (en) | Processing device, method and storage medium for block chain configuration file | |
| CN107682426B (en) | Interface proxy method and application server | |
| CN108415925B (en) | Electronic device, data call log generation and query method and storage medium | |
| CN110069925B (en) | Software monitoring method, system and computer readable storage medium | |
| WO2017219574A1 (en) | Version rollback method and system for base station | |
| CN108509322B (en) | Method for avoiding excessive return visit, electronic device and computer readable storage medium | |
| CN112001707A (en) | Business workflow generation method and system based on business data | |
| CN112559155A (en) | Batch job processing method, system, computer device and readable storage medium | |
| CN112559363A (en) | Test case generation method, system, equipment and medium based on full link marking | |
| CN111400085A (en) | Transaction rollback exception processing method and device, computer equipment and storage medium | |
| CN115729946B (en) | Method, device, equipment and storage medium for processing distributed transaction | |
| CN110704462A (en) | Data change notification method, electronic device, computer device, and storage medium | |
| CN111580894A (en) | Data analysis early warning method, device, computer system and readable storage medium | |
| CN111984484A (en) | Non-invasive micro-service access method and system | |
| CN114490196A (en) | Database switching method, system, device and medium | |
| CN111813600B (en) | Method, device, terminal and medium for recovering controller | |
| CN111488236B (en) | Order exception processing method, server, storage medium and processing device | |
| CN110825771B (en) | Batch data processing method, electronic device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |