CN115794347A

CN115794347A - Task processing method and device, computer equipment and storage medium

Info

Publication number: CN115794347A
Application number: CN202211560624.XA
Authority: CN
Inventors: 许浩奇
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2022-12-07
Filing date: 2022-12-07
Publication date: 2023-03-14

Abstract

The embodiment of the application belongs to the field of artificial intelligence and relates to a task processing method, which comprises the following steps: judging whether a fault event corresponding to the batch running task on the first appointed day exists in the target application at present; wherein the first specified day is the previous day separated from the current day; if so, switching the data source currently used by the target application to a preset target data source; acquiring target summarized data corresponding to a first appointed day from a target data source; generating a target batch running task corresponding to the target application based on the target summarized data; and running the target batch running task and generating result data corresponding to the target batch running task. The application also provides a task processing device, computer equipment and a storage medium. In addition, the present application also relates to block chain techniques, and the resulting data may be stored in a block chain. The batch running task processing method and the batch running task processing device can quickly recover batch running task processing of the target application, are favorable for improving the processing efficiency of the batch running task, and guarantee the successful operation of the batch running task.

Description

Task processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a task processing method and apparatus, a computer device, and a storage medium.

Background

In the financial field, batch processing, namely large data batch processing, is part of business which is unavailable in bank operation, each insurance company adopts a self batch program, each insurance company has the self batch program and the batch flow, and the business income and expenditure of the current month are generated in the accounts opened by a bank through an enterprise to form a system general ledger in batches, carry out mass transactions such as settlement, reference and payment and the like, and form a system statement in batches.

At present, batch running tasks adopted by most insurance companies are generally operation execution methods based on Hadoop MapReduce, the execution time of a single task is generally more than 30 minutes, and in specific applications, processing is completed by importing source-attached business data into a terminal index result, so that the number of layers is large, links are long, and 2-3 hours are often needed. Once a task failure occurs, the whole link needs to be rerun again, the task processing mode is long in time consumption and has uncertain factors, and the risk that the rerun task cannot be recovered even after multiple times of rerun tasks are executed is possibly caused.

Disclosure of Invention

The embodiment of the application aims to provide a task processing method, a task processing device, a computer device and a storage medium, so as to solve the technical problems that once a task fault occurs in an existing batch running task, the whole link needs to be rerun again, the task processing mode is long in time consumption, uncertain factors exist, and risks cannot be recovered even if repeated rerun tasks are executed possibly.

In order to solve the above technical problem, an embodiment of the present application provides a task processing method, which adopts the following technical solutions:

judging whether a fault event corresponding to the batch running task on the first appointed day exists in the target application at present; wherein the first specified day is the previous day separated from the current day;

if so, switching the data source currently used by the target application to a preset target data source;

acquiring target summarized data corresponding to the first appointed day from the target data source;

generating a target batch running task corresponding to the target application based on the target summary data;

and running the target batch running task, and generating result data corresponding to the target batch running task.

Further, before the step of switching the data source currently used by the target application to a preset target data source, the method further includes:

acquiring real-time service data of the first appointed day;

processing the index of the first appointed day based on the real-time service data in a preset time period to obtain real-time summarized data of the first appointed day;

acquiring first summarized data corresponding to a second appointed day; wherein the second designated day is the previous day spaced from the first designated day;

generating second summary data corresponding to the first designated day based on the real-time summary data and the first summary data;

and taking the second summarized data as the target summarized data, and storing the target summarized data into the target data source.

Further, the step of acquiring the real-time service data of the first specified day specifically includes:

acquiring an archive log corresponding to the target application from a preset source database;

inquiring target log data corresponding to the first appointed day from the filing log;

and extracting the real-time service data of the first appointed day from the target log data.

Further, after the step of querying the archived log for the target log data corresponding to the first specified day, the method further includes:

calling a preset abnormal prediction model;

inputting the target log data into the abnormity prediction model, and carrying out abnormity analysis on the target log data through the abnormity prediction model to generate an abnormity analysis result corresponding to the target log data;

acquiring communication information of a target user;

and sending the abnormal analysis result to the communication equipment of the target user based on the communication information.

Further, before the step of calling the preset abnormality prediction model, the method further includes:

acquiring preset sample data;

dividing the sample data into training sample data and test sample data according to a preset proportion; the training sample data comprises a plurality of log sample data and a category label corresponding to the log sample data;

inputting log sample data in the training sample data as a model, outputting a class label in the training sample data as a model, and training a preset machine learning model to obtain a trained machine learning model;

testing the trained machine learning model based on the test sample data;

and if the trained machine learning model passes the test, taking the trained machine learning model as the abnormal prediction model.

Further, the step of processing the index of the first specified day based on the real-time service data to obtain the real-time summary data of the first specified day specifically includes:

acquiring a preset index statistical rule;

calling a preset rule engine;

using the rule engine to perform index statistical processing on the real-time service data according to the index statistical rule to obtain a corresponding index statistical result;

and taking the index statistical result as the real-time summarized data.

Further, the step of generating second summarized data corresponding to the first specified day based on the real-time summarized data and the first summarized data specifically includes:

calling a preset index data calculation formula;

calculating the real-time summarized data and the first summarized data based on the index data calculation formula to obtain corresponding calculated data;

taking the calculated data as the second summary data.

In order to solve the above technical problem, an embodiment of the present application further provides a task processing device, which adopts the following technical solutions:

the judging module is used for judging whether a fault event corresponding to the batch running task on the first appointed day exists in the target application at present; wherein the first specified day is the previous day separated from the current day date;

the switching module is used for switching the data source currently used by the target application to a preset target data source if the data source is used;

a first obtaining module, configured to obtain, from the target data source, target summarized data corresponding to the first specified day;

a first generation module, configured to generate a target batch task corresponding to the target application based on the target summary data;

and the second generation module is used for operating the target batch running task and generating result data corresponding to the target batch running task.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:

In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:

Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:

when detecting whether a fault event corresponding to a batch task on a first appointed day exists in a target application at present, the embodiment of the application can switch a data source currently used by the target application to a preset target data source, then obtain target summarized data corresponding to the first appointed day from the target data source, subsequently generate a target batch task corresponding to the target application based on the target summarized data, finally run the target batch task, and generate result data corresponding to the target batch task. According to the method and the device, when the running batch task currently executed by the target application fails, the data source currently used by the target application is switched to the pre-calculated target summarized data target data source required by the running batch task on the same day, so that the target summarized data on the first appointed day can be rapidly stored by switching the data source currently used by the target application when the target application has a task fault, the running batch task processing of the target application can be rapidly recovered, the processing efficiency and the processing intelligence of the running batch task can be improved, and the successful operation of the running batch task is ensured.

Drawings

In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.

FIG. 1 is an exemplary system architecture diagram to which the present application may be applied;

FIG. 2 is a flow diagram of one embodiment of a task processing method according to the present application;

FIG. 3 is a schematic block diagram of one embodiment of a task processing device according to the present application;

FIG. 4 is a schematic block diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof in the description and claims of this application and the description of the figures above, are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. Network 104 is the medium used to provide communication links between

terminal devices

101, 102, 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the

terminal devices

101, 102, 103.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Mov I ng P I characters Experts G roup Aud I o Layer I, motion picture Experts compression standard audio Layer 3), an MP4 (Mov I ng P I characters Experts G roup Aud I o Layer I V, motion picture Experts compression standard audio Layer 4) player, a laptop portable computer, a desktop computer, and the like.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the

terminal devices

101, 102, 103.

It should be noted that the task processing method provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the task processing apparatus is generally disposed in the server/terminal device.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.

With continuing reference to FIG. 2, a flowchart of one embodiment of a task processing method according to the present application is shown. The task processing method comprises the following steps:

step S201, determining whether the target application currently has a fault event corresponding to the batching task on the first specified day.

In this embodiment, the electronic device (for example, the server/terminal device shown in fig. 1) on which the task processing method operates may acquire the fault event through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection manners may include, but are not limited to, 3G/4G/5G connection, wi-fi connection, bluetooth connection, wi-MAX connection, Z i gbee connection, UWB (u l t ra W i deband) connection, and other wireless connection manners now known or developed in the future. The fault event refers to an event that the running batch task on the first specified day currently executed by the target application fails. Wherein the first specified day is the previous day separated from the current day date. The batch running task can specifically refer to an H iv batch running task, and the batch running task is a Hadoop MapReduce-based operation execution method.

And S202, if so, switching the data source currently used by the target application to a preset target data source.

In this embodiment, the target data source is a pre-constructed data source storing summarized data of all dates before the current date.

Step S203, acquiring target summarized data corresponding to the first specified day from the target data source.

In this embodiment, date information corresponding to the first specified day may be acquired, and target summary data corresponding to the first specified day may be acquired from the target data source based on the date information.

And step S204, generating a target batch task corresponding to the target application based on the target summary data.

In this embodiment, a pre-constructed batch task template may be obtained, and then target summary data is input into the batch task template to generate a target batch task corresponding to the target application. The batch task template can be a template which is pre-constructed according to actual service use requirements and is applied to batch tasks.

And step S205, running the target batch running task and generating result data corresponding to the target batch running task.

In this embodiment, by running the target batching task, result data corresponding to the target batching task may be generated. And if the running of the target batch running task does not have a fault, generating corresponding result data after the target batch running task is executed. In addition, the generated result data can be stored in a node of a block chain, so as to ensure the privacy and the security of the result data.

When detecting whether a fault event corresponding to a running batch task on a first appointed day exists in a target application at present, the method can switch a data source currently used by the target application to a preset target data source, then obtains target summary data corresponding to the first appointed day from the target data source, subsequently generates a target running batch task corresponding to the target application based on the target summary data, finally runs the target running batch task, and generates result data corresponding to the target running batch task. According to the method and the device, when the running batch task currently executed by the target application fails, the data source currently used by the target application is switched to the pre-calculated target summarized data target data source required by the running batch task on the same day, so that the target summarized data on the first appointed day can be stored in the data source currently used by the target application in a switching mode quickly when the target application has a task fault, the running batch task processing of the target application can be recovered quickly, the processing efficiency and the processing intelligence of the running batch task can be improved, and the running batch task can be successfully operated.

In some optional implementations, before step S202, the electronic device may further perform the following steps:

and acquiring the real-time service data of the first appointed day.

In this embodiment, the real-time service data of the first designated day may be further synchronized to the pasting layer. The real-time data is imported into the source layer, and the data can be written into the data warehouse source layer by acquiring an archive log (orca l e: OGG/mysq l: bi n l og) from a source database through f l ume technology. In addition, the specific implementation process of acquiring the real-time service data on the first designated day is further described in detail in the following specific embodiments, and is not set forth herein in any greater detail.

And processing the index of the first appointed day based on the real-time service data in a preset time period to obtain real-time summarized data of the first appointed day.

In this embodiment, the preset time period is 24 minutes after the first specified day. By performing the calculation of the real-time summarized data 24 minutes after the first specified day, it can be ensured that the real-time data of the day has been completely put in storage, thereby ensuring the accuracy of the generated real-time summarized data. In the above specific implementation process, the real-time service data is processed on the basis of the index of the first specified day to obtain the real-time summary data of the first specified day, which will be described in further detail in the following specific embodiments and will not be elaborated herein. By using the day index required by real-time service data processing, the processing task can be completed quickly due to the fact that the day data of the first appointed day is few, and the processing time is short.

Acquiring first summarized data corresponding to a second appointed day; wherein the second designated day is the previous day spaced from the first designated day.

In this embodiment, data query processing may be performed on the target data source through the date information of the second specified day, so as to obtain the first summarized data corresponding to the second specified day from the target data source.

Generating second summary data corresponding to the first designated day based on the real-time summary data and the first summary data.

In this embodiment, the above-mentioned specific implementation process of generating the second summarized data corresponding to the first designated day based on the real-time summarized data and the first summarized data is further described in detail in the following specific embodiments, and will not be elaborated herein.

The method comprises the steps of obtaining real-time service data of a first appointed day, processing indexes of the first appointed day based on the real-time service data in a preset time period to obtain real-time summarized data of the first appointed day, then obtaining first summarized data corresponding to a second appointed day, wherein the second appointed day is the day before the interval of the first appointed day, subsequently generating second summarized data corresponding to the first appointed day based on the real-time summarized data and the first summarized data, and finally taking the second summarized data as target summarized data and storing the target summarized data into a target data source. This application is through utilizing the real-time service data calculation real-time summary data of first appointed day, thereby can be based on real-time summary data with first summary data generation with the target summary data that first appointed day corresponds to the required data of the task of batching that realizes calculating good the day in advance, thereby when the target application breaks down, can be quick switch the data source with the present use of target application and store the target summary data of this first appointed day, thereby can resume the processing of the task of batching of target application fast and handle, be favorable to improving the treatment effeciency and the processing intelligence of the task of batching.

In some optional implementation manners of this embodiment, the obtaining of the real-time service data of the first specified day includes the following steps:

and acquiring an archive log corresponding to the target application from a preset source database.

In this embodiment, the source database is a database that is constructed in advance and stores logs of each application.

And inquiring target log data corresponding to the first appointed day from the filing log.

In this embodiment, the archive log includes log data of each working day, and target log data corresponding to a first specified day may be searched for from all the archive logs based on date information of the first specified day.

In this embodiment, the target log data at least includes real-time service data of the first specified day.

According to the method and the device, the archival log corresponding to the target application is obtained from a preset source database, then the target log data corresponding to the first appointed day is inquired from the archival log, and then the real-time service data of the first appointed day is extracted from the target log data. The method and the device can quickly and accurately acquire the real-time service data of the first appointed day based on query processing of the source end database, and improve the acquisition efficiency of the real-time service data.

In some optional implementations, after the step of querying the archived log for the target log data corresponding to the first specified day, the electronic device may further perform the following steps:

and calling a preset abnormity prediction model.

In this embodiment, for the training and generating process of the above-mentioned anomaly prediction model, the present application will further describe this in detail in the following specific embodiments, which are not set forth herein too much.

And inputting the target log data into the abnormity prediction model, and carrying out abnormity analysis on the target log data through the abnormity prediction model to generate an abnormity analysis result corresponding to the target log data.

In this embodiment, the content of the anomaly analysis result includes that the log data is normal or that the log data is anomalous.

And acquiring the communication information of the target user.

In this embodiment, the target user may refer to a worker related to operation and maintenance of log data, and the communication information may include a mobile phone number or a mail address.

According to the method and the device, after target log data corresponding to a first appointed day are inquired from a filing log, the target log data can be intelligently input into a preset abnormity prediction model, abnormity analysis is carried out on the target log data through the abnormity prediction model, an abnormity analysis result corresponding to the target log data is generated, then communication information of a target user is obtained, and based on the communication information, the abnormity analysis result is sent to communication equipment of the target user. According to the method and the device, the target log data are subjected to the anomaly analysis processing by using the anomaly prediction model, so that a corresponding anomaly analysis result can be generated quickly and accurately, and the anomaly analysis result is sent to the target user, so that the user can analyze the reason of batch task failure in the same day according to the received anomaly analysis result and can execute corresponding repair processing, and the use experience of the target user is improved.

In some optional implementations, before the step of invoking the preset anomaly prediction model, the electronic device may further perform the following steps:

and acquiring preset sample data.

In this embodiment, the sample data may include log data collected in advance in a historical time period, and the log data is labeled with an exception tag. The label marking mode can adopt a machine marking mode, a manual marking mode and a marking mode combining the machine marking mode and the manual marking mode.

Dividing the sample data into training sample data and test sample data according to a preset proportion; the training sample data comprises a plurality of log sample data and a category label corresponding to the log sample data.

In this embodiment, the value of the preset ratio is not particularly limited, and may be set according to actual requirements, for example, may be set to 30%. The test sample data comprises a plurality of test log sample data and a category label corresponding to each test log sample data.

And taking the log sample data in the training sample data as model input, taking the class label in the training sample data as model output, and training a preset machine learning model to obtain the trained machine learning model.

In this embodiment, the machine learning model may include any one of a naive bayes model, a random forest model, and a logistic regression model. In addition, the training process for various machine learning models can refer to the existing model training process, and is not elaborated herein.

And testing the trained machine learning model based on the test sample data.

In this embodiment, the step of testing the trained machine learning model based on the test sample data set includes: and if the prediction accuracy is greater than the accuracy threshold, judging that the trained machine learning model is converged, thereby finishing the training of the trained machine learning model, and taking the trained machine learning model as the abnormal prediction model. In addition, if the prediction accuracy is smaller than the accuracy threshold, it indicates that the training of the trained machine learning model has not reached the preset standard, and it may be that the number of samples of training sample data for training is too small or the number of samples of test sample data is too small, so in this case, the number of samples of training sample data may be further increased, for example, a fixed number is further increased each time or a random number is increased each time, then the above-mentioned training process and test process are executed again on the basis, and the above-mentioned steps are executed in such a cycle until the prediction accuracy of the trained machine learning model meets the requirement that the prediction accuracy is larger than the accuracy threshold, and the model training is ended.

And if the trained machine learning model passes the test, taking the trained machine learning model as the abnormity prediction model.

According to the method and the device, the preset machine learning model is trained and tested by using the training sample data and the test sample data which comprise a plurality of log sample data and the category labels corresponding to the log sample data, so that the abnormity prediction model meeting the actual use requirement can be intelligently and quickly generated, the target log data can be subjected to abnormity prediction subsequently based on the abnormity prediction model obtained through training, the abnormity analysis result corresponding to the target log data can be quickly and accurately generated, and the intelligence and the efficiency of generating the abnormity analysis result are improved.

In some optional implementation manners of this embodiment, the processing the index of the first specified day based on the real-time service data to obtain the real-time summarized data of the first specified day includes the following steps:

and acquiring a preset index statistical rule.

In this embodiment, the index statistical rule may be an index statistical rule generated by writing in advance according to actual service requirements.

And calling a preset rule engine.

In this embodiment, the rule engine may be drool.

And using the rule engine to perform index statistical processing on the real-time service data according to the index statistical rule to obtain a corresponding index statistical result.

In this embodiment, the rule engine may perform a classification and aggregation operation on the real-time service data according to the time dimension information, so as to obtain index data corresponding to the real-time service data.

And taking the index statistical result as the real-time summarized data.

According to the method and the device, the preset index statistical rule is obtained, then the preset rule engine is called, then the rule engine is used, index statistical processing is carried out on the real-time business data according to the index statistical rule, a corresponding index statistical result is obtained, and the index statistical result is used as the real-time summarized data. The method and the device for generating the target summarized data based on the index statistical rule and the rule engine can quickly and accurately generate the real-time summarized data of the first appointed day, and are favorable for generating the target summarized data corresponding to the first appointed day based on the obtained first summarized data subsequently.

In some optional implementations of this embodiment, the generating second summarized data corresponding to the first specified day based on the real-time summarized data and the first summarized data includes:

and calling a preset index data calculation formula.

In this embodiment, the index data calculation formula may be a calculation formula generated by writing in advance according to actual service requirements. For example, in practical application, there are indexes that are accumulated annually and monthly, that is, T-1 year cumulative index = T-2 year cumulative index + current day real-time index; and when the statistics period needs to be considered, whether factors such as year-crossing, month-crossing and the like need to be processed or not is considered.

And calculating the real-time summarized data and the first summarized data based on the index data calculation formula to obtain corresponding calculated data.

In this embodiment, the calculation data may be obtained by adding the real-time summary data and the first summary data, and may be used as the second summary data.

Taking the calculated data as the second summary data.

According to the method and the device, a preset index data calculation formula is called, and then the index data calculation formula is based on the real-time summarized data and the first summarized data are subjected to calculation processing to obtain corresponding calculated data, and the calculated data serve as the second summarized data. According to the method and the device, based on the application of the index data calculation formula, the second summarized data corresponding to the first appointed day can be generated quickly and accurately, and the method and the device are favorable for generating the target summarized data corresponding to the first appointed day based on the obtained first summarized data in the follow-up process.

It is emphasized that, to further ensure the privacy and security of the result data, the result data may also be stored in a node of a blockchain.

The block chain referred by the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The block chain (B l ockcha i n), which is essentially a decentralized database, is a string of data blocks associated by using cryptography, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. The artificial intelligence (Art I f I c I a l I nte l I gene, AI) is a theory, method, technique and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, sensing environment, acquiring knowledge and obtaining the best result by using the knowledge.

The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a computer readable storage medium, and when executed, the processes of the embodiments of the methods described above can be included. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-only Memory (ROM), or a Random Access Memory (RAM).

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a task processing apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied in various electronic devices.

As shown in fig. 3, the task processing device 300 according to the present embodiment includes: the device comprises a judging module 301, a switching module 302, a first acquiring module 303, a first generating module 304 and a second generating module 305. Wherein:

the judging module 301 is configured to judge whether a fault event corresponding to a batching task on a first specified day exists in a target application at present; wherein the first specified day is the previous day separated from the current day;

a switching module 302, configured to switch, if yes, a data source currently used by the target application to a preset target data source;

a first obtaining module 303, configured to obtain, from the target data source, target summarized data corresponding to the first specified day;

a first generating module 304, configured to generate a target batch task corresponding to the target application based on the target summary data;

a second generating module 305, configured to run the target batching task and generate result data corresponding to the target batching task.

In this embodiment, the operations that the modules or units are respectively configured to execute correspond to the steps of the task processing method in the foregoing embodiment one to one, and are not described herein again.

In some optional implementations of this embodiment, the task processing device further includes:

the second acquisition module is used for acquiring the real-time service data of the first appointed day;

the processing module is used for processing the indexes of the first appointed day based on the real-time service data in a preset time period to obtain real-time summarized data of the first appointed day;

the third acquisition module is used for acquiring first summarized data corresponding to the second appointed day; wherein the second designated day is the previous day spaced from the first designated day;

a third generation module, configured to generate second summarized data corresponding to the first specified day based on the real-time summarized data and the first summarized data;

and the storage module is used for taking the second summarized data as the target summarized data and storing the target summarized data into the target data source.

In some optional implementation manners of this embodiment, the second obtaining module includes:

the first obtaining sub-module is used for obtaining an archiving log corresponding to the target application from a preset source database;

the query submodule is used for querying target log data corresponding to the first appointed day from the filing log;

and the extraction submodule is used for extracting the real-time service data of the first appointed day from the target log data.

In some optional implementation manners of this embodiment, the second obtaining module further includes:

the first calling submodule is used for calling a preset abnormity prediction model;

the analysis submodule is used for inputting the target log data into the abnormity prediction model, carrying out abnormity analysis on the target log data through the abnormity prediction model and generating an abnormity analysis result corresponding to the target log data;

the second acquisition submodule is used for acquiring the communication information of the target user;

and the sending submodule is used for sending the abnormity analysis result to the communication equipment of the target user based on the communication information.

the third acquisition submodule is used for acquiring preset sample data;

the dividing submodule is used for dividing the sample data into training sample data and test sample data according to a preset proportion; the training sample data comprises a plurality of log sample data and a category label corresponding to the log sample data;

the training submodule is used for inputting log sample data in the training sample data as a model, outputting a class label in the training sample data as the model, and training a preset machine learning model to obtain a trained machine learning model;

the test sub-module is used for testing the trained machine learning model based on the test sample data;

and the first determining submodule is used for taking the trained machine learning model as the abnormity prediction model if the trained machine learning model passes the test.

In some optional implementations of this embodiment, the processing module includes:

the fourth obtaining submodule is used for obtaining a preset index statistical rule;

the first calling submodule is used for calling a preset rule engine;

the first calculation submodule is used for performing index statistical processing on the real-time service data according to the index statistical rule by using the rule engine to obtain a corresponding index statistical result;

and the second determining submodule is used for taking the index statistical result as the real-time summarized data.

In some optional implementations of this embodiment, the third generating module includes:

the second calling submodule is used for calling a preset index data calculation formula;

the second calculation submodule is used for calculating the real-time summarized data and the first summarized data based on the index data calculation formula to obtain corresponding calculated data;

a third determining submodule, configured to use the calculated data as the second summarized data.

In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 4 in particular, fig. 4 is a block diagram of a basic structure of a computer device according to the embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only computer device 4 having components 41-43 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. AS will be understood by those skilled in the art, the computer device herein is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit (App I cat I on Spec I C I integrated C I rcu I, AS ic), a programmable Gate array (F I l D-programmable ab l Gate Ar ray, FPGA), a digital Processor (D I ta l S I gna l Processor, DSP), an embedded device, and the like.

The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user in a keyboard mode, a mouse mode, a remote controller mode, a touch panel mode or a voice control equipment mode.

The memory 41 includes at least one type of readable storage medium including flash memory, hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk provided on the computer device 4, a Smart Memory Card (SMC), a Secure digital (Secure D i g i ta l, SD) Card, a flash memory Card (F l ash Card), and so on. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as computer readable instructions of a task processing method. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions or processing data stored in the memory 41, for example, computer readable instructions for executing the task processing method.

The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing communication connection between the computer device 4 and other electronic devices.

in the embodiment of the application, when whether a fault event corresponding to a running batch task on a first appointed day exists at present in a target application is detected, a data source currently used by the target application is switched to a preset target data source, then target summary data corresponding to the first appointed day are obtained from the target data source, a target running batch task corresponding to the target application is generated based on the target summary data, and finally the target running batch task is operated to generate result data corresponding to the target running batch task. According to the method and the device, when the running batch task currently executed by the target application fails, the data source currently used by the target application is switched to the pre-calculated target summarized data target data source required by the running batch task on the same day, so that the target summarized data on the first appointed day can be rapidly stored by switching the data source currently used by the target application when the target application has a task fault, the running batch task processing of the target application can be rapidly recovered, the processing efficiency and the processing intelligence of the running batch task can be improved, and the successful operation of the running batch task is ensured.

The present application further provides another embodiment, which is to provide a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the task processing method as described above.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.

It should be understood that the above-described embodiments are merely exemplary of some, and not all, embodiments of the present application, and that the drawings illustrate preferred embodiments of the present application without limiting the scope of the claims appended hereto. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.

Claims

1. A task processing method, comprising the steps of:

2. The task processing method according to claim 1, further comprising, before the step of switching the data source currently used by the target application to a preset target data source:

acquiring real-time service data of the first appointed day;

3. The task processing method according to claim 2, wherein the step of acquiring the real-time service data on the first specified day specifically includes:

4. The task processing method according to claim 3, further comprising, after the step of querying the archive log for the target log data corresponding to the first specified day:

calling a preset abnormal prediction model;

acquiring communication information of a target user;

5. The task processing method according to claim 4, further comprising, before the step of calling a preset anomaly prediction model:

acquiring preset sample data;

testing the trained machine learning model based on the test sample data;

6. The task processing method according to claim 2, wherein the step of processing the index of the first specified day based on the real-time service data to obtain the real-time summary data of the first specified day specifically includes:

acquiring a preset index statistical rule;

calling a preset rule engine;

and taking the index statistical result as the real-time summarized data.

7. The task processing method according to claim 2, wherein the step of generating second summary data corresponding to the first specified day based on the real-time summary data and the first summary data includes:

calling a preset index data calculation formula;

taking the calculated data as the second summary data.

8. A task processing apparatus, characterized by comprising:

the judging module is used for judging whether a fault event corresponding to the batch running task on the first appointed day exists in the target application at present; wherein the first specified day is the previous day separated from the current day;

the switching module is used for switching the data source currently used by the target application to a preset target data source if the data source currently used by the target application is the preset target data source;

and the second generation module is used for operating the target batch task and generating result data corresponding to the target batch task.

9. A computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of a task processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, having computer-readable instructions stored thereon, which, when executed by a processor, implement the steps of the task processing method according to any one of claims 1 to 7.