CN113064661A - Memory processing method and device for batch tasks, electronic equipment and storage medium - Google Patents

Memory processing method and device for batch tasks, electronic equipment and storage medium Download PDF

Info

Publication number
CN113064661A
CN113064661A CN201911425296.0A CN201911425296A CN113064661A CN 113064661 A CN113064661 A CN 113064661A CN 201911425296 A CN201911425296 A CN 201911425296A CN 113064661 A CN113064661 A CN 113064661A
Authority
CN
China
Prior art keywords
task
callback
target batch
subtasks
subtask
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911425296.0A
Other languages
Chinese (zh)
Inventor
杨赛舟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Intellifusion Technologies Co Ltd
Original Assignee
Shenzhen Intellifusion Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Intellifusion Technologies Co Ltd filed Critical Shenzhen Intellifusion Technologies Co Ltd
Priority to CN201911425296.0A priority Critical patent/CN113064661A/en
Publication of CN113064661A publication Critical patent/CN113064661A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44557Code layout in executable memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention relates to the technical field of data processing, and provides a memory processing method, a memory processing device, electronic equipment and a storage medium for batch tasks, wherein the method comprises the following steps: the task logic processing area creates a target batch task and initializes the target batch task, wherein the target batch task comprises a plurality of subtasks; registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks; executing the action corresponding to the subtask in a task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter; and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished. The embodiment of the invention can reduce the coupling in the task processing process and improve the working efficiency.

Description

Memory processing method and device for batch tasks, electronic equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for processing a batch task memory, an electronic device, and a storage medium.
Background
At present, batch processing tasks such as batch file uploading, Excel batch processing tasks and the like are often required. In the existing processing mode, the logic processing of the task and the business code are coupled with each other, and although the logic processing of the task and the business code can be independent by using the middleware, the complexity and the management difficulty are increased when the amount of the processing task is small in the use place of the middleware. And the high coupling case will cause the interaction between the logical processing of the task and the business code. Therefore, the problems of high coupling and low working efficiency exist in the prior art.
Disclosure of Invention
The embodiment of the invention provides a memory processing method for batch tasks, which can reduce the coupling in the task processing process and improve the working efficiency.
In a first aspect, an embodiment of the present invention provides a method for processing a memory of a batch task, where the method includes the following steps:
creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks;
registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks;
executing the action corresponding to the subtask in a task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter;
and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
In a second aspect, an embodiment of the present invention further provides a device for processing a batch of tasks in a memory, including:
the system comprises a creating module, a task logic processing area and a task execution module, wherein the creating module is used for creating a target batch task in the task logic processing area and initializing the target batch task, and the target batch task comprises a plurality of subtasks;
the registration module is used for registering the subtasks of the target batch tasks and adding the monitoring parameters and the callback parameters of the subtasks;
the execution module is used for executing the action corresponding to the subtask through a memory, and performing state monitoring and state callback on the action corresponding to the subtask in a task logic processing area according to the monitoring parameter and the callback parameter;
and the detection module is used for judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks and archiving the target batch tasks after the target batch tasks are finished.
In a third aspect, an embodiment of the present invention further provides an electronic device, including: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps in the memory processing method of the batch tasks provided by the embodiment.
In a fourth aspect, a computer-readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the steps in the memory processing method for batch tasks provided by the embodiment.
In the embodiment of the invention, a target batch task is created in a task logic processing area and is initialized, wherein the target batch task comprises a plurality of subtasks; registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks; executing the action corresponding to the subtask in a task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter; and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished. The embodiment of the invention creates the task logical processing area and the task response area, and in the task logical processing area, creating target batch tasks, initializing the target batch tasks, registering subtasks in the target batch tasks and adding monitoring parameters and callback parameters, namely, completing the logic processing of the tasks in a task logic processing area in advance, transmitting the subtasks to a task response area after the logic processing is completed, the corresponding action of the subtask is executed through the memory, the monitoring process of the parameter is monitored, and the feedback is carried out through the callback parameter so as to complete the processing of the service code, by stripping the logic processing and the service code processing, the coupling in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch tasks is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a memory processing method for batch tasks according to an embodiment of the present invention;
FIG. 2 is a flowchart of another method for processing the memory of batch tasks according to the embodiment of the present invention;
FIG. 3 is a flowchart of another method for processing the memory of batch tasks according to the embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a memory processing apparatus for batch tasks according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the structure of the create module 401 in FIG. 4;
FIG. 6 is a schematic diagram of the structure of registration module 402 of FIG. 4;
FIG. 7 is a block diagram of the execution module 403 in FIG. 4;
FIG. 8 is a schematic diagram of another structure of the execution module 403 in FIG. 4;
FIG. 9 is a schematic diagram of another embodiment of a batch task memory processing device;
FIG. 10 is a schematic diagram of the structure of the detection module 404 in FIG. 4;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "comprising" and "having," and any variations thereof, in the description and claims of this application and the description of the figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
As shown in fig. 1, fig. 1 is a flowchart of a memory processing method for a batch task according to an embodiment of the present invention, where the memory processing method for the batch task includes the following steps:
101. and creating a target batch task in the task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks.
In this embodiment, the electronic device on which the memory processing method for batch tasks is executed may obtain information such as tasks in a wired connection manner or a wireless connection manner. It should be noted that the Wireless connection manner may include, but is not limited to, a 3G/4G connection, a WiFi (Wireless-Fidelity) connection, a bluetooth connection, a wimax (worldwide Interoperability for Microwave access) connection, a Zigbee (low power local area network protocol), a uwb (ultra wideband) connection, and other Wireless connection manners known now or developed in the future.
The task logical processing area may be a plug-in for logically processing a task. The task logic processing area can be configured with a task interface, which can be a network interface or a local interface, and the task interface can receive a request task sent by a user through the electronic device, and can transmit a final processing result back to the electronic device sending the task request by the user through the task interface for the user to use.
The embodiment of the invention can perform batch processing on the tasks under the condition of small data volume, can preset the data volume threshold value of the data volume received by the task interface for judging the size of the data volume, and can receive the data volume as long as the data volume threshold value is met. The target batch tasks may mean that after the batch task requests are obtained, corresponding target batch tasks may be created according to the batch task requests, and the created tasks facilitate subsequent logic processing and data calculation of the task requests included in the tasks in a task form.
The target batch task includes a plurality of subtasks, and the initializing the target batch task may indicate that data volumes of the plurality of subtasks included in the target batch task are counted, may further include performing fragmentation processing on the target batch task to decompose the target batch task into the plurality of subtasks, and may further include transmitting the initialized subtasks to a subtask part for registering the target batch task.
102. And registering the subtasks of the target batch task, and adding the monitoring parameters and callback parameters of the subtasks.
The task identification can be directly found by finding the task identification. The above monitoring parameters may be used to monitor the task status of each subtask, and the task monitoring parameters may be registered in the task listener. The Callback parameter may be a Callback Stub (Callback Stub), and adding the Callback parameter may perform a status Callback.
103. And executing the action corresponding to the subtask in the task response area through the memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter.
The task response area can be specially used for responding to the subtask, and data calculation is carried out on the subtask through the service code. Moreover, the actions corresponding to the execution of the subtasks may be executed in the memory of the task response area. The above-mentioned action corresponding to the execution of the subtask is to perform corresponding data calculation on the subtask, for example: if the subtask request is to acquire a picture, the process of performing data calculation may be to acquire the picture first, and then analyze the characteristics of the picture, and the like.
The status of executing the subtasks may be monitored by monitoring parameters, which may be timed monitoring or real-time monitoring. The current process state can be acquired at any time through monitoring the monitoring parameters, and the monitoring parameters can be in one-to-one correspondence with the subtasks, namely one monitoring parameter is added to one subtask. When the current state of the subtask is monitored to be the finished state, the Callback parameter can be called to call the finished state through the Callback inlet Callback, and the Callback is carried out through the Callback inlet Callback. One callback parameter for each subtask.
104. And judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
In the task response area, each time a subtask is acquired, a monitoring parameter corresponding to the subtask is called to perform state monitoring, and a callback parameter is called to acquire a processing result after an action corresponding to the subtask is completed to perform state callback. If the monitoring parameter does not continue to work, it may indicate that all the subtasks in the target batch task have been completely processed. At this time, all the already processed subtasks may be stored in the memory.
In the embodiment of the invention, a target batch task is created in a task logic processing area, and is initialized, wherein the target batch task comprises a plurality of subtasks; registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks; executing the action corresponding to the subtask in the task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter; and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished. The embodiment of the invention creates the task logical processing area and the task response area, and in the task logical processing area, creating target batch tasks, initializing the target batch tasks, registering subtasks in the target batch tasks and adding monitoring parameters and callback parameters, namely, completing the logic processing of the tasks in a task logic processing area in advance, transmitting the subtasks to a task response area after the logic processing is completed, the corresponding action of the subtask is executed through the memory, the monitoring process of the parameter is monitored, and the feedback is carried out through the callback parameter so as to complete the processing of the service code, by stripping the logic processing and the service code processing, the coupling in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch tasks is improved.
As shown in fig. 2, fig. 2 is a flowchart of another method provided by the embodiment of the present invention, which includes the following steps:
201. and establishing the batch tasks in the task logic processing area to obtain target batch tasks.
The batch task may mean that a task interface in the service logic processing area receives a large number of task requests at the same time, for example: the task requests sent by a plurality of users to the task interface respectively, or a plurality of task requests sent by one user to the task interface. In order to perform ordered and rapid processing on the batch task requests, the task requests of different types can be classified and labeled, the classification labeling can be distinguished according to the task identifiers, and the task requests belonging to the same data type in all the task identifiers are clustered, for example: if the data type is an acquisition picture class, clustering the requests belonging to picture acquisition in the task request into the acquisition picture class; and when the data type is a data structuring processing class, clustering the request classes which need to perform structuring processing on the data in all the task requests into the data structuring processing class. After the task requests are classified, corresponding tasks can be created corresponding to the task requests in each data type, and after all the tasks are created, target batch tasks can be extracted from the created tasks according to the time sequence.
202. Initializing statistical parameters of the target batch task and processing the initialized target batch task after the target batch task is completed, wherein the target batch task comprises a plurality of subtasks.
After the target batch tasks are obtained, the target batch tasks can be subjected to fragmentation processing, that is, one target batch task is divided into a plurality of subtasks according to different processing objects in the process to be executed, so that a pipeline of the subtasks corresponding to the target batch tasks is formed. Each subtask in the pipeline may be processed sequentially in the task response area.
The statistical parameter may represent the data amount of the subtask, and after the target batch task is decomposed into a plurality of subtasks, the data amount of the subtask may be counted, for example: the data amount of the subtasks is 10, 20, etc. The processing after the initialization of the target batch task is completed may mean that after the fragments are divided into a plurality of subtasks, the subtasks are sequentially transferred from the task logic processing area to the memory for storage.
203. And registering the subtasks of the target batch task, and adding the monitoring parameters and callback parameters of the subtasks.
204. And executing the action corresponding to the subtask in the task response area through the memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter.
205. And judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
Optionally, step 203 includes:
and respectively registering each subtask in the target batch task.
The registering may be expressed as registering a task identifier for each subtask, and the subtasks are distinguished by the task identifiers, for example: subtask-01, subtask-02, subtask-03 … … subtask-0 n.
And adding a monitoring parameter and a callback parameter which correspond to each subtask in number.
After registering each subtask, the task identifier of each subtask may be obtained, and a corresponding listening parameter and a callback parameter are added to the subtask corresponding to each task identifier, for example: the subtask is subtask-01, the corresponding monitoring parameter is monitor-01, the corresponding callback parameter is callback-01, the subtask is subtask-02, the corresponding monitoring parameter is monitor-02, and the corresponding callback parameter is callback-02. By registering the corresponding task identifier for each subtask and then adding the monitoring parameters and the callback parameters corresponding to each task identifier one to one, the states of the subtasks can be conveniently tracked and counted, and the whole processing process can be orderly executed.
Optionally, step 204 includes:
and analyzing the subtasks to obtain task data in the subtasks.
The task response area can actively extract the stored subtasks from the memory and analyze the subtasks. The analysis of the subtasks may obtain the actions to be executed in the subtasks, that is, the task data, for example: the task data is collected pictures, or the task data is used for target detection, or the task data is used for characteristic value extraction, or the task data is used for attribute analysis, and the like.
And according to the task data, performing data calculation corresponding to the task data in the memory of the task response area.
After the task data is analyzed, data calculation may be directly performed on the task data in the memory, where the data calculation may include extracting features in the subtask data and calculating a corresponding result, for example: if the task data is a search image, the image in the task data can be extracted, the extracted information is compared with image information stored in the memory in advance, and a comparison result is obtained, wherein the comparison result is used as a response result of the subtask corresponding to the task data.
In the embodiment of the invention, by creating a task logic processing area and a task response area, in the task logic processing area, creating a target batch task, initializing the target batch task, registering subtasks in the target batch task and adding monitoring parameters and callback parameters, namely, completing the logic processing of the task in advance in the task logic processing area, after the logic processing is completed, transmitting the subtasks to the task response area, the corresponding action of the subtask is executed through the memory, the monitoring process of the parameter is monitored, and the feedback is carried out through the callback parameter so as to complete the processing of the service code, by stripping the logic processing and the service code processing, the coupling in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch tasks is improved.
As shown in fig. 3, fig. 3 is a flowchart of another batch task memory processing method implemented and provided by the present invention, which specifically includes the following steps:
301. and creating a target batch task in the task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks.
302. And registering the subtasks of the target batch task, and adding the monitoring parameters and callback parameters of the subtasks.
303. And executing the action corresponding to the subtask in the task response area through the memory, and monitoring the state of the action corresponding to the subtask executed by the memory in the task response area according to the monitoring parameter.
The action state corresponding to the execution of the subtask can be monitored in real time through the monitoring parameter, so that the current state can be monitored in real time, whether the action corresponding to the subtask is executed is judged, or the action corresponding to the execution of the second subtask is executed, and the like. In the monitoring process, if data calculation of the subtask is started in the memory, the monitoring parameter generates corresponding execution information, and when the data calculation of the subtask is completed, the corresponding end information is also generated.
304. And if the action corresponding to the execution of the subtask in the memory is monitored to be completed, performing state callback on the completed state through the callback parameter, wherein the completed state refers to the action corresponding to the execution of the subtask in the memory.
When the memory is monitored to finish the action corresponding to the execution of the subtask, the memory is in a finished state, and the memory does not continue to respond to the current processed subtask but continues to respond to the next subtask. When the subtask is in the completed state, the Callback parameter may be called to Callback the completed state through the Callback entry Callback.
305. And if the memory is monitored to be in the state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
The monitoring function monitors in real time or at regular time, if the monitoring function is in the state of executing the action corresponding to the subtask, the monitoring is continued, and corresponding response information cannot be generated before the processing of the subtask is not finished, so that the callback function is not started.
306. And judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
Optionally, after the step 302, the method may further include:
and adding a data statistical table, wherein the data statistical table is used for counting the data volume of the state callback and the data volume of the subtask.
The data statistics table can be attached to a preset statistics monitor, and the statistics monitor can be called to dynamically update the data volume as long as the data volume changes. The data statistics table may be used to count the data amount of all the subtasks, and may be placed in the data statistics table after the callback parameter has called back the completed state. The data volume of the subtask and the data volume of the state callback can be respectively stored in different areas in the data statistical table, so that confusion is avoided.
And if the callback parameter is detected to carry out state callback on the finished state, updating the data volume of the state callback in the data statistical table.
When each state is called back into the data statistical table, the data quantity of the subtasks in the data statistical table is sequentially updated until the data quantity of the subtasks in the task data table is 0, which indicates that the data processing is completed for all the subtasks in the target batch of tasks. Wherein, the data amount of the sub-task may be the remaining unprocessed data amount of the sub-task, for example: the data volume of the subtasks is 10, if a state callback enters the data statistical table, the data volume of the subtasks is reduced by one and is changed into 9, and the data volume of the subtasks is decreased sequentially until the data volume of the subtasks is decreased to 0.
Optionally, the step 306 includes:
and detecting whether the subtask data volume in the data statistical table reaches the target data volume.
The target data amount may be a data amount of a subtask in the data statistics table of 0. Detecting whether the data volume of the subtasks reaches the target data volume may be used to determine whether to complete all responses to all subtasks in the target batch of tasks.
And if the data volume of the subtasks reaches the target data volume, generating response information for completing the processing of the target batch tasks.
If the data volume of the subtask is detected to reach the target data volume, the data volume of the subtask in the data statistical table is 0. At this time, response information may be generated in the memory, where the response information includes information that processing of the target batch task has been completed, for example: and confirming the completion of the processing for the target batch task and the like by the response information.
And triggering a preset function to send the response information according to the response information, and archiving the target batch tasks.
When the generation of the response information is detected, the preset function can be triggered immediately to send the response information to notify the calling party that the processing of the target batch tasks is completed, and the target batch tasks can be stored in the memory. In addition, there may be a case where a request timeout occurs when an action corresponding to a subtask is executed, and for this case, a corresponding preset function may be triggered to send out a request timeout message, and the like. The preset function may be a Hook function Hook, and the preset function may be triggered correspondingly according to specific situations, for example: if the response information is received, the Hook function Hook triggered to send the information that the response is successful, and if the delay request occurs, the Hook function Hook triggered correspondingly is the information that the request is delayed or the request is failed.
In the embodiment of the method, the target batch tasks are created, the target batch tasks are initialized, the subtasks in the target batch tasks are registered and the monitoring parameters and the callback parameters are added in the task logic processing area by creating the task logic processing area and the task response area, namely, the logic processing of the tasks is completed in the task logic processing area in advance, after the logic processing is completed, the subtasks are transmitted to the task response area, the corresponding action of the subtask is executed through the memory, the monitoring process of the parameter is monitored, and the feedback is carried out through the callback parameter so as to complete the processing of the service code, by stripping the logic processing and the service code processing, the coupling in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch tasks is improved.
As shown in fig. 4, fig. 4 is a schematic structural diagram of a memory processing apparatus for batch tasks according to an embodiment of the present invention, where the memory processing apparatus 400 for batch tasks includes:
a creating module 401, configured to create a target batch task in the task logic processing area, and initialize the target batch task, where the target batch task includes multiple subtasks;
a registering module 402, configured to register subtasks of the target batch task, and add a monitoring parameter and a callback parameter of the subtask;
the execution module 403 is configured to execute the action corresponding to the subtask through the memory, and perform state monitoring and state callback on the action corresponding to the subtask in the task logic processing area according to the monitoring parameter and the callback parameter;
and the detection module 404 is configured to determine whether the target batch task is completed through state monitoring and state callback of the subtasks, and archive the target batch task after completion.
Optionally, as shown in fig. 5, fig. 5 is a schematic structural diagram of the creating module 401 in fig. 4, where the creating module 401 includes:
the creating unit 4011 is configured to create batch tasks in the task logic processing area to obtain target batch tasks;
the initialization unit 4012 is configured to initialize statistical parameters of the target batch task and initialize post-processing of the target batch task after completion.
Optionally, as shown in fig. 6, fig. 6 is a schematic structural diagram of the registration module 402 in fig. 4, where the registration module 402 includes:
the registration unit 4021 is configured to register each subtask in the target batch task;
the adding unit 4022 is configured to add the monitoring parameters and the callback parameters corresponding to the number of each sub-task.
Optionally, as shown in fig. 7, fig. 7 is a schematic structural diagram of the execution module 403 in fig. 4, where the execution module 403 includes:
the analysis unit 4031 is used for analyzing the subtasks to acquire task data in the subtasks;
a calculating unit 4032, configured to perform data calculation corresponding to the task data in the memory of the task response area according to the task data.
Optionally, as shown in fig. 8, fig. 8 is another schematic structural diagram of the execution module 403 in fig. 4, where the execution module 403 further includes:
an execution unit 4033, configured to monitor, according to the monitoring parameter, a state of an action corresponding to the execution of the sub-task in the memory in the task response area;
a callback unit 4034, configured to perform state callback on a completed state through a callback parameter if it is monitored that the memory completes an action corresponding to the execution of the sub-task, where the completed state refers to an action corresponding to the completion of the execution of the sub-task by the memory;
a monitoring unit 4034, configured to, if it is monitored that the memory is in a state of executing an action corresponding to the sub task, continue monitoring according to the monitoring parameter, and not enable the callback parameter.
Optionally, as shown in fig. 9, fig. 9 is a schematic structural diagram of another batch task memory processing apparatus according to an embodiment of the present invention, and the apparatus 400 further includes:
an adding module 405, configured to add a data statistics table, where the data statistics table is used to count a data amount of a state callback and a data amount of a subtask;
the updating module 406 is configured to update the data amount of the state callback in the data statistics table if it is detected that the callback parameter performs state callback on the completed state.
Optionally, as shown in fig. 10, fig. 10 is a schematic structural diagram of the detection module 404 in fig. 4, where the detection module 404 includes:
a detecting unit 4041, configured to detect whether the sub-task data amount in the data statistics table reaches a target data amount;
a generating unit 4042, configured to generate response information that processing of the target batch tasks is completed if the data amount of the subtask reaches the target data amount;
and the storage unit 4043 is configured to trigger the preset function to send the response information according to the response information, and archive the target batch tasks.
The memory processing device for batch tasks provided by the embodiment of the invention can realize each process realized by the memory processing method for batch tasks in the method embodiment and can achieve the same beneficial effect, and in order to avoid repetition, the repeated description is omitted here.
As shown in fig. 11, fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 1100 includes: the memory 1102, the processor 1101, the network interface 1103 and a computer program stored on the memory 1102 and operable on the processor 1101 implement the steps in the memory processing method for batch tasks provided by the embodiment when the computer program is executed by the processor 1101.
Specifically, the processor 1101 is configured to perform the following steps:
creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks;
registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks;
executing the action corresponding to the subtask in the task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter;
and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
Optionally, the steps executed by the processor 1101 of creating the target batch task in the task logic processing area and initializing the target batch task include:
creating batch tasks in a task logic processing area to obtain target batch tasks;
initializing the statistical parameters of the target batch tasks and initializing the post-processing of the target batch tasks.
Optionally, the step of registering a subtask of the target batch task, and adding a monitoring parameter and a callback parameter of the subtask, executed by the processor 1101, includes:
registering each subtask in the target batch task respectively;
and adding the monitoring parameters and callback parameters corresponding to each subtask in quantity.
Optionally, the step of executing, by the processor 1101, an action corresponding to the subtask in the task response area through the memory includes:
analyzing the subtasks to obtain task data in the subtasks;
and according to the task data, performing data calculation corresponding to the task data in the memory of the task response area.
Optionally, the step of performing state monitoring and state callback on the action corresponding to the subtask by the processor 1101 according to the monitoring parameter and the callback parameter includes:
monitoring the state of the action corresponding to the execution of the subtask in the memory in the task response area according to the monitoring parameter;
if the action corresponding to the execution of the subtask in the memory is monitored, carrying out state callback on the finished state through the callback parameter, wherein the finished state refers to the action corresponding to the execution of the subtask in the memory;
and if the memory is monitored to be in the state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
Optionally, after registering the subtasks of the target batch task, the processor 1101 is further configured to:
adding a data statistical table, wherein the data statistical table is used for counting the data volume of the state callback and the data volume of the subtasks;
and if the callback parameter is detected to carry out state callback on the finished state, updating the data volume of the state callback in the data statistical table.
Optionally, the step of determining whether the target batch task is completed through state monitoring and state callback of the subtasks executed by the processor 1101, and archiving the target batch task after completion includes:
detecting whether the subtask data volume in the data statistical table reaches the target data volume;
if the data volume of the subtasks reaches the target data volume, response information for completing processing of the target batch tasks is generated;
and triggering a preset function to send the response information according to the response information, and archiving the target batch tasks.
The electronic device 1100 provided by the embodiment of the present invention can implement each implementation manner in the memory processing method embodiments of the batch task, and has corresponding beneficial effects, and for avoiding repetition, details are not described here again.
It is noted that only 1101-1103 with components are shown, but it is understood that not all of the shown components are required and that more or fewer components may alternatively be implemented. As will be understood by those skilled in the art, the electronic device 1100 is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable gate array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device 1100 may be a computing device such as a desktop computer, a notebook, or a palmtop computer. The electronic device 1100 may interact with a user through a keyboard, mouse, remote control, touch pad, voice control device, or the like.
The memory 1102 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 1102 may be an internal storage unit of the electronic device 1100, such as a hard disk or a memory of the electronic device 1100. In other embodiments, the memory 1102 may also be an external storage device of the electronic device 1100, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the electronic device 1100. Of course, the memory 1102 may also include both internal and external memory units of the electronic device 1100. In this embodiment, the storage 1102 is generally used for storing an operating system installed in the electronic device 1100 and various types of application software, such as program codes of a memory processing method of a batch task. In addition, the memory 1102 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 1101 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 1101 is typically used to control the overall operation of the electronic device 1100. In this embodiment, the processor 1101 is configured to execute the program code stored in the storage 1102 or process data, for example, execute the program code of the memory processing method of the batch task.
The network interface 1103 may include a wireless network interface or a wired network interface, and the network interface 1103 is typically used to establish communication connections between the electronic device 1100 and other electronic devices.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by the processor 1101, the computer program implements each process in the memory processing method for batch tasks provided in the embodiment, and can achieve the same technical effect, and is not described here again to avoid repetition.
It will be understood by those skilled in the art that all or part of the processes of the embodiments may be implemented by hardware related to instructions of a computer program, and the computer program may be stored in a computer readable storage medium, and when executed, may include processes such as those of the embodiments of the methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (10)

1. A memory processing method for batch tasks is characterized by comprising the following steps:
the task logic processing area creates a target batch task and initializes the target batch task, wherein the target batch task comprises a plurality of subtasks;
registering subtasks of the target batch tasks, and adding monitoring parameters and callback parameters of the subtasks;
executing the action corresponding to the subtask in a task response area through a memory, and performing state monitoring and state callback on the action corresponding to the subtask according to the monitoring parameter and the callback parameter;
and judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks, and archiving the target batch tasks after the target batch tasks are finished.
2. The method of claim 1, wherein the steps of creating a target batch of tasks in a task logic processing area and initializing the target batch of tasks comprise:
creating batch tasks in a task logic processing area to obtain target batch tasks;
initializing the statistical parameters of the target batch tasks and initializing the processing after the target batch tasks are completed.
3. The method of claim 1, wherein the step of registering the subtasks of the target batch task and adding the listening parameters and the callback parameters of the subtasks comprises:
registering each subtask in the target batch tasks respectively;
and adding the monitoring parameters and callback parameters corresponding to each subtask in quantity.
4. The memory processing method of batch tasks according to claim 1, wherein the step of executing the action corresponding to the subtask through the memory in the task response area comprises:
analyzing the subtasks to obtain task data in the subtasks;
and according to the task data, performing data calculation corresponding to the task data in the memory of the task response area.
5. The method according to claim 1, wherein the step of performing state interception and state callback on the action corresponding to the subtask according to the interception parameter and the callback parameter comprises:
monitoring the state of the memory executing the action corresponding to the subtask in the task response area according to the monitoring parameter;
if the fact that the memory finishes executing the action corresponding to the subtask is monitored, state callback is carried out on a finished state through the callback parameter, wherein the finished state refers to the fact that the memory finishes executing the action corresponding to the subtask;
and if the memory is monitored to be in the state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
6. The method of memory processing of batch tasks of claim 5, wherein after the registering of the subtasks of the target batch task, the method further comprises:
adding a data statistical table, wherein the data statistical table is used for counting the data volume of the state callback and the data volume of the subtask;
and if the callback parameter is detected to carry out state callback on the finished state, updating the data volume of the state callback in the data statistical table.
7. The method according to claim 6, wherein the step of determining whether the target batch task is completed through the status monitoring and status callback of the subtask and archiving the target batch task after completion comprises:
detecting whether the data volume of the subtasks in the data statistical table reaches a target data volume;
if the data volume of the subtasks reaches the target data volume, generating response information for completing processing of the target batch tasks;
and triggering a preset function to send the response information according to the response information, and archiving the target batch tasks.
8. A memory processing apparatus for batch tasks, comprising:
the system comprises a creating module, a task logic processing area and a task execution module, wherein the creating module is used for creating a target batch task in the task logic processing area and initializing the target batch task, and the target batch task comprises a plurality of subtasks;
the registration module is used for registering the subtasks of the target batch tasks and adding the monitoring parameters and the callback parameters of the subtasks;
the execution module is used for executing the action corresponding to the subtask through a memory, and performing state monitoring and state callback on the action corresponding to the subtask in a task logic processing area according to the monitoring parameter and the callback parameter;
and the detection module is used for judging whether the target batch tasks are finished or not through state monitoring and state callback of the subtasks and archiving the target batch tasks after the target batch tasks are finished.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, the processor implementing the steps in the method for memory processing of bulk tasks according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps in the method for memory processing of batch tasks according to any one of claims 1 to 7.
CN201911425296.0A 2019-12-31 2019-12-31 Memory processing method and device for batch tasks, electronic equipment and storage medium Pending CN113064661A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911425296.0A CN113064661A (en) 2019-12-31 2019-12-31 Memory processing method and device for batch tasks, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911425296.0A CN113064661A (en) 2019-12-31 2019-12-31 Memory processing method and device for batch tasks, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113064661A true CN113064661A (en) 2021-07-02

Family

ID=76557934

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911425296.0A Pending CN113064661A (en) 2019-12-31 2019-12-31 Memory processing method and device for batch tasks, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113064661A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113485808A (en) * 2021-07-06 2021-10-08 中国工商银行股份有限公司 System time callback method and device
CN113986379A (en) * 2021-10-26 2022-01-28 北京字节跳动网络技术有限公司 Application starting method and device, computer equipment and storage medium
CN114331382A (en) * 2021-12-31 2022-04-12 中国邮政储蓄银行股份有限公司 Batch file processing method, processing device, processor and batch processing system
CN115495252A (en) * 2022-11-18 2022-12-20 苏州成科自控设备有限公司 Design method of MCU background task pool
CN118093147A (en) * 2024-04-29 2024-05-28 浙江农商数字科技有限责任公司 Massive data summarizing method and system based on task chain and divide-and-conquer method

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100005472A1 (en) * 2008-07-07 2010-01-07 Infosys Technologies Ltd. Task decomposition with throttled message processing in a heterogeneous environment
US20100333097A1 (en) * 2009-06-29 2010-12-30 Sun Microsystems, Inc. Method and system for managing a task
CN102707993A (en) * 2012-04-11 2012-10-03 招商银行股份有限公司 System and method for centralized processing of service
CN103049164A (en) * 2011-10-14 2013-04-17 上海可鲁系统软件有限公司 Method for quick response of dynamic vector graphic and control device thereof
US20130173686A1 (en) * 2011-12-28 2013-07-04 Sybase, Inc. Monitoring Distributed Task Execution Using a Chained Text Messaging System
US8726278B1 (en) * 2004-07-21 2014-05-13 The Mathworks, Inc. Methods and system for registering callbacks and distributing tasks to technical computing works
US20150254105A1 (en) * 2012-10-31 2015-09-10 Nec Corporation Data processing system, data processing method, and program
CN104991821A (en) * 2015-06-29 2015-10-21 北京奇虎科技有限公司 Monitor task batch processing method and apparatus
CN105700958A (en) * 2016-01-07 2016-06-22 北京京东尚科信息技术有限公司 Method and system for automatic splitting of task and parallel execution of sub-task
US20160321104A1 (en) * 2014-04-03 2016-11-03 Tencent Technology (Shenzhen) Company Limited Method and apparatus for scheduling concurrent task
US20160371123A1 (en) * 2014-03-04 2016-12-22 Huawei Technologies Co., Ltd. Data Processing Method and Apparatus
CN107729139A (en) * 2017-09-18 2018-02-23 北京京东尚科信息技术有限公司 A kind of method and apparatus for concurrently obtaining resource
CN109918184A (en) * 2019-03-01 2019-06-21 腾讯科技(深圳)有限公司 Picture processing system, method and relevant apparatus and equipment
CN110287018A (en) * 2019-07-04 2019-09-27 中国工商银行股份有限公司 Batch tasks method of combination and device
CN110471718A (en) * 2019-07-04 2019-11-19 深圳市共进电子股份有限公司 A kind of task processing method and equipment

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8726278B1 (en) * 2004-07-21 2014-05-13 The Mathworks, Inc. Methods and system for registering callbacks and distributing tasks to technical computing works
US20100005472A1 (en) * 2008-07-07 2010-01-07 Infosys Technologies Ltd. Task decomposition with throttled message processing in a heterogeneous environment
US20100333097A1 (en) * 2009-06-29 2010-12-30 Sun Microsystems, Inc. Method and system for managing a task
CN103049164A (en) * 2011-10-14 2013-04-17 上海可鲁系统软件有限公司 Method for quick response of dynamic vector graphic and control device thereof
US20130173686A1 (en) * 2011-12-28 2013-07-04 Sybase, Inc. Monitoring Distributed Task Execution Using a Chained Text Messaging System
CN102707993A (en) * 2012-04-11 2012-10-03 招商银行股份有限公司 System and method for centralized processing of service
US20150254105A1 (en) * 2012-10-31 2015-09-10 Nec Corporation Data processing system, data processing method, and program
US20160371123A1 (en) * 2014-03-04 2016-12-22 Huawei Technologies Co., Ltd. Data Processing Method and Apparatus
US20160321104A1 (en) * 2014-04-03 2016-11-03 Tencent Technology (Shenzhen) Company Limited Method and apparatus for scheduling concurrent task
CN104991821A (en) * 2015-06-29 2015-10-21 北京奇虎科技有限公司 Monitor task batch processing method and apparatus
CN105700958A (en) * 2016-01-07 2016-06-22 北京京东尚科信息技术有限公司 Method and system for automatic splitting of task and parallel execution of sub-task
CN107729139A (en) * 2017-09-18 2018-02-23 北京京东尚科信息技术有限公司 A kind of method and apparatus for concurrently obtaining resource
CN109918184A (en) * 2019-03-01 2019-06-21 腾讯科技(深圳)有限公司 Picture processing system, method and relevant apparatus and equipment
CN110287018A (en) * 2019-07-04 2019-09-27 中国工商银行股份有限公司 Batch tasks method of combination and device
CN110471718A (en) * 2019-07-04 2019-11-19 深圳市共进电子股份有限公司 A kind of task processing method and equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
朱嘉舟;邵培南;陈景;: "影像数据分布并行计算处理平台体系架构研究", 计算机工程, no. 05, 15 May 2017 (2017-05-15) *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113485808A (en) * 2021-07-06 2021-10-08 中国工商银行股份有限公司 System time callback method and device
CN113485808B (en) * 2021-07-06 2024-06-28 中国工商银行股份有限公司 System time callback method and device
CN113986379A (en) * 2021-10-26 2022-01-28 北京字节跳动网络技术有限公司 Application starting method and device, computer equipment and storage medium
CN113986379B (en) * 2021-10-26 2023-12-15 抖音视界有限公司 Application starting method and device, computer equipment and storage medium
CN114331382A (en) * 2021-12-31 2022-04-12 中国邮政储蓄银行股份有限公司 Batch file processing method, processing device, processor and batch processing system
CN115495252A (en) * 2022-11-18 2022-12-20 苏州成科自控设备有限公司 Design method of MCU background task pool
CN115495252B (en) * 2022-11-18 2023-03-03 苏州成科自控设备有限公司 Design method of MCU background task pool
CN118093147A (en) * 2024-04-29 2024-05-28 浙江农商数字科技有限责任公司 Massive data summarizing method and system based on task chain and divide-and-conquer method

Similar Documents

Publication Publication Date Title
CN113064661A (en) Memory processing method and device for batch tasks, electronic equipment and storage medium
CN108804299B (en) Application program exception handling method and device
CN111415241A (en) Method, device, equipment and storage medium for identifying cheater
CN110780965B (en) Vision-based process automation method, equipment and readable storage medium
CN109040970B (en) Service planning handling method based on cloud computing and terminal equipment
EP2819015B1 (en) Method, terminal, and server for synchronizing terminal mirror
CN112328458A (en) Data processing method and device based on flink data engine
JP7389860B2 (en) Security information processing methods, devices, electronic devices, storage media and computer programs
CN110324208B (en) Data loss processing method, intelligent terminal and storage medium
CN116755844A (en) Data processing method, device and equipment of simulation engine and storage medium
CN113110965B (en) Monitoring method and device for abnormal information, computer storage medium and terminal
CN113660134B (en) Port detection method, device, electronic device and storage medium
CN110753090A (en) Task execution method and device of server cluster, computer equipment and storage medium
CN109766347B (en) Data updating method, device, system, computer equipment and storage medium
CN114048467A (en) Model data processing method, device, equipment and storage medium based on intrusion detection system
CN116932157A (en) Data acquisition method and device, electronic equipment and storage medium
CN114039765A (en) Safety management and control method and device for power distribution Internet of things and electronic equipment
CN109167808B (en) Data processing method, device and system
CN116820658B (en) Cloud application page rendering method, device, equipment and storage medium
CN107888445B (en) Method and device for analyzing performance state, computer equipment and storage medium
CN111756836A (en) Information sending method and device based on event management model
CN110516412A (en) The original information acquisition method of picture, device, computer equipment and storage medium
CN111931945B (en) Data processing method, device, equipment and storage medium based on label engine
CN113470701B (en) Audio and video editing method and device, computer equipment and storage medium
CN112463828B (en) Data processing method, device, equipment, system and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination