CN112307105A - Timing task running method, device, equipment and storage medium based on multithreading - Google Patents

Timing task running method, device, equipment and storage medium based on multithreading Download PDF

Info

Publication number
CN112307105A
CN112307105A CN202011207416.2A CN202011207416A CN112307105A CN 112307105 A CN112307105 A CN 112307105A CN 202011207416 A CN202011207416 A CN 202011207416A CN 112307105 A CN112307105 A CN 112307105A
Authority
CN
China
Prior art keywords
target
task
timing task
timing
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011207416.2A
Other languages
Chinese (zh)
Inventor
周光华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Puhui Enterprise Management Co Ltd
Original Assignee
Ping An Puhui Enterprise Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Puhui Enterprise Management Co Ltd filed Critical Ping An Puhui Enterprise Management Co Ltd
Priority to CN202011207416.2A priority Critical patent/CN112307105A/en
Publication of CN112307105A publication Critical patent/CN112307105A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/252Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • G06F16/2255Hash tables
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2308Concurrency control
    • G06F16/2336Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343Locking methods, e.g. distributed locking or locking implementation details
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/12Accounting
    • G06Q40/125Finance or payroll

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Finance (AREA)
  • Data Mining & Analysis (AREA)
  • Accounting & Taxation (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of cloud, and discloses a timing task operation method, a timing task operation device, timing task operation equipment and a timing task operation storage medium based on multiple threads, which are used for improving the utilization rate of resources and the working efficiency of data processing. The timing task running method based on multithreading comprises the following steps: acquiring a plurality of timing tasks and execution time; splitting the corresponding timing task into a plurality of sub-timing tasks based on each traffic data volume; establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads; when the target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the number of the plurality of temporary child nodes and the database lock; the state of the target sub-timing task is updated to be a working state through the database lock, the target sub-timing task is executed through the target running thread, and target task data are obtained.

Description

Timing task running method, device, equipment and storage medium based on multithreading
Technical Field
The invention relates to the technical field of cloud computing, in particular to a timing task running method, a timing task running device, timing task running equipment and a timing task running storage medium based on multithreading.
Background
The distributed system is a loosely coupled system formed by interconnecting a plurality of processors through communication lines, has various general physical and logical resources, can dynamically allocate tasks, and realizes information exchange through computer networks by the dispersed physical and logical resources.
At present, management systems of many companies execute a timing task in a distributed environment, a certain management system relates to a plurality of servers and a plurality of timing tasks, when a server needs to execute a certain timing task at a certain moment, two execution modes are generally provided, wherein one execution mode is to deploy the timing task to one of the servers for execution, and if the server fails, the timing task cannot be executed in time. Another way is to execute the timing task by multiple servers in a distributed environment, which results in repeated execution of the timing task, which wastes resources and reduces the work efficiency of processing data.
Disclosure of Invention
The invention provides a timing task running method, a timing task running device, timing task running equipment and a timing task running storage medium based on multiple threads, and the resource utilization rate and the data processing working efficiency are improved.
The invention provides a timing task running method based on multithreading in a first aspect, which comprises the following steps: acquiring a plurality of timing tasks and execution time, wherein the timing tasks are sent by a financial management system; reading corresponding service data volume based on each timing task, and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume; establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads; when a target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the temporary child node numbers and the database lock; and updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
Optionally, in a first implementation manner of the first aspect of the present invention, the creating a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads includes: reading a preset name of a database lock to obtain a name of the database lock, and creating a parent node of a path named by the name of the database lock; creating temporary child nodes corresponding to each running thread under the path of the father node, and reading the path of each temporary child node; and establishing temporary child node numbers corresponding to the temporary child nodes based on the path of each temporary child node to obtain a plurality of temporary child node numbers.
Optionally, in a second implementation manner of the first aspect of the present invention, when the target sub-timing task is triggered at the execution time, determining a target running thread among a plurality of running threads based on the plurality of temporary child node numbers and the database lock includes: when the target sub-timing task is triggered at the execution time, searching the smallest temporary sub-node number in the plurality of temporary sub-node numbers, and determining the smallest temporary sub-node number as the target sub-node number; and determining the corresponding running thread as a target running thread based on the target child node number.
Optionally, in a third implementation manner of the first aspect of the present invention, the updating, by the database lock, the state of the target sub-timing task to be a working state, and executing the target sub-timing task by the target running thread to obtain target task data corresponding to the target sub-timing task includes: adjusting the record of the database lock as a working record, so as to update the state of the target sub-timing task to be a working state; determining a target data hash value corresponding to the target sub-timing task through the target operation thread; and determining target task data corresponding to the target sub-timing task according to the target data hash value.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the determining, by the target operation thread, a target data hash value corresponding to the target sub-timing task includes: reading a target sub-timing task number from the target sub-timing task, and determining a target data partition corresponding to the target sub-timing task number in a plurality of preset data partitions through the target running thread, wherein the target sub-timing task number is the same as the number of the target data partition; determining a target data key name in the target data partition through the target running thread; and calculating a hash value based on a preset hash function by combining the target data key name and the number of the target sub-timing task to obtain a target hash value.
Optionally, in a fifth implementation manner of the first aspect of the present invention, before the obtaining of the multiple timing tasks and the execution time, where the multiple timing tasks are multiple tasks sent by the financial management system, the multithread-based timing task running method further includes: and adding a database lock in a preset database table, reading a thread threshold of a server, creating threads based on the thread threshold, and obtaining a plurality of running threads, wherein the database lock is used for locking the state of a task.
Optionally, in a sixth implementation manner of the first aspect of the present invention, after the updating, by the database lock, the state of the target sub-timing task to be a working state and the target running thread executes the target sub-timing task to obtain target task data corresponding to the target sub-timing task, the multithread-based timing task running method further includes: reading the state of the target sub-timing task in real time to obtain the state of the real-time target sub-timing task; judging whether the real-time target sub-timing task state is a disconnection state; and if the real-time target sub-timing task state is the disconnection state, determining a target operation thread in the plurality of operation threads again based on the database lock.
The second aspect of the present invention provides a timing task running device based on multiple threads, including: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of timing tasks and execution time, and the timing tasks are sent by a financial management system; the task splitting module is used for reading the corresponding service data volume based on each timing task and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume; the number creating module is used for creating a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads; a target thread determining module, configured to determine a target running thread among a plurality of running threads based on the plurality of temporary child node numbers and the database lock when a target child timing task is triggered at the execution time; and the task execution module is used for updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
Optionally, in a first implementation manner of the second aspect of the present invention, the number creating module is specifically configured to: reading a preset name of a database lock to obtain a name of the database lock, and creating a parent node of a path named by the name of the database lock; creating temporary child nodes corresponding to each running thread under the path of the father node, and reading the path of each temporary child node; and establishing temporary child node numbers corresponding to the temporary child nodes based on the path of each temporary child node to obtain a plurality of temporary child node numbers.
Optionally, in a second implementation manner of the second aspect of the present invention, the target thread determining module is specifically configured to: when the target sub-timing task is triggered at the execution time, searching the smallest temporary sub-node number in the plurality of temporary sub-node numbers, and determining the smallest temporary sub-node number as the target sub-node number; and determining the corresponding running thread as a target running thread based on the target child node number.
Optionally, in a third implementation manner of the second aspect of the present invention, the task execution module includes: the adjusting unit is used for adjusting the record of the database lock as a working record so as to update the state of the target sub-timing task to be a working state; a hash value determining unit, configured to determine, through the target operation thread, a target data hash value corresponding to the target sub-timing task; and the task data determining unit is used for determining target task data corresponding to the target sub-timing task according to the target data hash value.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the hash value determining unit is specifically configured to: reading a target sub-timing task number from the target sub-timing task, and determining a target data partition corresponding to the target sub-timing task number in a plurality of preset data partitions through the target running thread, wherein the target sub-timing task number is the same as the number of the target data partition; determining a target data key name in the target data partition through the target running thread; and calculating a hash value based on a preset hash function by combining the target data key name and the number of the target sub-timing task to obtain a target hash value.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the multithread-based timed task execution apparatus further includes: the creating module is used for adding a database lock in a preset database table, reading a thread threshold of the server, creating threads based on the thread threshold, and obtaining a plurality of running threads, wherein the database lock is used for locking the state of a task.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the multithread-based timed task execution apparatus further includes: the state reading module is used for reading the state of the target sub-timing task in real time to obtain the state of the real-time target sub-timing task; the judging module is used for judging whether the real-time target sub-timing task state is a disconnection state; and the other running thread determining module is used for determining a target running thread in the plurality of running threads again based on the database lock if the real-time target sub-timing task state is the disconnection state.
A third aspect of the present invention provides a multithread-based timed task execution apparatus, including: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line; the at least one processor calls the instructions in the memory to cause the multithreading-based timed task execution apparatus to execute the multithreading-based timed task execution method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the above-mentioned multithread-based timed task execution method.
According to the technical scheme, a plurality of timing tasks and execution time are obtained, wherein the timing tasks are sent by a financial management system; reading corresponding service data volume based on each timing task, and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume; establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads; when a target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the temporary child node numbers and the database lock; and updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task. In the embodiment of the invention, the timing task is divided into a plurality of sub-timing tasks, the target operation thread is determined for the target sub-timing task based on the database lock and the temporary sub-node number, and the state of the target sub-timing task is modified based on the database lock, so that the target sub-timing task is executed through the target operation thread to obtain the target data, and the resource utilization rate and the data processing working efficiency are improved.
Drawings
FIG. 1 is a diagram of an embodiment of a timing task execution method based on multithreading according to the embodiment of the invention;
FIG. 2 is a diagram of another embodiment of a method for timing task execution based on multithreading according to the embodiment of the present invention;
FIG. 3 is a diagram of an embodiment of a multithreading-based timed task execution apparatus according to an embodiment of the present invention;
FIG. 4 is a diagram of another embodiment of a multithread-based timed task execution apparatus according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an embodiment of a multithread-based timed task execution device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a timing task running method, a timing task running device, timing task running equipment and a timing task running storage medium based on multiple threads, which are used for improving the utilization rate of resources and the working efficiency of data processing.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a specific flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a method for running a multi-thread based timed task according to an embodiment of the present invention includes:
101. acquiring a plurality of timing tasks and execution time, wherein the timing tasks are a plurality of tasks sent by a financial management system;
the service acquires the execution time and a plurality of timed tasks issued by the financial management system.
It should be noted that the timing task is a business task sent by the financial management system, such as processing accounting data sent by accounting, compressing the current-month transaction data, and screening the current-month transaction reconciliation. The timed task may be a task executed at the execution time or a task triggered in the execution period, and in this embodiment, the timed task is mainly triggered at the execution time, and when the timed task is triggered at the execution time, operations such as data processing, data accounting, and data compression may be performed.
The execution main body of the present invention may be a timing task execution device based on multithreading, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
102. Reading corresponding service data volume based on each timing task, and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume;
and the server reads the data volume of each timing task to obtain the corresponding service data volume, and splits the corresponding timing task based on the service data volume of each timing task to obtain a plurality of sub-timing tasks.
The traffic data amount may be measured in terms of number of bytes or in terms of number of rows, for example, when the traffic data amount is measured in terms of the number of rows, the traffic data amount may be 10kw or 12kw, and when the traffic data amount is measured in terms of the number of bytes, the traffic data amount may be 500M or 1000M.
After the business data volume corresponding to each timing task is obtained, the server judges whether each business data volume is larger than a data volume threshold value; and when the service data volume corresponding to the target timing task is larger than the data volume threshold value, the server splits the target timing task into a plurality of sub timing tasks.
103. Establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
and the server creates a plurality of temporary child node numbers based on a plurality of preset running threads of the database lock.
When a plurality of temporary child node numbers are created, a database lock is used as a reference basis to create a path, and then numbering is carried out based on the created path to obtain the plurality of temporary child node numbers.
It should be noted that one running thread corresponds to one temporary child node number, the temporary child node number may be understood as a suffix of a temporary child node path, and the temporary child node number may be regarded as an identifier of the running thread, for example, if the temporary child node number is 001, then the corresponding running thread is a running thread with number 001. In this embodiment, a distributed application program coordination service zookeeper is mainly used to create a temporary child node number, and a path generated by the zookeeper is incremented, so that the obtained temporary child node number is also incremented.
104. When the target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the number of the plurality of temporary child nodes and the database lock;
when the target sub-timing task is triggered at the execution time, the multiple running threads try to acquire the database lock, the server determines the target running thread in the multiple running threads, and the server determines the basis of the target running thread as the number of the temporary sub-node.
When the target sub-timing task is triggered, if each running thread executes the target sub-timing task, the target sub-timing task is repeatedly executed, which wastes resources. At the moment, the database lock plays a role in reducing resource waste, namely before the running threads execute the target sub-timing task, each running thread prepares for executing the target sub-timing task, the preparation is used for each running thread to try to acquire the database lock, the server judges which running thread can acquire the database lock according to the temporary sub-node number corresponding to each running thread, and finally the server determines the running thread acquiring the database lock as the target running thread.
105. And updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
The server updates the state of the target sub-timing task through the database lock, updates the state of the target sub-timing task to be a working state, and then executes the target sub-timing task through the target running thread, so that target task data corresponding to the target sub-timing task are obtained.
When the state of the target sub-timing task is a working state, it is equivalent to that other running threads indicate that the execution qualification of the target sub-timing task is met, the other running threads do not execute the target sub-timing task any more, the target running thread executes the target sub-timing task at the moment to obtain corresponding target task data, and the target task data are service data in a database, such as billing data, transaction data, reconciliation data and the like.
In the embodiment of the invention, the timing task is divided into a plurality of sub-timing tasks, the target operation thread is determined for the target sub-timing task based on the database lock and the temporary sub-node number, and the state of the target sub-timing task is modified based on the database lock, so that the target sub-timing task is executed through the target operation thread to obtain the target data, and the resource utilization rate and the data processing working efficiency are improved.
Referring to fig. 2, another embodiment of the method for timing task execution based on multithreading according to the embodiment of the present invention includes:
201. adding a database lock in a preset database table, reading a thread threshold of a server, creating threads based on the thread threshold, and obtaining a plurality of running threads, wherein the database lock is used for locking the state of a task;
the method comprises the steps that a server adds a database lock for locking a task state in a preset database table, then reads a thread threshold of the server, creates threads based on the thread threshold, and obtains a plurality of running threads.
The server adds a record in a database table, the field name is lack _ name, namely a database lock, only one database lock is provided, only one running thread can acquire the database lock at the same time, the state of the task can be updated to be a working state through the database lock, and other threads cannot repeatedly execute the task. Each server has a thread threshold, for example, if the thread threshold of server a is 10, 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 running threads can be created, and 11 running threads cannot be created.
202. Acquiring a plurality of timing tasks and execution time, wherein the timing tasks are a plurality of tasks sent by a financial management system;
the service acquires the execution time and a plurality of timed tasks issued by the financial management system.
It should be noted that the timing task is a business task sent by the financial management system, such as processing accounting data sent by accounting, compressing the current-month transaction data, and screening the current-month transaction reconciliation. The timed task may be a task executed at the execution time or a task triggered in the execution period, and in this embodiment, the timed task is mainly triggered at the execution time, and when the timed task is triggered at the execution time, operations such as data processing, data accounting, and data compression may be performed.
203. Reading corresponding service data volume based on each timing task, and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume;
and the server reads the data volume of each timing task to obtain the corresponding service data volume, and splits the corresponding timing task based on the service data volume of each timing task to obtain a plurality of sub-timing tasks.
The traffic data amount may be measured in terms of number of bytes or in terms of number of rows, for example, when the traffic data amount is measured in terms of the number of rows, the traffic data amount may be 10kw or 12kw, and when the traffic data amount is measured in terms of the number of bytes, the traffic data amount may be 500M or 1000M.
After the business data volume corresponding to each timing task is obtained, the server judges whether each business data volume is larger than a data volume threshold value; and when the service data volume corresponding to the target timing task is larger than the data volume threshold value, the server splits the target timing task into a plurality of sub timing tasks.
204. Establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
and the server creates a plurality of temporary child node numbers based on a plurality of preset running threads of the database lock.
When a plurality of temporary child node numbers are created, a database lock is used as a reference basis to create a path, and then numbering is carried out based on the created path to obtain the plurality of temporary child node numbers.
It should be noted that one running thread corresponds to one temporary child node number, the temporary child node number may be understood as a suffix of a temporary child node path, and the temporary child node number may be regarded as an identifier of the running thread, for example, if the temporary child node number is 001, then the corresponding running thread is a running thread with number 001. In this embodiment, a distributed application program coordination service zookeeper is mainly used to create a temporary child node number, and a path generated by the zookeeper is incremented, so that the obtained temporary child node number is also incremented.
Specifically, the server reads a preset name of the database lock to obtain a name of the database lock, and creates a parent node of a path named by the name of the database lock; the server creates a temporary child node corresponding to each running thread under the path of the father node and reads the path of each temporary child node; and the server creates temporary child node numbers corresponding to the temporary child nodes based on the paths of the temporary child nodes to obtain a plurality of temporary child node numbers.
The method comprises the steps that a server reads a name of a preset database lock, wherein the name is lack _ name, the server takes the lack _ name as a path to create a father node, and the type of the father node is a persistent node; then when the multiple running threads try to acquire the database lock, child nodes corresponding to each running thread are created by taking the father nodes as the basis to obtain multiple temporary child nodes, wherein the types of the temporary child nodes are temporary self-numbering nodes; when the temporary child nodes are created, paths of the temporary child nodes are automatically created according to the paths of the father nodes, and at the moment, the server reads the paths of the temporary child nodes corresponding to each temporary child node; and the server acquires the temporary child node number corresponding to each temporary child node through the path suffix of each temporary child node.
For example, the path of the parent node is lack _ name, the paths of 6 temporary child nodes obtained based on the parent node are lack _ name _ base0001, lack _ name _ base0002, lack _ name _ base0003, lack _ name _ base0004, lack _ name _ base0005 and lack _ name _ base0006, respectively, and the server extracts the numbers from the path suffixes of each temporary child node to obtain the temporary child node number 0001, the temporary child node number 0002, the temporary child node number 0003, the temporary child node number 0004, the temporary child node number 0005 and the temporary child node number 0006, respectively.
205. When the target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the number of the plurality of temporary child nodes and the database lock;
when the target sub-timing task is triggered at the execution time, the multiple running threads try to acquire the database lock, the server determines the target running thread in the multiple running threads, and the server determines the basis of the target running thread as the number of the temporary sub-node.
When the target sub-timing task is triggered, if each running thread executes the target sub-timing task, the target sub-timing task is repeatedly executed, which wastes resources. At the moment, the database lock plays a role in reducing resource waste, namely before the running threads execute the target sub-timing task, each running thread prepares for executing the target sub-timing task, the preparation is used for each running thread to try to acquire the database lock, the server judges which running thread can acquire the database lock according to the temporary sub-node number corresponding to each running thread, and finally the server determines the running thread acquiring the database lock as the target running thread.
Specifically, when the target sub-timing task is triggered, the smallest temporary sub-node number is searched for from the plurality of temporary sub-node numbers, the smallest temporary sub-node number is determined as the target sub-node number, and then the corresponding running thread is determined based on the target sub-node number, so that the target running thread is obtained.
It should be noted that, in this embodiment, since the way that the zookeeper creates the self-number has an incrementing characteristic, it is described that the running thread corresponding to the smallest temporary child node number is closest to the database lock, and therefore the smallest temporary child node number is determined as the target child node number.
206. And updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
The server updates the state of the target sub-timing task through the database lock, updates the state of the target sub-timing task to be a working state, and then executes the target sub-timing task through the target running thread, so that target task data corresponding to the target sub-timing task are obtained.
When the state of the target sub-timing task is a working state, it is equivalent to that other running threads indicate that the execution qualification of the target sub-timing task is met, the other running threads do not execute the target sub-timing task any more, the target running thread executes the target sub-timing task at the moment to obtain corresponding target task data, and the target task data are service data in a database, such as billing data, transaction data, reconciliation data and the like.
Specifically, the server adjusts the record of the database lock as a working record, so that the state of the target sub-timing task is updated to be a working state; then the server determines a hash value corresponding to the target sub-timing task through the target thread to obtain a target data hash value; and finally, the server determines target task data in a database according to the target data hash value.
The specific process of determining the hash value corresponding to the target sub-timing task through the target thread is as follows:
the server reads a task number from the target sub-timing task to obtain a target sub-timing task number, determines a target data partition corresponding to the target sub-timing task number from a plurality of preset data partitions through a target operation thread, and the target sub-timing task number is the same as the number of the target data partition; then the server determines a target data key name in the target data partition through the target operation thread; and finally, the server calculates a hash value based on a preset hash function in combination with the key name of the target data and the number of the target sub-timing task to obtain the target hash value.
It should be noted that each sub-timing task includes a corresponding number, the data partition also has a corresponding number, and the number value of the sub-timing task is the same as the number of the data partition, if the number values of the plurality of sub-timing tasks are 1, 2, 3, 4, 5, and 5, respectively. The number of data partitions is 5, wherein the sub-timing task numbered 1 is located in data partition 1, the sub-timing task numbered 2 is located in data partition 2, the sub-timing task numbered 3 is located in data partition 3, the sub-timing task numbered 4 is located in data partition 4, and the sub-timing task numbered 5 is located in data partition 5. If the number of the target sub-timing task is 2, determining that the target data partition is a data partition 2, determining, by the server, a target data key name corresponding to the target sub-timing task in the data partition 2 through the target running thread, and then calculating, by the server, a target hash value by using a preset hash function, where the hash function is ora _ hash (id _ key, # count #) # index #, where id _ key is the target data key name and index is the number of the corresponding sub-task, thereby obtaining the target hash value. And finally, the server determines target task data according to the target hash value.
Besides the steps, the timing task running method based on multithreading also comprises the following steps:
the server reads the state of the target sub-timing task in real time to obtain the state of the real-time target sub-timing task; the server judges whether the real-time target sub-timing task state is a disconnection state; and if the real-time target sub-timing task state is a disconnected state, the server determines a target operation thread in the multiple operation threads again based on the database lock.
After the target running program executes the target subtask, the database lock is released, that is, the record of the database lock is updated to be the disconnection record, so that the state of the target subtask is in a disconnection state, and at the moment, the server can determine that other target running threads execute other target subtasks.
It should be noted that the time length for the target running program to execute the target subtask is 30s, and the server may determine whether the target subtask state is the disconnected state in real time, or may determine whether the target subtask state is the disconnected state every 30 s.
In the embodiment of the invention, the timing task is divided into a plurality of sub-timing tasks, the target operation thread is determined for the target sub-timing task based on the database lock and the temporary sub-node number, and the state of the target sub-timing task is modified based on the database lock, so that the target sub-timing task is executed through the target operation thread to obtain the target data, and the resource utilization rate and the data processing working efficiency are improved.
With reference to fig. 3, the timing task execution device based on multiple threads in the embodiment of the present invention includes:
an obtaining module 301, configured to obtain multiple timing tasks and execution time, where the multiple timing tasks are multiple tasks sent by a financial management system;
a task splitting module 302, configured to read a corresponding service data volume based on each timing task, and split the corresponding timing task into multiple sub-timing tasks based on each service data volume;
a number creating module 303, configured to create a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
a target thread determining module 304, configured to determine a target running thread among a plurality of running threads based on the plurality of temporary child node numbers and the database lock when a target child timing task is triggered at the execution time;
the task execution module 305 is configured to update the state of the target sub-timing task to be a working state through the database lock, and execute the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
In the embodiment of the invention, the timing task is divided into a plurality of sub-timing tasks, the target operation thread is determined for the target sub-timing task based on the database lock and the temporary sub-node number, and the state of the target sub-timing task is modified based on the database lock, so that the target sub-timing task is executed through the target operation thread to obtain the target data, and the resource utilization rate and the data processing working efficiency are improved.
Referring to fig. 4, another embodiment of the apparatus for timing task execution based on multiple threads according to the present invention includes:
an obtaining module 301, configured to obtain multiple timing tasks and execution time, where the multiple timing tasks are multiple tasks sent by a financial management system;
a task splitting module 302, configured to read a corresponding service data volume based on each timing task, and split the corresponding timing task into multiple sub-timing tasks based on each service data volume;
a number creating module 303, configured to create a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
a target thread determining module 304, configured to determine a target running thread among a plurality of running threads based on the plurality of temporary child node numbers and the database lock when a target child timing task is triggered at the execution time;
the task execution module 305 is configured to update the state of the target sub-timing task to be a working state through the database lock, and execute the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
Optionally, the number creating module 303 may be further specifically configured to:
reading a preset name of a database lock to obtain a name of the database lock, and creating a parent node of a path named by the name of the database lock;
creating temporary child nodes corresponding to each running thread under the path of the father node, and reading the path of each temporary child node;
and establishing temporary child node numbers corresponding to the temporary child nodes based on the path of each temporary child node to obtain a plurality of temporary child node numbers.
Optionally, the target thread determining module 304 may be further specifically configured to:
when the target sub-timing task is triggered at the execution time, searching the smallest temporary sub-node number in the plurality of temporary sub-node numbers, and determining the smallest temporary sub-node number as the target sub-node number;
and determining the corresponding running thread as a target running thread based on the target child node number.
Optionally, the task execution module 305 includes:
an adjusting unit 3051, configured to adjust a record of the database lock as a working record, so as to update the state of the target sub-timing task to a working state;
a hash value determination unit 3052, configured to determine, through the target operation thread, a target data hash value corresponding to the target sub-timing task;
a task data determining unit 3053, configured to determine, according to the target data hash value, target task data corresponding to the target sub-timing task.
Optionally, the hash value determination unit 3052 may be further specifically configured to:
reading a target sub-timing task number from the target sub-timing task, and determining a target data partition corresponding to the target sub-timing task number in a plurality of preset data partitions through the target running thread, wherein the target sub-timing task number is the same as the number of the target data partition;
determining a target data key name in the target data partition through the target running thread;
and calculating a hash value based on a preset hash function by combining the target data key name and the number of the target sub-timing task to obtain a target hash value.
Optionally, the multithreading-based timed task execution apparatus further includes:
the creating module 306 is configured to add a database lock in a preset database table, read a thread threshold of a server, create a thread based on the thread threshold, and obtain multiple running threads, where the database lock is used to lock a state of a task.
Optionally, the multithreading-based timed task execution apparatus further includes:
a state reading module 307, configured to read the state of the target sub-timing task in real time to obtain a real-time target sub-timing task state;
the judging module 308 is configured to judge whether the real-time target sub-timing task state is a disconnected state;
the other running thread determining module 309, if the real-time target sub-timing task state is the disconnected state, is configured to determine a target running thread among the multiple running threads again based on the database lock.
In the embodiment of the invention, the timing task is divided into a plurality of sub-timing tasks, the target operation thread is determined for the target sub-timing task based on the database lock and the temporary sub-node number, and the state of the target sub-timing task is modified based on the database lock, so that the target sub-timing task is executed through the target operation thread to obtain the target data, and the resource utilization rate and the data processing working efficiency are improved.
Fig. 3 and 4 above describe the multithreading-based timed task execution device in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the multithreading-based timed task execution device in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 5 is a schematic structural diagram of a multithreading-based apparatus 500 for executing timed tasks according to an embodiment of the present invention, which may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, one or more storage media 530 (e.g., one or more mass storage devices) for storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the multi-thread based timed task execution apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the multi-thread based timed task execution apparatus 500.
The multi-thread based timed task execution apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the multithreading-based timed task execution apparatus configuration shown in FIG. 5 does not constitute a limitation of multithreading-based timed task execution apparatus and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
Further, the computer usable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and which may also be a volatile computer readable storage medium, having stored therein instructions, which, when executed on a computer, cause the computer to perform the steps of the multithreading-based timed task execution method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A timing task running method based on multithreading is characterized by comprising the following steps:
acquiring a plurality of timing tasks and execution time, wherein the timing tasks are sent by a financial management system;
reading corresponding service data volume based on each timing task, and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume;
establishing a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
when a target sub-timing task is triggered at the execution time, determining a target operation thread in a plurality of operation threads based on the temporary child node numbers and the database lock;
and updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
2. The multithreading-based timed task execution method of claim 1, wherein the creating a plurality of temporary child node numbers based on a preset database lock and a plurality of preset execution threads comprises:
reading a preset name of a database lock to obtain a name of the database lock, and creating a parent node of a path named by the name of the database lock;
creating temporary child nodes corresponding to each running thread under the path of the father node, and reading the path of each temporary child node;
and establishing temporary child node numbers corresponding to the temporary child nodes based on the path of each temporary child node to obtain a plurality of temporary child node numbers.
3. The method of claim 2, wherein determining a target run thread among a plurality of run threads based on the plurality of temporary child node numbers and the database lock when the target child timed task is triggered at the execution time comprises:
when the target sub-timing task is triggered at the execution time, searching the smallest temporary sub-node number in the plurality of temporary sub-node numbers, and determining the smallest temporary sub-node number as the target sub-node number;
and determining the corresponding running thread as a target running thread based on the target child node number.
4. The multithreading-based timed task operating method according to claim 1, wherein the updating, by the database lock, the state of the target sub-timed task to be a working state, and executing the target sub-timed task by the target operating thread to obtain target task data corresponding to the target sub-timed task comprises:
adjusting the record of the database lock as a working record, so as to update the state of the target sub-timing task to be a working state;
determining a target data hash value corresponding to the target sub-timing task through the target operation thread;
and determining target task data corresponding to the target sub-timing task according to the target data hash value.
5. The multithreading-based timed task execution method of claim 4, wherein the determining, by the target execution thread, the target data hash value corresponding to the target sub-timed task comprises:
reading a target sub-timing task number from the target sub-timing task, and determining a target data partition corresponding to the target sub-timing task number in a plurality of preset data partitions through the target running thread, wherein the target sub-timing task number is the same as the number of the target data partition;
determining a target data key name in the target data partition through the target running thread;
and calculating a hash value based on a preset hash function by combining the target data key name and the number of the target sub-timing task to obtain a target hash value.
6. The multithreading-based timed task execution method of claim 1, wherein before the obtaining of the plurality of timed tasks and the execution time, the plurality of timed tasks being a plurality of tasks issued by a financial management system, the multithreading-based timed task execution method further comprises:
and adding a database lock in a preset database table, reading a thread threshold of a server, creating threads based on the thread threshold, and obtaining a plurality of running threads, wherein the database lock is used for locking the state of a task.
7. The multithreading-based timed task execution method of any one of claims 1 to 6, wherein after the updating of the state of the target sub-timed task by the database lock is a working state and the target sub-timed task is executed by the target execution thread to obtain target task data corresponding to the target sub-timed task, the multithreading-based timed task execution method further comprises:
reading the state of the target sub-timing task in real time to obtain the state of the real-time target sub-timing task;
judging whether the real-time target sub-timing task state is a disconnection state;
and if the real-time target sub-timing task state is the disconnection state, determining a target operation thread in the plurality of operation threads again based on the database lock.
8. A multithread-based timed task execution apparatus, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of timing tasks and execution time, and the timing tasks are sent by a financial management system;
the task splitting module is used for reading the corresponding service data volume based on each timing task and splitting the corresponding timing task into a plurality of sub-timing tasks based on each service data volume;
the number creating module is used for creating a plurality of temporary child node numbers based on a preset database lock and a plurality of preset running threads;
a target thread determining module, configured to determine a target running thread among a plurality of running threads based on the plurality of temporary child node numbers and the database lock when a target child timing task is triggered at the execution time;
and the task execution module is used for updating the state of the target sub-timing task to be a working state through the database lock, and executing the target sub-timing task through the target running thread to obtain target task data corresponding to the target sub-timing task.
9. A multithread-based timed task execution apparatus, characterized in that the multithread-based timed task execution apparatus comprises: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;
the at least one processor invokes the instructions in the memory to cause the multithreading-based timed task execution apparatus to perform the multithreading-based timed task execution method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for multithread-based timed task execution according to any one of claims 1 to 7.
CN202011207416.2A 2020-11-03 2020-11-03 Timing task running method, device, equipment and storage medium based on multithreading Pending CN112307105A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011207416.2A CN112307105A (en) 2020-11-03 2020-11-03 Timing task running method, device, equipment and storage medium based on multithreading

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011207416.2A CN112307105A (en) 2020-11-03 2020-11-03 Timing task running method, device, equipment and storage medium based on multithreading

Publications (1)

Publication Number Publication Date
CN112307105A true CN112307105A (en) 2021-02-02

Family

ID=74333901

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011207416.2A Pending CN112307105A (en) 2020-11-03 2020-11-03 Timing task running method, device, equipment and storage medium based on multithreading

Country Status (1)

Country Link
CN (1) CN112307105A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113806056A (en) * 2021-10-08 2021-12-17 腾讯科技(深圳)有限公司 Timed task processing method and device, computer equipment and storage medium
CN114428671A (en) * 2021-12-23 2022-05-03 杭州贝嘟科技有限公司 Data processing method, data processing device, electronic device and storage medium
CN115185673A (en) * 2022-05-17 2022-10-14 贝壳找房(北京)科技有限公司 Distributed timed task scheduling method, system, storage medium and program product

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107688500A (en) * 2017-07-26 2018-02-13 阿里巴巴集团控股有限公司 A kind of distributed task scheduling processing method, device, system and equipment
CN110474963A (en) * 2019-07-17 2019-11-19 中国平安人寿保险股份有限公司 A kind of resource access method based on zookeeper, system, medium and electronic equipment
CN110677453A (en) * 2019-08-15 2020-01-10 平安普惠企业管理有限公司 ZooKeeper-based distributed lock service implementation method, device, equipment and storage medium
CN111400012A (en) * 2020-03-20 2020-07-10 中国建设银行股份有限公司 Data parallel processing method, device, equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107688500A (en) * 2017-07-26 2018-02-13 阿里巴巴集团控股有限公司 A kind of distributed task scheduling processing method, device, system and equipment
CN110474963A (en) * 2019-07-17 2019-11-19 中国平安人寿保险股份有限公司 A kind of resource access method based on zookeeper, system, medium and electronic equipment
CN110677453A (en) * 2019-08-15 2020-01-10 平安普惠企业管理有限公司 ZooKeeper-based distributed lock service implementation method, device, equipment and storage medium
CN111400012A (en) * 2020-03-20 2020-07-10 中国建设银行股份有限公司 Data parallel processing method, device, equipment and storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113806056A (en) * 2021-10-08 2021-12-17 腾讯科技(深圳)有限公司 Timed task processing method and device, computer equipment and storage medium
CN113806056B (en) * 2021-10-08 2023-10-27 腾讯科技(深圳)有限公司 Timed task processing method, device, computer equipment and storage medium
CN114428671A (en) * 2021-12-23 2022-05-03 杭州贝嘟科技有限公司 Data processing method, data processing device, electronic device and storage medium
CN115185673A (en) * 2022-05-17 2022-10-14 贝壳找房(北京)科技有限公司 Distributed timed task scheduling method, system, storage medium and program product
CN115185673B (en) * 2022-05-17 2023-10-31 贝壳找房(北京)科技有限公司 Distributed timing task scheduling method, system, storage medium and program product

Similar Documents

Publication Publication Date Title
CN112307105A (en) Timing task running method, device, equipment and storage medium based on multithreading
CN109325854B (en) Block chain network, deployment method and storage medium
Tsai et al. Two-tier multi-tenancy scaling and load balancing
CN111400112B (en) Writing method and device of storage system of distributed cluster and readable storage medium
Chen et al. AAGA: Affinity-aware grouping for allocation of virtual machines
CN111459641B (en) Method and device for task scheduling and task processing across machine room
US20200052885A1 (en) File storage method and apparatus for blockchain network
CN111880939A (en) Container dynamic migration method and device and electronic equipment
CN111639309B (en) Data processing method and device, node equipment and storage medium
CN112307066B (en) Distributed data aggregation method, system, device and storage medium
Han et al. Cloud-aware processing of MapReduce-based OLAP applications
CN110233802A (en) A method of the block chain framework of the building more side chains of one main chain
CN115134371A (en) Scheduling method, system, equipment and medium containing edge network computing resources
KR102191586B1 (en) Fog computing method with blockchain-based idle resoure sharing
CN113127199A (en) Load balancing configuration method, device, equipment and storage medium
CN110602227A (en) Intelligent contract management method and related device
CN111858659A (en) Data query method, device and equipment based on row key salt value and storage medium
CN114416490A (en) Monitoring alarm method and device based on block chain, electronic equipment and storage medium
CN113342526A (en) Dynamic management and control method, system, terminal and medium for cloud computing mobile network resources
KR100609459B1 (en) parallel distributed processing system and method
CN111752961A (en) Data processing method and device
US8495634B2 (en) Method for the management of tasks in a decentralized data network
CN110928679B (en) Resource allocation method and device
de Rezende et al. MapReduce with components for processing big graphs
CN112749197A (en) Data fragment refreshing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination