CN112965828A - Multithreading data processing method, device, equipment and storage medium - Google Patents
Multithreading data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112965828A CN112965828A CN202110150242.9A CN202110150242A CN112965828A CN 112965828 A CN112965828 A CN 112965828A CN 202110150242 A CN202110150242 A CN 202110150242A CN 112965828 A CN112965828 A CN 112965828A
- Authority
- CN
- China
- Prior art keywords
- information
- data processing
- processing resource
- account
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 30
- 230000008014 freezing Effects 0.000 claims abstract description 13
- 238000007710 freezing Methods 0.000 claims abstract description 13
- 230000015654 memory Effects 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000010365 information processing Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/12—Accounting
- G06Q40/125—Finance or payroll
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Development Economics (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Complex Calculations (AREA)
Abstract
The application discloses a multithreading data processing method, a multithreading data processing device, a multithreading data processing equipment and a storage medium, which relate to the technical field of information processing technology, and the method comprises the following steps: receiving a data processing request, acquiring total amount information of data to be processed and data processing resource allowance information of each effective account, and acquiring initial data processing resource mean value information; respectively comparing the initial data processing resource mean value information with the data processing resource allowance information of each effective account, selecting a first target account set, and recording first quantity information; judging whether the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed; and if the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed, outputting the current data processing resource mean value information of each first target account. The method and the device efficiently and accurately complete the calculation and freezing operations of the data processing resources.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a multithread data processing method, apparatus, device, and storage medium.
Background
With the popularization and application of big data, it is necessary to perform more intelligent big data processing (for example, data transfer, etc.), and therefore, it is very important to collect, count, and monitor the relevant content of the big data. In the traditional database technology, mass data needs to be processed when data transfer is carried out, and due to the fact that the user data volume in the database is huge, the dynamic change of the user data is large, the reading and writing pressure of the system database is huge, the system load is high, the calculation speed is slow, and the data processing efficiency is low.
Therefore, how to provide a convenient and efficient data processing scheme is a technical problem to be solved urgently.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, an apparatus, a device, and a storage medium for processing multithread data, so as to efficiently and accurately complete calculation and freeze operations of data processing resources.
A first aspect of an embodiment of the present application provides a multithreading data processing method, including: receiving a data processing request; starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and dividing the total amount information of the data to be processed by the total amount of all the effective accounts to obtain initial data processing resource mean value information, wherein the effective accounts are accounts which are activated and the activation date of which is before the current date; starting multithreading, respectively and synchronously comparing the initial data processing resource average value information with the data processing resource allowance information of each effective account by a plurality of threads, selecting a first target account set of which the data processing resource allowance information is greater than or equal to the initial data processing resource average value information, and recording the total number of the first target accounts of the first target account set as first quantity information; judging whether the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed; and if the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed, outputting the initial data processing resource mean value information allocated to each first target account.
In an embodiment, if the product of the initial data processing resource mean information and the first quantity information is smaller than the total quantity information of the data to be processed, adding preset data processing resource quantity information to the initial data processing resource mean information to obtain second data processing resource mean information, and respectively and synchronously comparing the second data processing resource mean information with the data processing resource margin information of each effective account by a plurality of threads, selecting a second target account set in which the data processing resource margin information is greater than or equal to the second data processing resource mean information, and recording the total second target accounts of the second target account set as second quantity information; judging whether the product of the second data processing resource mean information and the second quantity information is larger than or equal to the total quantity information of the data to be processed, if the product of the second data processing resource mean information and the second quantity information is smaller than the total quantity information of the data to be processed, adding the second data processing resource mean information to the preset data processing resource quantity information to obtain third data processing resource mean information, starting multithreading to respectively and synchronously compare the third data processing resource mean information with the data processing resource surplus information of each effective account, selecting a third target account set of which the data processing resource surplus information is larger than or equal to the third data processing resource mean information, and recording the total number of the third target accounts of the third target account set as third quantity information, and judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, and circularly executing until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information distributed by each current target account.
In one embodiment, an account information operation instruction is received; starting a single thread to acquire the current data processing resource mean value information in the target database and the current data processing resource allowance information of each effective account according to the operation instruction; and starting multithreading to synchronously process the data of each effective account, and limiting the use permission of the current data processing resource allowance information of the effective account according to the current data processing resource average value information.
In an embodiment, the limiting the usage right of the current data processing resource allowance information of the valid account according to the current data processing resource average information includes: multithreading respectively and synchronously judges whether the current data processing resource allowance information of each effective account is larger than or equal to the current data processing resource mean value information; if the current data processing resource allowance information is larger than or equal to the current data processing resource average value information, closing the use permission of the effective account on part of the data processing resource information, which is equal to the current data processing resource average value information, in the current data processing resource allowance information, and recording permission closing information into a preset freezing table.
In an embodiment, after the closing the usage right of the effective account for the partial data processing resource information, in which the current data processing resource surplus information is equal to the current data processing resource mean information, and recording right closing information in a preset freeze table, the method further includes: after a preset time period, deducting the current data processing resource mean information from the current data processing resource surplus information of the active account.
In an embodiment, the limiting the usage right of the current data processing resource allowance information of the valid account according to the current data processing resource average information further includes: and if the current data processing resource allowance information is smaller than the current data processing resource mean value information, recording the account state as the data processing resource allowance information is insufficient, and sending prompt information.
A second aspect of the embodiments of the present application provides a multithreading data processing apparatus, including: the first receiving module is used for receiving a data processing request; the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and dividing the total amount information of the data to be processed by the total amount of all the effective accounts to obtain initial data processing resource average value information, wherein the effective accounts are accounts which are activated and the activation date of which is before the current date; a first comparing module, configured to enable multithreading, where multiple threads respectively and synchronously compare the initial data processing resource mean information with the data processing resource surplus information of each active account, select a first target account set where the data processing resource surplus information is greater than or equal to the initial data processing resource mean information, and record a total number of the first target accounts of the first target account set as first quantity information; a first judging module, configured to judge whether a product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of the to-be-processed data; and the output module is used for outputting the initial data processing resource mean information allocated to each first target account if the product of the initial data processing resource mean information and the first quantity information is larger than or equal to the total quantity information of the data to be processed.
In an embodiment, the second comparing module is configured to, if a product of the initial data processing resource mean information and the first quantity information is smaller than the total quantity information of the to-be-processed data, add preset data processing resource quantity information to the initial data processing resource mean information to obtain second data processing resource mean information, and synchronously compare the second data processing resource mean information with the data processing resource surplus information of each active account by a plurality of threads respectively, select a second target account set in which the data processing resource surplus information is greater than or equal to the second data processing resource mean information, and record a total second target account number of the second target account set as the second quantity information; a second determining module, configured to determine whether a product of the second data processing resource mean information and the second amount information is greater than or equal to total amount information of the to-be-processed data, if the product of the second data processing resource mean information and the second amount information is smaller than the total amount information of the to-be-processed data, add the second data processing resource mean information to the preset data processing resource amount information to obtain third data processing resource mean information, enable multithreading to respectively and synchronously compare the third data processing resource mean information with the data processing resource surplus information of each active account, select a third target account set in which the data processing resource surplus information is greater than or equal to the third data processing resource mean information, and record that a total number of the third target accounts of the third target account set is third amount information, and judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, and circularly executing until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information distributed by each current target account.
In one embodiment, the multithreaded data processing device further comprises: the second receiving module is used for receiving an account information operation instruction; the second acquisition module starts a single thread to acquire the current data processing resource mean value information in the target database and the current data processing resource allowance information of each effective account according to the operation instruction; and the limiting module is used for enabling multithreading to synchronously process the data of each effective account and limiting the use permission of the current data processing resource allowance information of the effective accounts according to the current data processing resource average information.
In an embodiment, the limiting module is configured to respectively and synchronously determine, by multiple threads, whether current data processing resource allowance information of each active account is greater than or equal to current data processing resource mean information; if the current data processing resource allowance information is larger than or equal to the current data processing resource average value information, closing the use permission of the effective account on part of the data processing resource information, which is equal to the current data processing resource average value information, in the current data processing resource allowance information, and recording permission closing information into a preset freezing table.
In an embodiment, the restriction module is configured to deduct the current data processing resource mean information from the current data processing resource surplus information of the active account after a preset time period.
In an embodiment, the limitation module is configured to record the account status as insufficient data processing resource allowance information and send a prompt message if the current data processing resource allowance information is smaller than the current data processing resource average information.
A third aspect of embodiments of the present application provides an electronic device, including: a memory to store a computer program; a processor configured to perform the method of the first aspect of the embodiments of the present application and any of the embodiments of the present application to process asset data.
A fourth aspect of embodiments of the present application provides a non-transitory electronic device-readable storage medium, including: a program which, when run by an electronic device, causes the electronic device to perform the method of the first aspect of an embodiment of the present application and any embodiment thereof.
In the embodiment of the application, the mean value information of the data processing resources finally allocated to each target account can be calculated in a polling mode, after the mean value information of the current data processing resources is checked and confirmed, the target accounts are frozen, then the information related in the whole process of balancing the data processing resources is disclosed, after the disclosure is completed, the mean value information of the data processing resources allocated to the target accounts is deducted, and meanwhile, the accounts which do not meet the conditions are marked, so that the whole process of balancing the data processing resources is more efficient and accurate.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a multithreading data processing method according to an embodiment of the present disclosure;
FIG. 3 is a flowchart illustrating a multithreading data processing method according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating a multithreading data processing method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a multithread data processing apparatus according to an embodiment of the present application.
Icon: 100-an electronic device; 110-a bus; 120-a processor; 130-a memory; 600-a multi-threaded data processing apparatus; 601-a first receiving module; 602-a first obtaining module; 603-a first comparison module; 604-a first determination module; 605-an output module; 606-a second comparison module; 607-second judging module; 608-a second receiving module; 609-a second acquisition module; 610-restriction module.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
In the description of the present application, the terms "first," "second," and the like are used for distinguishing between descriptions and do not denote an order of magnitude, nor are they to be construed as indicating or implying relative importance.
In the description of the present application, the terms "comprises," "comprising," and/or the like, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof.
In the description of the present application, the terms "mounted," "disposed," "provided," "connected," and "configured" are to be construed broadly unless expressly stated or limited otherwise. For example, it may be a fixed connection, a removable connection, or a unitary construction; can be mechanically or electrically connected; either directly or indirectly through intervening media, or may be internal to two devices, elements or components. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
Please refer to fig. 1, which is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application, and includes at least one processor 120 and a memory 130, where fig. 1 illustrates one processor as an example. The processors 120 and the memory 130 are coupled by a bus 110, and the memory 130 stores instructions executable by at least one of the processors 120, the instructions being executable by at least one of the processors 120 to cause the at least one of the processors 120 to perform a multi-threaded data processing method as in the embodiments described below.
In one embodiment, the Processor 120 may be a general-purpose Processor, including but not limited to a Central Processing Unit (CPU), a Network Processor (NP), etc., a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 120 is the control center of the electronic device 100 and connects the various parts of the entire electronic device 100 using various interfaces and lines. The processor 120 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application.
In one embodiment, the Memory 130 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, including but not limited to Random Access Memory (RAM), Read Only Memory (ROM), Static Random Access Memory (SRAM), Programmable Read-Only Memory (PROM), Erasable Read-Only Memory (EPROM), electrically Erasable Read-Only Memory (EEPROM), and the like.
In one embodiment, the electronic device 100 may also communicate with one or more external devices, such as a keyboard, a mouse, a bluetooth device, a pointing device, etc., so that the account may interact with the electronic device 100.
The structure of the electronic device 100 shown in fig. 1 is merely illustrative, and the electronic device 100 may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
Referring to fig. 2, a multithread data processing method according to an embodiment of the present invention is executed by the electronic device 100 shown in fig. 1, and includes the following steps:
step 210: a data processing request is received.
Step 220: starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and calculating to obtain initial data processing resource mean value information.
In this step, for example, the data processing resource surplus information data of the valid accounts are sequentially loaded into the memory from the target database at a speed of 10 ten thousand pieces each time until the number of the loaded data pieces is less than 10 ten thousand pieces, the total amount information of the data to be processed is the total amount information of the data to be processed which is planned to be deducted at this stage minus the balance resource amount information of the previous stage, the initial data processing resource mean value information is equal to the data processing resource mean value information accumulated preset data processing resource amount information, and the data processing resource mean value information is equal to the total amount information of the data to be processed divided by the total amount of all the valid accounts. The preset data processing resource amount information may be 1 minute, 2 minutes, 3 minutes or other numbers, and in this embodiment, the preset data processing resource amount information is 1 minute.
A valid account is an account whose account status is already activated and whose activation date is prior to the current date.
Step 230: and starting multithreading, respectively and synchronously comparing the initial data processing resource mean value information with the data processing resource allowance information of each effective account by a plurality of threads, selecting a target account set of which the data processing resource allowance information is greater than or equal to the initial data processing resource mean value information, and recording the number of target accounts.
In this step, while reading the loaded data, the total number of accounts is obtained by accumulation, that is, when the initial data processing resource mean value information is greater than the data processing resource surplus information of the effective accounts, the number of the target accounts is increased by 1, and when the data processing resource surplus information of each effective account is compared with the initial data processing resource mean value information, the accumulated number of the target accounts is the total number of accounts capable of participating in the task.
Step 240: and judging whether the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed.
In this step, if the product of the initial data processing resource mean value information and the number of the first number information is greater than or equal to the total amount information of the data to be processed, step 250 is performed; if the product of the initial data processing resource mean information and the number of the target accounts is less than the total amount information of the data to be processed, step 220 is entered.
Step 250: and outputting the average value information of the initial data processing resources allocated to each first target account.
In an embodiment, a first target account set with data processing resource margin information greater than or equal to initial data processing resource mean information is selected, the total number of first target accounts of the first target account set is recorded as first quantity information, whether the product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of data to be processed is judged, and if the product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of the data to be processed, current data processing resource mean information of each first target account is output as the initial data processing resource mean information.
If the product of the initial data processing resource mean value information and the first quantity information is smaller than the total quantity information of the data to be processed, adding preset data processing resource quantity information to the initial data processing resource mean value information to obtain second data processing resource mean value information, respectively comparing the second data processing resource mean value information with the data processing resource surplus information of each effective account, selecting a second target account set of which the data processing resource surplus information is larger than or equal to the second data processing resource mean value information, and recording the total number of second target accounts of the second target account set as the second quantity information; then judging whether the product of the second data processing resource mean value information and the second quantity information is larger than or equal to the total quantity information of the data to be processed, if the product of the second data processing resource mean value information and the second quantity information is smaller than the total quantity information of the data to be processed, executing in a circulating way: adding the second data processing resource average value information to the preset data processing resource amount information to obtain third data processing resource average value information, respectively comparing the third data processing resource mean value information with the data processing resource surplus information of each effective account, selecting a third target account set of which the data processing resource surplus information is greater than or equal to the third data processing resource mean value information, recording the total number of third target accounts of the third target account set as third quantity information, and judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, executing in a circulating way until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information as the current data processing resource mean value information.
As shown in fig. 3, which is a multithread data processing method according to an embodiment of the present application, the method may be executed by the electronic device 100 shown in fig. 1, and the method includes the following steps:
step 310: and receiving an account information operation instruction.
In an embodiment, before the step is implemented, the number of the target accounts and the current data processing resource mean value information of each account may be checked, and after a result of the checking is obtained, an account freezing operation is initiated to the digital resource processing system through the management background.
Step 320: and starting a single thread to acquire the current data processing resource mean value information in the target database and the current data processing resource allowance information of each effective account.
In this step, one thread is started to load the effective account data into the memory sequentially at a speed of 10 ten thousand pieces at a time from the target database until the number of the loaded data is less than 10 ten thousand pieces, and a plurality of threads are started to synchronously process the loaded account data, wherein the account data comprises the current data processing resource allowance information of the effective account.
Step 330: and respectively and synchronously judging whether the current data processing resource allowance information of each effective account is larger than or equal to the current data processing resource mean value information by multiple threads, if so, entering a step 340, and otherwise, entering a step 360.
Step 340: and limiting the current data processing resource allowance information use permission of the effective account.
In this step, limiting the current data processing resource surplus information usage right of the valid account includes: and closing the use permission of the effective account on partial data processing resource information, which is equal to the average value information of the current data processing resource, in the residual information of the current data processing resource, and recording permission closing information into a preset freezing table.
Step 350: and after a preset time period, deducting the current data processing resource mean value information from the current data processing resource surplus information of the effective account.
In this step, the preset time period may be set to be one week, two weeks, half a month, or another time period, in this embodiment, the preset time period is one week, and in the preset time period, the total amount information of the data to be processed, the current data processing resource average information of each target account, the total number of the target accounts, and other data are disclosed. After a preset time period, namely after the completion of the announcement, if no objection is proposed, the digital resource processing system automatically executes resource deduction operation at regular time, loads data in the freezing table by using a multi-path thread synchronous processing mode, deducts corresponding current data processing resource mean value information from the data processing resource surplus information of each target account according to the freezing data, writes the data into the record table, deletes the freezing record at the same time, and ends the whole digital resource processing flow.
Step 360: and recording the account state as insufficient data processing resource allowance information, and sending prompt information.
In one embodiment, an account whose account status is marked as insufficient data processing resource balance information will send a notification to remind the account to recharge, if the account is not recharged within 30 days, the account status is modified to exit and resource balance is returned for the account. If the account is charged within 30 days, the account state is changed into the activated state again, and the next digital resource processing flow can be participated.
Fig. 4 is a block diagram of a multithread data processing method according to an embodiment of the present invention, which can be executed by the electronic device 100 shown in fig. 1, and includes the following steps:
step 501: a data processing request is received.
Step 502: starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and calculating to obtain initial data processing resource mean value information. See the description of step 220 in the above embodiments for details.
Step 503: and starting multithreading, respectively and synchronously comparing the initial data processing resource mean value information with the data processing resource allowance information of each effective account by a plurality of threads, selecting a target account set of which the data processing resource allowance information is greater than or equal to the initial data processing resource mean value information, and recording the number of target accounts. See the description of step 230 in the above embodiments for details.
Step 504: and judging whether the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed or not, if so, entering a step 505, and otherwise, entering a step 502.
Step 505: and outputting the average value information of the initial data processing resources allocated to each first target account. See the description of step 250 in the above embodiments for details.
Step 506: and receiving an account information operation instruction. See the description of step 310 in the above embodiments for details.
Step 507: and starting a single thread to acquire the current data processing resource mean value information in the target database and the current data processing resource allowance information of each effective account. See the description of step 320 in the above embodiments for details.
Step 508: and respectively and synchronously judging whether the current data processing resource allowance information of each effective account is larger than or equal to the current data processing resource average value information or not by multithreading, if so, entering a step 509, and otherwise, entering a step 511.
Step 509: and limiting the current data processing resource allowance information use permission of the effective account. See the description of step 340 in the above embodiments for details.
Step 510: and after a preset time period, deducting the current data processing resource mean value information from the current data processing resource surplus information of the effective account. See the description of step 350 in the above embodiments for details.
Step 511: and recording the account state as insufficient data processing resource allowance information, and sending prompt information. See the description of step 360 in the above embodiments for details.
In this step, if the account status is that the valid account with insufficient data processing resource allowance information is not recharged within a first preset time limit, the account status is modified to exit and the resource allowance of the valid account is returned, and if the account status is that the valid account with insufficient data processing resource allowance information is recharged within the first preset time limit, the account status is changed to an activated status to participate in next digital resource processing.
The multithreading data processing method solves the problems that the prior method for freezing the account data processing resource allowance information needs to be operated in the daytime, and when the account has actions of recharging, quitting and the like, the resource allowance in the account can be changed at any time to cause inaccurate calculation; the problem of high reading and writing pressure of a target database is solved; meanwhile, the problems of high load and low calculation speed of a digital resource processing system due to the fact that a large amount of calculation is needed are solved.
Please refer to fig. 5, which is a multi-threaded data processing apparatus 600 according to an embodiment of the present application, the apparatus being applicable to the electronic device 100 shown in fig. 1, and the apparatus including: the device comprises a first receiving module 601, a first obtaining module 602, a first comparing module 603, a first judging module 604, an output module 605, a second comparing module 606, a second judging module 607, a second receiving module 608, a second obtaining module 609 and a limiting module 610, wherein the principle relationship of each module is as follows:
the first receiving module 601 is configured to receive a data processing request, which is described in detail in the foregoing embodiment for step 210.
A first obtaining module 602, configured to enable a single thread, obtain total amount information of data to be processed in the target database and data processing resource surplus information of each valid account, and divide the total amount information of the data to be processed by the total amount of all valid accounts to obtain initial data processing resource mean value information, where a valid account is an account that has been activated and has an activation date before a current date, which is described in detail in the foregoing embodiment for step 220.
The first comparing module 603 is configured to compare the initial data processing resource average information with the data processing resource surplus information of each valid account, select a first target account set in which the data processing resource surplus information is greater than or equal to the initial data processing resource average information, and record a total number of first target accounts of the first target account set as first quantity information, which is described in detail in the foregoing embodiment with reference to step 230.
A first determining module 604, configured to enable multithreading, determine whether a product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of the to-be-processed data, add the initial data processing resource mean information to preset data processing resource quantity information to obtain second data processing resource mean information if the product of the initial data processing resource mean information and the first quantity information is smaller than the total quantity information of the to-be-processed data, compare the second data processing resource mean information with data processing resource surplus information of each active account, select a second target account set whose data processing resource surplus information is greater than or equal to the second data processing resource mean information, and record a total second target account number of the second target account set as the second quantity information; and starting multithreading, judging whether the product of the second data processing resource mean value information and the second quantity information is larger than or equal to the total quantity information of the data to be processed, and if the product of the second data processing resource mean value information and the second quantity information is smaller than the total quantity information of the data to be processed, executing in a circulating way: adding the second data processing resource average value information to the preset data processing resource amount information to obtain third data processing resource average value information, respectively comparing the third data processing resource mean value information with the data processing resource surplus information of each effective account, selecting a third target account set of which the data processing resource surplus information is greater than or equal to the third data processing resource mean value information, recording the total number of third target accounts of the third target account set as third quantity information, and starting multithreading, judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, executing in a circulating way until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information as the current data processing resource mean value information.
An output module 605, configured to, if a product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of the data to be processed, output the current data processing resource mean information of each first target account as the initial data processing resource mean information, which is described in detail in the foregoing embodiment for step 250.
A second comparing module 606, configured to, if a product of the initial data processing resource mean information and the first quantity information is smaller than total quantity information of the data to be processed, add preset data processing resource quantity information to the initial data processing resource mean information to obtain second data processing resource mean information, and respectively and synchronously compare the second data processing resource mean information with data processing resource surplus information of each active account by multiple threads, select a second target account set in which the data processing resource surplus information is greater than or equal to the second data processing resource mean information, and record a total number of second target accounts of the second target account set as the second quantity information.
A second determination module 607, configured to determine whether a product of the second data processing resource mean information and the second quantity information is greater than or equal to total quantity information of the to-be-processed data, and if the product of the second data processing resource mean information and the second quantity information is smaller than the total quantity information of the to-be-processed data, add the second data processing resource mean information to preset data processing resource quantity information to obtain third data processing resource mean information, and enable multithreading to respectively and synchronously compare the third data processing resource mean information with data processing resource surplus information of each active account, select a third target account set whose data processing resource surplus information is greater than or equal to the third data processing resource mean information, record a total number of third target accounts of the third target account set as the third quantity information, determine whether a product of the third data processing resource mean information and the third quantity information is greater than or equal to the total quantity information of the to-be-processed data, and circularly executing until the product of the current data processing resource mean value information and the current number of the current target accounts is larger than the total amount information of the data to be processed, and outputting the current data processing resource mean value information distributed by each current target account.
The second receiving module 608 receives the account information freezing request, which is described in detail in the foregoing embodiment with reference to step 310.
The second obtaining module 609 obtains the current data processing resource average information in the target database and the current data processing resource surplus information of each valid account, which refer to the description of step 320 in the foregoing embodiment in detail.
The limiting module 610 limits, for each valid account, the usage right of the current data processing resource allowance information of the valid account according to the current data processing resource average information. In one embodiment, the limiting module 610, when multithreading is enabled, respectively determines whether the current data processing resource allowance information of each active account is greater than or equal to the current data processing resource mean information; if the current data processing resource allowance information is larger than or equal to the current data processing resource mean value information, closing the use permission of the effective account on part of data processing resource information, which is equal to the current data processing resource mean value information, in the current data processing resource allowance information, and recording permission closing information into a preset freezing table; after a preset time period, deducting current data processing resource mean value information from current data processing resource surplus information of the effective account; and if the current data processing resource allowance information is smaller than the current data processing resource average value information, recording the account state as the data processing resource allowance information is insufficient, and sending prompt information.
An embodiment of the present invention further provides a storage medium readable by an electronic device, including: a program that, when run on an electronic device, causes the electronic device to perform all or part of the procedures of the methods in the above-described embodiments. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like. The storage medium may also comprise a combination of memories of the kind described above.
In the embodiments provided in the present application, the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s).
In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The above description is only a preferred embodiment of the present application, and is only for the purpose of illustrating the technical solutions of the present application, and not for the purpose of limiting the present application. Any modification, equivalent replacement, improvement or the like, which would be obvious to one of ordinary skill in the art and would be within the spirit and principle of the present application, should be included within the scope of the present application.
Claims (10)
1. A method of multithreaded data processing, comprising:
receiving a data processing request;
starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and dividing the total amount information of the data to be processed by the total amount of all the effective accounts to obtain initial data processing resource mean value information, wherein the effective accounts are accounts which are activated and the activation date of which is before the current date;
starting multithreading, respectively and synchronously comparing the initial data processing resource average value information with the data processing resource allowance information of each effective account by a plurality of threads, selecting a first target account set of which the data processing resource allowance information is greater than or equal to the initial data processing resource average value information, and recording the total number of the first target accounts of the first target account set as first quantity information;
judging whether the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed;
and if the product of the initial data processing resource mean value information and the first quantity information is larger than or equal to the total quantity information of the data to be processed, outputting the initial data processing resource mean value information allocated to each first target account.
2. The method of claim 1, further comprising:
if the product of the initial data processing resource mean value information and the first quantity information is smaller than the total quantity information of the data to be processed, adding preset data processing resource quantity information to the initial data processing resource mean value information to obtain second data processing resource mean value information, respectively and synchronously comparing the second data processing resource mean value information with the data processing resource allowance information of each effective account by a plurality of threads, selecting a second target account set of which the data processing resource allowance information is larger than or equal to the second data processing resource mean value information, and recording the total second target account number of the second target account set as second quantity information;
judging whether the product of the second data processing resource mean information and the second quantity information is larger than or equal to the total quantity information of the data to be processed, if the product of the second data processing resource mean information and the second quantity information is smaller than the total quantity information of the data to be processed, adding the second data processing resource mean information to the preset data processing resource quantity information to obtain third data processing resource mean information, starting multithreading to respectively and synchronously compare the third data processing resource mean information with the data processing resource surplus information of each effective account, selecting a third target account set of which the data processing resource surplus information is larger than or equal to the third data processing resource mean information, and recording the total number of the third target accounts of the third target account set as third quantity information, and judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, and circularly executing until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information distributed by each current target account.
3. The method of claim 2, further comprising:
receiving an account information operation instruction;
starting a single thread to acquire the current data processing resource mean value information in the target database and the current data processing resource allowance information of each effective account according to the operation instruction;
and starting multithreading to synchronously process the data of each effective account, and limiting the use permission of the current data processing resource allowance information of the effective account according to the current data processing resource average value information.
4. The method of claim 3, wherein limiting the usage right of the current data processing resource allowance information of the valid account according to the current data processing resource average information comprises:
multithreading respectively and synchronously judges whether the current data processing resource allowance information of each effective account is larger than or equal to the current data processing resource mean value information;
if the current data processing resource allowance information is larger than or equal to the current data processing resource average value information, closing the use permission of the effective account on part of the data processing resource information, which is equal to the current data processing resource average value information, in the current data processing resource allowance information, and recording permission closing information into a preset freezing table.
5. The method according to claim 4, wherein after the closing the usage right of the valid account for the partial data processing resource information in the current data processing resource surplus information equal to the current data processing resource mean information, and recording right closing information into a preset freezing table, further comprises:
after a preset time period, deducting the current data processing resource mean information from the current data processing resource surplus information of the active account.
6. The method of claim 4, wherein limiting the current data processing resource balance information usage rights of the active account according to the current data processing resource mean information further comprises:
and if the current data processing resource allowance information is smaller than the current data processing resource mean value information, recording the account state as the data processing resource allowance information is insufficient, and sending prompt information.
7. A multi-threaded data processing apparatus, comprising:
the first receiving module is used for receiving a data processing request;
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for starting a single thread, acquiring total amount information of data to be processed in a target database and data processing resource allowance information of each effective account, and dividing the total amount information of the data to be processed by the total amount of all the effective accounts to obtain initial data processing resource average value information, wherein the effective accounts are accounts which are activated and the activation date of which is before the current date;
a first comparing module, configured to enable multithreading, where multiple threads respectively and synchronously compare the initial data processing resource mean information with the data processing resource surplus information of each active account, select a first target account set where the data processing resource surplus information is greater than or equal to the initial data processing resource mean information, and record a total number of the first target accounts of the first target account set as first quantity information;
a first judging module, configured to judge whether a product of the initial data processing resource mean information and the first quantity information is greater than or equal to total quantity information of the to-be-processed data;
and the output module is used for outputting the initial data processing resource mean information allocated to each first target account if the product of the initial data processing resource mean information and the first quantity information is larger than or equal to the total quantity information of the data to be processed.
8. The apparatus of claim 7, further comprising:
a second comparing module, configured to, if a product of the initial data processing resource mean information and the first quantity information is smaller than total quantity information of the data to be processed, add preset data processing resource quantity information to the initial data processing resource mean information to obtain second data processing resource mean information, and respectively and synchronously compare the second data processing resource mean information with the data processing resource surplus information of each active account by multiple threads, select a second target account set in which the data processing resource surplus information is greater than or equal to the second data processing resource mean information, and record a total number of the second target accounts of the second target account set as second quantity information;
a second determining module, configured to determine whether a product of the second data processing resource mean information and the second amount information is greater than or equal to total amount information of the to-be-processed data, if the product of the second data processing resource mean information and the second amount information is smaller than the total amount information of the to-be-processed data, add the second data processing resource mean information to the preset data processing resource amount information to obtain third data processing resource mean information, enable multithreading to respectively and synchronously compare the third data processing resource mean information with the data processing resource surplus information of each active account, select a third target account set in which the data processing resource surplus information is greater than or equal to the third data processing resource mean information, and record that a total number of the third target accounts of the third target account set is third amount information, and judging whether the product of the third data processing resource mean value information and the third quantity information is larger than or equal to the total quantity information of the data to be processed, and circularly executing until the product of the current data processing resource mean value information and the current quantity of the current target account is larger than the total quantity information of the data to be processed, and outputting the current data processing resource mean value information distributed by each current target account.
9. An electronic device, comprising:
a memory to store a computer program;
a processor to perform the method of any one of claims 1 to 6 to process asset data.
10. A non-transitory electronic device readable storage medium, comprising: program which, when run by an electronic device, causes the electronic device to perform the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110150242.9A CN112965828B (en) | 2021-02-03 | 2021-02-03 | Multithreading data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110150242.9A CN112965828B (en) | 2021-02-03 | 2021-02-03 | Multithreading data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112965828A true CN112965828A (en) | 2021-06-15 |
| CN112965828B CN112965828B (en) | 2024-03-19 |
Family
ID=76274441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110150242.9A Active CN112965828B (en) | 2021-02-03 | 2021-02-03 | Multithreading data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112965828B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130283276A1 (en) * | 2012-04-20 | 2013-10-24 | Qualcomm Incorporated | Method and system for minimal set locking when batching resource requests in a portable computing device |
| US20170195994A1 (en) * | 2016-01-04 | 2017-07-06 | Bank Of America Corporation | Resource optimization allocation system |
| CN108427604A (en) * | 2018-02-06 | 2018-08-21 | 华为技术有限公司 | Resource adjusting method, device and the cloud platform of cluster |
| US20190095950A1 (en) * | 2016-07-01 | 2019-03-28 | Tencent Technology (Shenzhen) Company Limited | Data-processing method and apparatus, and computer storage medium |
| CN109685642A (en) * | 2018-11-16 | 2019-04-26 | 北京三快在线科技有限公司 | Credit card method for processing resource, device, electronic equipment and storage medium |
| US20190391980A1 (en) * | 2018-06-26 | 2019-12-26 | Live Nation Entertainment, Inc. | Systems methods and apparatus for generation of a database object |
| CN111105146A (en) * | 2019-12-02 | 2020-05-05 | 网联清算有限公司 | Method, device and equipment for allocating resources to multiple accounts |
| CN111240841A (en) * | 2020-01-10 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Method and system for executing new task or processing resource revocation request |
| CN111966757A (en) * | 2020-10-20 | 2020-11-20 | 支付宝(杭州)信息技术有限公司 | Method and device for managing storage space of intelligent contract account |
| CN112232809A (en) * | 2020-09-24 | 2021-01-15 | 支付宝(杭州)信息技术有限公司 | Data management method, device and equipment for frozen account and storage medium |
| US10896430B1 (en) * | 2018-11-21 | 2021-01-19 | Coupa Software Incorporated | Detecting supplier fraud from digital transactional data |
-
2021
- 2021-02-03 CN CN202110150242.9A patent/CN112965828B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130283276A1 (en) * | 2012-04-20 | 2013-10-24 | Qualcomm Incorporated | Method and system for minimal set locking when batching resource requests in a portable computing device |
| US20170195994A1 (en) * | 2016-01-04 | 2017-07-06 | Bank Of America Corporation | Resource optimization allocation system |
| US20190095950A1 (en) * | 2016-07-01 | 2019-03-28 | Tencent Technology (Shenzhen) Company Limited | Data-processing method and apparatus, and computer storage medium |
| CN108427604A (en) * | 2018-02-06 | 2018-08-21 | 华为技术有限公司 | Resource adjusting method, device and the cloud platform of cluster |
| US20190391980A1 (en) * | 2018-06-26 | 2019-12-26 | Live Nation Entertainment, Inc. | Systems methods and apparatus for generation of a database object |
| CN109685642A (en) * | 2018-11-16 | 2019-04-26 | 北京三快在线科技有限公司 | Credit card method for processing resource, device, electronic equipment and storage medium |
| US10896430B1 (en) * | 2018-11-21 | 2021-01-19 | Coupa Software Incorporated | Detecting supplier fraud from digital transactional data |
| CN111105146A (en) * | 2019-12-02 | 2020-05-05 | 网联清算有限公司 | Method, device and equipment for allocating resources to multiple accounts |
| CN111240841A (en) * | 2020-01-10 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Method and system for executing new task or processing resource revocation request |
| CN112232809A (en) * | 2020-09-24 | 2021-01-15 | 支付宝(杭州)信息技术有限公司 | Data management method, device and equipment for frozen account and storage medium |
| CN111966757A (en) * | 2020-10-20 | 2020-11-20 | 支付宝(杭州)信息技术有限公司 | Method and device for managing storage space of intelligent contract account |
Non-Patent Citations (3)
| Title |
|---|
| 徐秀娟: "多用户操作系统避免死锁的矩阵搜索法", 机电工程技术, no. 07, 15 July 2004 (2004-07-15), pages 34 - 36 * |
| 潘文高 等: "基于相对资源量的集群系统的研究与实现", 现代计算机, no. 14, pages 1 - 8 * |
| 苑迎: "云环境下面向多租赁的虚拟资源分配关键技术研究", 中国博士学位论文电子期刊网, pages 41 - 61 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112965828B (en) | 2024-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108876560B (en) | Method and device for performing credit evaluation on work publisher based on block chain | |
| EP3129880B1 (en) | Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system | |
| CN110659137B (en) | Processing resource allocation method and system for offline tasks | |
| CN112749013B (en) | Thread load detection method and device, electronic equipment and storage medium | |
| CN107247665B (en) | Method and system for testing C interface function | |
| CN108459905B (en) | Resource pool capacity planning method and server | |
| CN109470918B (en) | Electric quantity data processing method of IR46 electric energy meter and IR46 electric energy meter | |
| CN108228679B (en) | Time series data metering method and time series data metering device | |
| CN113836184A (en) | Service persistence method and device | |
| CN115794342A (en) | Method and device for estimating remaining time of pipeline task and electronic equipment | |
| CN109656789A (en) | IO method for testing performance and system | |
| CN112965828A (en) | Multithreading data processing method, device, equipment and storage medium | |
| CN113326114A (en) | Batch task processing method and device | |
| CN115757066A (en) | Hard disk performance test method, device, equipment, storage medium and program product | |
| CN115984022A (en) | Unified account checking method and device for distributed payment system | |
| CN112529703B (en) | Method and device for selecting accounting node of blockchain | |
| CN115168005A (en) | Processing method and processing device for delayed task and electronic equipment | |
| CN114048231A (en) | Data processing method, device and computer program product | |
| CN111309475B (en) | Detection task execution method and equipment | |
| CN110083624B (en) | Stream data processing method, stream data processing apparatus, and computer medium | |
| CN113468398A (en) | Book level determination method, computing device and computer storage medium | |
| CN111967767A (en) | Business risk identification method, device, equipment and medium | |
| CN109993501B (en) | Demand process management method and device, storage medium and electronic equipment | |
| CN115269208B (en) | Resource processing method and system based on formula configuration | |
| CN114598705B (en) | Message load balancing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: Room 716, 7 / F, building 2, 28 Andingmen East Street, Dongcheng District, Beijing Applicant after: Beijing Easy Yikang Information Technology Co.,Ltd. Address before: Room 716, 7 / F, building 2, 28 Andingmen East Street, Dongcheng District, Beijing Applicant before: BEIJING QINGSONGCHOU INFORMATION TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |