CN109885580A - A kind of hot spot account batch updating method, apparatus, server and storage medium - Google Patents
A kind of hot spot account batch updating method, apparatus, server and storage medium Download PDFInfo
- Publication number
- CN109885580A CN109885580A CN201910114763.1A CN201910114763A CN109885580A CN 109885580 A CN109885580 A CN 109885580A CN 201910114763 A CN201910114763 A CN 201910114763A CN 109885580 A CN109885580 A CN 109885580A
- Authority
- CN
- China
- Prior art keywords
- account
- task
- hot spot
- subregion
- task data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000000638 solvent extraction Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 32
- 230000015654 memory Effects 0.000 claims abstract description 26
- 238000005192 partition Methods 0.000 claims abstract description 24
- 241001269238 Data Species 0.000 claims description 22
- 238000012790 confirmation Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Abstract
The embodiment of the invention discloses a kind of hot spot account batch updating method, apparatus, server and storage mediums.This method, comprising: task data is divided into multiple subregion tasks according to preset partitioning strategies；Multiple groups holding account is generated based on the associated multiple hot spot accounts of the task data according to preset partitioning strategies；The multiple subregion tasks of parallel processing to update the multiple groups holding account parallel；Updated multiple groups holding account is incorporated into hot spot account.Embodiment provided by the invention, by carrying out partitioning strategies to data, it is multiple groups holding account that hot spot account, which is corresponded to image copying according to data subregion, respectively using partition data to every group of holding account parallel processing, the execution speed of hot spot account more new data is greatly improved, realizes the ability that the concurrent processing hot spot account that can be adjusted configuration according to server cpu, memory and disk I/O ability updates.
Description
Technical field
The present embodiments relate to data processing technique more particularly to a kind of hot spot account batch updating method, apparatus, clothes
Business device and storage medium.
Background technique
With the rapid development of technology, enterprise's batch application is more and more extensive.Wherein, based on the complexity of software development
And the Spring frame created becomes the outstanding person in batch application.
Spring Batch sub-project under Spring frame is a lightweight, perfect batch processing frame, it is intended to
Enterprise is helped to establish healthy and strong, efficient batch application.Spring Batch is using Java language and based on Spring frame
Foundational development so that the developer of Spring frame or enterprise has been used to be easier to access and utilize enterprises service.
In Spring Batch batch processing, speed is executed in order to improve batch processing, can generally be wanted in business without special
The data asked carry out multidomain treat-ment, for needing to update high-volume detailed data into same record after calculating in business
Processing, such business are belonged to hot spot account replacement problem, are generally still handled using single thread, avoid update inconsistency.
Only one update of account synchronization is guaranteed using single thread processing for the processing of hot spot account, is not had dirty
Data or update inconsistency, but have the defects that as follows:
1, single thread handles all data one by one, and execution speed is slow, overall execution low efficiency.
2, single thread is handled, and cannot sufficiently be used cpu, memory, disk I/O, the presence server wasting of resources.
Summary of the invention
The embodiment of the present invention provides a kind of hot spot account batch updating method, apparatus, server and storage medium, to realize
The ability that the concurrent processing hot spot account of configuration updates can be adjusted according to server cpu, memory and disk I/O ability.
In a first aspect, the embodiment of the invention provides a kind of hot spot account batch updating methods, comprising:
Task data is divided into multiple subregion tasks according to preset partitioning strategies；
It is interim that multiple groups are generated based on the associated multiple hot spot accounts of the task data according to preset partitioning strategies
Account；
The multiple subregion tasks of parallel processing to update the multiple groups holding account parallel；
Updated multiple groups holding account is incorporated into hot spot account.
Optionally, the partitioning strategies include:
The total number for obtaining task data carries out drawing for the multiple subregion task according to the total number of the task data
Point；Or
The computing resource for executing task data is obtained, drawing for the multiple subregion task is carried out according to the computing resource
Point；Or
It obtains the total number of task data and executes the computing resource of task data；According to total item of the task data
Number and the division that the multiple subregion task is carried out according to the computing resource.
Optionally, the partitioning strategies include:
The total number for obtaining task data, the life of the multiple groups holding account is carried out according to the total number of the task data
At；Or
The computing resource for executing task data is obtained, the life of the multiple groups holding account is carried out according to the computing resource
At；Or
It obtains the total number of task data and executes the computing resource of task data；According to total item of the task data
Number and the generation that the multiple groups holding account is carried out according to the computing resource.
Optionally, the holding account is the multiple groups mirror image account replicated based on the hot spot account.
It is optionally, described that task data is divided into multiple subregion tasks according to preset partitioning strategies, comprising:
Obtain the total number of task data；
The corresponding configuration parameter of partitioning strategies is obtained, the configuration parameter includes at least the number of subregion；
The item number of each subregion task is calculated according to the total number of the task data and the configuration parameter；
It is arranged and saves the parameter of each subregion task, to generate the contextual information of subregion task executed in batches.
Optionally, described that task data is divided into multiple subregion tasks according to preset partitioning strategies, it later include: root
The database of store tasks data is divided into multiple partition spaces according to preset partitioning strategies comprising:
Obtain total mark that all task datas correspond to ID value；
The total number of task data is obtained according to total mark；
The corresponding number of partitions of the configuration parameter is obtained, the configuration parameter includes at least the number of subregion；
The item number of the task of each subregion is determined according to the total number and the number of partitions；
It is arranged and saves the parameter of each subregion task, to generate the contextual information of subregion task executed in batches.
Optionally, described raw based on the associated multiple hot spot accounts of the data task according to preset partitioning strategies
At multiple groups holding account, include: later
Whether the corresponding account information of confirmation task data is hot spot account in multiple groups holding account；
If the corresponding account information of task data is not the hot spot account in current partition, corresponding with task data
New holding account is generated based on account information.
Optionally, the multiple subregion tasks of the parallel processing to update the multiple groups holding account parallel, comprising:
Task name is obtained according to the contextual information；
The holding account for needing to update according to the confirmation of current partition task；
Data update is carried out to the holding account for needing to update according to the task name corresponding task.
Second aspect, the embodiment of the invention also provides a kind of hot spot account batch updating devices, comprising:
Subregion TU task unit, for task data to be divided into multiple subregion tasks according to preset partitioning strategies；
Holding account unit, for being with the associated multiple hot spot accounts of the task data according to preset partitioning strategies
Basis generates multiple groups holding account；
Parallel updating unit, for the multiple subregion tasks of parallel processing to be carried out parallel more to the multiple groups holding account
Newly；
Account combining unit, for updated multiple groups holding account to be incorporated into hot spot account.
The third aspect the embodiment of the invention also provides a kind of server, including memory, processor and is stored in storage
On device and the computer program that can run on a processor, the processor are realized in the embodiment of the present invention when executing described program
Any hot spot account batch updating method.
Fourth aspect, the embodiment of the invention also provides a kind of storage mediums, are stored thereon with computer program, the program
Any hot spot account batch updating method in the embodiment of the present invention is realized when being executed by processor.
Embodiment provided by the invention, it is by carrying out partitioning strategies to data, hot spot account is corresponding according to data subregion
Image copying is multiple groups holding account, then that multiple groups are interim respectively using partition data to every group of holding account parallel processing
The data summarization of account avoids multithreading and directly updates the conflict that will appear to hot spot account, greatly promote to hot spot account
The execution speed of hot spot account more new data, realization can be adjusted configuration according to server cpu, memory and disk I/O ability
Concurrent processing hot spot account update ability.
Detailed description of the invention
Fig. 1 is a kind of flow diagram for hot spot account batch updating method that the embodiment of the present invention one provides；
Fig. 2 is a kind of structural schematic diagram of hot spot account batch updating device provided by Embodiment 2 of the present invention；
Fig. 3 is a kind of structural schematic diagram for server that the embodiment of the present invention three provides；
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
It should be mentioned that some exemplary embodiments are described as before exemplary embodiment is discussed in greater detail
The processing or method described as flow chart.Although each step is described as the processing of sequence by flow chart, many of these
Step can be implemented concurrently, concomitantly or simultaneously.In addition, the sequence of each step can be rearranged.When its operation
The processing can be terminated when completion, it is also possible to have the additional step being not included in attached drawing.The processing can be with
Corresponding to method, function, regulation, subroutine, subprogram etc..
In addition, term " first ", " second " etc. can be used to describe herein various directions, movement, step or element etc.,
But these directions, movement, step or element should not be limited by these terms.These terms are only used to by first direction, movement, step
Rapid or element and another direction, movement, step or element are distinguished.For example, the case where not departing from scope of the present application
Under, it can be second speed difference by First Speed difference, and similarly, it is poor second speed difference can be known as First Speed
Value.First Speed difference and second speed difference both speed difference, but it is not same speed difference.Term " the
One ", " second " etc. is not understood to indicate or imply relative importance or implicitly indicates the number of indicated technical characteristic
Amount." first " is defined as a result, the feature of " second " can explicitly or implicitly include one or more of the features.
In the description of the present invention, the meaning of " plurality " is at least two, such as two, three etc., unless otherwise clearly specific limit
It is fixed.
Embodiment one
Fig. 1 is the flow chart for the hot spot account batch updating method that the embodiment of the present invention one provides, and the present embodiment is applicable
In hot spot account batch updating situation, this method can be executed by hot spot account batch updating device, specifically include following step
It is rapid:
Task data is divided into multiple subregion tasks according to preset partitioning strategies by step 110.
The task data of the present embodiment can be account pipelined data, is also possible to transaction data the present embodiment and does not limit
System.
Optionally, the partitioning strategies include: the total number for obtaining task data, according to the total number of the task data
Carry out the division of the multiple subregion task；Or the computing resource for executing task data is obtained, it is carried out according to the computing resource
The division of the multiple subregion task；Or it obtains the total number of task data and executes the computing resource of task data；According to
The total number of the task data and the division that the multiple subregion task is carried out according to the computing resource.The present embodiment
Computing resource includes but is not limited to server cpu resource, memory source and disk I/O resource or storage resource.It can in one embodiment
100,000 task datas of the corresponding 100 hot spot accounts of 1,000 ten thousand funds are divided into 10 subregions, each subregion includes
10000 task datas.100,000 task datas can certainly be divided into 100 or more subregions according to computing resource,
The present embodiment is with no restrictions.
Optionally, described that task data is divided into multiple subregion tasks according to preset partitioning strategies, comprising: to obtain and appoint
The total number for data of being engaged in；The corresponding configuration parameter of partitioning strategies is obtained, the configuration parameter includes at least the number of subregion；According to
The total number of the task data and the configuration parameter calculate the item number of each subregion task；It is arranged and saves each subregion and appoints
The parameter of business, to generate the contextual information of subregion task executed in batches.In one embodiment, according to above-mentioned corresponding 100
100,000 task datas of a hot spot account obtain the corresponding configuration parameter of partitioning strategies, and 100,000 task datas are divided by determination
10 subregions, then the task data item number for calculating each subregion is 10,000, generates contextual information according to calculated result, i.e.,
100,000 ID (can be the continuous number between from 1 to 100000) of corresponding 100,000 task datas are generated, and are each ID
It divides corresponding subregion (1 to 10 subregion), such as the ID of the 50003rd article of task data is 50003, the subregion of distribution is the 6th subregion.
100000 task datas generate total mark of corresponding all 100,000 task data ID after the completion of dividing, and generating includes total number
It is 10 for 100,000, number of partitions, each subregion task data item number is the letter such as title and ID of 10,000, every task datas
The subregion inventory of breath, i.e. contextual information.
Optionally, described that task data is divided into multiple subregion tasks according to preset partitioning strategies, it later include: root
The database of store tasks data is divided into multiple partition spaces according to preset partitioning strategies comprising: obtain all tasks
Data correspond to total mark of ID value；The total number of task data is obtained according to total mark；It is corresponding to obtain the configuration parameter
Number of partitions, the configuration parameter include at least subregion number；It is determined according to the total number and the number of partitions every
The item number of the task of a subregion；It is arranged and saves the parameter of each subregion task, it is upper to generate executing in batches for subregion task
Context information.In the present embodiment, by obtaining total mark of task data ID, to needing the data for carrying out partitioned storage to know
Not, such as confirmation is divided into that the total number of 100,000 task datas of 10 subregions is 100,000, number of partitions is 10, each subregion
Task data item number is the title and ID of 10,000, every task datas.Data are stored and are generated according to these information and are deposited
Store up inventory, i.e. contextual information.
Step 120 is generated based on the associated multiple hot spot accounts of the task data according to preset partitioning strategies
Multiple groups holding account.
The hot spot account of the present embodiment can be the huge multiple accounts of flowing water capital quantity, such as odd-numbered day cash flow water inventory
Multiple accounts more than 10,000,000 are also possible to multiple accounts that transaction stroke count compares concentration, such as odd-numbered day transaction stroke count is more than
100000 multiple accounts.
Optionally, the partitioning strategies include: the total number for obtaining task data, according to the total number of the task data
Carry out the generation of the multiple groups holding account；Or the computing resource for executing task data is obtained, it is carried out according to the computing resource
The generation of the multiple groups holding account；Or it obtains the total number of task data and executes the computing resource of task data；According to
The total number of the task data and the generation that the multiple groups holding account is carried out according to the computing resource.
Optionally, the holding account is the multiple groups mirror image account replicated based on the hot spot account.One embodiment
In, if 100,000 task datas of the corresponding 100 hot spot accounts of 1,000 ten thousand funds are divided into 10 subregions, each subregion
Include 10,000 task datas, then corresponds to 100 10 parts of hot spot account duplication generations, 10 groups of holding accounts, every group of interim account
Family is all the image copying of 100 hot spot accounts.
Optionally, described raw based on the associated multiple hot spot accounts of the data task according to preset partitioning strategies
It later include: to confirm whether the corresponding account information of task data is hot spot in multiple groups holding account at multiple groups holding account
Account；If the corresponding account information of task data is not the hot spot account in current partition, with the corresponding account of task data
New holding account is generated based on the information of family.
In the present embodiment, after the completion of 100,000 task datas all subregions of 100 hot spot accounts, in times new to other
During the business further subregion of data, if the corresponding account of new task data is in 100 current hot spot accounts
In, then without generating new holding account, if the corresponding account of new task data is not in 100 current hot spot accounts
In family, then need to generate new holding account.
Step 130, the multiple subregion tasks of parallel processing to update the multiple groups holding account parallel.
Optionally, the multiple subregion tasks of the parallel processing to update the multiple groups holding account parallel, comprising:
Task name is obtained according to the contextual information；The holding account for needing to update according to the confirmation of current partition task；According to described
The corresponding task of task name carries out data update to the holding account for needing to update.
In one embodiment, if 100,000 articles of task datas of the corresponding 100 hot spot accounts of 1,000 ten thousand funds are divided into
1 to the 10th subregion, each subregion include 10,000 task datas, then correspond to and generate the 1st for 100 10 parts of hot spot account duplication
To the 10th group of holding account.Then and be about to the 1st subregion 10,000 articles of task datas update to the 1st group of holding account, by the 2nd subregion
10,000 articles of task datas update to the 2nd group of holding account ... to update 10,000 articles of task datas of the 10th subregion to the 10th group and face
When account.
Subregion extension is carried out using Spring-batch tool in the present embodiment, different cachings is divided in memory and is used for
The data of holding account and corresponding subregion task direction are stored, the data that then parallel calling subregion task is directed toward are to multiple
Holding account carries out more capable.
Updated multiple groups holding account is incorporated into hot spot account by step 140.
In one embodiment, the data correspondence for the 1st to the 10th group of holding account that above-mentioned completion updates is incorporated into 100 heat
In point account, the movement once updated parallel is completed.Then repeat step 110-140 to other hot spot accounts for updating of needs into
Row is processed similarly, or is processed similarly to other task datas of identical hot spot account.
Embodiment provided by the invention, it is by carrying out partitioning strategies to data, hot spot account is corresponding according to data subregion
Image copying is multiple groups holding account, then that multiple groups are interim respectively using partition data to every group of holding account parallel processing
The data summarization of account avoids multithreading and directly updates the conflict that will appear to hot spot account, greatly promote to hot spot account
The execution speed of hot spot account more new data, realization can be adjusted configuration according to server cpu, memory and disk I/O ability
Concurrent processing hot spot account update ability.
Embodiment two
Hot spot account batch updating device provided by the embodiment of the present invention can be performed any embodiment of that present invention and be provided
Hot spot account batch updating method, have the corresponding functional module of execution method and beneficial effect.Fig. 2 is the embodiment of the present invention
A kind of structural schematic diagram of the two hot spot account batch updating devices provided, the device include:
Subregion TU task unit 210, for task data to be divided into multiple subregion tasks according to preset partitioning strategies；
Holding account unit 220 is used for according to preset partitioning strategies with the associated multiple hot spot accounts of the task data
Multiple groups holding account is generated based on family；
Parallel updating unit 230 carries out the multiple groups holding account for the multiple subregion tasks of parallel processing parallel
It updates；
Account combining unit 240, for updated multiple groups holding account to be incorporated into hot spot account.
Optionally, the partitioning strategies include:
The total number for obtaining task data carries out drawing for the multiple subregion task according to the total number of the task data
Point；Or
The computing resource for executing task data is obtained, drawing for the multiple subregion task is carried out according to the computing resource
Point；Or
It obtains the total number of task data and executes the computing resource of task data；According to total item of the task data
Number and the division that the multiple subregion task is carried out according to the computing resource.
Optionally, the partitioning strategies include:
The total number for obtaining task data, the life of the multiple groups holding account is carried out according to the total number of the task data
At；Or
The computing resource for executing task data is obtained, the life of the multiple groups holding account is carried out according to the computing resource
At；Or
It obtains the total number of task data and executes the computing resource of task data；According to total item of the task data
Number and the generation that the multiple groups holding account is carried out according to the computing resource.
Optionally, the holding account is the multiple groups mirror image account replicated based on the hot spot account.
Optionally, the subregion TU task unit 210 is also used to obtain the total number of task data；It is corresponding to obtain partitioning strategies
Configuration parameter；The item number of each subregion task is calculated according to the total number of the task data and the configuration parameter；Setting
And the parameter of each subregion task is saved, to generate the contextual information of subregion task executed in batches.
Optionally, the subregion TU task unit 210 is also used to described be divided task data according to preset partitioning strategies
It later include: that the database of store tasks data is divided into multiple points according to preset partitioning strategies for multiple subregion tasks
Area space comprising: obtain total mark that all task datas correspond to ID value；The total of task data is obtained according to total mark
Item number；Obtain the corresponding number of partitions of the configuration parameter；Each subregion is determined according to the total number and the number of partitions
Task item number；It is arranged and saves the parameter of each subregion task, to generate the context letter of subregion task executed in batches
Breath.
Optionally, holding account unit 220 is also used to described be associated with according to preset partitioning strategies with the data task
Multiple hot spot accounts based on generate multiple groups holding account, include: later the corresponding account information of confirmation task data whether
For the hot spot account in multiple groups holding account；If the corresponding account information of task data is not the hot spot account in current partition
Family then generates new holding account based on the corresponding account information of task data.
Optionally, the parallel updating unit 230 is also used to obtain task name according to the contextual information；According to working as
The holding account that preceding subregion task confirmation needs to update；The needs are updated according to the task name corresponding task interim
Account carries out data update.
Embodiment provided by the invention, it is by carrying out partitioning strategies to data, hot spot account is corresponding according to data subregion
Image copying is multiple groups holding account, then that multiple groups are interim respectively using partition data to every group of holding account parallel processing
The data summarization of account avoids multithreading and directly updates the conflict that will appear to hot spot account, greatly promote to hot spot account
The execution speed of hot spot account more new data, realization can be adjusted configuration according to server cpu, memory and disk I/O ability
Concurrent processing hot spot account update ability.
Embodiment three
Fig. 3 is a kind of structural schematic diagram for server that the embodiment of the present invention five provides, as shown in figure 3, the server packet
Include processor 30, memory 31, input unit 32 and output device 33；In server the quantity of processor 30 can be one or
It is multiple, in Fig. 3 by taking a processor 30 as an example；Processor 30, memory 31, input unit 32 and output device in server
33 can be connected by bus or other modes, in Fig. 3 for being connected by bus.
Memory 31 is used as a kind of computer readable storage medium, can be used for storing software program, journey can be performed in computer
Sequence and module, if the corresponding program instruction/module of hot spot account batch updating method in the embodiment of the present invention is (for example, heat
Subregion TU task unit 210, holding account unit 220, parallel updating unit 230 and account in point account batch updating device are closed
And unit 240).Software program, instruction and the module that processor 30 is stored in memory 31 by operation, thereby executing clothes
The various function application and data processing of business device, that is, realize above-mentioned hot spot account batch updating method.
Memory 31 can mainly include storing program area and storage data area, wherein storing program area can store operation system
Application program needed for system, at least one function；Storage data area, which can be stored, uses created data etc. according to terminal.This
Outside, memory 31 may include high-speed random access memory, can also include nonvolatile memory, for example, at least a magnetic
Disk storage device, flush memory device or other non-volatile solid state memory parts.In some instances, memory 31 can be further
Including the memory remotely located relative to processor 30, these remote memories can pass through network connection to server.On
The example for stating network includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Input unit 32 can be used for receiving the number or character information of input, and generate with the user setting of server with
And the related key signals input of function control.Output device 33 may include that display screen etc. shows equipment.
Example IV
The embodiment of the present invention four also provides a kind of storage medium comprising computer executable instructions, and the computer can be held
Row instruction is used to execute a kind of hot spot account batch updating method when being executed by computer processor, this method comprises:
Task data is divided into multiple subregion tasks according to preset partitioning strategies；
It is interim that multiple groups are generated based on the associated multiple hot spot accounts of the task data according to preset partitioning strategies
Account；
The multiple subregion tasks of parallel processing to update the multiple groups holding account parallel；
Updated multiple groups holding account is incorporated into hot spot account.
Certainly, a kind of storage medium comprising computer executable instructions, computer provided by the embodiment of the present invention
The method operation that executable instruction is not limited to the described above, can also be performed hot spot account provided by any embodiment of the invention
Relevant operation in batch updating method
By the description above with respect to embodiment, it is apparent to those skilled in the art that, the present invention
It can be realized by software and required common hardware, naturally it is also possible to which by hardware realization, but in many cases, the former is more
Good embodiment.Based on this understanding, technical solution of the present invention substantially in other words contributes to the prior art
Part can be embodied in the form of software products, which can store in computer readable storage medium
In, floppy disk, read-only memory (Read-Only Memory, ROM), random access memory (Random such as computer
Access Memory, RAM), flash memory (FLASH), hard disk or CD etc., including some instructions are with so that a computer is set
Standby (can be personal computer, server or the network equipment etc.) executes method described in each embodiment of the present invention.
It is worth noting that, in the embodiment of above-mentioned hot spot account batch updating device, included each unit and mould
Block is only divided according to the functional logic, but is not limited to the above division, and is as long as corresponding functions can be realized
It can；In addition, the specific name of each functional unit is also only for convenience of distinguishing each other, the protection model being not intended to restrict the invention
It encloses.
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (11)
1. a kind of hot spot account batch updating method characterized by comprising
Task data is divided into multiple subregion tasks according to preset partitioning strategies；
Multiple groups holding account is generated based on the associated multiple hot spot accounts of the task data according to preset partitioning strategies；
The multiple subregion tasks of parallel processing to update the multiple groups holding account parallel；
Updated multiple groups holding account is incorporated into hot spot account.
2. hot spot account batch updating method according to claim 1, which is characterized in that the partitioning strategies include:
The total number for obtaining task data, the division of the multiple subregion task is carried out according to the total number of the task data；
Or
The computing resource for executing task data is obtained, the division of the multiple subregion task is carried out according to the computing resource；Or
It obtains the total number of task data and executes the computing resource of task data；According to the total number of the task data with
And the division of the multiple subregion task is carried out according to the computing resource.
3. hot spot account batch updating method according to claim 1, which is characterized in that the partitioning strategies include:
The total number for obtaining task data, the generation of the multiple groups holding account is carried out according to the total number of the task data；
Or
The computing resource for executing task data is obtained, the generation of the multiple groups holding account is carried out according to the computing resource；Or
It obtains the total number of task data and executes the computing resource of task data；According to the total number of the task data with
And the generation of the multiple groups holding account is carried out according to the computing resource.
4. hot spot account batch updating method according to claim 1, which is characterized in that the holding account is with described
The multiple groups mirror image account replicated based on hot spot account.
5. hot spot account batch updating method according to claim 1, which is characterized in that described according to preset subregion plan
Task data is slightly divided into multiple subregion tasks, comprising:
Obtain the total number of task data；
The corresponding configuration parameter of partitioning strategies is obtained, the configuration parameter includes at least the number of subregion；
The item number of each subregion task is calculated according to the total number of the task data and the configuration parameter；
It is arranged and saves the parameter of each subregion task, to generate the contextual information of subregion task executed in batches.
6. hot spot account batch updating method according to claim 5, which is characterized in that described according to preset subregion plan
Task data is slightly divided into multiple subregion tasks, later includes: according to preset partitioning strategies by the number of store tasks data
Multiple partition spaces are divided into according to library comprising:
Obtain total mark that all task datas correspond to ID value；
The total number of task data is obtained according to total mark；
The corresponding number of partitions of the configuration parameter is obtained, the configuration parameter includes at least the number of subregion；
The item number of the task of each subregion is determined according to the total number and the number of partitions；
It is arranged and saves the parameter of each subregion task, to generate the contextual information of subregion task executed in batches.
7. hot spot account batch updating method according to claim 1, which is characterized in that described according to preset subregion plan
Multiple groups holding account is generated slightly based on the associated multiple hot spot accounts of the task data, includes: later
Whether the corresponding account information of confirmation task data is hot spot account in multiple groups holding account；
If the corresponding account information of task data is not the hot spot account in current partition, with the corresponding account of task data
New holding account is generated based on information.
8. hot spot account batch updating method according to claim 5 or 6, which is characterized in that the parallel processing is multiple
Subregion task to update the multiple groups holding account parallel, comprising:
Task name is obtained according to the contextual information；
The holding account for needing to update according to the confirmation of current partition task；
Data update is carried out to the holding account for needing to update according to the task name corresponding task.
9. a kind of hot spot account batch updating device characterized by comprising
Subregion TU task unit, for task data to be divided into multiple subregion tasks according to preset partitioning strategies；
Holding account unit is used for according to preset partitioning strategies based on the associated multiple hot spot accounts of the task data
Generate multiple groups holding account；
Parallel updating unit, for the multiple subregion tasks of parallel processing to be updated parallel to the multiple groups holding account；
Account combining unit, for updated multiple groups holding account to be incorporated into hot spot account.
10. a kind of server including memory, processor and stores the computer that can be run on a memory and on a processor
Program, which is characterized in that the processor realizes such as hot spot account described in any one of claims 1-8 when executing described program
Batch updating method.
11. a kind of storage medium, is stored thereon with computer program, which is characterized in that the realization when program is executed by processor
Such as hot spot account batch updating method described in any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910114763.1A CN109885580A (en) | 2019-02-14 | 2019-02-14 | A kind of hot spot account batch updating method, apparatus, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910114763.1A CN109885580A (en) | 2019-02-14 | 2019-02-14 | A kind of hot spot account batch updating method, apparatus, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109885580A true CN109885580A (en) | 2019-06-14 |
Family
ID=66928095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910114763.1A Pending CN109885580A (en) | 2019-02-14 | 2019-02-14 | A kind of hot spot account batch updating method, apparatus, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109885580A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347513A (en) * | 2019-07-15 | 2019-10-18 | 中国工商银行股份有限公司 | Hot spot data lot size scheduling method and device |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
| CN112307126A (en) * | 2020-11-24 | 2021-02-02 | 上海浦东发展银行股份有限公司 | Batch processing method and system for credit card account management data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105765952A (en) * | 2015-10-13 | 2016-07-13 | 深圳还是威健康科技有限公司 | Account registering method, terminal and server |
| CN106034149A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Account identification method and device |
| CN106649396A (en) * | 2015-11-03 | 2017-05-10 | 腾讯科技（深圳）有限公司 | Game level data sorting method, system and apparatus |
-
2019
- 2019-02-14 CN CN201910114763.1A patent/CN109885580A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106034149A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Account identification method and device |
| CN105765952A (en) * | 2015-10-13 | 2016-07-13 | 深圳还是威健康科技有限公司 | Account registering method, terminal and server |
| CN106649396A (en) * | 2015-11-03 | 2017-05-10 | 腾讯科技（深圳）有限公司 | Game level data sorting method, system and apparatus |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347513A (en) * | 2019-07-15 | 2019-10-18 | 中国工商银行股份有限公司 | Hot spot data lot size scheduling method and device |
| CN110347513B (en) * | 2019-07-15 | 2022-05-20 | 中国工商银行股份有限公司 | Hot data batch scheduling method and device |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
| CN111966289B (en) * | 2020-08-13 | 2024-02-09 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
| CN112307126A (en) * | 2020-11-24 | 2021-02-02 | 上海浦东发展银行股份有限公司 | Batch processing method and system for credit card account management data |
| CN112307126B (en) * | 2020-11-24 | 2022-09-27 | 上海浦东发展银行股份有限公司 | Batch processing method and system for credit card account management data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fanjul-Peyro et al. | Reformulations and an exact algorithm for unrelated parallel machine scheduling problems with setup times | |
| US11347481B2 (en) | Support system for designing an artificial intelligence application, executable on distributed computing platforms | |
| US20060229924A1 (en) | Data driven dynamic workflow | |
| CN109885580A (en) | A kind of hot spot account batch updating method, apparatus, server and storage medium | |
| Thevenin et al. | Makespan minimisation for a parallel machine scheduling problem with preemption and job incompatibility | |
| Yang | Single machine rescheduling with new jobs arrivals and processing time compression | |
| Jovanovic et al. | Fixed set search application for minimizing the makespan on unrelated parallel machines with sequence-dependent setup times | |
| Cano et al. | An algorithm to compute the minimal siphons in S 4 PR nets | |
| CN111949832A (en) | Method and device for analyzing dependency relationship of batch operation | |
| Davari et al. | Exact algorithms for single-machine scheduling with time windows and precedence constraints | |
| Abreu et al. | A new hybridization of adaptive large neighborhood search with constraint programming for open shop scheduling with sequence-dependent setup times | |
| Ferone et al. | Combining simulation with a GRASP metaheuristic for solving the permutation flow-shop problem with stochastic processing times | |
| Beaty | Genetic algorithms versus tabu search for instruction scheduling | |
| JP2845618B2 (en) | Expert system | |
| Shuvo et al. | A hybrid metaheuristic method for solving resource constrained project scheduling problem | |
| Kaler et al. | Communication-efficient graph neural networks with probabilistic neighborhood expansion analysis and caching | |
| Ecker et al. | Scheduling tasks on a flexible manufacturing machine to minimize tool change delays | |
| US20050278303A1 (en) | System and method for production planning utilizing on-line state-space planning | |
| WO2012027523A1 (en) | Job schedule generation using historical decision database | |
| CN110134434B (en) | Application generation processing method and system and application generation system | |
| US20160004982A1 (en) | Method and system for estimating the progress and completion of a project based on a bayesian network | |
| US7159012B2 (en) | Computational data processing system and computational process implemented by means of such a system | |
| CN114089889A (en) | Model training method, device and storage medium | |
| Sahu et al. | A task-level parallelism approach for process discovery | |
| CN112363819A (en) | Big data task dynamic scheduling method and device and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190614 |
|
| RJ01 | Rejection of invention patent application after publication |