CN109284177A - A kind of data-updating method and device - Google Patents
A kind of data-updating method and device Download PDFInfo
- Publication number
- CN109284177A CN109284177A CN201710594880.3A CN201710594880A CN109284177A CN 109284177 A CN109284177 A CN 109284177A CN 201710594880 A CN201710594880 A CN 201710594880A CN 109284177 A CN109284177 A CN 109284177A
- Authority
- CN
- China
- Prior art keywords
- data
- execution machine
- processing
- processing authority
- thread
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5022—Mechanisms to release resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data-updating method and devices, are related to field of computer technology.One specific embodiment of this method includes: that the data distribution of identical services number will be corresponded in the data of input to same execution machine using preset algorithm；The processing authority of data in this execution machine, the preset rules are configured according to preset rules in each execution machine are as follows: an only item data is configured with the processing authority in the data that synchronization corresponds to identical services number；The respective data transmission for being configured with the processing authority is executed into data update processing to data store set group by each execution machine；After the data for being configured with the processing authority in an execution machine are executed data update processing, which discharges the processing authority of the data.The embodiment can reduce processing delay, lifting system handling capacity under the premise of guaranteeing that data correctly update.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of data-updating methods and device.
Background technique
Nowadays big data is applied more and more in practical business, and technology relevant to data processing is also rapidly sent out
Exhibition.Under big data scene, in order to match the processing speed of not homologous ray, it will usually number is cached and transmitted using pipeline
According to, while for the processing capacity of lifting system, data can be handled using more example parallel forms in practical applications, it is this
Although mode accelerates the processing speed of system, but also bring the synchronized update problem of data.
In electric business business, the update of order status is common business scenario, and business side always wants to get order
Newest state, and the more new information of a plurality of order status is possible in more examples processed simultaneously, therefore system is true
Order status is protected to be updated correctly.Existing technical solution uses to entire order update method or carries out thing to each order
Business processing, i.e. affairs lock.Wherein, it is either explained using the affairs of Spring, or uses Synchronized keyword
(being a kind of locking method that Spring frame provides, bottom layer realization is the packaging to Synchronized), more to entire order
New method is locked, and whole system is become serial process by parallel processing, and need to consider the expense of locking and de-locking,
And each order lock and needs to carry out each order plus unlock operation, performance cost is larger, while different instances pair
Same order, which lock, is related to distributed lock, increases the complexity of system.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery:
The processing capacity that existing scheme will cause system declines to a great extent, and increases the delay of state update.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of data-updating method and device, it can guarantee data correctly more
Under the premise of new, processing delay, lifting system handling capacity are reduced.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of data-updating method is provided.
A kind of data-updating method, comprising: the number of identical services number will be corresponded in the data of input using preset algorithm
According to being distributed to same execution machine；The processing authority of data in this execution machine, institute are configured according to preset rules in each execution machine
State preset rules are as follows: an only item data is configured with the disposal right in the data that synchronization corresponds to identical services number
Limit；By each execution machine by the respective data transmission for being configured with the processing authority to data store set group to execute number
It is handled according to update；Processing is updated when the data for being configured with the processing authority in an execution machine are executed the data
Afterwards, which discharges the processing authority of the data.
Optionally, the preset algorithm is hash algorithm or consistency hash algorithm.
Optionally, the step of configuring the processing authority of data in this execution machine according to preset rules in each execution machine,
It include: the processing authority for configuring all data one by one according to the time that data reach this execution machine, wherein configuring every item data
Processing authority when, execute following steps: step 1, judge whether have in other data for numbering with the item data identical services
Data have been configured with the processing authority, if so, step 2 is executed, if it is not, thening follow the steps three；Step 2 waits pre-
If time interval, then returns and execute the step 1；Step 3 makes the item data be configured with the processing authority.
Optionally, the step of data of the input include timestamp information, and the execution data update processing includes: pair
Each single item is transmitted to the data of the data store set group, and the data store set group judges whether there is the data with the transmission
The canned data of identical services number；If so, the timestamp for comparing both data of the canned data and the transmission is believed
Breath, using the newest data of timestamp in described the two as new canned data；If it is not, the data of the transmission are then stored in institute
State data store set group.
Optionally, the processing authority is the permission using thread, is configured by each execution machine by respective
The step of data transmission of processing authority is stated to data store set group, comprising: be configured with by each execution machine by respective
Pass through the thread being randomly assigned using the data of the permission of thread and be sent to data store set group, the described execution machine discharges the number
According to the processing authority the step of, comprising: the described execution machine removes the data to the occupancy of thread, judgement and the item data
Whether there is the step of data have been configured with the processing authority in other data of identical services number, comprising: judge and be somebody's turn to do
Whether there are data occupying thread in other data of item data identical services number.
Optionally, it is originally held using the multithreading Hash set built in Java according to preset rules configuration in each execution machine
The processing authority of data in row machine.
According to another aspect of an embodiment of the present invention, a kind of data update apparatus is provided.
A kind of data update apparatus, comprising: data distribution module, for that will be corresponded in the data of input using preset algorithm
The data distribution of identical services number is to same execution machine；Permission configuration module is used in each execution machine according to default rule
Then configure the processing authority of data in this execution machine, the preset rules are as follows: correspond to the number of identical services number in synchronization
Only have an item data to be configured with the processing authority in；Data transmitting module, being used for will be respective by each execution machine
The data transmission for being configured with the processing authority is handled to data store set group with executing data update；Permission release module,
For when the data that the processing authority is configured in an execution machine be executed the data update processing after, the execution
Machine discharges the processing authority of the data.
Optionally, the preset algorithm is hash algorithm or consistency hash algorithm.
Optionally, the permission configuration module is also used to: configuring items one by one according to the time that data reach this execution machine
The processing authority of data, wherein when configuring the processing authority of every item data, execute following steps: step 1, judge and this
Whether there are data to be configured with the processing authority in other data of data identical services number, if so, thening follow the steps
Two, if it is not, thening follow the steps three；Step 2 waits prefixed time interval, then returns and executes the step 1；Step 3 makes
The item data is configured with the processing authority.
Optionally, the step of data of the input include timestamp information, and the execution data update processing includes: pair
Each single item is transmitted to the data of the data store set group, and the data store set group judges whether there is the data with the transmission
The canned data of identical services number；If so, the timestamp for comparing both data of the canned data and the transmission is believed
Breath, using the newest data of timestamp in described the two as new canned data；If it is not, the data of the transmission are then stored in institute
State data store set group.
Optionally, the processing authority is the permission using thread, and the data transmitting module is also used to: being held by each
Respective be configured with is sent to data store set group by the thread being randomly assigned using the data of the permission of thread by row machine,
The permission release module is also used to: the described execution machine removes the data to the occupancy of thread, and the permission configuration module is also
For: whether there are data occupying thread in judgement and other data of item data identical services number.
Optionally, the permission configuration module is also used to: the multithreading Hash built in Java is used in each execution machine
Set configures the processing authority of data in this execution machine according to preset rules.
Another aspect according to an embodiment of the present invention, provides a kind of server.
A kind of server, comprising: one or more processors；Memory, for storing one or more programs, when described
When one or more programs are executed by one or more of processors, so that one or more of processors realize data more
New method.
Another aspect according to an embodiment of the present invention, provides a kind of computer-readable medium.
A kind of computer-readable medium is stored thereon with computer program, number is realized when described program is executed by processor
According to update method.
One embodiment in foregoing invention has the following advantages that or the utility model has the advantages that using preset algorithm by the data of input
The data distribution of middle corresponding identical services number is to same execution machine；This execution is configured according to preset rules in each execution machine
The processing authority of data in machine, wherein preset rules are an only item number in the data that synchronization corresponds to identical services number
According to being configured with processing authority；The respective data transmission for being configured with processing authority to data is stored by each execution machine
Cluster is to execute data update processing；When the data for being configured with processing authority in an execution machine are executed data update processing
Later, which discharges the processing authority of the data.Can be under the premise of guaranteeing that data correctly update, reduction processing is prolonged
Late, lifting system handling capacity.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the key step schematic diagram of data-updating method according to an embodiment of the present invention；
Fig. 2 is the block schematic illustration of order slip condition update method according to an embodiment of the present invention；
Fig. 3 is the order status number according to an embodiment of the present invention based on consistency hash algorithm distribution order slip condition
According to update flow diagram；
Fig. 4 is the main modular schematic diagram of data update apparatus according to an embodiment of the present invention；
Fig. 5 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the server of the embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
Fig. 1 is the key step schematic diagram of data-updating method according to an embodiment of the present invention.
As shown in Figure 1, the data-updating method of the embodiment of the present invention mainly includes the following steps, namely S101 to step S104.
Step S101: the data distribution that identical services number is corresponded in the data of input is held to same using preset algorithm
Row machine.
Wherein, the data of input may include timestamp information, and the data of the embodiment of the present invention update mainly according to input
Data canned data is updated to the newest data of timestamp, therefore, the data-updating method of the embodiment of the present invention is suitable for
The scene that various data update, such as the scene that updates of order slip condition in electric business field, correspondingly, the data of input can be with
For order slip condition, business number can be order number.
Preset algorithm can be hash algorithm or consistency hash algorithm.It should be noted that Hash described herein is calculated
Method can be the various hash algorithms in addition to consistency hash algorithm.
Step S102: the processing authority of data in this execution machine is configured according to preset rules in each execution machine.
The time that this execution machine can specifically be reached according to data configures the processing authority of all data one by one, wherein is matching
When setting the processing authority of every item data, following steps are executed:
Whether there are data to be configured with processing in step 1, judgement and other data of item data identical services number
Permission, if so, step 2 is executed, if it is not, thening follow the steps three；
Step 2 waits prefixed time interval, then returns to step one；
Step 3 makes the item data be configured with processing authority.
Wherein, preset rules are as follows: an only item data is configured in the data that synchronization corresponds to identical services number
There is processing authority.
In each execution machine, the multithreading Hash set built in Java can be used to configure this execution machine according to preset rules
The processing authority of middle data.
Processing authority is specifically as follows the permission using thread.
Step S103: by each execution machine by the respective data transmission for being configured with processing authority to data store set
Group is to execute data update processing.
By each execution machine by the respective data transmission for being configured with processing authority to data store set group the step of,
Can specifically include: by each execution machine by it is respective be configured with to pass through using the data of the permission of thread be randomly assigned
Thread is sent to data store set group.
After an item data is configured with the permission using thread, a thread can be probabilistically assigned to handle the item number
According to the item data is mainly transmitted to data store set group and executed in data store set group by the thread process item data
Data update processing, specifically, the thread can call and execute the method that data update processing in data store set group, by data
It is transmitted to data store set group, this method is executed by data store set group to realize that data update is handled.
Data store set group such as Redis cluster.
Executing the step of data update processing can specifically include: the data of data store set group are transmitted to each single item,
Data store set group judges whether there is the canned data with the data identical services of transmission number；It has been deposited if so, comparing
The timestamp information of both data and the data of transmission, using the newest data of timestamp in the two as new canned data；If
It is no, then the data of transmission are stored in data store set group.
Step S104: after the data for being configured with processing authority in an execution machine are executed data update processing,
The execution machine discharges the processing authority of the data.
The processing authority of execution machine release data mainly removes the data to the occupancy of thread.
Judge whether there are data to be configured in other data numbered with the item data identical services in step S102
The step of processing authority, can specifically include: whether have data in judgement and other data of item data identical services number
Occupying thread.
Implementation of the present invention is discussed in detail so that the data of input are order slip condition, business number is order number as an example below
The main flow of the data-updating method of example.
Fig. 2 is the block schematic illustration of order slip condition update method according to an embodiment of the present invention.As shown in Fig. 2, with
The data of input is for order slip condition, the order slip condition update method of the embodiment of the present invention can be held by server
Row is passed through wherein may include primary server and multiple execution machines (execution machine is alternatively referred to as example) by primary server reception
Each order slip condition pipeline (order by order slip condition pipeline 1, order slip condition pipeline 2, order slip condition pipeline 3
Single status data pipeline 4 ... ...) input order slip condition, the data of input are distributed again using hash algorithm, by it
The order slip condition of middle corresponding same order number is distributed to same execution machine, by each execution machine (execute machine 1, execute machine 2,
Execution machine 3, executes machine 4 ... ...) respective order slip condition is transmitted to data store set group to execute at data update
Reason.
It, can be by quantity N of the quantity M of order slip condition to execute machine in the realization of hash algorithm (N is natural number)
Order slip condition is distributed to each execution machine by the method for remainder, with 10 execution machines (N=10), 20 order slip condition (M
=20) for, on 10 remainders, then the 1st article and Sub_clause 11 order slip condition are distributed to the 1st execution machine, and the 2nd article and the
12 articles of order slip conditions are distributed to the 2nd execution machine ... ..., and so on.Hash used in the embodiment of the present invention is calculated
Method is specially highly reliable consistency hash algorithm highly fault tolerant, and consistency hash algorithm is a kind of better Hash calculation of stability
Method, it will be appreciated by persons skilled in the art that the embodiment of the present invention also may be implemented by other hash algorithms.
Fig. 3 is the order status number according to an embodiment of the present invention based on consistency hash algorithm distribution order slip condition
According to update flow diagram.As shown in figure 3, when execution machine receives the order slip condition after the distribution of consistency hash algorithm
Later, pass through the ConcurrentHashMap (i.e. multithreading Hash set, also referred to as the Hash set of thread-safe) built in Java
Processing authority (processing authority is specifically as follows the permission using thread) is configured for the order slip condition in the execution machine,
In, when the execution machine handles a certain order slip condition, ConcurrentHashMap judges whether to configure processing authority
This order slip condition is given, specifically, ConcurrentHashMap, which judges whether there is other threads, has identical order in processing
Other order slip conditions of odd numbers, that is, judge whether there is other order slip conditions with same order number and occupying
Thread, if so, then waiting prefixed time interval (i.e. blocking time), prefixed time interval (i.e. blocking time) be may be configured as
1ms (millisecond), then ConcurrentHashMap judges whether there is other threads again has its of same order number in processing
His order slip condition, until other thread process complete have other order slip conditions of the same order number,
Other order slip conditions of same order number (are removed the phase using the permission release of thread by ConcurrentHashMap
With order number other order slip conditions to the occupancy of thread) after, ConcurrentHashMap is (specific by processing authority
For the permission for using thread) it is allocated to this order slip condition, so that this order slip condition is configured with using thread
Permission, then pass through thread process this order slip condition that is randomly assigned.
It is as follows using the specific algorithm of the permission of thread for the order slip condition configuration in the execution machine:
One, ConcurrentHashMap is initialized, which for example applies for the initialization process such as memory source；
Two, the putIfAbsent method of recursive call ConcurrentHashMap.There are two ginsengs for putIfAbsent method
Number key and value.It is to lock single status data and delete single shape with two order slip conditions being distributed in the execution machine simultaneously
State data instance, then the execution machine calls putIfAbsent when single status data is locked in processing and deletes single status data respectively
Method locks single status data and deletes single status data to configure the respective permission using thread to be respectively, wherein by seizing
Mode, which determines to lock single status data and delete only one in single status data the two, is configured with the permission using thread.Assuming that working as
When locking single status data and being configured with the permission using thread, the putIfAbsent that is called when handling the lock list status data
Method is to lock the order number of single status data as key, and using random value as value, wherein the random value is specifically as follows lock
The state value of single status data may be other customized random numbers, and the return value of the putIfAbsent method is
NULL, and the return value for the putIfAbsent method called when single status data is deleted in processing is above-mentioned random value value (tool
Body for example locks the state value or other customized random numbers of single status data)；
Three, the putIfAbsent method called if it is certain order slip condition configuration using the permission of thread is returned
Returning value is NULL, indicates that other order slip conditions not with this order slip condition same order number are configured with use
The permission of thread, but this order slip condition is configured with the permission using thread, then for this order slip condition with
Machine distributes a thread, this order slip condition is sent to data store set group to execute at data update by the thread
Reason.The remove method for completing to call ConcurrentHashMap after data update is handled is being executed, is being deleted as this order
The key and corresponding order number for the putIfAbsent method that status data configuration is called using the permission of thread, to discharge this
The permission using thread of order slip condition, to remove this order slip condition to the occupancy of thread；
Four, the putIfAbsent method called if it is certain order slip condition configuration using the permission of thread is returned
Returning is not NULL, indicates to be configured with other order slip conditions of the order slip condition same order number using thread
Permission, that is, there is thread handling other status informations of the order number, then etc. at the thread of other status informations to be processed
Reason complete and discharge using thread permission and then call ConcurrentHashMap putIfAbsent method again into
The configuration of the permission with thread is exercised, if configuration successful, this order status information is configured with the permission using thread,
If configuration failure, a predetermined time interval (such as 1 millisecond) is waited, then re-call ConcurrentHashMap's
PutIfAbsent method is to reconfigure above-mentioned permission, until being configured with the permission.Whether there can be thread just with automatic regular polling
In other order slip conditions for handling the order number, the interval time of poll can be set to 1 millisecond or be set as needed
Other values extract the order number of this order slip condition, judge that the corresponding key of the order number whether there is, if it is present
Show other order slip conditions for thering is thread handling the order number, if the key is not present, shows to handle the order
Number the threads of other order slip conditions processed finish.
The putIfAbsent method of ConcurrentHashMap is the method for data synchronization realized by segmentation lock, to order
The order slip condition a of odd numbers 1, the order slip condition b of order number 2 are distributed to for the execution machine that (order number 1 is different from
Order number 2), due to the segmentation lock mechanism of ConcurrentHashMap, configure when for order slip condition a using thread
Permission and while call putIfAbsent method, can also be called for order slip condition b configuration using the permission of thread
PutIfAbsent method, the latter, which withouts waiting for the former calling, to terminate, and if the two calls putIfAbsent method to return
Value is NULL, then, order slip condition a and order slip condition b can be sent to data by respective thread simultaneously
Storage cluster efficiently realizes data synchronization updating so as to the order slip condition between synchronous different threads.
Data store set group such as Redis cluster.Below by taking data store set group is Redis cluster as an example, introduces and execute
Data update the detailed process of processing.The data executed by Redis cluster update processing and mainly check that two steps updated operate
(whether the data i.e. in inspection Redis cluster need to update and execute the two steps operation updated when needing to update), considers
To there is Multi-thread synchronization, the two steps operation which updates needs to be packaged into atomic operation, and (atomic operation is most basic
Operation, it will not be interrupted by other threads when being executed).
Redis cluster execute data update processing process it is as follows: to this order slip condition sent,
Redis cluster first checks for whether the canned data in Redis cluster needs to update, wherein when in Redis cluster not with this
The identical canned data of order slip condition order number, alternatively, having and this bar order slip condition order when in Redis cluster
Number identical canned data, but the timestamp of the canned data earlier than this order slip condition timestamp when, Redis cluster
In data need to update, wherein the get method of Redis cluster can be used to judge whether have and this in Redis cluster
The identical canned data of order slip condition order number.Then, Redis cluster executes update when needing more new data, accordingly
Ground, when canned data not identical with this order slip condition order number in Redis cluster, then Redis cluster need to should
Order slip condition insertion Redis cluster in the storage record of each order status information to save, when in Redis cluster
There is canned data identical with this order slip condition order number, but the timestamp of the canned data is earlier than this order status
When the timestamp of data, show that is recorded in Redis cluster is ordering for the same order number than this order slip condition earlier
The canned data of the order number recorded in Redis cluster is then updated to this order slip condition by single state.If
Data in Redis cluster do not need to update, i.e., have deposited identical with this order slip condition order number in Redis cluster
Data, but the timestamp of the canned data is later than the timestamp of this order slip condition, and show to record in Redis cluster is
This order slip condition corresponds to the newest order status of order number, then keep in Redis cluster with this order slip condition
The identical canned data of order number is constant.
Data update scheme in the prior art malfunctions in order to avoid data update, and needs to lock in Redis cluster, with
So that checking that other order status are waited for having updated until the order status inspection when updating a certain order status
Finish the inspection update processing for carrying out other order status again, the performance of entire Redis cluster, and Java can be seriously affected in this way
Affairs locking mechanisms there is a problem of lock long flow path and performance cost it is big.It, can for the drawbacks described above for overcoming the prior art
Handle the status information with same order number by same thread to pass through, however this mode is needed to each example
The order slip condition that (i.e. execution machine) receives is distributed again according to Thread Id (mark), and implementation complexity is higher, and
And data are repeatedly distributed, and processing delay is increased.According to embodiments of the present invention, it is calculated when execution machine is received by consistency Hash
After order slip condition after method distribution, by ConcurrentHashMap built in Java (i.e. multithreading Hash set,
The also referred to as Hash set of thread-safe) it is permission of the order slip condition configuration in the execution machine using thread, so that same
One moment, which corresponded in the order slip condition of same order number, only has an order slip condition to be configured with the power using thread
Limit, that is, in the order slip condition of only one thread process of synchronization order number, so that passing through unique thread
After the order slip condition of the order number of transmission reaches Redis cluster, order status number of the Redis cluster to the order number
It will not malfunction according to updating, and without being locked when Redis cluster updates order slip condition, overcome and lock band in the prior art
The many defects come.
Using the technical solution of the embodiment of the present invention, the big affairs locking operation of performance cost is converted into being based on
The distribution of the processing authority (being specifically as follows the permission using thread) of ConcurrentHashMap improves system processing energy
And performance, the average handling time of single order slip condition can be made by under 10ms of the prior art based on the scheme of locking
It is down to 3ms, and system is spat the amount of gulping down and is substantially improved, in addition, then further from data distribution angle using consistency hash algorithm
Improve the stability of system.
Fig. 4 is the main modular schematic diagram of data update apparatus according to an embodiment of the present invention.
As shown in figure 4, the data update apparatus 400 of the embodiment of the present invention is mainly matched including data distribution module 401, permission
Set module 402, data transmitting module 403, permission release module 404.
Data distribution module 401, for the data of identical services number will to be corresponded in the data of input using preset algorithm
It is distributed to same execution machine.
Wherein, preset algorithm can be hash algorithm or consistency hash algorithm.
The data of input may include timestamp information.
Permission configuration module 402, for configuring the place of data in this execution machine according to preset rules in each execution machine
Manage permission, preset rules are as follows: an only item data is configured with processing in the data that synchronization corresponds to identical services number
Permission.
Processing authority can be the permission for using thread.
Permission configuration module 402 can be also used for: configure all data one by one according to the time that data reach this execution machine
Processing authority, wherein when configuring the processing authority of every item data, execute following steps: step 1, judgement and the item data
Whether there are data to be configured with processing authority in other data of identical services number, if so, step 2 is executed, if it is not,
Then follow the steps three；Step 2 waits prefixed time interval, then returns to step one；Step 3 matches the item data
It is equipped with processing authority.
Permission configuration module 402 can be also used for: judgement with the item data identical services number other data in whether
There are data occupying thread.
Permission configuration module 402 can be also used for: the multithreading Hash set built in Java is used in each execution machine
The processing authority of data in this execution machine is configured according to preset rules.
Data transmitting module 403, for by each execution machine by the respective data transmission for being configured with processing authority
To data store set group to execute data update processing.
Data transmitting module 403 can be also used for: by each execution machine by the respective power being configured with using thread
The data of limit pass through the thread being randomly assigned and are sent to data store set group.
Executing the step of data update processing can specifically include: the data of data store set group are transmitted to each single item,
Data store set group judges whether there is the canned data with the data identical services of transmission number；It has been deposited if so, comparing
The newest data of timestamp in the two have been deposited number by the timestamp information of data and both data of the transmission
According to；If it is not, the data of transmission are then stored in data store set group.
Permission release module 404, for being executed data more when the data for being configured with processing authority in an execution machine
After new processing, which discharges the processing authority of the data, wherein the processing authority of execution machine release data is mainly
The data are removed to the occupancy of thread.
Fig. 5 is shown can be using the data-updating method of the embodiment of the present invention or the exemplary system of data update apparatus
Framework 500.
As shown in figure 5, system architecture 500 may include terminal device 501,502,503, network 504 and server 505.
Network 504 between terminal device 501,502,503 and server 505 to provide the medium of communication link.Network 504 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 501,502,503 and be interacted by network 504 with server 505, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 501,502,503
The application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform software etc..
Terminal device 501,502,503 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 505 can be to provide the server of various services, such as utilize terminal device 501,502,503 to user
The shopping class website browsed provides the back-stage management server supported.Back-stage management server can believe the product received
The data such as breath inquiry request carry out the processing such as analyzing, and processing result (order slip condition of product) is fed back to terminal and is set
It is standby.
It should be noted that data-updating method provided by the embodiment of the present invention is generally executed by server 505, accordingly
Ground, data update apparatus are generally positioned in server 505.
It should be understood that the number of terminal device, network and server in Fig. 5 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Below with reference to Fig. 6, it illustrates the computer systems 600 for the server for being suitable for being used to realize the embodiment of the present application
Structural schematic diagram.Server shown in Fig. 6 is only an example, should not function and use scope band to the embodiment of the present application
Carry out any restrictions.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and
Execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interface 605 is connected to lower component: the importation 606 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon
Computer program be mounted into storage section 608 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer
Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.?
In such embodiment, which can be downloaded and installed from network by communications portion 609, and/or from can
Medium 611 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 601, the system that executes the application
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In this application, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In application, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
Include data distribution module 401, permission configuration module 402, data transmitting module 403, permission release module 404.Wherein, these moulds
The title of block does not constitute the restriction to the module itself under certain conditions, for example, data distribution module 401 can also be retouched
It states as " for the data distribution of identical services number will to be corresponded in the data of input to the mould of same execution machine using preset algorithm
Block ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
The equipment includes: will correspond to data distribution that identical services are numbered in the data of input to same execution using preset algorithm
Machine；The processing authority of data in this execution machine, preset rules are as follows: in same a period of time are configured according to preset rules in each execution machine
It carves an only item data in the data of corresponding identical services number and is configured with processing authority；It will be respective by each execution machine
The data transmission for being configured with processing authority is handled to data store set group with executing data update；It is configured when in an execution machine
There are the data of processing authority to be executed after data update processing, which discharges the processing authority of the data.
Technical solution according to an embodiment of the present invention will correspond to identical services using preset algorithm and number in the data of input
Data distribution to same execution machine；The disposal right of data in this execution machine is configured according to preset rules in each execution machine
Limit, wherein preset rules are that an only item data is configured with disposal right in the data that synchronization corresponds to identical services number
Limit；The respective data transmission for being configured with processing authority is executed into data to data store set group more by each execution machine
New processing；After the data for being configured with processing authority in an execution machine are executed data update processing, which is released
Put the processing authority of the data.Using the technical solution of the embodiment of the present invention, can under the premise of guaranteeing that data correctly update,
Reduce processing delay, lifting system handling capacity.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (14)
1. a kind of data-updating method characterized by comprising
The data distribution of identical services number will be corresponded in the data of input to same execution machine using preset algorithm；
The processing authority of data in this execution machine, the preset rules are configured according to preset rules in each execution machine are as follows:
Synchronization corresponds to an only item data in the data of identical services number and is configured with the processing authority；
By each execution machine by the respective data transmission for being configured with the processing authority to data store set group to execute
Data update processing；
After the data for being configured with the processing authority in an execution machine are executed data update processing, the execution
Machine discharges the processing authority of the data.
2. the method according to claim 1, wherein the preset algorithm is that hash algorithm or consistency Hash are calculated
Method.
3. the method according to claim 1, wherein configuring this execution according to preset rules in each execution machine
In machine the step of the processing authority of data, comprising:
Configure the processing authority of all data one by one according to the time that data reach this execution machine, wherein configuring every item data
Processing authority when, execute following steps:
Whether there are data to be configured with the processing in step 1, judgement and other data of item data identical services number
Permission, if so, step 2 is executed, if it is not, thening follow the steps three；
Step 2 waits prefixed time interval, then returns and executes the step 1；
Step 3 makes the item data be configured with the processing authority.
4. the method according to claim 1, wherein the data of the input include timestamp information,
The execution data update the step of processing and include:
The data of the data store set group are transmitted to each single item, the data store set group judges whether there is and the transmission
Data identical services number canned data；
If so, the timestamp information of both data of the canned data and the transmission is compared, by the time in described the two
Newest data are stabbed as new canned data；
If it is not, the data of the transmission are then stored in the data store set group.
5. according to the method described in claim 3, it is characterized in that, the processing authority be using thread permission,
By each execution machine by the respective data transmission for being configured with the processing authority to data store set group the step of,
It include: that respective be configured with is passed through using the data of the permission of thread by the thread being randomly assigned transmission by each execution machine
To data store set group,
The step of described execution machine discharges the processing authority of the data, comprising: the described execution machine removes the data pair
The occupancy of thread,
Whether there are data to be configured with the processing authority in judgement and other data of item data identical services number
Step, comprising: whether there are data occupying thread in judgement and other data of item data identical services number.
6. the method according to claim 1, wherein being breathed out in each execution machine using the multithreading built in Java
Uncommon set configures the processing authority of data in this execution machine according to preset rules.
7. a kind of data update apparatus characterized by comprising
Data distribution module, for the data distribution of identical services number will to be corresponded in the data of input to together using preset algorithm
One executes machine；
Permission configuration module, for the processing authority of data in this execution machine to be configured according to preset rules in each execution machine,
The preset rules are as follows: an only item data is configured with the processing in the data that synchronization corresponds to identical services number
Permission；
Data transmitting module, for by each execution machine by the respective data transmission for being configured with the processing authority to number
According to storage cluster to execute data update processing；
Permission release module, for being executed the data more when the data for being configured with the processing authority in an execution machine
After new processing, which discharges the processing authority of the data.
8. device according to claim 7, which is characterized in that the preset algorithm is that hash algorithm or consistency Hash are calculated
Method.
9. device according to claim 7, which is characterized in that the permission configuration module is also used to:
Configure the processing authority of all data one by one according to the time that data reach this execution machine, wherein configuring every item data
Processing authority when, execute following steps:
Whether there are data to be configured with the processing in step 1, judgement and other data of item data identical services number
Permission, if so, step 2 is executed, if it is not, thening follow the steps three；
Step 2 waits prefixed time interval, then returns and executes the step 1；
Step 3 makes the item data be configured with the processing authority.
10. device according to claim 7, which is characterized in that the data of the input include timestamp information,
The execution data update the step of processing and include:
The data of the data store set group are transmitted to each single item, the data store set group judges whether there is and the transmission
Data identical services number canned data；
If so, the timestamp information of both data of the canned data and the transmission is compared, by the time in described the two
Newest data are stabbed as new canned data；
If it is not, the data of the transmission are then stored in the data store set group.
11. device according to claim 9, which is characterized in that the processing authority is the permission using thread,
The data transmitting module is also used to: by each execution machine by the respective data for being configured with the permission using thread
It is sent to data store set group by the thread being randomly assigned,
The permission release module is also used to: the described execution machine removes the data to the occupancy of thread,
The permission configuration module is also used to: whether having data just in judgement and other data of item data identical services number
Occupying thread.
12. device according to claim 7, which is characterized in that the permission configuration module is also used to: in each execution machine
The middle multithreading Hash set using built in Java configures the processing authority of data in this execution machine according to preset rules.
13. a kind of server characterized by comprising
One or more processors；
Memory, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors
Realize such as method as claimed in any one of claims 1 to 6.
14. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
Such as method as claimed in any one of claims 1 to 6 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710594880.3A CN109284177B (en) | 2017-07-20 | 2017-07-20 | Data updating method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710594880.3A CN109284177B (en) | 2017-07-20 | 2017-07-20 | Data updating method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109284177A true CN109284177A (en) | 2019-01-29 |
| CN109284177B CN109284177B (en) | 2021-03-30 |
Family
ID=65185367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710594880.3A Active CN109284177B (en) | 2017-07-20 | 2017-07-20 | Data updating method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109284177B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825762A (en) * | 2019-11-14 | 2020-02-21 | 北京融易做科技有限公司 | Data storage method, device and system |
| CN114358779A (en) * | 2022-03-18 | 2022-04-15 | 汇承金融科技服务(南京)有限公司 | Order processing method and device, storage medium and electronic device |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100095291A1 (en) * | 2008-10-10 | 2010-04-15 | Sony Corporation | Software upgrade using single source channel |
| CN102354316A (en) * | 2011-09-22 | 2012-02-15 | 用友软件股份有限公司 | Data updating device and data updating method |
| CN102467570A (en) * | 2010-11-17 | 2012-05-23 | 日电（中国）有限公司 | Connection query system and method for distributed data warehouse |
| CN103632250A (en) * | 2013-12-06 | 2014-03-12 | 湖南御家汇化妆品有限公司 | Quick sales order sorting, grouping and screening method |
| US20150199318A1 (en) * | 2011-09-02 | 2015-07-16 | Micah Lemonik | System and Method for Using a Third-Party Add-On in a Collaborative On-Line Software Development Environment |
| US20150294196A1 (en) * | 2014-04-15 | 2015-10-15 | Kyocera Document Solutions Inc. | Electronic device and image forming apparatus |
| CN105045791A (en) * | 2014-03-26 | 2015-11-11 | 日本电气株式会社 | Database device |
| CN105139151A (en) * | 2015-09-25 | 2015-12-09 | 佛山市中格威电子有限公司 | Information management system based on two-dimensional code technology |
| US20160224393A1 (en) * | 2015-02-03 | 2016-08-04 | Ca, Inc. | System and method of distributing processes stored in a common database |
| CN106651402A (en) * | 2016-12-29 | 2017-05-10 | 江西博瑞彤芸科技有限公司 | Data processing method for goods return operation |
-
2017
- 2017-07-20 CN CN201710594880.3A patent/CN109284177B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100095291A1 (en) * | 2008-10-10 | 2010-04-15 | Sony Corporation | Software upgrade using single source channel |
| CN102467570A (en) * | 2010-11-17 | 2012-05-23 | 日电（中国）有限公司 | Connection query system and method for distributed data warehouse |
| US20150199318A1 (en) * | 2011-09-02 | 2015-07-16 | Micah Lemonik | System and Method for Using a Third-Party Add-On in a Collaborative On-Line Software Development Environment |
| CN102354316A (en) * | 2011-09-22 | 2012-02-15 | 用友软件股份有限公司 | Data updating device and data updating method |
| CN103632250A (en) * | 2013-12-06 | 2014-03-12 | 湖南御家汇化妆品有限公司 | Quick sales order sorting, grouping and screening method |
| CN105045791A (en) * | 2014-03-26 | 2015-11-11 | 日本电气株式会社 | Database device |
| US20150294196A1 (en) * | 2014-04-15 | 2015-10-15 | Kyocera Document Solutions Inc. | Electronic device and image forming apparatus |
| US20160224393A1 (en) * | 2015-02-03 | 2016-08-04 | Ca, Inc. | System and method of distributing processes stored in a common database |
| CN105139151A (en) * | 2015-09-25 | 2015-12-09 | 佛山市中格威电子有限公司 | Information management system based on two-dimensional code technology |
| CN106651402A (en) * | 2016-12-29 | 2017-05-10 | 江西博瑞彤芸科技有限公司 | Data processing method for goods return operation |
Non-Patent Citations (1)
| Title |
|---|
| 张鹏等: "基于数据服务的数据组合视图的优化更新", 《计算机学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825762A (en) * | 2019-11-14 | 2020-02-21 | 北京融易做科技有限公司 | Data storage method, device and system |
| CN114358779A (en) * | 2022-03-18 | 2022-04-15 | 汇承金融科技服务(南京)有限公司 | Order processing method and device, storage medium and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109284177B (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106447246A (en) | Inventory data account checking method and inventory data account checking device | |
| CN103426072B (en) | The order processing system of a kind of high concurrent competition stock and disposal route thereof | |
| CN109274731A (en) | Deployment, call method and the device of web services based on multi-tenant technology | |
| CN110245011A (en) | A kind of method for scheduling task and device | |
| CN106375241B (en) | Batch data processing method, front-end system, host and batch data processing system | |
| CN108874513A (en) | Handle method, system, electronic equipment and the computer-readable medium of timed task | |
| CN108920222A (en) | A kind of method and device for business processing of rule-based engine | |
| CN109413127A (en) | A kind of method of data synchronization and device | |
| US8874587B2 (en) | Tenant placement in multitenant cloud databases with one-to-many data sharing | |
| CN108874828A (en) | The method and apparatus of Data Migration | |
| US20130013549A1 (en) | Hardware-assisted approach for local triangle counting in graphs | |
| CN108897854A (en) | A kind of monitoring method and device of overtime task | |
| CN109408286A (en) | Data processing method, device, system, computer readable storage medium | |
| CN107844324A (en) | Customer terminal webpage redirects treating method and apparatus | |
| CN110502572A (en) | A kind of method and server that data are synchronous | |
| CN110377416A (en) | Distributed subregion method for scheduling task and device | |
| CN110135925A (en) | Order processing system, method and apparatus | |
| CN110135770A (en) | The generation method and device of outbound scheme | |
| CN109284177A (en) | A kind of data-updating method and device | |
| CN110019539A (en) | A kind of method and apparatus that the data of data warehouse are synchronous | |
| CN110083457A (en) | A kind of data capture method, device and data analysing method, device | |
| CN108984197A (en) | A kind of code update method and device | |
| CN109840815A (en) | System and method for order processing | |
| CN109144992A (en) | A kind of method and apparatus of data storage | |
| US8141103B2 (en) | Solution for modifying a queue manager to support smart aliasing which permits extensible software to execute against queued data without application modifications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |