CN108345643A - A kind of data processing method and device - Google Patents
A kind of data processing method and device Download PDFInfo
- Publication number
- CN108345643A CN108345643A CN201810031750.3A CN201810031750A CN108345643A CN 108345643 A CN108345643 A CN 108345643A CN 201810031750 A CN201810031750 A CN 201810031750A CN 108345643 A CN108345643 A CN 108345643A
- Authority
- CN
- China
- Prior art keywords
- data
- mark
- node
- data block
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data processing method and devices.The method includes：The data processing request sent according to the client device received includes the mark of pending data, and the mark of the corresponding first node of mark of the pending data is determined in the multiple nodes for including from group system；And then according to the mark of the first node, indicate that the first node handles the pending data.Since there are the correspondences of the mark of data and the mark of node in the embodiment of the present invention, therefore, after receiving data processing request, corresponding node can be specified to be handled according to the mark of data, all the data processing request is responded without all nodes in group system, can so make the data of the different node processing different identifications in group system, the case where so as to effectively avoid data redundancy, the space for saving node, improves the utilization rate of group system.
Description
Technical field
The present invention relates to field of communication technology more particularly to a kind of data processing method and devices.
Background technology
With the rapid development of internet technology, the clustered deploy(ment) of node is a kind of common sharing system operating pressure
Method.In the environment of big data quantity, high concurrent, usually integrated cache of memory layer promotes response speed for node in cluster, and
It reduces to the pressure of back-end data base layer, currently, generally use data synchronization technology ensures the data handled by each node
Consistency.
After existing data synchronization technology receives data processing request by a node, needing successively will be at the data
Reason request is sent to other nodes in cluster.In this way, the data volume for including in data processing request it is larger or
It is larger to the consumption of network bandwidth in the case that node in cluster is more, and, the identical data of each node processing are easy to make
At data redundancy, node space is wasted.
Based on this, there is an urgent need for a kind of data processing methods at present, and phase is handled for solving each back end in the prior art
The problem of same data cause node space to waste.
Invention content
A kind of data processing method of offer of the embodiment of the present invention and device are protected with solving each back end in the prior art
Deposit the technical issues of identical data lead to waste of storage space.
The embodiment of the present invention provides a kind of data processing method, the method includes：
The data processing request that client device is sent is received, the data processing request includes the mark of pending data
Know；
According to the mark of the pending data, the pending number is determined in the multiple nodes for including from group system
According to the corresponding first node of mark mark；
According to the mark of the first node, indicate that the first node handles the pending data.
Optionally, it according to the mark of the pending data, is determined in the multiple nodes for including from group system described
The mark of the corresponding first node of mark of pending data, including：
According to the mark of the data to be stored, the mark of corresponding first data block of mark of the data to be stored is obtained
Know；
According to the default correspondence of the mark of data block and the mark of node, the mark of first data block is corresponded to
Node mark of the mark as the first node.
Optionally, according to the mark of the data to be stored, corresponding first number of mark of the data to be stored is obtained
According to the mark of block, including：
The mark of the data to be stored is calculated using consistency hash algorithm, obtains the data to be stored
Identify corresponding target cryptographic Hash；
According to the default correspondence of the mark of cryptographic Hash and data block, corresponding first number of the target cryptographic Hash is obtained
According to the mark of block.
Optionally, the data processing request is data storage request, further includes described wait in the data processing request
Handle the corresponding value of data；
Indicate that the first node handles the pending data, including：
Indicate that the first node arrives the mark of the data to be stored and the corresponding value storage of the data to be stored
First data block.
Optionally, the data processing request is data inquiry request；
Indicate that the first node handles the pending data, including：
Indicate that the first node waits for described in being inquired in first data block according to the mark of the pending data
Handle the corresponding value of data.
The embodiment of the present invention provides a kind of device, and described device includes：
Receiving unit, the data processing request for receiving client device transmission, the data processing request include
The mark of pending data；
Processing unit is determined for the mark according to the pending data in the multiple nodes for including from group system
Go out the mark of the corresponding first node of mark of the pending data；
Indicating unit indicates the first node to the pending data for the mark according to the first node
It is handled.
Optionally, the processing unit is specifically used for：
According to the mark of the data to be stored, the mark of corresponding first data block of mark of the data to be stored is obtained
Know；
According to the default correspondence of the mark of data block and the mark of node, the mark of first data block is corresponded to
Node mark of the mark as the first node.
Optionally, the processing unit is specifically used for：
The mark of the data to be stored is calculated using consistency hash algorithm, obtains the data to be stored
Identify corresponding target cryptographic Hash；
According to the default correspondence of the mark of cryptographic Hash and data block, corresponding first number of the target cryptographic Hash is obtained
According to the mark of block.
Optionally, the data processing request is data storage request, further includes described wait in the data processing request
Handle the corresponding value of data；
The indicating unit is specifically used for：
Indicate that the first node arrives the mark of the data to be stored and the corresponding value storage of the data to be stored
First data block.
Optionally, the data processing request is data inquiry request；
The indicating unit is specifically used for：
Indicate that the first node waits for described in being inquired in first data block according to the mark of the pending data
Handle the corresponding value of data.
In the embodiment of the present invention, the data processing request sent according to the client device received includes pending number
According to mark, the corresponding first node of mark of the pending data is determined in the multiple nodes for including from group system
Mark；And then according to the mark of the first node, indicate that the first node handles the pending data.This hair
Bright embodiment can determine the mark of corresponding node according to the mark of data, compared with the existing technology each of middle group system
For node will handle identical data processing request, the embodiment of the present invention only needs the transmission data to corresponding node to ask
Ask, can saving network flow, alleviate the pressure of group system；Further, since there are data in the embodiment of the present invention
The correspondence of mark and the mark of node therefore, can be specified pair according to the mark of data after receiving data processing request
The node answered is handled, and is all responded to the data processing request without all nodes in group system, can so be made
The data of different node processing different identifications in group system the case where so as to effectively avoid data redundancy, save section
The space of point, improves the utilization rate of group system.
Description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without having to pay creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of schematic diagram for system architecture that the embodiment of the present invention is applicable in；
Fig. 2 is the flow diagram corresponding to a kind of data processing method provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram of flow involved in the embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of device provided in an embodiment of the present invention.
Specific implementation mode
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention make into
It is described in detail to one step, it is clear that the described embodiments are only some of the embodiments of the present invention, rather than whole implementation
Example.Based on the embodiments of the present invention, obtained by those of ordinary skill in the art without making creative efforts
All other embodiment, shall fall within the protection scope of the present invention.
Fig. 1 illustrates a kind of schematic diagram of the applicable system architecture of the embodiment of the present invention, as shown in Figure 1, this is
Include client device 101, device 102, at least one node, node 103, node 104 and node as shown in Figure 1 in system
105；Wherein, described device 102 is used to determine the node for handling the request according to the data processing request received.User passes through
The processing request of 101 transmission data of client device, is interacted between device 102 and client device 101 by network, and from
The information such as mark or the corresponding value of data of the data included by data processing request are obtained in client device.Device 102
It can be communicated between any one node.
In the embodiment of the present invention, multiple nodes form group system 100, may include multiple data blocks in each node,
The data block can be used for storing the corresponding value of data, and the memory capacity of each data block can be identical.As shown in Fig. 1
Data block 1031, data block 1032, data block 1041, data block 1042, data block 1051 and the data block 1052 gone out；Wherein,
Node 103 includes data block 1031 and data block 1032, and node 104 includes data block 1041 and data block 1042, and node 105 wraps
Include data block 1051 and data block 1052.
Further, the number for the data block that each node includes can be consistent, and each be saved as shown in table 1
Point includes two data blocks；Alternatively, the number for the data block that each node includes can be inconsistent, people in the art
Member can rule of thumb set the data block data that each node includes with actual conditions, as long as meeting the storage of each node
Capacity meets preset ratio, does not limit specifically.
In the embodiment of the present invention, there can be multiple client equipment, for ease of description, only with one of visitor in Fig. 1
For the end equipment of family.Client device 101 can be a plurality of types of equipment, such as laptop, smart mobile phone, tablet electricity
Brain, smart television etc..
Based on system architecture shown in figure 1, a kind of data processing method provided in an embodiment of the present invention, this method can be with
The device 102 shown in Fig. 1 executes, as shown in Fig. 2, being corresponding to a kind of data processing method provided in an embodiment of the present invention
Flow diagram, specifically comprise the following steps：
Step 201, the data processing request that client is sent is received, the data processing request includes pending data
Mark.
Step 202, it according to the mark of the pending data, is determined in the multiple nodes for including from group system described
The mark of the corresponding first node of mark of pending data.
Step 203, according to the mark of the first node, indicate the first node to the pending data at
Reason.
The embodiment of the present invention can determine the mark of corresponding node according to the mark of data, compared with the existing technology middle cluster
For each node in system will handle identical data processing request, the embodiment of the present invention is only needed to corresponding node
Transmission data request, can saving network flow, alleviate the pressure of group system；Further, due to the embodiment of the present invention
Middle there are the correspondences of the mark of data and the mark of node, therefore, can be according to data after receiving data processing request
Mark specify corresponding node to be handled, all the data processing request is rung without all nodes in group system
It answers, can so make the data of the different node processing different identifications in group system, so as to effectively avoid data redundancy
Situation saves the space of node, improves the utilization rate of group system.
Specifically, in step 201 and step 202, the mark of pending data for distinguishing different pending datas,
For example, the mark of pending data is using the key in key-value pair, the i.e. mark by " key " of pending data as pending data
Know, be a kind of example of key-value pair as shown in table 1, is the example of storage location coordinate class data, the 1st article of number shown in table 1
The data for including according to processing request are { A, (0,1) }, indicate that the coordinate of position A is (0,1), then key (key) is A, value
(value) it is (0,1), that is to say, that the pending data that the data processing request includes is identified as A；At 2nd data
The data that reason request includes are { B, (7,2) }, indicate that the coordinate of position B is (7,2), then key (key) is B, and value (value) is
(7,2), that is to say, that the pending data that the data processing request includes is identified as B；In the processing request of 3rd data
Including data be { C, (5,3) }, indicate position C coordinate be (5,3), then key (key) be C, value (value) be (5,3),
That is the pending data that the data processing request includes is identified as C.
Table 1：A kind of example of key-value pair
| Data processing request | Data | Key (key) | It is worth (value) |
| 1 | { A, (0,1) } | A | (0,1) |
| 2 | { B, (7,2) } | B | (7,2) |
| 3 | { C, (5,3) } | C | (5,3) |
In the embodiment of the present invention, data processing request can be the request of diversified forms, for example, it may be data storage is asked
It asks, or can also be data inquiry request.The request of both forms is specifically described separately below.
(1) data processing request is data storage request
In the embodiment of the present invention, data storage request include data to be stored mark and data to be stored it is corresponding
Value is as shown in table 2 a kind of example of data to be stored, for details, reference can be made to the descriptions in table 1, and details are not described herein.
Table 2：A kind of example of data to be stored
| Serial number | Data to be stored | The mark of data to be stored | The corresponding value of data to be stored |
| 1 | { A, (0,1) } | A | (0,1) |
| 2 | { B, (7,2) } | B | (7,2) |
| 3 | { C, (5,3) } | C | (5,3) |
Further, there are many modes that the mark of first node is determined according to the mark of data to be stored, a kind of possibility
Realization method be that data to be stored is stored in the data block in node.Specifically, according to the mark of data to be stored,
The mark of corresponding first data block of mark of data to be stored is obtained, and then according to the mark of the mark and node of data block
Default correspondence, the mark by the mark of the corresponding node of mark of the first data block as first node.As shown in table 3,
For a kind of example of the default correspondence of the mark of the mark and node of data block, the data of node being identified as corresponding to a
Block is identified as 0,1,2；The data block of node being identified as corresponding to b is identified as 3,4,5；Node is identified as corresponding to c
Data block be identified as 6,7,8,9.Wherein, data block for storing data, and, the amount of storage of each data block is identical.
Table 3：A kind of example of the default correspondence of the mark of data block and the mark of node
In the embodiment of the present invention, the mode for the mark for obtaining corresponding first data block according to the mark of data to be stored has
It is a variety of, for example, it may be being obtained according to the correspondence of the mark of preset data and the mark of data block；Alternatively, can also
It is to be obtained according to preset algorithm.By taking consistency hash algorithm as an example, the embodiment of the present invention can be used consistency hash algorithm and treat
The mark of storage data is calculated, and the corresponding target cryptographic Hash of mark of data to be stored is obtained；And then according to cryptographic Hash with
The default correspondence of the mark of data block obtains the mark of corresponding first data block of target cryptographic Hash.As shown in table 4, it is
A kind of example of the default correspondence of the mark of cryptographic Hash and data block, if cryptographic Hash is 0, the mark of corresponding data block
It is 0；If cryptographic Hash is 1, corresponding data block is identified as 1；If cryptographic Hash is 2, corresponding data block is identified as 2；
If cryptographic Hash is 3, corresponding data block is identified as 3；If cryptographic Hash is 4, corresponding data block is identified as 4；If breathing out
Uncommon value is 5, then corresponding data block is identified as 5；If cryptographic Hash is 6, corresponding data block is identified as 6；If cryptographic Hash
It is 7, then corresponding data block is identified as 7；If cryptographic Hash is 8, corresponding data block is identified as 8；If cryptographic Hash is 9,
Then corresponding data block is identified as 9.
Table 4：A kind of example of the default correspondence of the mark of cryptographic Hash and data block
| Cryptographic Hash | The mark of data block |
| Value=0 | 0 |
| Value=1 | 1 |
| Value=2 | 2 |
| Value=3 | 3 |
| Value=4 | 4 |
| Value=5 | 5 |
| Value=6 | 6 |
| Value=7 | 7 |
| Value=8 | 8 |
| Value=9 | 9 |
It is stored in the realization process in the block of the data in node in order to illustrate more clearly of above-mentioned data to be stored, below will
It illustrates.There are three nodes for being identified as a, b, c in setting group system, the mark of corresponding data block such as above-mentioned table 3
Shown in, and, the cryptographic Hash corresponding to the mark of each data block is as shown in table 1；If data to be stored is upper
State the 2nd data shown in table 2, i.e., the data are { B, (7,2) }, that is to say, that the data are identified as B, be worth for (7,
2)；If it is 2 that the corresponding cryptographic Hash of the data, which is calculated, according to consistency hash algorithm, according to the content shown in table 4
It is found that the corresponding data block of the data is identified as 2, and then the content according to table 3 is it is found that the corresponding section of the data
Point is identified as a.
In this way, the mark of corresponding node is determined by the mark of data, compared with the existing technology in middle group system
For each node will handle identical data storage request, the embodiment of the present invention only needs to indicate that corresponding node is treated to deposit
Storage data are handled, and so as to saving network flow, alleviate the pressure of group system.
In other possible realization methods, can also there is no data block, data to be stored to be directly stored in section in node
Point in.Specifically, according to the correspondence of the mark of data and the mark of node, in the multiple nodes for including from group system
Determine the mark of the corresponding first node of mark of pending data.The embodiment of the present invention can pre-set the mark of data
It is as shown in table 5 the one of the correspondence of the mark of the mark and node of preset data with the correspondence of the mark of node
Corresponding node when being identified as A of kind of example, data is identified as a；Corresponding node when being identified as B of data is identified as b；
Corresponding node when being identified as C of data is identified as c.For example, by taking data to be stored shown in table 2 as an example, if waiting for
It is { A, (0,1) } to store data, then can determine the A that is identified as of the data, and then can determine the corresponding node of the data
It is identified as a.
Table 5：A kind of example of the correspondence of the mark of preset data and the mark of node
| The mark of data | The mark of node |
| A | a |
| B | b |
| C | c |
Further, after determining first node, it may indicate that first node by the mark of data to be stored and to be stored
The corresponding value storage of data all stores the data without all nodes in group system to the first data block, to
The case where data redundancy can effectively be avoided, the memory space of node is saved, improves the utilization rate of group system.
Further, it after determining first node, can also further determine that between the corresponding network of first node mutually
Address agreement (Internet Protocol, IP) even, so as to be convenient for through the corresponding IP address of first node and first
Node interacts.
(2) data processing request is data inquiry request
In the embodiment of the present invention, data inquiry request includes the mark of data to be checked, is to be checked as shown in table 6
A kind of example of data, the data to be checked that the 1st data inquiry request includes are identified as A；2nd data inquiry request
The data to be checked for including are identified as B；The data to be checked that 3rd data inquiry request includes are identified as C.
Table 6：A kind of example of data to be checked
| Serial number | The mark of data to be stored |
| 1 | A |
| 2 | B |
| 3 | C |
Further, there are many modes that the mark of first node is determined according to the mark of data to be checked, for example, can be with
It is the mark that corresponding node is directly inquired by the mark of data to be checked；Alternatively, can also be first determining data to be checked
The corresponding data block of mark mark, and then according to the mark correspondence of data block and node, determine and data to be checked
The corresponding node of mark mark.The specific implementation process of above two mode can refer to content as described above,
This is repeated no more.
The specific implementation process that will be exemplified below data query, in order to which data processing request is more clearly understood
For data inquiry request when implementation process.There are three nodes for being identified as a, b, c, corresponding data in setting group system
Block is identified as shown in above-mentioned table 3, and, the cryptographic Hash corresponding to the mark of each data block is as shown in table 1；
If data to be checked are identified as C, it is 3 that the corresponding cryptographic Hash of the data, which is calculated, according to consistency hash algorithm, then root
According to the content shown in table 4 it is found that the corresponding data block of the data is identified as 3, and then the content according to table 3
It is found that the corresponding node of the data is identified as b.
It in this way, determining the mark of data block by the mark of data, and then can determine the mark of corresponding node, use
This method, for each node in middle group system will handle identical data inquiry request compared with the existing technology,
The embodiment of the present invention only needs the step of being inquired from node corresponding with the mark of data to be checked, save inquiry, greatly
Improve search efficiency greatly.
Further, after determining first node, it may indicate that first node according to the mark of data to be checked first
The corresponding value of the data to be checked is inquired in data block.If not inquiring the corresponding value of data to be checked in first node,
It can then continue to inquire the corresponding relevant database of group system；If finding data to be checked in relevant database to correspond to
Value, then may indicate that first node by the mark of data to be checked and the storage of data to be checked corresponding value to the first data block.
In this way, after receiving data inquiry request, corresponding node can be specified to be inquired according to the mark of data, nothing
It needs all nodes in group system all to be responded to the data inquiry request, so as to simplify operating procedure, improves system
System response speed.
Method in order to more clearly introduce above-mentioned data processing, with reference to Fig. 3, to involved in the embodiment of the present invention
The flow arrived carries out globality explanation.As shown in figure 3, may comprise steps of：
Step 301, the data processing request that client is sent is received.
Step 302, judge that the data processing request is data storage request either data inquiry request, if data
Storage request, thens follow the steps 303；If data inquiry request, 307 are thened follow the steps.
Step 303, include the mark of data to be stored according to data storage request, using consistency hash algorithm to institute
The mark for stating data to be stored is calculated, and the corresponding first object cryptographic Hash of mark of the data to be stored is obtained.
Step 304, according to the default correspondence of the mark of cryptographic Hash and data block, the first object cryptographic Hash is obtained
The mark of corresponding first data block.
Step 305, according to the default correspondence of the mark of data block and the mark of node, by first data block
Identify mark of the mark of corresponding node as the first node.
Step 306, indicate that the first node is corresponding by the mark of the data to be stored and the data to be stored
First data block is arrived in value storage.
Step 307, include the mark of data to be checked according to data inquiry request, using consistency hash algorithm to institute
The mark for stating data to be checked is calculated, and the corresponding second target cryptographic Hash of mark of the data to be checked is obtained.
Step 308, according to the default correspondence of the mark of cryptographic Hash and data block, the second target cryptographic Hash is obtained
The mark of corresponding second data block.
Step 309, according to the default correspondence of the mark of data block and the mark of node, by second data block
Identify mark of the mark of corresponding node as the second node.
Step 310, indicate that the second node is looked into according to the mark of the data to be checked in second data block
Ask the corresponding value of the data to be checked.
The embodiment of the present invention can determine the mark of corresponding node according to the mark of data, compared with the existing technology middle cluster
For each node in system will handle identical data processing request, the embodiment of the present invention is only needed to corresponding node
Transmission data request, can saving network flow, alleviate the pressure of group system；Further, due to the embodiment of the present invention
Middle there are the correspondences of the mark of data and the mark of node, therefore, can be according to data after receiving data processing request
Mark specify corresponding node to be handled, all the data processing request is rung without all nodes in group system
It answers, can so make the data of the different node processing different identifications in group system, so as to effectively avoid data redundancy
Situation saves the space of node, improves the utilization rate of group system.
In view of the node in group system may be changed due to equipment adjustment etc., the embodiment of the present invention
The method that data after a kind of variation of node are redistributed also is provided, it is specific as follows：
After node in group system has variation, the mark for the node that group system includes and the mark of data block are determined
The correspondence of knowledge.
Specifically, the node in group system include variation before node and the node after variation, and change after section
Point can be newly-increased node；Or, or delete node.In one example, as shown in table 7a, for node before and after variation
A kind of example with the correspondence of data block identifier is identified, shown by table 7a is the example of newly-increased node, changes front nodal point
Mark include node 1 and node 2, wherein 1 corresponding data block of node is identified as data block 1, data block 2 and data block
3,2 corresponding data block of node is identified as data block 4, data block 5 and data block 6；The mark for changing posterior nodal point includes node
1, node 2 and node 3；Wherein, 1 corresponding data block of node is identified as data block 1 and data block 2,2 corresponding data of node
Block is identified as data block 3 and data block 4, and 3 corresponding data block of node is identified as data block 5 and data block 6.Namely
It says, node 1 and node 2 are ancestor node, and node 3 is newly-increased node.
Table 7a：A kind of example of the mark of the front and back node of variation and the correspondence of data block identifier
In another example, as shown in table 7b, for the correspondence of the mark and data block identifier of node before and after variation
Another example, shown by table 7b is the example for deleting node, and the mark for changing front nodal point includes node 1, node 2 and node
3, wherein 1 corresponding data block of node is identified as data block 1 and data block 2, and 2 corresponding data block of node is identified as number
According to block 3 and data block 4,3 corresponding data block of node is identified as data block 5 and data block 6；The mark packet of node after variation
Include node 1 and node 2, wherein 1 corresponding data block of node is identified as data block 1, data block 2 and data block 3, and node 2 is right
The data block answered is identified as data block 4, data block 5 and data block 6.That is, node 1 and node 2 are ancestor node, section
Point 3 is to delete node.
Table 7b：Another example of the mark of the front and back node of variation and the correspondence of data block identifier
It is directed to third node, according to the correspondence of the mark for the node determined and the mark of data block, obtains institute
The mark of the corresponding data block of mark of third node is stated, and according to the mark pair of the third node in the third node
The mark for the data block answered redistributes data；The third node is any one node in the node after variation.
Specifically, by taking newly-increased node as an example, the mark for changing posterior nodal point includes node 1, node 2 and node 3；Wherein,
1 corresponding data block of node is identified as data block 1 and data block 2, and 2 corresponding data block of node is identified as 3 He of data block
Data block 4,3 corresponding data block of node are identified as data block 5 and data block 6.If node 1 is used as the third node,
It can be by the data storage in the 1 corresponding data block identifier 1 of node before changing shown in table 7a to the corresponding number of third node
It is 2 by the data storage in data block identifier 2 to the corresponding data block identifier of third node in the data block for being 1 according to block identification
Data block in.Further, data are being redistributed, can also by be identified as in the node 1 before variation 1 data block and mark
It deletes for 2 data block, is redistributed so as to complete data quickly, in an orderly manner.
Based on same inventive concept, the structure that Fig. 4 illustrates a kind of device provided in an embodiment of the present invention is shown
It is intended to, which can be device 102 shown in Fig. 1, as shown in figure 4, the device includes receiving unit 401, processing unit
402, indicating unit 403；Wherein,
Receiving unit 401, the data processing request for receiving client device transmission are wrapped in the data processing request
Include the mark of pending data；
Processing unit 402, for the mark according to the pending data, in the multiple nodes for including from group system really
Make the mark of the corresponding first node of mark of the pending data；
Indicating unit 403 indicates the first node to the pending number for the mark according to the first node
According to being handled.
Optionally, the processing unit 402 is specifically used for：
According to the mark of the data to be stored, the mark of corresponding first data block of mark of the data to be stored is obtained
Know；
According to the default correspondence of the mark of data block and the mark of node, the mark of first data block is corresponded to
Node mark of the mark as the first node.
Optionally, the processing unit 402 is specifically used for：
The mark of the data to be stored is calculated using consistency hash algorithm, obtains the data to be stored
Identify corresponding target cryptographic Hash；
According to the default correspondence of the mark of cryptographic Hash and data block, corresponding first number of the target cryptographic Hash is obtained
According to the mark of block.
Optionally, the data processing request is data storage request, further includes described wait in the data processing request
Handle the corresponding value of data；
The indicating unit 403 is specifically used for：
Indicate that the first node arrives the mark of the data to be stored and the corresponding value storage of the data to be stored
First data block.
Optionally, the data processing request is data inquiry request；
The indicating unit 403 is specifically used for：
Indicate that the first node waits for described in being inquired in first data block according to the mark of the pending data
Handle the corresponding value of data.
In the embodiment of the present invention, the data processing request sent according to the client device received includes pending number
According to mark, the corresponding first node of mark of the pending data is determined in the multiple nodes for including from group system
Mark；And then according to the mark of the first node, indicate that the first node handles the pending data.This hair
Bright embodiment can determine the mark of corresponding node according to the mark of data, compared with the existing technology each of middle group system
For node will handle identical data processing request, the embodiment of the present invention only needs the transmission data to corresponding node to ask
Ask, can saving network flow, alleviate the pressure of group system；Further, since there are data in the embodiment of the present invention
The correspondence of mark and the mark of node therefore, can be specified pair according to the mark of data after receiving data processing request
The node answered is handled, and is all responded to the data processing request without all nodes in group system, can so be made
The data of different node processing different identifications in group system the case where so as to effectively avoid data redundancy, save section
The space of points improves the utilization rate of group system.
It should be understood by those skilled in the art that, the embodiment of the present invention can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, the present invention can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The present invention be with reference to according to the method for the embodiment of the present invention, the flow of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
God and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of data processing method, which is characterized in that the method includes：
The data processing request that client device is sent is received, the data processing request includes the mark of pending data；
According to the mark of the pending data, the pending data is determined in the multiple nodes for including from group system
Identify the mark of corresponding first node；
According to the mark of the first node, indicate that the first node handles the pending data.
2. according to the method described in claim 1, it is characterized in that, according to the mark of the pending data, from group system
Including multiple nodes in determine the pending data the corresponding first node of mark mark, including：
According to the mark of the data to be stored, the mark of corresponding first data block of mark of the data to be stored is obtained；
According to the default correspondence of the mark of data block and the mark of node, by the corresponding section of mark of first data block
Mark of the mark of point as the first node.
3. according to the method described in claim 2, it is characterized in that, according to the mark of the data to be stored, described wait for is obtained
The mark of corresponding first data block of mark of data is stored, including：
The mark of the data to be stored is calculated using consistency hash algorithm, obtains the mark of the data to be stored
Corresponding target cryptographic Hash；
According to the default correspondence of the mark of cryptographic Hash and data block, corresponding first data block of the target cryptographic Hash is obtained
Mark.
4. according to the method described in claim 2, it is characterized in that, the data processing request be data storage request, it is described
It further include the corresponding value of the pending data in data processing request；
Indicate that the first node handles the pending data, including：
Indicate the first node by the mark of the data to be stored and the corresponding value storage of the data to be stored described in
First data block.
5. according to the method described in claim 2, it is characterized in that, the data processing request is data inquiry request；
Indicate that the first node handles the pending data, including：
It is described pending to indicate that the first node is inquired according to the mark of the pending data in first data block
The corresponding value of data.
6. a kind of device, which is characterized in that described device includes：
Receiving unit, the data processing request for receiving client device transmission, the data processing request includes waiting locating
Manage the mark of data；
Processing unit determines institute for the mark according to the pending data in the multiple nodes for including from group system
State the mark of the corresponding first node of mark of pending data；
Indicating unit indicates that the first node carries out the pending data for the mark according to the first node
Processing.
7. device according to claim 6, which is characterized in that the processing unit is specifically used for：
According to the mark of the data to be stored, the mark of corresponding first data block of mark of the data to be stored is obtained；
According to the default correspondence of the mark of data block and the mark of node, by the corresponding section of mark of first data block
Mark of the mark of point as the first node.
8. device according to claim 7, which is characterized in that the processing unit is specifically used for：
The mark of the data to be stored is calculated using consistency hash algorithm, obtains the mark of the data to be stored
Corresponding target cryptographic Hash；
According to the default correspondence of the mark of cryptographic Hash and data block, corresponding first data block of the target cryptographic Hash is obtained
Mark.
9. device according to claim 7, which is characterized in that the data processing request is data storage request, described
It further include the corresponding value of the pending data in data processing request；
The indicating unit is specifically used for：
Indicate the first node by the mark of the data to be stored and the corresponding value storage of the data to be stored described in
First data block.
10. device according to claim 7, which is characterized in that the data processing request is data inquiry request；
The indicating unit is specifically used for：
It is described pending to indicate that the first node is inquired according to the mark of the pending data in first data block
The corresponding value of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810031750.3A CN108345643A (en) | 2018-01-12 | 2018-01-12 | A kind of data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810031750.3A CN108345643A (en) | 2018-01-12 | 2018-01-12 | A kind of data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108345643A true CN108345643A (en) | 2018-07-31 |
Family
ID=62961392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810031750.3A Pending CN108345643A (en) | 2018-01-12 | 2018-01-12 | A kind of data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108345643A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109032804A (en) * | 2018-08-01 | 2018-12-18 | 阿里巴巴集团控股有限公司 | Data processing method and device, server |
| CN109032803A (en) * | 2018-08-01 | 2018-12-18 | 阿里巴巴集团控股有限公司 | Data processing method and device, client |
| CN109145053A (en) * | 2018-08-01 | 2019-01-04 | 阿里巴巴集团控股有限公司 | Data processing method and device, client, server |
| CN110275924A (en) * | 2019-05-09 | 2019-09-24 | 阿里巴巴集团控股有限公司 | Asynchronous data processing method, device and computer readable storage medium |
| CN110659272A (en) * | 2019-08-30 | 2020-01-07 | 深圳壹账通智能科技有限公司 | Data cleaning method and system |
| CN111782610A (en) * | 2020-06-03 | 2020-10-16 | 中移（杭州）信息技术有限公司 | Log processing method, device, server, system network and storage medium |
| CN111796950A (en) * | 2020-07-16 | 2020-10-20 | 网易（杭州）网络有限公司 | Data processing method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831120A (en) * | 2011-06-15 | 2012-12-19 | 腾讯科技（深圳）有限公司 | Data processing method and system |
| CN102968498A (en) * | 2012-12-05 | 2013-03-13 | 华为技术有限公司 | Method and device for processing data |
| CN104298687A (en) * | 2013-07-18 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Hash partitioning management method and device |
| CN104391863A (en) * | 2014-10-23 | 2015-03-04 | 中国建设银行股份有限公司 | Data storage method and device |
| CN106254240A (en) * | 2016-09-18 | 2016-12-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method and routing layer equipment and system |
-
2018
- 2018-01-12 CN CN201810031750.3A patent/CN108345643A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831120A (en) * | 2011-06-15 | 2012-12-19 | 腾讯科技（深圳）有限公司 | Data processing method and system |
| CN102968498A (en) * | 2012-12-05 | 2013-03-13 | 华为技术有限公司 | Method and device for processing data |
| CN104298687A (en) * | 2013-07-18 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Hash partitioning management method and device |
| CN104391863A (en) * | 2014-10-23 | 2015-03-04 | 中国建设银行股份有限公司 | Data storage method and device |
| CN106254240A (en) * | 2016-09-18 | 2016-12-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method and routing layer equipment and system |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145053B (en) * | 2018-08-01 | 2021-03-23 | 创新先进技术有限公司 | Data processing method and device, client and server |
| CN109032803A (en) * | 2018-08-01 | 2018-12-18 | 阿里巴巴集团控股有限公司 | Data processing method and device, client |
| CN109145053A (en) * | 2018-08-01 | 2019-01-04 | 阿里巴巴集团控股有限公司 | Data processing method and device, client, server |
| CN109032804A (en) * | 2018-08-01 | 2018-12-18 | 阿里巴巴集团控股有限公司 | Data processing method and device, server |
| US11563805B2 (en) | 2018-08-01 | 2023-01-24 | Advanced New Technologies Co., Ltd. | Method, apparatus, client terminal, and server for data processing |
| US11233878B2 (en) | 2018-08-01 | 2022-01-25 | Advanced New Technologies Co., Ltd. | Data processing method, apparatus, and client device |
| US11106488B2 (en) | 2018-08-01 | 2021-08-31 | Advanced New Technologies Co., Ltd. | Blockchain read/write data processing method, apparatus, and server |
| CN110275924A (en) * | 2019-05-09 | 2019-09-24 | 阿里巴巴集团控股有限公司 | Asynchronous data processing method, device and computer readable storage medium |
| CN110275924B (en) * | 2019-05-09 | 2023-02-03 | 创新先进技术有限公司 | Asynchronous data processing method and device and computer readable storage medium |
| WO2021036319A1 (en) * | 2019-08-30 | 2021-03-04 | 深圳壹账通智能科技有限公司 | Data cleaning method and system |
| CN110659272A (en) * | 2019-08-30 | 2020-01-07 | 深圳壹账通智能科技有限公司 | Data cleaning method and system |
| CN111782610A (en) * | 2020-06-03 | 2020-10-16 | 中移（杭州）信息技术有限公司 | Log processing method, device, server, system network and storage medium |
| CN111782610B (en) * | 2020-06-03 | 2024-03-22 | 中移（杭州）信息技术有限公司 | Log processing method, device, server, system network and storage medium |
| CN111796950A (en) * | 2020-07-16 | 2020-10-20 | 网易（杭州）网络有限公司 | Data processing method and system |
| CN111796950B (en) * | 2020-07-16 | 2023-06-30 | 网易（杭州）网络有限公司 | Data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108345643A (en) | A kind of data processing method and device | |
| JP5514041B2 (en) | Identifier assignment method and program | |
| JP5719323B2 (en) | Distributed processing system, dispatcher and distributed processing management device | |
| CN105450705B (en) | Business data processing method and equipment | |
| EP3236365A1 (en) | Data query method and device | |
| CN106101232A (en) | Load-balancing method and device | |
| US10848407B2 (en) | Efficient density based geo clustering | |
| CN103942209B (en) | Data processing method | |
| US20140025723A1 (en) | Cloud storage system and data storage and sharing method based on the system | |
| CN103338252A (en) | Distributed database concurrence storage virtual request mechanism | |
| CN109085999A (en) | data processing method and processing system | |
| CN106790552B (en) | A kind of content providing system based on content distributing network | |
| CN102891872B (en) | The method and system of data storage and query in a kind of peer-to-peer network | |
| CN107451853A (en) | Method, apparatus, system and the storage medium that a kind of red packet distributes in real time | |
| CN111723073B (en) | Data storage processing method, device, processing system and storage medium | |
| CN105491078B (en) | Data processing method and device, SOA system in SOA system | |
| CN108563697A (en) | A kind of data processing method, device and storage medium | |
| CN107330680A (en) | Red packet control method, device, computer equipment and computer-readable recording medium | |
| CN105471700B (en) | A kind of methods, devices and systems of Message Processing | |
| CN106815258A (en) | A kind of date storage method and coordinator node | |
| CN109361625A (en) | Verify the method, apparatus and controller of forwarding-table item | |
| CN103248636B (en) | The system and method downloaded offline | |
| CN109144690B (en) | Task processing method and device | |
| CN108153494B (en) | A kind of I/O request processing method and processing device | |
| CN106446050A (en) | Method and system for subscription by aiming at changed data of database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180731 |