CN108763381A - Divide table method and apparatus based on consistency hash algorithm - Google Patents
Divide table method and apparatus based on consistency hash algorithm Download PDFInfo
- Publication number
- CN108763381A CN108763381A CN201810481880.7A CN201810481880A CN108763381A CN 108763381 A CN108763381 A CN 108763381A CN 201810481880 A CN201810481880 A CN 201810481880A CN 108763381 A CN108763381 A CN 108763381A
- Authority
- CN
- China
- Prior art keywords
- business datum
- hash algorithm
- serial number
- consistency hash
- divide
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
What the invention discloses a kind of based on consistency hash algorithm divide table method and apparatus, wherein this divide the table method to include：Obtaining record has the data summary table of business datum；The business datum in data summary table is stored in multiple first tables using consistency hash algorithm；It changes in business datum, and the destination number of table for the business datum after storage change with the quantity of the first table when differing, the business datum in the first table of part in multiple first tables is migrated using consistency hash algorithm, it can be when business datum variation needs the quantity of adjustment table, the amount of migration for reducing business datum, is rapidly completed Data Migration.
Description
Technical field
The present invention relates to technical field of data processing more particularly to it is a kind of based on consistency hash algorithm divide table method and
Equipment.
Background technology
With the fast development of Internet service, internet services data is increasing, and the data storage of individual table is
The development need of business cannot be met.Because if the data volume of a table is more than a certain amount of, then inquiry, insertion uniform velocity meeting
Entire lowering, so needing to carry out a point table to single table.The quantity that table can be specifically estimated according to data volume, is divided into multiple tables, makes every
The data volume for opening table lowers, to restore single table performance.
At present in the fractionation to database table, the method used is the table number obtained to some modulus where it.Such as
User's table is divided into 100 sublists, table serial number 0~99 by plan now, it is assumed that and the identity number (id) of user A is 1234,
By 1234%100=34, obtaining user A should be inserted into inside the table of serial number 34.Although can be with according to id modulo operations
Realize the purpose for dividing table, but when business datum variation needs the quantity of adjustment table, substantially all services data are required for moving
It moves, causes Data Migration slow.
Invention content
Dividing table method and apparatus based on consistency hash algorithm the main purpose of the present invention is to provide a kind of, to solve
When business datum variation needs the quantity of adjustment table, substantially all services data are required for migrating, and cause Data Migration slow
Problem.
To achieve the above object, dividing table method based on consistency hash algorithm the present invention provides a kind of, this divides table side
Method includes：
Obtaining record has the data summary table of business datum；
The business datum in data summary table is stored in multiple first tables using consistency hash algorithm；
It changes in business datum, and the destination number of the table for the business datum after storage change and the first table
When the quantity of lattice differs, using consistency hash algorithm to the business datum in the first table of part in multiple first tables
It is migrated.
Wherein, the step being stored in the business datum in data summary table using consistency hash algorithm in multiple first tables
Suddenly, including：
According to the quantity for dividing table key and the first table for storage service data of setting, consistency hash algorithm is used
It obtains and each divides table key assignments corresponding first table serial number in data summary table；Wherein, it refers to specific in data summary table to divide table key
Row, it refers to the value of the particular column in data summary table to divide table key assignments；
According to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key assignments, respectively will
Divide the corresponding business datum of table key assignments to be stored in this in data summary table to divide in corresponding first table of table key assignments；Wherein, Mei Ge
One table serial number corresponds to first table.
Wherein, the business datum in the first table of part in multiple first tables is carried out using consistency hash algorithm
The step of migration, including：
According to destination number and multiple first tables, using consistency hash algorithm to the business number in the first table of part
According to being migrated.
Wherein, according to destination number and multiple first tables, using consistency hash algorithm in the first table of part
The step of business datum is migrated, including：
Following steps are executed respectively for each first table：
According to dividing table key to traverse the first table, obtain the first table divides table key assignments；
Divide table key assignments according to destination number and the first table, divides table using what consistency hash algorithm obtained the first table
The corresponding second table serial number of key assignments；
If the second table serial number the first table serial number corresponding with the first table is identical, not to the business in the first table
Data are migrated；
If the second table serial number the first table serial number corresponding with the first table differs, to the business in the first table
Data are migrated.
Wherein, the step of business datum in the first table being migrated, including：
Business datum in first table is inserted into corresponding second table of the second table serial number；
Delete the business datum in the first table.
Wherein, according to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key assignments,
The corresponding business datum of table key assignments will be divided to be stored in the step divided in corresponding first table of table key assignments in data summary table respectively
Later, the table method is divided to further include：
For each first table, respectively according to the corresponding first table serial number gencration structured query language of the first table
SQL statement.
Wherein, the step being inserted into the business datum in the first table in corresponding second table of the second table serial number it
Afterwards, the table method is divided to further include：
For the second table, according to the corresponding second table serial number gencration SQL statement of the second table.
Wherein, destination number is no more than 5,000,000.
Table equipment that the present invention also provides a kind of based on consistency hash algorithm divide, including memory, processor and
The computer program that can be run in memory and on a processor is stored, which realizes above-mentioned when executing computer program
Based on consistency hash algorithm the step of dividing table method.
The present invention also provides a kind of computer readable storage medium, computer-readable recording medium storage has computer journey
Sequence, the computer program realized when being executed by processor it is above-mentioned based on consistency hash algorithm the step of dividing table method.
The said program of the present invention includes at least following advantageous effect：
In an embodiment of the present invention, by using consistency hash algorithm to record have the data summary table of business datum into
It goes and divides table, the business datum in data summary table is stored in multiple first tables, wherein since consistency hash algorithm can incite somebody to action
Business datum is distributed evenly in multiple first tables, so that when business datum changes, and it is used for storage change
When the destination number of the table of business datum afterwards and the quantity of the first table differ, using consistency hash algorithm to multiple
The business datum in the first table of part in first table is migrated, and then is reached and needed to adjust in business datum variation
When the quantity of whole table, the amount of migration of business datum is reduced, the effect of Data Migration is rapidly completed.
Description of the drawings
Fig. 1 is the flow chart for dividing table method based on consistency hash algorithm in first embodiment of the invention；
Fig. 2 is the flow chart for dividing table method based on consistency hash algorithm in second embodiment of the invention；
Fig. 3 is in second embodiment of the invention be on consistency Hash rings Object Selection machine schematic diagram；
Fig. 4 is the structural schematic diagram of consistency Hash rings after increasing machine in second embodiment of the invention；
Fig. 5 is the structural schematic diagram for dividing table equipment based on consistency hash algorithm in third embodiment of the invention.
Specific implementation mode
Exemplary embodiment of the present invention is more fully described below with reference to accompanying drawings.Although showing the present invention in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the present invention without should be by embodiments set forth here
It is limited.It is to be able to be best understood from the present invention on the contrary, providing these embodiments, and can be by the scope of the present invention
Completely it is communicated to those skilled in the art.
First embodiment
As shown in Figure 1, specific embodiments of the present invention provide it is a kind of dividing table method based on consistency hash algorithm, should
Point table method includes：
Step 11, obtaining record has the data summary table of business datum.
It wherein, in a specific embodiment of the present invention, can be total obtain data by way of outside reception data summary table
Table.Specifically, the data summary table can be input by user, the transmission of other external equipments can also be.Of course, it should be understood that
In a specific embodiment of the present invention, the specific acquisition modes of above-mentioned data summary table are not limited.
Step 12, the business datum in data summary table is stored in multiple first tables using consistency hash algorithm.
Wherein, in a specific embodiment of the present invention, the Hash method in guava packets can be used in above-mentioned consistency hash algorithm
(hashing) class, it is accomplished that a kind of consistency Hash (hash) algorithm of entitled jump consistent hash.And
The design object of jump consistent hash is：Balance is evenly distributed in object in all buckets；Monotonicity, when
When the quantity variation of bucket, it is only necessary to some objects be moved to new bucket from old bucket, other movements need not be done.Therefore, by upper
The business datum in data summary table can be uniformly distributed in multiple first tables by stating consistency hash algorithm, and subsequently to work as
When the quantity of table changes, it is only necessary to mobile a part of data.
Step 13, it changes in business datum, and the destination number of the table for the business datum after storage change
When being differed with the quantity of the first table, using consistency hash algorithm in the first table of part in multiple first tables
Business datum is migrated.
Wherein, above-mentioned business datum, which changes, extends or is reduced including business datum, and above-mentioned destination number is by business
Development speed determines, is likely larger than the quantity of the first table, it is also possible to it is less than the quantity of the first table, it specifically can be according to current
Business day increased number determines table number to estimate the total amount of data of the following 2-3.Here, as a preferred example, on
It states destination number and is no more than 5,000,000.Because rule of thumb, the single table data volume of Relational DBMS (mySQL) is maximum
5,000,000 are not exceeded, once data volume is more than 5,000,000, inquiry velocity will be decreased obviously.
Wherein, in a specific embodiment of the present invention, since consistency hash algorithm can uniformly divide all business datums
It is distributed in table, therefore when business datum changes, and the destination number of the table for the business datum after storage change
When being differed with the quantity of the first table, Data Migration can be completed in the business datum in the first table of movable part.
As it can be seen that in a specific embodiment of the present invention, having business datum to record by using consistency hash algorithm
Data summary table carries out a point table, and the business datum in data summary table is stored in multiple first tables, wherein since consistency is breathed out
Business datum can be distributed evenly in multiple first tables by uncommon algorithm, so that when business datum changes, and use
When the destination number of the table of business datum after storage change and the quantity of the first table differ, consistency Hash is used
Algorithm migrates the business datum in the first table of part in multiple first tables, and then reaches in business datum
When variation needs the quantity of adjustment table, the amount of migration of business datum is reduced, the effect of Data Migration is rapidly completed.
Second embodiment
As shown in Fig. 2, specific embodiments of the present invention provide it is a kind of dividing table method based on consistency hash algorithm, should
Point table method includes：
Step 21, obtaining record has the data summary table of business datum.
It wherein, in a specific embodiment of the present invention, can be total obtain data by way of outside reception data summary table
Table.Specifically, the data summary table can be input by user, the transmission of other external equipments can also be.Of course, it should be understood that
In a specific embodiment of the present invention, the specific acquisition modes of above-mentioned data summary table are not limited.And preferably in this as one
Example, can be realized by interface (Dao Impl) class algorithm of operating database.
Step 22, according to the quantity for dividing table key and the first table for storage service data of setting, consistency is used
Hash algorithm, which obtains, each divides table key assignments corresponding first table serial number in data summary table.
Wherein, it refers to the particular column in data summary table to divide table key, and it refers to the value of the particular column in data summary table to divide table key assignments,
And divide table key that can be determined according to type of service.It illustrates, it is assumed that data summary table is user's order table, and user's order table includes
The corresponding order time of origins of multiple user id and each user id, trade name corresponding with each user id and quantity
Deng, it at this time can be using user id as dividing table key, and the numerical value of user's id row is to divide table key assignments.
Step 23, according to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key assignments,
The corresponding business datum of table key assignments will be divided to be stored in this in data summary table respectively to divide in corresponding first table of table key assignments.
Wherein, each first table serial number corresponds to first table, i.e., multiple first table serial numbers and multiple first tables
Lattice correspond, consequently facilitating business datum is uniformly distributed in multiple first tables.
It should be noted that in a specific embodiment of the present invention, after having executed above-mentioned steps 23, above-mentioned point of table side
Method further includes following steps：For each first table, respectively according to the corresponding first table serial number gencration structure of the first table
Change query language SQL statement, and by the structured query language of generation (SQL, Structured Query Language) sentence
It is inserted into inquiry database, facilitates the data in the first table of subsequent query, promote search efficiency.
Step 24, it changes in business datum, and the destination number of the table for the business datum after storage change
When being differed with the quantity of the first table, using consistency hash algorithm in the first table of part in multiple first tables
Business datum is migrated.
Wherein, above-mentioned business datum, which changes, extends or is reduced including business datum, and above-mentioned destination number is by business
Development speed determines, is likely larger than the quantity of the first table, it is also possible to it is less than the quantity of the first table, it specifically can be according to current
Business day increased number determines table number to estimate the total amount of data of the following 2-3.Here, as a preferred example, on
It states destination number and is no more than 5,000,000.Because rule of thumb, the mono- table data volume maximums of mySQL do not exceed 5,000,000, once data
Amount is more than 5,000,000, then inquiry velocity will be decreased obviously.
Wherein, in a specific embodiment of the present invention, above-mentioned consistency hash algorithm can be used in guava packets
Hashing classes, it is accomplished that a kind of consistent hashing algorithm of entitled jump consistent hash.And jump
The design object of consistent hash is：Balance is evenly distributed in object in all buckets；Monotonicity, when the number of bucket
When amount variation, it is only necessary to some objects be moved to new bucket from old bucket, other movements need not be done.Therefore, by above-mentioned consistent
Property hash algorithm the business datum in data summary table can be uniformly distributed in multiple first tables, and make follow-up when table
When quantity changes, it is only necessary to mobile a part of data.It can be directly invoked in guava packets when wherein specifically used
Consistent hash (long input, int buckets) method in hashing classes.Input is incoming to divide table key assignments,
Buckets is passed to table number (being normally defined constant), realizes according to table number and divides table key assignments, is determined using consistent hashing algorithm
Table number, and return to table number.If dividing table key to be character string type certainly, another method consistent can be called
Hash (HashCode hashCode, int buckets), incoming to divide the value after table key assignments hash, calculating hashCode can
With the MurMurHash algorithms for using guava to encapsulate.
It is simple to illustrate consistency Hash principle herein by taking hash rings as an example.Consistent hashing algorithm is called one by one
The data structure of cause property hash rings is realized.As shown in figure 3, the starting point of this ring is 0, terminal is 2^32-1, and starting point and end
Point connection, the integer of the centre of ring is by distribution counterclockwise, therefore the integer distribution of this ring is [0,2^32-1]。o1、o2、
O3, o4 are object, and m1, m2, m3, m4 are the value after hash；C1, c2, c3 are machine node, after t1, t2, t3 are its hash
Value.After they being put into a hash ring, the nearest machine of hash values apart from this object is searched clockwise on hash rings
Device is the machine belonging to this object.
For example, for object o2, it is c1 that sequence needle, which finds nearest machine, therefore machine c1 can cache object o2.And machine
C2 then caches o3, o4, machine c3 then cache object o1.Now, it is assumed that increasing a machine c4, and by machine c4
It is added between the machine c3 and c2 of hash rings.At this moment, the object only between machine c3 and c4 needs to redistribute new machine
Device.It should be noted that the present invention's divides table method to be namely based on what this consistent hashing principle was realized, wherein object is seen
Work divides table key assignments, the serial number of table to regard machine as, then may be implemented that, when the quantity of tables of data needs adjustment, only a small amount of number need to be moved
According to purpose.
It can be seen that in a specific embodiment of the present invention, since consistency hash algorithm can be all objects equably
It is distributed in all buckets.Therefore when increasing to two buckets by a bucket, it is only necessary to mobile 1/2 object, and so on, when me
Newly plus when a table, it is only necessary to the data of mobile 1/ (n+1), the data of n/ (n+1) need not move, and wherein n is present
Table number, i.e., the quantity of above-mentioned first table.And found through lot of experiment validation, in table data variation, such as operation expanding (or subtract
It is few) when, a large amount of Data Migration need not be carried out by increasing table (or reducing table), it is only necessary to mobile a part of data.Specifically
, when business expands demand, when table number increases or decreases, need to only move the data volume of 1/n, n be table number after expanding or
Table number after reduction, i.e., above-mentioned destination number.
Wherein, in a specific embodiment of the present invention, it changes in business datum, and for the business after storage change
When the destination number of the table of data and the quantity of the first table differ, consistency hash algorithm pair is used in above-mentioned steps 24
The specific implementation for the step of business datum in the first table of part in multiple first tables is migrated is：According to mesh
Quantity and multiple first tables are marked, the business datum in the first table of part is migrated using consistency hash algorithm.
Specifically, in a specific embodiment of the present invention, according to destination number and multiple first tables, being breathed out using consistency
The specific implementation that uncommon algorithm migrates the business datum in the first table of part is as follows：
Following steps are executed respectively for each first table：
Step 1, according to dividing table key to traverse the first table, obtain the first table divides table key assignments.
Step 2 divides table key assignments according to destination number and the first table, consistency hash algorithm is used to obtain the first table
Lattice divide table key assignments corresponding second table serial number.
Step 3, if the second table serial number the first table serial number corresponding with the first table is identical, not to the first table
In business datum migrated.
Step 4, if the second table serial number the first table serial number corresponding with the first table differs, to the first table
In business datum migrated.
That is, in a specific embodiment of the present invention, by there is the first table to execute above-mentioned steps one respectively to step more
Four, only when dividing the corresponding first table serial number of table key assignments to be differed with the second table serial number, divide table key assignments corresponding this
Data in first table are migrated, to greatly reduce Data Migration amount, it is ensured that Data Migration can be rapidly completed.
And in a specific embodiment of the present invention, the business datum in first table is migrated in above-mentioned steps four
The specific implementation of step is：Business datum in first table is inserted into corresponding second table of the second table serial number,
And delete the business datum in the first table.That is, after the business datum in the first table is inserted into the second table, delete former
Business datum in first table is the Data Migration completed for the first table.It is above-mentioned in this as a preferred example
Step 4 can be realized by Data Migration (Data Migration) class algorithm.
Wherein, in a specific embodiment of the present invention, the business datum in the first table is being inserted into the second table serial number
After in corresponding second table, above-mentioned point of table method further includes following steps：It is corresponding according to the second table for the second table
Second table serial number gencration SQL statement, and the SQL statement of generation is inserted into inquiry database, so as to the second table of subsequent query
Data in lattice promote search efficiency.
It can be seen that in a specific embodiment of the present invention, by dividing table key and for storage service number according to setting
According to the first table quantity, using consistency hash algorithm to record there is the data summary table of business datum to carry out a point table, will count
It is stored in multiple first tables according to the business datum in summary table, wherein since consistency hash algorithm can be uniform by business datum
Ground is distributed in multiple first tables, so that when business datum changes, and for the business datum after storage change
The destination number of table and the quantity of the first table when differing, according to destination number and multiple first tables, using consistent
Property hash algorithm the business datum in the first table of part is migrated, and then reach business datum variation need to adjust
When the quantity of whole table, the amount of migration of business datum is reduced, the effect of Data Migration is rapidly completed.
3rd embodiment
As shown in figure 5, specific embodiments of the present invention provide it is a kind of dividing table equipment based on consistency hash algorithm, packet
It includes memory 51, processor 52 and is stored in the computer program 53 that can be run in memory 51 and on the processor 52, it should
Processor 52 execute computer program 53 when realize it is above-mentioned based on consistency hash algorithm the step of dividing table method.
Specifically, in a specific embodiment of the present invention, processor 52 realizes following steps when executing computer program 53：
Obtaining record has the data summary table of business datum；The business datum in data summary table is stored in using consistency hash algorithm more
In a first table；Change in business datum, and the destination number of the table for the business datum after storage change with
When the quantity of first table differs, using consistency hash algorithm to the industry in the first table of part in multiple first tables
Business data are migrated.
Optionally, following steps are also realized when processor 52 executes computer program 53：Divide table key and use according to setting
In the quantity of the first table of storage service data, is obtained using consistency hash algorithm and each divide table key-value pair in data summary table
The the first table serial number answered；According to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key
Value, will divide the corresponding business datum of table key assignments to be stored in this and divides in corresponding first table of table key assignments respectively in data summary table；Its
In, it refers to the particular column in data summary table to divide table key, and it refers to the value of the particular column in data summary table to divide table key assignments；Each first table
Lattice serial number corresponds to first table.
Optionally, following steps are also realized when processor 52 executes computer program 53：According to destination number and multiple
One table migrates the business datum in the first table of part using consistency hash algorithm.
Optionally, following steps are also realized when processor 52 executes computer program 53：Distinguish for each first table
Execute following steps：According to dividing table key to traverse the first table, obtain the first table divides table key assignments；According to destination number and first
Table divides table key assignments, divides table key assignments corresponding second table serial number using consistency hash algorithm the first table of acquisition；If
Second table serial number the first table serial number corresponding with the first table is identical, then is not moved to the business datum in the first table
It moves；If the second table serial number the first table serial number corresponding with the first table differs, to the business datum in the first table
It is migrated.
Optionally, following steps are also realized when processor 52 executes computer program 53：By the business number in the first table
According in corresponding second table of the second table serial number of insertion；Delete the business datum in the first table.
Optionally, following steps are also realized when processor 52 executes computer program 53：For each first table, respectively
According to the corresponding first table serial number gencration structured query language SQL statement of the first table.
Optionally, following steps are also realized when processor 52 executes computer program 53：For the second table, according to second
The corresponding second table serial number gencration SQL statement of table.
That is, in a specific embodiment of the present invention, being realized when the processor 52 of table equipment 5 being divided to execute computer program 53
State based on consistency hash algorithm the step of dividing table method, can business datum change need the quantity of adjustment table when, subtract
The amount of migration of few business datum, is rapidly completed Data Migration.
Illustratively, above-mentioned point of table equipment 5 can be desktop PC, notebook, palm PC and cloud server
Equal computing devices.This divides table equipment 5 may include, but is not limited only to, processor 52, memory 51.Those skilled in the art can be with
Understand, the schematic diagram is only a point example for table equipment 5, does not constitute the restriction to dividing table equipment 5, may include than diagram
More or fewer components either combine certain components or different components, such as divide table equipment 5 that can also include input
Output equipment, network access equipment, bus etc..
It should be noted that due to divide the processor 52 of table equipment 5 execute computer program 53 when realize it is above-mentioned based on
The step of dividing table method of consistency hash algorithm, therefore the above-mentioned all implementations for dividing table method based on consistency hash algorithm
Example divides table equipment 5 suitable for this, and can reach same or analogous advantageous effect.
Fourth embodiment
The fourth embodiment of the present invention provides a kind of computer readable storage medium, computer-readable recording medium storage
There is computer program, which realizes when being executed by processor above-mentioned divides table method based on consistency hash algorithm
The step of.
Specifically, in a specific embodiment of the present invention, following steps are realized when computer program is executed by processor：It obtains
Record is taken to have the data summary table of business datum；The business datum in data summary table is stored in using consistency hash algorithm multiple
In first table；It changes in business datum, and the destination number of the table for the business datum after storage change and the
When the quantity of one table differs, using consistency hash algorithm to the business in the first table of part in multiple first tables
Data are migrated.
Optionally, following steps are also realized when computer program is executed by processor：Dividing table key and be used for according to setting
The quantity of first table of storage service data is obtained in data summary table using consistency hash algorithm and table key-value pair is each divided to answer
The first table serial number；According to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key assignments,
The corresponding business datum of table key assignments will be divided to be stored in this in data summary table respectively to divide in corresponding first table of table key assignments；Wherein,
It refers to the particular column in data summary table to divide table key, and it refers to the value of the particular column in data summary table to divide table key assignments；Each first table
Serial number corresponds to first table.
Optionally, following steps are also realized when computer program is executed by processor：According to destination number and multiple first
Table migrates the business datum in the first table of part using consistency hash algorithm.
Optionally, following steps are also realized when computer program is executed by processor：It is held respectively for each first table
Row following steps：According to dividing table key to traverse the first table, obtain the first table divides table key assignments；According to destination number and the first table
Lattice divide table key assignments, divide table key assignments corresponding second table serial number using consistency hash algorithm the first table of acquisition；If the
Two table serial numbers the first table serial number corresponding with the first table is identical, then is not moved to the business datum in the first table
It moves；If the second table serial number the first table serial number corresponding with the first table differs, to the business datum in the first table
It is migrated.
Optionally, following steps are also realized when computer program is executed by processor：By the business datum in the first table
It is inserted into corresponding second table of the second table serial number；Delete the business datum in the first table.
Optionally, following steps are also realized when computer program is executed by processor：For each first table, difference root
According to the corresponding first table serial number gencration structured query language SQL statement of the first table.
Optionally, following steps are also realized when computer program is executed by processor：For the second table, according to the second table
The corresponding second table serial number gencration SQL statement of lattice.
That is, in a specific embodiment of the present invention, the computer program of computer readable storage medium is executed by processor
Shi Shixian it is above-mentioned based on consistency hash algorithm the step of dividing table method, can change in business datum and need the number of adjustment table
When amount, the amount of migration of business datum is reduced, Data Migration is rapidly completed.
Illustratively, the computer program of computer readable storage medium includes computer program code, the computer
Program code can be source code form, object identification code form, executable file or certain intermediate forms etc..The computer can
Reading medium may include：Any entity or device, recording medium, USB flash disk, mobile hard of the computer program code can be carried
Disk, magnetic disc, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory
(RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..
It should be noted that the computer program due to computer readable storage medium realized when being executed by processor it is above-mentioned
Based on consistency hash algorithm the step of dividing table method, therefore the above-mentioned institute for dividing table method based on consistency hash algorithm
There is embodiment to be suitable for the computer readable storage medium, and can reach same or analogous advantageous effect.
It these are only the preferred embodiment of the present invention, be not intended to limit the scope of the invention, it is every to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of dividing table method based on consistency hash algorithm, which is characterized in that described to divide the table method to include：
Obtaining record has the data summary table of business datum；
The business datum in the data summary table is stored in multiple first tables using consistency hash algorithm；
It changes in the business datum, and the destination number of the table for the business datum after storage change and the first table
When the quantity of lattice differs, using consistency hash algorithm to the business datum in the first table of part in multiple first tables
It is migrated.
2. according to claim 1 point of table method, which is characterized in that described to use consistency hash algorithm by the data
The business datum in summary table is stored in the step in multiple first tables, including：
Divide table key and the quantity of the first table for storing the business datum according to setting, uses consistency hash algorithm
It obtains and each divides table key assignments corresponding first table serial number in the data summary table；Wherein, described that table key is divided to refer to the data
Particular column in summary table, it refers to the value of the particular column in the data summary table to divide table key assignments；
According to each dividing table key assignments corresponding first table serial number in data summary table, for each point of table key assignments, respectively by data
Divide the corresponding business datum of table key assignments to be stored in this in summary table to divide in corresponding first table of table key assignments；Wherein, each first table
Lattice serial number corresponds to first table.
3. according to claim 2 point of table method, which is characterized in that described to use consistency hash algorithm to multiple first
The step of business datum in the first table of part in table is migrated, including：
According to the destination number and multiple first tables, using consistency hash algorithm to the business number in the first table of part
According to being migrated.
4. according to claim 3 point of table method, which is characterized in that described according to the destination number and multiple first tables
Lattice, the step of migration to the business datum in the first table of part using consistency hash algorithm, including：
Following steps are executed respectively for each first table：
Table key is divided to traverse the first table according to described, obtain the first table divides table key assignments；
Divide table key assignments according to the destination number and the first table, divides table using what consistency hash algorithm obtained the first table
The corresponding second table serial number of key assignments；
If the second table serial number the first table serial number corresponding with first table is identical, not to first table
In business datum migrated；
If the second table serial number the first table serial number corresponding with first table differs, to first table
In business datum migrated.
5. according to claim 4 point of table method, which is characterized in that the business datum in first table into
The step of row migration, including：
Business datum in first table is inserted into corresponding second table of the second table serial number；
Delete the business datum in first table.
6. according to claim 2 point of table method, which is characterized in that each divide table key assignments in the summary table according to data
Corresponding first table serial number will divide the corresponding business datum of table key assignments to deposit respectively for each point of table key assignments in data summary table
This is stored in divide after the step in corresponding first table of table key assignments, it is described to divide the table method to further include：
For each first table, respectively according to the corresponding first table serial number gencration structured query language SQL of the first table
Sentence.
7. according to claim 5 point of table method, which is characterized in that in the business datum by first table
It is inserted into after the step in corresponding second table of the second table serial number, it is described to divide the table method to further include：
For the second table, according to the corresponding second table serial number gencration SQL statement of the second table.
8. according to claim 1 point of table method, which is characterized in that the destination number is no more than 5,000,000.
9. a kind of dividing table equipment based on consistency hash algorithm, including memory, processor and it is stored in the memory
In and the computer program that can run on the processor, which is characterized in that the processor executes the computer program
Shi Shixian such as claim 1 to 8 any one of them based on consistency hash algorithm the step of dividing table method.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, feature to exist
In realization such as claim 1 to 8 any one of them is calculated based on consistency Hash when the computer program is executed by processor
The step of dividing table method of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810481880.7A CN108763381B (en) | 2018-05-18 | 2018-05-18 | Table dividing method and device based on consistent Hash algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810481880.7A CN108763381B (en) | 2018-05-18 | 2018-05-18 | Table dividing method and device based on consistent Hash algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108763381A true CN108763381A (en) | 2018-11-06 |
| CN108763381B CN108763381B (en) | 2022-05-17 |
Family
ID=64007359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810481880.7A Active CN108763381B (en) | 2018-05-18 | 2018-05-18 | Table dividing method and device based on consistent Hash algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108763381B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111125114A (en) * | 2019-12-25 | 2020-05-08 | 中国建设银行股份有限公司 | Database horizontal extension method and device, and data table access method and device |
| CN112699133A (en) * | 2021-03-23 | 2021-04-23 | 浙江太美医疗科技股份有限公司 | Database operation method and device and computer readable medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120054467A1 (en) * | 2010-09-01 | 2012-03-01 | International Business Machines Corporation | Real-time hash map |
| US20120179723A1 (en) * | 2011-01-11 | 2012-07-12 | Hitachi, Ltd. | Data replication and failure recovery method for distributed key-value store |
| JP2013025497A (en) * | 2011-07-19 | 2013-02-04 | Nippon Telegr & Teleph Corp <Ntt> | Distributed processing system, distributed processing method, load distribution device, load distribution method, and load distribution program |
| CN104077423A (en) * | 2014-07-23 | 2014-10-01 | 山东大学（威海） | Consistent hash based structural data storage, inquiry and migration method |
| CN104714972A (en) * | 2013-12-17 | 2015-06-17 | 中国银联股份有限公司 | Database sub-table establishing and searching method |
| CN105488174A (en) * | 2015-12-01 | 2016-04-13 | 北京浩瀚深度信息技术股份有限公司 | Multi-distribution-type data source management method and system based on BS (Browser/Server) structure |
| CN106339475A (en) * | 2016-08-29 | 2017-01-18 | 联众智慧科技股份有限公司 | Distributed storage system for mass data |
| CN106503139A (en) * | 2016-10-20 | 2017-03-15 | 上海携程商务有限公司 | Dynamic data access method and system |
-
2018
- 2018-05-18 CN CN201810481880.7A patent/CN108763381B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120054467A1 (en) * | 2010-09-01 | 2012-03-01 | International Business Machines Corporation | Real-time hash map |
| US20120179723A1 (en) * | 2011-01-11 | 2012-07-12 | Hitachi, Ltd. | Data replication and failure recovery method for distributed key-value store |
| JP2013025497A (en) * | 2011-07-19 | 2013-02-04 | Nippon Telegr & Teleph Corp <Ntt> | Distributed processing system, distributed processing method, load distribution device, load distribution method, and load distribution program |
| CN104714972A (en) * | 2013-12-17 | 2015-06-17 | 中国银联股份有限公司 | Database sub-table establishing and searching method |
| CN104077423A (en) * | 2014-07-23 | 2014-10-01 | 山东大学（威海） | Consistent hash based structural data storage, inquiry and migration method |
| CN105488174A (en) * | 2015-12-01 | 2016-04-13 | 北京浩瀚深度信息技术股份有限公司 | Multi-distribution-type data source management method and system based on BS (Browser/Server) structure |
| CN106339475A (en) * | 2016-08-29 | 2017-01-18 | 联众智慧科技股份有限公司 | Distributed storage system for mass data |
| CN106503139A (en) * | 2016-10-20 | 2017-03-15 | 上海携程商务有限公司 | Dynamic data access method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111125114A (en) * | 2019-12-25 | 2020-05-08 | 中国建设银行股份有限公司 | Database horizontal extension method and device, and data table access method and device |
| CN111125114B (en) * | 2019-12-25 | 2023-06-16 | 中国建设银行股份有限公司 | Method and device for laterally expanding database and accessing data table |
| CN112699133A (en) * | 2021-03-23 | 2021-04-23 | 浙江太美医疗科技股份有限公司 | Database operation method and device and computer readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108763381B (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102134952B1 (en) | Data processing method and system | |
| JP6940239B2 (en) | Methods and systems for anonymizing data stock | |
| US10552397B2 (en) | Merging of sorted lists using array pair | |
| US11074246B2 (en) | Cluster-based random walk processing | |
| CN105989015B (en) | Database capacity expansion method and device and method and device for accessing database | |
| CN111061758B (en) | Data storage method, device and storage medium | |
| CN104915717A (en) | Data processing method, knowledge base reasoning method and related device | |
| US11899810B2 (en) | Systems and methods for determining database permissions | |
| CN107229674A (en) | A kind of data migration device, server and method | |
| CN114429222A (en) | Model training method, device and equipment | |
| CN114090695A (en) | Query optimization method and device for distributed database | |
| CN108763381A (en) | Divide table method and apparatus based on consistency hash algorithm | |
| CN112445776B (en) | Presto-based dynamic barrel dividing method, system, equipment and readable storage medium | |
| US20170091244A1 (en) | Searching a Data Structure | |
| CN106980673A (en) | Main memory database table index updating method and system | |
| US9298505B2 (en) | Time and space-deterministic task scheduling apparatus and method using multi-dimensional scheme | |
| CN104598567B (en) | A kind of method of the data statistics re-scheduling based on Hadoop MapReduce programming frameworks | |
| CN111475511A (en) | Data storage method, data access method, data storage device, data access device and data access equipment based on tree structure | |
| Mishra et al. | RUSLI: real-time updatable spline learned index | |
| US11435926B2 (en) | Method, device, and computer program product for managing storage system | |
| CN112639761B (en) | Method and device for establishing index for data | |
| CN110659286B (en) | Dynamic space index method based on weakly balanced space tree and storage medium and device thereof | |
| CN105468603A (en) | Data selection method and apparatus | |
| CN114780541B (en) | Data partitioning method, device, equipment and medium in micro batch flow processing system | |
| KR102247032B1 (en) | Apparatus and method for sorting item using longest increasing subsequence of item group |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 R & D room 3501, block a, building 7, Vanke Cloud City Phase I, Xingke 1st Street, Xili community, Xili street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Tubatu Group Co.,Ltd. Address before: 1001-a, 10th floor, bike technology building, No.9, Keke Road, high tech Zone, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN BINCENT TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |