Summary of the invention
The object of the embodiment of the present invention is to provide a kind of data processing method based on key-value pair and system, the data processing performance being intended to solve above-mentioned PostgreSQL database Redis is lower, and need input a large amount of time and manpower to monitor and O&M database, add the problem of the burden of data base administration.
The embodiment of the present invention is achieved in that a kind of data processing method based on key-value pair, comprising:
Any node in server cluster receives the database key order that client sends;
The Hash groove corresponding to key value carried in database key order described in described node calculate, and judge whether described Hash groove is responsible for by oneself;
If so, then described node performs described database key order;
If not, then described node returns the information of moved mistake to described client, makes described client turn to node correct in described server cluster according to described information.
Described in the embodiment of the present invention based in the data processing method of key-value pair, any node in server cluster also comprises before receiving the database key order that client sends:
Multiple Hash grooves in described server cluster are assigned to respectively each node in described server cluster, and the assignment information of described multiple Hash groove is stored to each node in described server cluster.
Described in the embodiment of the present invention based in the data processing method of key-value pair, described information comprises the nodename that the Hash groove corresponding to described key value is assigned.
Described in the embodiment of the present invention based in the data processing method of key-value pair, the Hash groove corresponding to key value carried in database key order described in described node calculate specifically comprises:
Described node employing cluster use formula CRC16 (key) %16384 calculates the Hash groove corresponding to key value, and wherein, CRC16 (key) statement is for calculating the CRC16 School Affairs of key value.
Described in the embodiment of the present invention based in the data processing method of key-value pair, described database key order comprises get order, set order, del order and hget order.
Another object of the embodiment of the present invention is to provide a kind of data handling system based on key-value pair, comprise the receiver module, computing module, judge module, execution module and the reminding module that are positioned at server cluster interior joint inside, described receiver module, described computing module, described judge module and described execution module connect successively, described judge module is also connected with described reminding module, wherein:
Described receiver module, for receiving the database key order that client sends;
Described computing module, for calculating the Hash groove corresponding to the key value of carrying in described database key order;
Described judge module, for judging whether described Hash groove is responsible for by described node;
Described execution module, if for being responsible for by described node, then performs described database key order;
Described reminding module, for if not be responsible for by described node, then returns the information of moved mistake to described client, make described client turn to node correct in described server cluster according to described information.
Described in the embodiment of the present invention based in the data handling system of key-value pair, also comprise the distribution module that is connected with the node in described server cluster and be positioned at the memory module that described intra-node is connected with described distribution module and described judge module, wherein:
Described distribution module, for being assigned to each node in described server cluster respectively by the multiple Hash grooves in described server cluster;
Described memory module, for storing the assignment information of described multiple Hash groove.
Described in the embodiment of the present invention based in the data handling system of key-value pair, described information comprises the nodename that the Hash groove corresponding to described key value is assigned.
Described in the embodiment of the present invention based in the data handling system of key-value pair, described computing module specifically for:
Employing cluster use formula CRC16 (key) %16384 calculates the Hash groove corresponding to key value, and wherein, CRC16 (key) statement is for calculating the CRC16 School Affairs of key value.
Described in the embodiment of the present invention based in the data handling system of key-value pair, described database key order comprises get order, set order, del order and hget order.
The data processing method based on key-value pair that the enforcement embodiment of the present invention provides and system have following beneficial effect:
The database key order that by any node in server cluster received client transmission of the embodiment of the present invention owing to first adopting; Then the Hash groove corresponding to the key value of carrying in database key order described in described node calculate, and judge whether described Hash groove is responsible for by oneself; If so, then described node performs described database key order; If not, then described node returns the information of moved mistake to described client, described client is made to turn to node correct in described server cluster according to described information, thus make PostgreSQL database Redis that the multinuclear advantage of processor can be utilized to carry out concurrent processing, improve the handling property of database, make only to dispose a database instance and can reach service feature index, decrease the deployment of database instance, alleviate the burden of data base administration.
Embodiment
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
Fig. 1 shows the specific implementation flow process of the data processing method based on key-value pair that the embodiment of the present invention provides.Shown in Figure 1, a kind of data processing method based on key-value pair that the embodiment of the present invention provides, comprising:
In S101, any node in server cluster receives the database key order that client sends.
In embodiments of the present invention, any node in server cluster also comprises before receiving the database key order of client transmission: the multiple Hash grooves in described server cluster are assigned to respectively each node in described server cluster, and the assignment information of described multiple Hash groove is stored to each node in described server cluster.Further, each Hash groove in server cluster correspondence can only be assigned to a node, but each node all may correspond to is responsible for the multiple Hash groove of process.
In S102, the Hash groove corresponding to key value carried in database key order described in described node calculate, and judge whether described Hash groove is responsible for by oneself.
In embodiments of the present invention, include 16384 Hash grooves (hashslot) in the server cluster of database, each key value in database belongs to one of them of this 16384 Hash grooves.Described node is after the data key order receiving client transmission, can resolve described data key order, to obtain the key value of carrying in described data key order, then calculate the Hash groove corresponding to described key value, and judge whether described Hash groove is responsible for by oneself.
Wherein, the Hash groove corresponding to key value carried in database key order described in described node calculate specifically comprises: described node adopts cluster to use formula CRC16 (key) %16384 to calculate the Hash groove corresponding to key value, wherein, CRC16 (key) statement is for calculating the CRC16 School Affairs of key value.
Wherein, judge that whether described Hash groove is responsible for specifically comprising by oneself: search according to the assignment information of the multiple Hash grooves prestored in the Hash groove calculated and described node the nodename that the Hash groove corresponding to described key value is assigned, judge that whether described nodename is identical with the nodename that self stores, if identical, then illustrate that the Hash groove corresponding to described key value is responsible for by oneself; Otherwise the Hash groove corresponding to described key value is not responsible for by oneself.
In S103-1, if so, then described node performs described database key order.
In embodiments of the present invention, if described node judges that the Hash groove corresponding to described key value is responsible for by oneself, then described database key order is performed.Wherein, described database key order includes but not limited to get order, set order, del order and hget order.
In S103-2, if not, then described node returns the information of moved mistake to described client, makes described client turn to node correct in described server cluster according to described information.
In the present embodiment, described information comprises the nodename that the Hash groove corresponding to described key value is assigned.Described node returns the information of moved mistake to described client, described client is made to turn to node correct in described server cluster specifically to comprise according to described information: if the Hash groove corresponding to described key value is not be responsible for by described node, the nodename that then the Hash groove of described node automatically corresponding to described key value is assigned generates the information of moved mistake, and described information is back to described client, the nodename that the Hash groove of described client corresponding to the described key value in described information is assigned turns to node correct in described server cluster.
More than can find out, the database key order that by any node in server cluster received client transmission of data processing method owing to first adopting based on key-value pair that the embodiment of the present invention provides; Then the Hash groove corresponding to the key value of carrying in database key order described in described node calculate, and judge whether described Hash groove is responsible for by oneself; If so, then described node performs described database key order; If not, then described node returns the information of moved mistake to described client, described client is made to turn to node correct in described server cluster according to described information, thus make PostgreSQL database Redis that the multinuclear advantage of processor can be utilized to carry out concurrent processing, improve the handling property of database, make only to dispose a database instance and can reach service feature index, decrease the deployment of database instance, alleviate the burden of data base administration.
Fig. 2 shows the structured flowchart of the data handling system based on key-value pair that the embodiment of the present invention provides, and this system is for running the method provided embodiment illustrated in fig. 1.Illustrate only the part relevant to the embodiment of the present invention for convenience of explanation.
Shown in Figure 2, a kind of data handling system based on key-value pair that the embodiment of the present invention provides, comprise the receiver module 1, computing module 2, judge module 3, execution module 4 and the reminding module 5 that are positioned at server cluster interior joint inside, described receiver module 1, described computing module 2, described judge module 3 and described execution module 4 connect successively, described judge module 3 is also connected with described reminding module 5, wherein:
Described receiver module 1, for receiving the database key order that client sends;
Described computing module 2, for calculating the Hash groove corresponding to the key value of carrying in described database key order;
Described judge module 3, for judging whether described Hash groove is responsible for by described node;
Described execution module 4, if for being responsible for by described node, then performs described database key order;
Described reminding module 5, for if not be responsible for by described node, then returns the information of moved mistake to described client, make described client turn to node correct in described server cluster according to described information.
Optionally, the described data handling system based on key-value pair also comprises the distribution module 6 that is connected with the node in described server cluster and is positioned at the memory module 7 that described intra-node is connected with described distribution module 6 and described judge module 3, wherein:
Described distribution module 6, for being assigned to each node in described server cluster respectively by the multiple Hash grooves in described server cluster;
Described memory module 7, for storing the assignment information of described multiple Hash groove.
Optionally, described information comprises the nodename that the Hash groove corresponding to described key value is assigned.
Optionally, described computing module 2 specifically for:
Employing cluster use formula CRC16 (key) %16384 calculates the Hash groove corresponding to key value, and wherein, CRC16 (key) statement is for calculating the CRC16 School Affairs of key value.
Optionally, described database key order comprises get order, set order, del order and hget order.
It should be noted that, modules in the said system that the embodiment of the present invention provides, due to the inventive method embodiment based on same design, its technique effect brought is identical with the inventive method embodiment, particular content see describing in the inventive method embodiment, can repeat no more herein.
Therefore, can find out that the data handling system based on key-value pair that the embodiment of the present invention provides can make PostgreSQL database Redis that the multinuclear advantage of processor can be utilized to carry out concurrent processing equally, improve the handling property of database, and the deployment of database instance can be reduced, alleviate the burden of data base administration.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.