CN107346258A - A kind of reading and writing data separation method and device - Google Patents
A kind of reading and writing data separation method and device Download PDFInfo
- Publication number
- CN107346258A CN107346258A CN201710549825.2A CN201710549825A CN107346258A CN 107346258 A CN107346258 A CN 107346258A CN 201710549825 A CN201710549825 A CN 201710549825A CN 107346258 A CN107346258 A CN 107346258A
- Authority
- CN
- China
- Prior art keywords
- redis
- read
- sliced fashion
- database
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Embodiments of the invention disclose a kind of reading and writing data separation method and device, are related to reading and writing data technology, can lift data read-write efficiency.The reading and writing data separation method includes：Redis requests are received, the sliced fashion configuration information in Redis data base configuration files is read, obtains sliced fashion corresponding to the sliced fashion configuration information；The class file of the sliced fashion mapping obtained is called, is calculated based on the class file and Redis requests, obtains Redis databases to be visited；The method name included in extraction Redis requests, it is to read or write to load the read and write abruption method configuration file that method name maps described in Redis data base configuration files and obtain this Redis requests；Position the read from database in the Redis databases to be visited or write database, the Redis read from database writes database from Redis can be located at different Redis servers.The present invention is adapted to carry out the read and write abruption of data manipulation.
Description
Technical field
The present invention relates to reading and writing data technology, more particularly to a kind of reading and writing data separation method and device.
Background technology
Long-range dictionary services (Redis, REmote DIctionary Server) are the use an increased income American Nationals
ANSI (ANSIC, American National Standards Institute) database that language is write, can be used as
Cache and message queue agency, support character string, Hash table, list, set, ordered set, bitmap, can be held based on internal memory
The data types such as the daily record changed long, data are stored and read and write using Key-Value modes, and the application of multilingual can be provided
Program Interfaces (API, Application Programming Interface), so that developer can be based on
Application program access routine, without accessing source code or understanding the details of internal work mechanism.
At present, when carrying out Redis configurations, it is necessary to pre-set primary database (Redis host) and one or more
From database, wherein, for primary database to provide a user read-write capability, user can carry out the reading of data in primary database
Write operation, data syn-chronization is kept from the running status of database monitoring primary database and with primary database, when monitoring master data
When storehouse is not normally functioning extremely, according to the takeover strategy pre-set, determine that one takes over primary database from database, so as to
To continue to provide a user normal service, to avoid the interruption of business.
But the existing method that reading and writing data is carried out by primary database, in primary database normal operation, due to data
Read operation and write operation are all directed to primary database, and are only used for backup from database and use, thus, concurrent user's visit capacity compared with
In the case of the operation of more or concurrent reading and writing is more frequent, cause the operating pressure of primary database larger, blocked so as to cause.
For example, when concurrently relatively up to the performance bottleneck of primary database, processing request number of times (QPS, the Query per second of primary database
Per Second) it can decline therewith so that user's request extends from the response time of primary database read-write data, reading and writing data effect
Rate is relatively low.
The content of the invention
In view of this, the embodiment of the present invention provides a kind of reading and writing data separation method and device, can lift reading and writing data
Efficiency, to solve in existing data read-write method, only as caused by primary database provides read-write data manipulation during reading and writing data
Between it is longer the problem of.
In a first aspect, the embodiment of the present invention provides a kind of reading and writing data separation method, including：
Redis requests are received, read the sliced fashion configuration information in Redis data base configuration files, obtain described point
Sliced fashion corresponding to sheet mode configuration information；
The class file of the sliced fashion mapping obtained is called, is counted based on the class file and Redis requests
Calculate, obtain Redis databases to be visited；
The method name included in extraction Redis requests, loads method name mapping described in Redis data base configuration files
It is to read or write that read and write abruption method configuration file, which obtains this request,；
Position the Redis read from database in the Redis databases to be visited or Redis writes database, the Redis
Read from database writes database from Redis can be located at different Redis servers.
With reference in a first aspect, in the first embodiment of first aspect, the sliced fashion obtained is called to reflect described
The class file penetrated, based on the class file and the Redis request calculated, obtain Redis databases to be visited it
Before, methods described also includes：
Whether the Redis databases number under the sliced fashion for judging to obtain is more than one group, is obtained if it is, performing the calling
The class file of the sliced fashion mapping taken, is calculated based on the class file and Redis requests, is obtained to be visited
The step of Redis databases.
With reference to the first of first aspect or first aspect embodiment, in second of embodiment of first aspect,
The sliced fashion is uniformity Hash sliced fashion, the class file for calling the sliced fashion mapping obtained, based on described
Class file and Redis requests are calculated, and obtaining Redis databases to be visited includes：
The threshold value cut-off pre-set is distributed on the round model by the circle of structure one as circle model；
Hash operation is carried out to every group of Redis database under uniformity Hash sliced fashion successively, obtains each group Hash
Computing point, Hash number virtualization is carried out respectively to every group of Hash operation point, each group Hash operation value that Hash number virtualizes is asked
It is total with virtualization is obtained, virtualization sum is evenly distributed to the model and justified；
The key assignments (Key) included in the Redis requests is obtained, Hash operation is carried out to the key assignments of acquisition, obtains a value
For the inquiry request point of positive integer, the model circle is corresponded to；
Using drop point site of the inquiry request point on model circle as starting point, obtained according to the lookup algorithm pre-set
Take the reference address of the Redis databases to be visited.
With reference to second of embodiment of first aspect, in the third embodiment of first aspect, the threshold value is 2
32 powers, the Hash number be 128.
It is described according to pre- in the 4th kind of embodiment of first aspect with reference to the third embodiment of first aspect
The reference address that the lookup algorithm first set obtains the Redis databases to be visited includes：
The sub- point of Hash operation being distributed on model circle is searched clockwise, and first Hash operation found is put to wait to visit
The reference address for the Redis databases asked, if it exceeds 2^32 does not find the sub- point of Hash operation, give tacit consent on model circle
First Hash operation son point is the reference address of Redis databases to be visited.
With reference to the first of first aspect or first aspect embodiment, in the 5th kind of embodiment of first aspect,
The sliced fashion is modulus sliced fashion, the class file for calling the sliced fashion mapping obtained, based on the class file
And the Redis requests are calculated, obtaining Redis databases to be visited includes：
The key assignments (Key) included in the Redis requests is obtained, Hash operation is carried out to the key assignments of acquisition, obtains a value
For the inquiry request point of positive integer；
The Redis database group numbers configured under inquiry modulus sliced fashion；
The Redis databases group number complementation is carried out to the inquiry request point, obtains the Redis to be visited
The reference address of database.
With reference in a first aspect, in the 6th kind of embodiment of first aspect, methods described name includes：Write method name and
Method name is read, the read and write abruption method configuration file includes reading separation method configuration file and writes separation method configuration text
Part.
Second aspect, the embodiment of the present invention provide a kind of reading and writing data separator, including：Sliced fashion acquisition module,
Reference address computing module, separation method configuration file load-on module and read-write locating module, wherein,
Sliced fashion acquisition module, for receiving Redis requests, read the burst side in Redis data base configuration files
Formula configuration information, obtain sliced fashion corresponding to the sliced fashion configuration information；
Reference address computing module, for call obtain sliced fashion mapping class file, based on the class file with
And the Redis requests are calculated, and obtain Redis databases to be visited；
Separation method configuration file load-on module, for extracting the method name included in Redis requests, load Redis numbers
The read and write abruption method configuration file mapped according to method name described in base configuration file；
Locating module is read and write, data are write for positioning read from database or Redis in the Redis databases to be visited
Storehouse, the Redis read from database write database from Redis and are located at different Redis servers.
With reference to second aspect, in the first embodiment of second aspect, the sliced fashion is uniformity Hash point
Sheet mode, the reference address computing module include：Cut-off map unit, Hash operation point map unit, inquiry request point
Map unit and reference address computing unit, wherein,
Cut-off map unit, for building a circle as circle model, the threshold value cut-off pre-set is distributed on
On the round model；
Hash operation point map unit, for entering successively to every group of Redis database under uniformity Hash sliced fashion
Row Hash operation, each group Hash operation point is obtained, Hash number virtualization is carried out respectively to every group of Hash operation point, Hash number is empty
The each group Hash operation value of planization sums to obtain virtualization sum, and virtualization sum is evenly distributed into the model justifies；
Inquiry request point map unit, for described in inquiry request point map unit and the acquisition of reference address computing unit
The key assignments (Key) included in Redis requests, Hash operation is carried out to the key assignments of acquisition, obtain the inquiry that a value is positive integer and ask
Ask a little, correspond to the model circle；
Reference address computing unit, for using the inquiry request point model circle on drop point site as starting point, press
The reference address of the Redis databases to be visited is obtained according to the lookup algorithm pre-set.
With reference to second aspect, in second of embodiment of second aspect, the sliced fashion is modulus sliced fashion,
The reference address computing module includes：Inquiry request point computing unit, query unit and reference address acquiring unit, its
In,
Inquiry request point computing unit, for obtaining the key assignments (Key) included in the Redis requests, to the key of acquisition
Value carries out Hash operation, obtains the inquiry request point that a value is positive integer；
Query unit, for inquiring about under modulus sliced fashion the Redis database group numbers configured；
Reference address acquiring unit, for carrying out the Redis databases group number complementation to the inquiry request point,
Obtain the reference address of the Redis databases to be visited.
A kind of reading and writing data separation method and device provided in an embodiment of the present invention, by receiving Redis requests, read
Sliced fashion configuration information in Redis data base configuration files, obtain burst side corresponding to the sliced fashion configuration information
Formula；The class file of the sliced fashion mapping obtained is called, is calculated, obtained based on the class file and Redis requests
To Redis databases to be visited；The method name included in extraction Redis requests, loads institute in Redis data base configuration files
State the read and write abruption method configuration file of method name mapping；Position in the Redis databases to be visited read from database or
Redis writes database, and the Redis read from database writes database from Redis and is located at different Redis servers, can be lifted
Data read-write efficiency, to solve in existing data read-write method, the number only as caused by primary database provides read-write data manipulation
According to access time it is longer the problem of.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the reading and writing data separation method schematic flow sheet of embodiments of the invention one；
Fig. 2 is the reading and writing data separator structural representation of embodiments of the invention two.
Embodiment
The embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.
It will be appreciated that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.Base
Embodiment in the present invention, those of ordinary skill in the art obtained under the premise of creative work is not made it is all its
Its embodiment, belongs to the scope of protection of the invention.
Fig. 1 is the reading and writing data separation method schematic flow sheet of embodiments of the invention one, as shown in figure 1, the present embodiment
Method can include：
Step 101, Redis requests are received, the sliced fashion configuration information in Redis data base configuration files is read, obtains
Take sliced fashion corresponding to the sliced fashion configuration information；
In the present embodiment, as an alternative embodiment, Redis requests include but is not limited to：Redis read data requests and
Redis write data requests.
In the present embodiment, Redis data base configuration files are the files pre-defined, in the Redis databases
In configuration file, sliced fashion configuration information is provided with, sliced fashion can be obtained by being loaded directly into this document.
In the present embodiment, as an alternative embodiment, sliced fashion (type) configuration information is used to show Redis databases
It is to be conducted interviews using uniformity Hash (hash) sliced fashion or conducted interviews using modulus (modulo) sliced fashion.
In the present embodiment, sliced fashion includes but is not limited to：Uniformity Hash sliced fashion and modulus sliced fashion, often
One sliced fashion maps a class file to handle the related burst logic of the sliced fashion.Certainly, can also spirit in practical application
Other sliced fashions are easily set up in work, it is necessary to which corresponding increase what the sliced fashion mapped after a kind of every sliced fashion of increase
Class file handles related burst logic, and the present embodiment is not construed as limiting to this.
In the present embodiment, as an alternative embodiment, the class file of uniformity Hash sliced fashion mapping route for Hash
(hashRouter) class file, the class file of modulus sliced fashion mapping route (moduloRouter) class file for modulus.
Step 102, the class file for calling the sliced fashion of acquisition to map, please based on the class file and the Redis
Ask and calculated, obtain Redis databases to be visited；
In the present embodiment, to every kind of sliced fashion, one group or multigroup Redis databases, every group of Redis database are set
Comprising a read-write storehouse pair, to writing database comprising a Redis read from database and a Redis, every Redis is read in each read-write storehouse
Database or Redis write database has corresponding reference address (host addresses) respectively, wherein, the category of Redis read from database
Property to read, Redis writes the attribute of database to write.
In the present embodiment, as another alternative embodiment, every Redis read from database or Redis are write database and also distinguished
With information such as corresponding port ports, time-out time (timeout), prefixes (prefix).Host addresses, port ports, surpass
When the time (timeout), prefix (prefix) composition read-write subpool (RW subpool) pair configuration information.
In the present embodiment, Redis data base configuration files are set and define the specific procedure code segment of Redis databases
It is as follows：
In the present embodiment, as an alternative embodiment, in the class file of the sliced fashion mapping called and obtained, it is based on
The class file and Redis requests are calculated, and before obtaining Redis databases to be visited, this method also includes：
Whether the undefined Redis databases number of sliced fashion for judging to obtain is more than one group, if it is, performing the tune
The class file mapped with the sliced fashion of acquisition, calculated based on the class file and Redis requests, obtain waiting to visit
The Redis databases asked.
In the present embodiment, if the undefined Redis databases number of sliced fashion obtained is equal to one group, the group
Redis is directly as Redis databases to be visited.
In the present embodiment, if the undefined Redis databases number of sliced fashion obtained is more than one group, according to this point
Class file corresponding to sheet mode, and according to the fixed-point computation method pre-set in the class file, calculating use which
Group Redis databases, so as to obtain Redis databases to be visited.
In the present embodiment, as an alternative embodiment, sliced fashion is uniformity Hash sliced fashion, and described call obtains
Sliced fashion mapping class file, based on the class file and the Redis request calculated, obtain to be visited
Redis databases include：
The threshold value cut-off pre-set is distributed on the round model by A11, the circle of structure one as circle model；
In the present embodiment, as an alternative embodiment, threshold value 2^32, i.e., 2 32 powers.A circle is chosen as mould
Type, with 2^32 cut-off uniformly cut the model justify, will the model even circumferential be cut into 232Part.
A12, Hash operation is carried out to every group of Redis database under uniformity Hash sliced fashion successively, obtains each group
Hash operation point, Hash number virtualization is carried out respectively to every group of Hash operation point, each group Hash operation that Hash number is virtualized
Value summation obtains virtualization sum, and virtualization sum is evenly distributed into the model justifies；
In the present embodiment, as an alternative embodiment, Hash number is 128.For example, it is assumed that under uniformity Hash sliced fashion
Comprising 3 groups of Redis database, every group of Redis database is subjected to Hash (hash) respectively and set, obtains a Hash operation point,
3 Hash operation points are always obtained, each Hash operation point is carried out to 128 virtualization, i.e., each Hash operation point includes 128
The individual sub- point of Hash operation, obtains the sub- point of 3*128 Hash operation after summation, then, 3*128 Hash operation is put uniform minute
Cloth to model justify on.
In the present embodiment, as an alternative embodiment, first cut-off and first Hash operation point are in the circle
Overlapped on model.
A13, obtains the key assignments (Key) included in the Redis requests, carries out Hash operation to the key assignments of acquisition, obtains
One value is the inquiry request point of positive integer, corresponds to the model circle；
In the present embodiment, as an alternative embodiment, according to positive integer corresponding to inquiry request point, found on model circle
Cut-off corresponding to the positive integer.
A14, using drop point site of the inquiry request point on model circle as starting point, calculated according to the lookup pre-set
Method obtains the reference address of the Redis databases to be visited.
In the present embodiment, using drop point site of the inquiry request point on model circle as starting point, search be distributed in clockwise
The sub- point of Hash operation on model circle, the first Hash operation son point found are the access of Redis databases to be visited
Location, if it exceeds 2^32 does not find the sub- point of Hash operation, first Hash operation point on model circle is given tacit consent to treat
The reference address of the Redis databases of access.
In the present embodiment, as an alternative embodiment, the reference address of Redis databases to be visited includes：Redis is read
The reference address and Redis of database write the reference address of database.
In the present embodiment, as another alternative embodiment, sliced fashion is modulus sliced fashion, described to call point obtained
The class file of sheet mode mapping, calculated based on the class file and Redis requests, obtain Redis to be visited
Database includes：
B11, obtains the key assignments (Key) included in the Redis requests, carries out Hash operation to the key assignments of acquisition, obtains
One value is the inquiry request point of positive integer；
In the present embodiment, client initiates a Redis request, and Redis databases, will after Redis requests are received
Key corresponding to Redis requests is hashed into a point, for example, being designated as numA (positive integer).
B12, the Redis database group numbers configured are inquired about under modulus sliced fashion；
In the present embodiment, the Redis database groups number scale configured under modulus sliced fashion is (numB).
B13, the Redis databases group number complementation is carried out to the inquiry request point, obtained described to be visited
The reference address of Redis databases.
In the present embodiment, the host addresses that the value that numA%numB is obtained is Redis databases to be visited are calculated.
Step 103, the method name included in Redis requests is extracted, loads method described in Redis data base configuration files
It is to read or write that the read and write abruption method configuration file of name mapping, which obtains this request,；
In the present embodiment, as an alternative embodiment, method name includes：Write method name and reading method name.Please in Redis
In asking, the method name for being shown to be and reading still to write is carried.
In the present embodiment, according to the method name that includes in Redis requests, loading party religious name mapping edits in advance
Read and write abruption method configuration file, wherein, read and write abruption method configuration file is the file in Redis data base configuration files,
Including reading separation method configuration file and writing separation method configuration file, for will access the read operations of Redis databases with
Write operation is separated so that and read operation can perform in different primary databases and from database respectively from write operation,
So as to avoid read-write efficiency caused by read-write operation performs in primary database relatively low.If for example, entitled write method of method
Separation method configuration file is write in name, then loading.In the present embodiment, application program can be directly by read and write abruption method configuration file
It is loaded into session.
In the present embodiment, the program code segments for loading read and write abruption method configuration file are as follows：
' writeFun '=＞ [' WYset ', ' WYrPush ' ...]
' readFun '=＞ [' WYget ', ' WYlRange ' ...]
Step 104, position the read from database in the Redis databases to be visited or Redis writes database, it is described
Redis read from database writes database from Redis and is located at different Redis servers.
In the present embodiment, when client initiates a Redis request, the method name in being asked according to Redis, go to compare
Search read and write abruption method configuration file and load, meanwhile, according to method name, determine that the Redis of party's religious name mapping reads data
Storehouse or Redis write database, if for example, the entitled reading method name of method, accesses Redis read from database.
In the present embodiment, after Redis requests are received, with the sliced fashion defined in Redis data base configuration files
Exemplified by hash, then call hashRouter classes corresponding to uniformity Hash, algorithm in hashRouter classes and
The key assignments included in Redis requests, find out the reference address for the Redis databases for needing to access, for example, assuming that being consistent
Property Hash sliced fashion under first group of Redis database, then, extraction Redis request in include method name (read method
Name), the read and write abruption method configuration file that party's religious name defines in Redis data base configuration files is loaded, selectes first group
Attribute is then attached use, due to Redis read from database to read the Redis read from database of (read) in Redis databases
Database is write positioned at different Redis servers (primary database and from database) from Redis, support read-write so as to pass through
The Redis caching methods of separation, Redis read and write abruptions are supported, the pressure of Redis (primary database) is effectively reduced, lifts it
Runnability, data read-write efficiency can be lifted, to solve in existing data read-write method, primary database provides read-write data
The problem of reading and writing data time is longer caused by operation；Further, the Redis data base configuration files of setting support a variety of points
Sheet mode, being capable of effective decentralized data storage so that each database loads are more balanced；Moreover, instantiated by singleton pattern
Class file, read and write access speed can be improved with fast positioning Redis databases to be visited.
Fig. 2 is the reading and writing data separator structural representation of embodiments of the invention two, as shown in Fig. 2 the present embodiment
Device can include：Sliced fashion acquisition module 21, reference address computing module 22, separation method configuration file load-on module 23
And read-write locating module 24, wherein,
Sliced fashion acquisition module 21, for receiving Redis requests, read the burst in Redis data base configuration files
Mode configuration information, obtain sliced fashion corresponding to the sliced fashion configuration information；
In the present embodiment, as an alternative embodiment, Redis requests include but is not limited to：Redis read data requests and
Redis write data requests.
In the present embodiment, sliced fashion includes but is not limited to：Uniformity Hash sliced fashion and modulus sliced fashion, often
One sliced fashion maps a class file to handle the related burst logic of the sliced fashion.
In the present embodiment, as an alternative embodiment, the class file of uniformity Hash sliced fashion mapping route for Hash
Class file, the class file of modulus sliced fashion mapping route class file for modulus.
Reference address computing module 22, for calling the class file of the sliced fashion obtained mapping, based on the class file
And the Redis requests are calculated, and obtain Redis databases to be visited；
In the present embodiment, in Redis databases, to every kind of sliced fashion, one group or multigroup Redis data are set
Storehouse, every group of Redis database include one or more read-write subpool (RW subpool)s pair, and each read-write subpool (RW subpool) is to including a Redis read from database
Database is write with a Redis, every Redis read from database or Redis write database has corresponding reference address (host respectively
Address), wherein, for the attribute of Redis read from database to read, the attribute that Redis writes database is to write.
In the present embodiment, as an alternative embodiment, the sliced fashion is uniformity Hash sliced fashion, reference address
Computing module 22 includes：Cut-off map unit, Hash operation point map unit, request point map unit and reference address meter
Unit (not shown) is calculated, wherein,
Cut-off map unit, for building a circle as circle model, the threshold value cut-off pre-set is distributed on
On the round model；
Hash operation point map unit, for entering successively to every group of Redis database under uniformity Hash sliced fashion
Row Hash operation, each group Hash operation point is obtained, Hash number virtualization is carried out respectively to every group of Hash operation point, Hash number is empty
The each group Hash operation value of planization sums to obtain virtualization sum, and virtualization sum is evenly distributed into the model justifies；
Inquiry request point map unit, for described in inquiry request point map unit and the acquisition of reference address computing unit
The key assignments (Key) included in Redis requests, Hash operation is carried out to the key assignments of acquisition, obtain the inquiry that a value is positive integer and ask
Ask a little, correspond to the model circle；
Reference address computing unit, for using the inquiry request point model circle on drop point site as starting point, press
The reference address of the Redis databases to be visited is obtained according to the lookup algorithm pre-set.
In the present embodiment, as an alternative embodiment, the reference address of Redis databases to be visited includes：Redis is read
The reference address and Redis of database write the reference address of database.
It is described described to be visited according to the lookup algorithm pre-set acquisition as an alternative embodiment in the present embodiment
The reference address of Redis databases include：
The sub- point of Hash operation being distributed on model circle is searched clockwise, and first Hash operation found is put to wait to visit
The reference address for the Redis databases asked, if it exceeds 2^32 does not find the sub- point of Hash operation, give tacit consent on model circle
First Hash operation son point is the reference address of Redis databases to be visited.
In the present embodiment, as an alternative embodiment, the threshold value is 2 32 powers, and the Hash number is 128.
In the present embodiment, as another alternative embodiment, the sliced fashion is modulus sliced fashion, and reference address calculates
Module 22 includes：Inquiry request point computing unit, query unit and reference address acquiring unit, wherein,
Inquiry request point computing unit, for obtaining the key assignments (Key) included in the Redis requests, to the key of acquisition
Value carries out Hash operation, obtains the inquiry request point that a value is positive integer；
Query unit, for inquiring about under modulus sliced fashion the Redis database group numbers configured；
Reference address acquiring unit, for carrying out the Redis databases group number complementation to the inquiry request point,
Obtain the reference address of the Redis databases to be visited.
In the present embodiment, the host addresses that the value that numA%numB is obtained is Redis databases to be visited are calculated.Its
In, numA be by Redis ask corresponding to the inquiry request points that are hashed into of Key, numB is Redis database group numbers.
Separation method configuration file load-on module 23, for extracting the method name included in Redis requests, load Redis
It is to read or write that the read and write abruption method configuration file of the mapping of method name described in data base configuration file, which obtains this request,；
In the present embodiment, as an alternative embodiment, methods described name includes：Write method name and reading method name, it is described
Read and write abruption method configuration file includes reading separation method configuration file and writes separation method configuration file.
Locating module 24 is read and write, number is write for positioning read from database in the Redis databases to be visited or Redis
According to storehouse, the Redis read from database writes database from Redis and is located at different Redis servers.
In the present embodiment, when client initiates a Redis request, the method name in being asked according to Redis, go to compare
Search read and write abruption method configuration file and load, meanwhile, according to method name, determine that the Redis of party's religious name mapping reads data
Storehouse or Redis write database, if for example, the entitled reading method name of method, accesses Redis read from database.
In the present embodiment, as an alternative embodiment, the device also includes：
Whether judge module (not shown), the Redis databases number under sliced fashion for judging acquisition are more than
One group, if it is, notice reference address computing module.
In the present embodiment, if under the sliced fashion obtained, i.e., it define only in the db columns of Redis data base configuration files
One group of Redis database and its host addresses, then when receiving Redis request call Redis databases, can directly it utilize
Method name (reading or writing) in Redis requests determines to write using the Redis read from database or Redis of this group of Redis database
Database, and according to the corresponding Redis servers of host addresses connection (primary database or from database) configured in corresponding word bank
Data reading operation or write operation are carried out, without performing other operations.
The device of the present embodiment, it can be used for the technical scheme for performing embodiment of the method shown in Fig. 1, its realization principle and skill
Art effect is similar, and here is omitted.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply and deposited between these entities or operation
In any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Nonexcludability includes, so that process, method, article or equipment including a series of elements not only will including those
Element, but also the other element including being not expressly set out, or it is this process, method, article or equipment also to include
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence " including one ... ", it is not excluded that
Other identical element in the process including the key element, method, article or equipment also be present.
Each embodiment in this specification is described by the way of related, identical similar portion between each embodiment
Divide mutually referring to what each embodiment stressed is the difference with other embodiment.
For device embodiment, because it is substantially similar to embodiment of the method, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
Put.The more specifically example (non-exhaustive list) of computer-readable medium includes following：Electricity with one or more wiring
Connecting portion (electronic installation), portable computer diskette box (magnetic device), random access memory (RAM), read-only storage
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device, and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium, which can even is that, to print the paper of described program thereon or other are suitable
Medium, because can then enter edlin, interpretation or if necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.
In the above-described embodiment, multiple steps or method can use storage to be performed in memory and by suitable instruction
The software or firmware that system performs are realized.If for example, being realized with hardware, with another embodiment, can use
Any one of following technology well known in the art or their combination are realized：With for realizing logic work(to data-signal
The discrete logic of the logic gates of energy, there is the application specific integrated circuit of suitable combinational logic gate circuit, programmable gate
Array (PGA), field programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that to realize all or part of step that above-described embodiment method carries
Suddenly it is that by program the hardware of correlation can be instructed to complete, described program can be stored in a kind of computer-readable storage medium
In matter, the program upon execution, including one or a combination set of the step of embodiment of the method.
For convenience of description, it is to be divided into various units/modules with function to describe respectively to describe apparatus above.Certainly, exist
The function of each unit/module can be realized in same or multiple softwares and/or hardware when implementing of the invention.
As seen through the above description of the embodiments, those skilled in the art can be understood that the present invention can
Realized by the mode of software plus required general hardware platform.Based on such understanding, technical scheme essence
On the part that is contributed in other words to prior art can be embodied in the form of software product, the computer software product
It can be stored in storage medium, such as ROM/RAM, magnetic disc, CD, including some instructions are causing a computer equipment
(can be personal computer, server, either network equipment etc.) performs some of each embodiment of the present invention or embodiment
Method described in part.
The foregoing is only a specific embodiment of the invention, but protection scope of the present invention is not limited thereto, any
Those familiar with the art the invention discloses technical scope in, the change or replacement that can readily occur in, all should
It is included within the scope of the present invention.Therefore, protection scope of the present invention should be defined by scope of the claims.
Claims (10)
- A kind of 1. method of reading and writing data separation, it is characterised in that including：Redis requests are received, the sliced fashion configuration information in Redis data base configuration files is read, obtains the burst side Sliced fashion corresponding to formula configuration information；The class file of the sliced fashion mapping obtained is called, is calculated, obtained based on the class file and Redis requests To Redis databases to be visited；The method name included in extraction Redis requests, loads the read-write of method name mapping described in Redis data base configuration files It is to read or write that separation method configuration file, which obtains this request,；Position the Redis read from database in the Redis databases to be visited or Redis writes database, the Redis readings Different Redis servers can be located at by writing database from Redis according to storehouse.
- 2. reading and writing data separation method according to claim 1, it is characterised in that in the sliced fashion for calling and obtaining The class file of mapping, calculated based on the class file and Redis requests, obtain Redis databases to be visited Before, methods described also includes：Whether the Redis databases number under the sliced fashion for judging to obtain is more than one group, if it is, performing what the calling obtained The class file of sliced fashion mapping, calculated, obtained to be visited based on the class file and Redis requests The step of Redis databases.
- 3. reading and writing data separation method according to claim 1 or 2, it is characterised in that the sliced fashion is uniformity Hash sliced fashion, the class file for calling the sliced fashion mapping obtained, please based on the class file and the Redis Ask and calculated, obtaining Redis databases to be visited includes：The threshold value cut-off pre-set is distributed on the round model by the circle of structure one as circle model；Hash operation is carried out to every group of Redis database under uniformity Hash sliced fashion successively, obtains each group Hash operation Point, Hash number virtualization is carried out respectively to every group of Hash operation point, each group Hash operation value that Hash number virtualizes is summed To virtualization sum, virtualization sum is evenly distributed to the model and justified；The key assignments (Key) included in the Redis requests is obtained, Hash operation is carried out to the key assignments of acquisition, obtains a value as just The inquiry request point of integer, correspond to the model circle；Using drop point site of the inquiry request point on model circle as starting point, institute is obtained according to the lookup algorithm pre-set State the reference address of Redis databases to be visited.
- 4. reading and writing data separation method according to claim 3, it is characterised in that the threshold value is 2 32 powers, described Hash number is 128.
- 5. reading and writing data separation method according to claim 4, it is characterised in that described to be calculated according to the lookup pre-set The reference address that method obtains the Redis databases to be visited includes：The sub- point of Hash operation being distributed on model circle is searched clockwise, and the first Hash operation point found is to be visited The reference address of Redis databases, if it exceeds 2^32 does not find the sub- point of Hash operation, give tacit consent to first on model circle Individual Hash operation point is the reference address of Redis databases to be visited.
- 6. reading and writing data separation method according to claim 1 or 2, it is characterised in that the sliced fashion is modulus point Sheet mode, the class file for calling the sliced fashion mapping obtained, is carried out based on the class file and Redis requests Calculate, obtaining Redis databases to be visited includes：The key assignments (Key) included in the Redis requests is obtained, Hash operation is carried out to the key assignments of acquisition, obtains a value as just The inquiry request point of integer；The Redis database group numbers configured under inquiry modulus sliced fashion；The Redis databases group number complementation is carried out to the inquiry request point, obtains the Redis data to be visited The reference address in storehouse.
- 7. reading and writing data separation method according to claim 1, it is characterised in that methods described name includes：Write method name And method name is read, the read and write abruption method configuration file includes reading separation method configuration file and writes separation method configuration File.
- A kind of 8. reading and writing data separator, it is characterised in that including：Sliced fashion acquisition module, reference address computing module, Separation method configuration file load-on module and read-write locating module, wherein,Sliced fashion acquisition module, for receiving Redis requests, the sliced fashion read in Redis data base configuration files is matched somebody with somebody Confidence ceases, and obtains sliced fashion corresponding to the sliced fashion configuration information；Reference address computing module, for calling the class file of the sliced fashion obtained mapping, based on the class file and institute State Redis requests to be calculated, obtain Redis databases to be visited；Separation method configuration file load-on module, for extracting the method name included in Redis requests, load Redis databases It is to read or write that the read and write abruption method configuration file of the mapping of method name described in configuration file, which obtains this request,；Locating module is read and write, number is write for positioning Redis read from database in the Redis databases to be visited or Redis According to storehouse, the Redis read from database writes database from Redis can be located at different Redis servers.
- 9. reading and writing data separator according to claim 8, it is characterised in that the sliced fashion is uniformity Hash Sliced fashion, the reference address computing module include：Cut-off map unit, Hash operation point map unit, inquiry request Point map unit and reference address computing unit, wherein,Cut-off map unit, for building a circle as circle model, the threshold value cut-off pre-set is distributed on described On circle model；Hash operation point map unit, for being breathed out successively to every group of Redis database under uniformity Hash sliced fashion Uncommon computing, obtains each group Hash operation point, carries out Hash number virtualization respectively to every group of Hash operation point, Hash number is virtualized Each group Hash operation value sum to obtain virtualization sum, virtualization sum is evenly distributed to the model circle；Inquiry request point map unit, for described in inquiry request point map unit and the acquisition of reference address computing unit The key assignments (Key) included in Redis requests, Hash operation is carried out to the key assignments of acquisition, obtain the inquiry that a value is positive integer and ask Ask a little, correspond to the model circle；Reference address computing unit, for using the inquiry request point model circle on drop point site as starting point, according to pre- The lookup algorithm first set obtains the reference address of the Redis databases to be visited.
- 10. reading and writing data separator according to claim 9, it is characterised in that the sliced fashion is modulus burst Mode, the reference address computing module include：Inquiry request point computing unit, query unit and reference address obtain single Member, wherein,Inquiry request point computing unit, for obtaining the key assignments (Key) included in the Redis requests, the key assignments of acquisition is entered Row Hash operation, obtain the inquiry request point that a value is positive integer；Query unit, for inquiring about under modulus sliced fashion the Redis database group numbers configured；Reference address acquiring unit, for carrying out the Redis databases group number complementation to the inquiry request point, obtain The reference address of the Redis databases to be visited.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710549825.2A CN107346258A (en) | 2017-07-06 | 2017-07-06 | A kind of reading and writing data separation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710549825.2A CN107346258A (en) | 2017-07-06 | 2017-07-06 | A kind of reading and writing data separation method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107346258A true CN107346258A (en) | 2017-11-14 |
Family
ID=60257613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710549825.2A Pending CN107346258A (en) | 2017-07-06 | 2017-07-06 | A kind of reading and writing data separation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107346258A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664601A (en) * | 2018-05-09 | 2018-10-16 | 广州市冰海网络技术有限公司 | A kind of method of front and back end reading and writing data separation |
| CN109032511A (en) * | 2018-07-09 | 2018-12-18 | 武汉斗鱼网络科技有限公司 | Data storage method, server and storage medium |
| CN109542872A (en) * | 2018-10-26 | 2019-03-29 | 金蝶软件（中国）有限公司 | Method for reading data, device, computer equipment and storage medium |
| CN112231300A (en) * | 2020-10-21 | 2021-01-15 | 北京人大金仓信息技术股份有限公司 | Function analysis method and device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473272A (en) * | 2013-08-20 | 2013-12-25 | 小米科技有限责任公司 | Data processing method, device and system |
| CN103473267A (en) * | 2013-08-09 | 2013-12-25 | 深圳市中科新业信息科技发展有限公司 | Data storage query method and system |
| CN105260136A (en) * | 2015-09-24 | 2016-01-20 | 北京百度网讯科技有限公司 | Data read-write method and distributed storage system |
| CN106254240A (en) * | 2016-09-18 | 2016-12-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method and routing layer equipment and system |
-
2017
- 2017-07-06 CN CN201710549825.2A patent/CN107346258A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473267A (en) * | 2013-08-09 | 2013-12-25 | 深圳市中科新业信息科技发展有限公司 | Data storage query method and system |
| CN103473272A (en) * | 2013-08-20 | 2013-12-25 | 小米科技有限责任公司 | Data processing method, device and system |
| CN105260136A (en) * | 2015-09-24 | 2016-01-20 | 北京百度网讯科技有限公司 | Data read-write method and distributed storage system |
| CN106254240A (en) * | 2016-09-18 | 2016-12-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method and routing layer equipment and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664601A (en) * | 2018-05-09 | 2018-10-16 | 广州市冰海网络技术有限公司 | A kind of method of front and back end reading and writing data separation |
| CN109032511A (en) * | 2018-07-09 | 2018-12-18 | 武汉斗鱼网络科技有限公司 | Data storage method, server and storage medium |
| CN109032511B (en) * | 2018-07-09 | 2022-05-13 | 武汉斗鱼网络科技有限公司 | Data storage method, server and storage medium |
| CN109542872A (en) * | 2018-10-26 | 2019-03-29 | 金蝶软件（中国）有限公司 | Method for reading data, device, computer equipment and storage medium |
| CN109542872B (en) * | 2018-10-26 | 2021-01-22 | 金蝶软件（中国）有限公司 | Data reading method and device, computer equipment and storage medium |
| CN112231300A (en) * | 2020-10-21 | 2021-01-15 | 北京人大金仓信息技术股份有限公司 | Function analysis method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11068187B2 (en) | Systems and methods for data migration in a clustered file system | |
| US11068455B2 (en) | Mapper tree with super leaf nodes | |
| US7415490B2 (en) | Data storage system | |
| US7853770B2 (en) | Storage system, data relocation method thereof, and recording medium that records data relocation program | |
| CN107346258A (en) | A kind of reading and writing data separation method and device | |
| CN106909317B (en) | Storing data on storage nodes | |
| US20100312749A1 (en) | Scalable lookup service for distributed database | |
| US20130179495A1 (en) | System and method for alerting leakage of personal information in cloud computing environment | |
| CN106445944A (en) | Data query request processing method and apparatus, and electronic device | |
| CN107180113A (en) | A kind of big data searching platform | |
| CN116226166A (en) | Data query method and system based on data source | |
| WO2022175080A1 (en) | Cache indexing using data addresses based on data fingerprints | |
| US8589652B2 (en) | Reorganization of a fragmented directory of a storage data structure comprised of the fragmented directory and members | |
| CN111399999B (en) | Computer resource processing method, device, readable storage medium and computer equipment | |
| CN107895044A (en) | A kind of database data processing method, device and system | |
| CN112528327A (en) | Data desensitization method and device and data restoration method and device | |
| JP2006134191A (en) | Document retrieval method and its system | |
| US20200334302A1 (en) | Automatic check of search configuration changes | |
| CN111680069B (en) | Database access method and device | |
| WO2022089249A1 (en) | Information acquisition method and apparatus, server, and storage medium | |
| US11580082B2 (en) | Object storage system with control entity quota usage mapping | |
| CN108664601A (en) | A kind of method of front and back end reading and writing data separation | |
| CN106527960A (en) | Management method for multi-memory disk loads, device, document system and memory network system | |
| US20140115246A1 (en) | Apparatus, system and method for managing empty blocks in a cache | |
| US20190303037A1 (en) | Using sequential read intention to increase data buffer reuse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20210212 |
|
| AD01 | Patent right deemed abandoned |