CN108829798A - Date storage method and system based on distributed data base - Google Patents
Date storage method and system based on distributed data base Download PDFInfo
- Publication number
- CN108829798A CN108829798A CN201810570806.2A CN201810570806A CN108829798A CN 108829798 A CN108829798 A CN 108829798A CN 201810570806 A CN201810570806 A CN 201810570806A CN 108829798 A CN108829798 A CN 108829798A
- Authority
- CN
- China
- Prior art keywords
- data
- target data
- layer
- back end
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013500 data storage Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 230000029058 respiratory gaseous exchange Effects 0.000 claims 1
- 238000006467 substitution reaction Methods 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 15
- 230000004044 response Effects 0.000 abstract description 5
- 239000003795 chemical substances by application Substances 0.000 description 80
- 238000012546 transfer Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000004602 germ cell Anatomy 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 210000000352 storage cell Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Present disclose provides a kind of date storage methods based on distributed data base, belong to technical field of data processing.The distributed data base includes Agent layer, the first data Layer and the second data Layer；The method includes：The Agent layer receives the target data of request write-in, and the target data includes at least first property value and the second attribute value；The Agent layer distributes the target data to the first object back end into the first data Layer according to the first property value, and the target data is distributed to the second target data node into the second data Layer according to second attribute value；The first object back end and the second target data node store the assigned target data respectively.The disclosure can reduce the back end quantity being related to when reading data, reduce system workload, shorten the response time, and improve the reliability of system.
Description
Technical field
This disclosure relates to technical field of data processing more particularly to a kind of date storage method based on distributed data base
And system, data distributing method and device, electronic equipment, computer readable storage medium.
Background technique
With the development of data processing technique, traditional centralized data base more and more turns to distributed data base
Become.In distributed data base, data dispersion is stored onto multiple back end by network connection, it is bigger to obtain
Memory capacity and higher concurrent amount of access, therefore have more advantages compared to centralized data base.
In existing distributed data base, when data are written in external request, usually have to the storage method of data following
It is several：Request sequence or time series based on write-in data etc., data are stored in order in corresponding back end, example
As first back end is preferentially written, after first back end is write completely, second back end is written in new data；Or
Person splits data according to attribute, is split as multiple discrete numerical value, is respectively stored into multiple back end.Above-mentioned side
The shortcomings that case is：When reading multiple data, it usually needs from multiple back end called datas, increase system processing task
Workload and process response time；In order to guarantee the ability to ward off risks of system, usually each back end configures one
Standby node, however when host node and standby node are all abnormal, such as place the computer room hair of host node and standby node
Biological rationality disaster, it would still be possible to lead to loss of data.
It should be noted that information is only used for reinforcing the reason to the background of the disclosure disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
The disclosure is designed to provide a kind of date storage method based on distributed data base and system, data distribution
Method and device, electronic equipment, computer readable storage medium, and then overcome at least to a certain extent due to the prior art
The problem of method of distributed data base storing data is unfavorable for reading data caused by limitation and defect.
Other characteristics and advantages of the disclosure will be apparent from by the following detailed description, or partially by the disclosure
Practice and acquistion.
According to one aspect of the disclosure, a kind of date storage method is provided, distributed data base, the distribution are based on
Database includes Agent layer, the first data Layer and the second data Layer；The method includes：The Agent layer receives request write-in
Target data, the target data includes at least first property value and the second attribute value；The Agent layer is according to described first
Attribute value distributes the target data to the first object back end into the first data Layer, and according to second attribute
The target data is distributed the second target data node into the second data Layer by value；The first object back end and institute
It states the second target data node and stores the assigned target data respectively.
In a kind of exemplary embodiment of the disclosure, the Agent layer is according to the first property value by the number of targets
The target data is divided according to first object back end of the distribution into the first data Layer, and according to second attribute value
The second target data node being assigned in the second data Layer includes：The Agent layer is according to the first property value to the target
Data carry out the first subseries, and carry out the second subseries to the target data according to second attribute value；Described
One data Layer is according to first subseries as a result, specifying the first object back end；Second data Layer according to
Second subseries as a result, specifying the second target data node；The Agent layer by the target data distribute to
The first object back end and the second target data node.
In a kind of exemplary embodiment of the disclosure, first subseries and second subseries include that Hash turns
Hair.
In a kind of exemplary embodiment of the disclosure, the Agent layer includes at least a home agent node and one standby
Agent node；Wherein, the home agent node is for periodically sending state representation information to the standby agent node；It is described
Standby agent node is used for when judging the home agent node exception according to the state representation information, replaces the primary proxy section
Point.
In a kind of exemplary embodiment of the disclosure, first data Layer and the second data Layer respectively include multiple numbers
According to node, wherein each back end includes at least a primary data node and one from back end；The primary data node
For periodically sending state representation information from back end to described；It is described to be used for from back end when according to the state
When characterization information judges the primary data node exception, replace the primary data node.
According to one aspect of the disclosure, a kind of data distributing method based on distributed data base is provided, including：It receives
The target data of write-in is requested, the target data includes at least first property value and the second attribute value；Belong to according to described first
Property value, the back end that the target data is distributed into the first data Layer of the distributed data base is stored；Root
According to second attribute value, the target data is distributed to the back end into the second data Layer of the distributed data base
It is stored.
According to one aspect of the disclosure, a kind of data-storage system based on distributed data base is provided, including：Agency
Layer carries out the first sub-distribution, Yi Jigen for receiving the target data of write-in, and according to the first property value of the target data
The second sub-distribution is carried out according to the second attribute value of the target data；First data Layer, for according to first sub-distribution
As a result, specified first object back end stores the target data；Second data Layer, for according to second sub-distribution
As a result, specified second target data node stores the target data.
According to one aspect of the disclosure, a kind of data distribution device is provided, applied to the Agent layer of distributed data base,
The device includes：Target data receiving module, for receiving the target data of request write-in, the target data includes at least the
One attribute value and the second attribute value；First distribution module, for according to the first property value, by the target data distribute to
Back end in first data Layer of the distributed data base is stored；Second distribution module, for according to described the
Two attribute values deposit the back end that the target data is distributed into the second data Layer of the distributed data base
Storage.
According to one aspect of the disclosure, a kind of electronic equipment is provided, including：Processor；And memory, for storing
The executable instruction of the processor；Wherein, the processor is configured to above-mentioned to execute via the executable instruction is executed
Method described in any one.
According to one aspect of the disclosure, a kind of computer readable storage medium is provided, computer program is stored thereon with,
The computer program realizes method described in above-mentioned any one when being executed by processor.
The exemplary embodiment of the disclosure has the advantages that：
In the above method, system and device, the Agent layer of distributed data base is receiving external request write-in target
After the task of data, first object data section is assigned it to according to the first property value of target data and the second attribute value respectively
It puts and is stored on the second target data node.On the one hand, target data is respectively according to first property value and the second attribute value
Distributed storage is formed on two data Layers, it, can be according to specific demand flexible choice when external request reading data
The called data from the first data Layer or the second data Layer, so as to reduce the points of data section involved in data read request
Amount reduces the workload that system processes data reads task, shortens the response time of the process.On the other hand, in the present embodiment
In, each target data is stored respectively in two back end, equivalent to increase a data backup, when the first data
When disaster occurs for the back end in layer, the back end that disaster occurs can be rebuild by the second data Layer, when the
When disaster occurs for the back end in two data Layers, weight can also be carried out to the back end that disaster occurs by the first data Layer
It builds, to improve the reliability of system.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 schematically shows a kind of system architecture diagram for data storage in disclosure exemplary embodiment；
Fig. 2 schematically shows a kind of flow chart of date storage method in disclosure exemplary embodiment；
Fig. 3 schematically shows a kind of sub-process figure of date storage method in disclosure exemplary embodiment；
Fig. 4 schematically shows a kind of structural block diagram of data distribution device in disclosure exemplary embodiment；
Fig. 5 schematically shows a kind of electronic equipment for realizing the above method in disclosure exemplary embodiment；
Fig. 6 schematically shows a kind of computer-readable storage medium for realizing method in disclosure exemplary embodiment
Matter.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described attribute, knot
Structure or characteristic can be incorporated in any suitable manner in one or more embodiments.
It should be noted that in the disclosure, term "one", " one ", "the" and " described " indicating there are one or
Multiple element/component parts/etc.；Term "include", "comprise" and " having " are to indicate the open meaning being included
And refer to the element in addition to listing/component part/also may be present other than waiting other element/component part/etc.；Term " the
One " and " second " etc. are only used as label, are not the limitations to quantity, the order of its object etc..
In an exemplary embodiment of the disclosure, a kind of date storage method based on distributed data base is provided.Its
In, distributed data base can be the common distributed data base such as Hive, MongoDB, be also possible to using distributed structure/architecture
The time series databases such as InfluxDB, KairosDB.Fig. 1 show the germline in the present exemplary embodiment for data storage
System architecture diagram, which may include Agent layer 101, the first data Layer 102 and the second data Layer 103.Wherein, it acts on behalf of
Layer 101 includes at least an agent node, the read-write data task of external request can be forwarded or be distributed to the first data Layer 102
In the second data Layer 103, multiple back end are separately included in the first data Layer 102 and the second data Layer 103, surprise can be used
Number is numbered respectively with even number to be identified, can also distinguish by other means, each back end can be according to Agent layer 101
Forwarding or allocation result carry out distributed storage to data.
It should be appreciated that the quantity of agent node, the first data Layer and the back end in the second data Layer is not limited to Fig. 1 institute
Any number of agent node or back end can be set in the case where showing according to the needs of actual conditions.
In one exemplary embodiment, refering to what is shown in Fig. 2, the date storage method may comprise steps of：
In step S210, Agent layer receive request write-in target data, target data include at least first property value with
Second attribute value.
Target data is usually the data being of practical significance, and can be tuple, data list including multi-component system etc., is asking
When seeking write-in Agent layer, can be converted to can the identification of proxied layer form.Two categories are included at least in the tuple of target data
Property value.As shown in Table 1 insure records as a kind of target data of data, wherein every a line is a tuple, and table 1 contains 5
A tuple, then the tables of data can be split as 5 discrete tuples and writes respectively using each tuple as target data by Agent layer
Enter.If having numerical value missing in row, Agent layer can be identified as null value."Order No.","Time","Location",
These metadata for representing numerical value meaning of " Applicant ", " Type " are attribute, and the target data in table 1 has included five
Attribute value, then first property value and the second attribute value can be any two therein.The attribute value number of data in practical application
Measure related with its type, having one in the attribute of usual data is primary attribute for unique identification, as long as furthermore data are again
Has another attribute value being of practical significance, that is, meeting in step S210 includes wanting for two attribute values for target data
It asks, therefore the method for the present embodiment is all applicable for most of data.
| Order No. | Time | Location | Applicant | Type |
| 14330 | 20180308 140003 | Shenzhen | Jia | car |
| 14333 | 20180308 140006 | Shenzhen | Yi | life |
| 14334 | 20180308 140007 | Shanghai | Bing | property |
| 14337 | 20180308 140009 | Shanghai | Ding | life |
| 14342 | 20180308 140012 | Shenzhen | Wu | property |
Table 1
In step S220, target data is distributed the first object into the first data Layer according to first property value by Agent layer
Back end, and target data is distributed to the second target data node into the second data Layer according to the second attribute value.
Agent layer usually has the function of routing in distributed data base, and data Layer can not directly be handed over external node
Mutually, task is assigned to each back end by Agent layer.Allocation rule is usually preset distribution logic, in the present embodiment,
Allocation rule may include two parts：Mapping relations are established between the back end in first property value and the first data Layer,
And mapping relations are established between the back end in the second attribute value and the second data Layer, so as to be mapped according to two kinds
Relationship distributes target data twice.Such as the target data in table 1 is distributed into each back end shown in FIG. 1, first
Data Layer 102 can correspond to Location attribute, wherein the target data of " Location value=Shenzhen " is distributed to number
It distributes according to the target data of node 01, " Location value=Shanghai " to back end 03；Second data Layer 103 can be right
Should be in Type attribute, wherein the target data of " Type value=car " is distributed to back end 02, the target of " Type value=life "
Data are distributed to back end 04, and the target data of " Type value=property " is distributed to back end 06.Then allocation result
As shown in table 2, wherein each target data has been assigned in two back end, such as " 14330 " have been assigned to number simultaneously
According in node 01 and back end 02.
Table 2
In step S230, first object back end and the second target data node store assigned number of targets respectively
According to.
After determining first object back end and the second target data node, two nodes are received respectively by Agent layer point
The target data matched, and stored, then target data can be stored respectively in two back end.With the distribution in table 2
It as a result,, can flexible choice data according to the actual situation when external request reads data after completing target data storage for
Node is to transfer target data, such as request reads insuring for Shenzhen and records data, can transfer from back end 01
The target data of " Location value=Shenzhen ", request read insuring for property insurance type and record data, can be from data
The target data etc. of " Type value=property " is transferred in node 06, so that reading number can be met by a back end
According to request, reduce Agent layer and interact caused high workload and long response time with multiple back end.In addition, first belongs to
Property value can be different according to the actual situation from the selection of the second attribute value, are not limited to situation shown in table 2, such as table 1
For, if more in practical application retrieve data according to time or insurer, first property value can also be chosen
Time value, the second attribute value can also choose Applicant value etc..The present embodiment is not specially limited this.
Based on the setting of two data Layers, data disaster tolerance may be implemented.Such as disaster occurs for back end 01, causes wherein
Data when all lossing, Agent layer can traverse all back end of the second data Layer, therefrom read " Location value=
The data of Shenzhen ", in the back end 01 after being copied into calamity, to help back end 01 to rebuild；Work as back end
When data in 06 are loss because of disaster, Agent layer can traverse all back end of the first data Layer, therefrom read
The data of " Type value=property ", to complete to rebuild in the back end 06 after being copied into calamity.
In above-mentioned date storage method, the Agent layer of distributed data base is receiving external request write-in target data
Task after, respectively according to the first property value of target data and the second attribute value assign it to first object back end with
It is stored on second target data node.On the one hand, target data is respectively according to first property value and the second attribute value two
Distributed storage is formed on a data Layer, it, can be according to specific demand flexible choice from when external request reading data
Called data in one data Layer or the second data Layer, so as to reduce back end quantity involved in data read request,
The workload that system processes data reads task is reduced, the response time of the process is shortened.On the other hand, in the present embodiment,
Each target data is stored respectively in two back end, equivalent to increase a data backup, when a data Layer
In back end occur disaster when, can by another data Layer to occur disaster back end rebuild, with realize
The long-distance disaster of back end, to improve the reliability of system.
It in one exemplary embodiment, can also be in distributed data base when target data has multiple attribute values
Third data Layer or even the four, the 5th data Layers etc. are added, to meet the diversified demand of reading data in practical application.
By increase back end, and in Agent layer configure about third attribute value, the 4th attribute value etc. allocation rule and with
The mapping relations of newly-increased back end, it may be convenient to realize the increase of data Layer and the dilatation of system.
In one exemplary embodiment, step S220 can be realized by the step S301 to step S304 in Fig. 3：Step
In S301, Agent layer carries out the first subseries to target data according to first property value, and according to the second attribute value to target
Data carry out the second subseries；In step S302, the first data Layer is according to the first subseries as a result, specified first object data
Node；In step S303, the second data Layer is according to the second subseries as a result, specified second target data node；Step S304
In, Agent layer distributes target data to first object back end and the second target data node.Classification refers in attribute value
The mapping relations of many-one (or one-to-one) are established between back end, can be not exactly the same by attribute value but have general character
Target data is distributed into same back end, to improve the utilization rate of back end.In the present embodiment, responsible pair of Agent layer
First property value is classified with the second attribute value, obtains classification results.And which data section specific classification results correspond to
Point then matches classification results with the property value set of itself by each back end, wherein one in the first data Layer
Back end will match the success of first time classification results, can declare to become first object back end, in the second data Layer
One back end will match second of classification results success, can declare to become the second target data node.Then Agent layer
Distribution can be completed respectively using first object back end and the second target data node as destination.
In one exemplary embodiment, the first subseries and the second subseries may include Hash forwarding.Hash forwards
A kind of implementation of field formula classification is carried out to first property value and the second attribute value, its main feature is that the input space is much larger than defeated
Space out is a kind of many-to-one mapping relations.In the present embodiment, in order to guarantee the first subseries to target data and
Secondary classification result is different, can be by being turned from the second subseries using different hash algorithms in the first subseries
Hair makes two kinds of Hash forwarding values have apparent difference (digit difference, the system difference of such as numerical value), such as the first hash algorithm
All use middle square method with the second hash algorithm, but wherein the first hash algorithm take the 7th be and the 9th bit value, the second Hash
Algorithm takes the bit value of the 6th, the 8th and the 10th, then the output result of two kinds of hash algorithms has apparent difference.
The Hash repeating process of first subseries is illustrated：Agent layer can be in advance each number in the first data Layer
Different Hash forwarding values are distributed according to node, then it, can be according to target data after Agent layer receives the target data in such as table 1
" Location " value, Hash translation is obtained by preset first hash algorithm as a result, if the Hash translation result with for certain
The Hash forwarding values of one back end distribution are identical, then distribute target data to the back end.Alternatively, can also be first
The first hash algorithm of configured in advance and Hash forward the value set (Kazakhstan of different data node in each back end in data Layer
Without intersection between uncommon forwarding value set), then it, can be by target data after Agent layer receives the target data in such as table 1
" Location " value is advertised to all back end in the first data Layer, and each back end should according to the first hash algorithm
" Location " value be converted to Hash translation as a result, and the Hash forwarding values sets match with itself, wherein there is usually one
Back end can be with successful match, and back end declaration at this time receives target data.Hash about the second data Layer forwards
Process can realize that similar to the above process, details are not described herein again by preset second hash algorithm.
In order to improve the redundancy ability of system, in one exemplary embodiment, Agent layer can include at least a main generation
Manage node and a standby agent node；Wherein, home agent node is for periodically sending state table reference to standby agent node
Breath；Standby agent node is used for when judging home agent node abnormal state according to state representation information, replaces home agent node.Shape
Whether state characterization information is normal for characterizing the current state of home agent node, can be heartbeat message quasi-periodic information,
It can be the triggering property information such as abnormality alarm information.The interaction of state representation information can be it is two-way, i.e., for agent node
The information of characterization oneself state can be periodically sent to home agent node.In addition it is also possible to by the mesh in home agent node
The backup of property information cycles such as data allocation rule, back end configuration status is marked in standby agent node, so as to home agent node
Can smoothly it switch when abnormal for agent node.In other embodiments, Agent layer can also include multiple standby agent nodes,
Such as one Agent layer may include home agent node, first for agent node and second for agent node, home agent node can be with
The heartbeat of itself is sent for agent node to first, first can send the heart of itself to second for agent node for agent node
It jumps, second can send the heartbeat of itself for agent node to home agent node, thus in real now all of agent node mutually
Logical status information can set when home agent node exception and preferentially upgrade to home agent node for first for agent node.It is right
In particular number, sequence and the interactive mode of state representation information etc. of standby agent node, the present embodiment is not specially limited.
In one exemplary embodiment, the first data Layer and the second data Layer can respectively include multiple back end,
In, each back end includes at least a primary data node and one from back end；Primary data node is for periodic
State representation information is sent to from back end；It is used to work as from back end and judges that primary data node is different according to state representation information
Chang Shi replaces primary data node.In the present embodiment, whether state representation information is for characterizing the current state of primary data node
Normally, it can be heartbeat message quasi-periodic information, be also possible to the triggering property information such as abnormality alarm information.One master data section
Point can also configure multiple from back end.Respectively the data in primary data node can be periodically replicated from back end to deposit
Storage, to realize backup.By the master/slave configuration structure of back end, the redundancy ability of data Layer is increased.In addition, in data
Node can separate the read/write of data, it can write data into master data section using in the system of master/slave configuration structure
Point, and the data from back end are read, to reduce the access pressure of back end.
In an exemplary embodiment of the disclosure, a kind of data distributing method is additionally provided, can be applied to distributed number
According to the Agent layer in library, such as Agent layer 101 shown in Fig. 1.The data distributing method may comprise steps of：Receive request
The target data of write-in, target data include at least first property value and the second attribute value；According to first property value, by number of targets
It is stored according to back end of the distribution into the first data Layer of distributed data base；According to the second attribute value, by number of targets
It is stored according to back end of the distribution into the second data Layer of distributed data base.It can be in the default distribution rule of Agent layer
Then, allocation rule may include：First property value and the second attribute value are specifically which kind of attribute, the forwarding rule of first property value
Then, the forward rule etc. of the second attribute value.Based on allocation rule, above-mentioned distribution method is may be implemented in Agent layer.Wherein divide twice
With not distinguishing precedence, and can also carry out simultaneously.
In an exemplary embodiment of the disclosure, a kind of data-storage system based on distributed data base is additionally provided,
Refering to what is shown in Fig. 1, the data-storage system 100 may include：Agent layer 101, for receive write-in target data, and according to
The first property value of target data carries out the first sub-distribution, and carries out second point according to the second attribute value of target data
Match；First data Layer 102, for according to the first sub-distribution as a result, specified first object back end stores target data；The
Two data Layers 103, for according to the second sub-distribution as a result, specified second target data node stores target data.
It should be noted that Agent layer is when configuring the back end information of the first data Layer and the second data Layer, usually
It needs to carry out unique identification to back end, it, can be in order to distinguish the back end in the first data Layer and the second data Layer
The different unique identification of form is respectively allocated to the back end in two data Layers, such as identifies the first data with number
The back end (01,02,03 etc.) of layer, with the back end (AA, AB, AC etc.) of letter the second data Layer of mark；It can also make
It is distinguish with the unique identification of same form and by ad hoc rules, odd-numbered represents the first data Layer to example as shown in figure 1
Back end, even-numbered represents the back end etc. of the second data Layer.The present embodiment is not specially limited this.
Further, since target data is stored in the back end of the first data Layer and the back end of the second data Layer simultaneously
In, when reading target data, it can determine to retrieve from the back end of which data Layer according to following rule and transfer target
Data：When the indexing key words of reading order are related to first property value, corresponding number can be retrieved in the first data Layer
According to node, and transfer target data；It, can be in the second data when the indexing key words of reading order are related to the second attribute value
Corresponding back end is retrieved in layer, and transfers target data；When the indexing key words of reading order and two attribute values not
When related, it can preset and preferentially transfer target data from the first data Layer, can also preset and preferentially transfer mesh from the second data Layer
Mark data；It, can be according to associated order or related journey when the indexing key words of reading order and two attribute values related simultaneously
Degree determines, can also preset and preferentially transfer target data from the first data Layer or the second data Layer.
In an exemplary embodiment of the disclosure, a kind of data distribution device is additionally provided, can be applied to system shown in Fig. 1
The Agent layer 101 of system.As shown in figure 4, the data distribution system 400 may include：Target data receiving module 410, for connecing
The target data of request write-in is received, target data includes at least first property value and the second attribute value；First distribution module 420,
For according to first property value, the back end that target data is distributed into the first data Layer of distributed data base to be deposited
Storage；Second distribution module 430, for according to the second attribute value, target data to be distributed to the second data of distributed data base
Back end in layer is stored.The detail of each module has been described in detail in the embodiment of method part, therefore
It repeats no more.
The exemplary embodiment of the disclosure additionally provides a kind of electronic equipment that can be realized the above method.
Person of ordinary skill in the field it is understood that various aspects of the disclosure can be implemented as system, method or
Program product.Therefore, various aspects of the disclosure can be with specific implementation is as follows, i.e.,：It is complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 500 of this exemplary embodiment according to the disclosure is described referring to Fig. 5.What Fig. 5 was shown
Electronic equipment 500 is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in figure 5, electronic equipment 500 is showed in the form of universal computing device.The component of electronic equipment 500 can wrap
It includes but is not limited to：At least one above-mentioned processing unit 510, at least one above-mentioned storage unit 520, the different system components of connection
The bus 530 of (including storage unit 520 and processing unit 510), display unit 540.
Wherein, the storage unit is stored with program code, and said program code can be held by the processing unit 510
Row, so that various according to the disclosure described in the execution of the processing unit 510 above-mentioned " illustrative methods " part of this specification
The step of illustrative embodiments.For example, the processing unit 510 can execute step as shown in Figure 2：Step S210,
Agent layer receives the target data of request write-in, and target data includes at least first property value and the second attribute value；Step S220,
Agent layer distributes target data to the first object back end into the first data Layer according to first property value, and according to
Target data is distributed the second target data node into the second data Layer by two attribute values；Step S230, first object data
Node and the second target data node store assigned target data respectively.
Storage unit 520 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 521 and/or cache memory unit 522, it can further include read-only memory unit (ROM) 523.
Storage unit 520 can also include program/utility 524 with one group of (at least one) program module 525,
Such program module 525 includes but is not limited to：Operating system, one or more application program, other program modules and
It may include the realization of network environment in program data, each of these examples or certain combination.
Bus 530 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 500 can also be with one or more external equipments 700 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 500 communicate, and/or with make
Any equipment (such as the router, modulation /demodulation that the electronic equipment 500 can be communicated with one or more of the other calculating equipment
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 550.Also, electronic equipment 500 can be with
By network adapter 560 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.As shown, network adapter 560 is communicated by bus 530 with other modules of electronic equipment 500.
It should be understood that although not shown in the drawings, other hardware and/or software module can not used in conjunction with electronic equipment 500, including but not
It is limited to：Microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and
Data backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, terminal installation or network equipment etc.) is executed according to the exemplary implementation of the disclosure
The method of example.
The exemplary embodiment of the disclosure additionally provides a kind of computer readable storage medium, and being stored thereon with can be realized
The program product of this specification above method.In some possible embodiments, various aspects of the disclosure can also be realized
For a kind of form of program product comprising program code, when described program product is run on the terminal device, described program
Code is various according to the disclosure described in above-mentioned " illustrative methods " part of this specification for executing the terminal device
The step of illustrative embodiments.
It is produced refering to what is shown in Fig. 6, describing the program according to the exemplary embodiment of the disclosure for realizing the above method
Product 600, can be using portable compact disc read only memory (CD-ROM) and including program code, and can set in terminal
It is standby, such as run on PC.However, the program product of the disclosure is without being limited thereto, in this document, readable storage medium storing program for executing can
With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
Person is in connection.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or any above combination.The more specific example of readable storage medium storing program for executing is (non exhaustive
List) include：It is electrical connection, portable disc, hard disk, random access memory (RAM) with one or more conducting wires, read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable signal medium can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.
The program code for including on readable medium can transmit with any suitable medium, including but not limited to wirelessly, have
Line, optical cable, RF etc. or above-mentioned any appropriate combination.
Can with any combination of one or more programming languages come write for execute the disclosure operation program
Code, described program design language include object oriented program language-Java, C++ etc., further include conventional
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in user
It calculates and executes in equipment, partly executes on a user device, being executed as an independent software package, partially in user's calculating
Upper side point is executed on a remote computing or is executed in remote computing device or server completely.It is being related to far
Journey calculates in the situation of equipment, and remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In addition, above-mentioned attached drawing is only the schematic theory of the processing according to included by the method for disclosure exemplary embodiment
It is bright, rather than limit purpose.It can be readily appreciated that the time that above-mentioned processing shown in the drawings did not indicated or limited these processing is suitable
Sequence.In addition, be also easy to understand, these processing, which can be, for example either synchronously or asynchronously to be executed in multiple modules.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to an exemplary embodiment of the present disclosure, above-described two or
More multimode or the feature and function of unit can embody in a module or unit.Conversely, above-described one
A module or the feature and function of unit can be to be embodied by multiple modules or unit with further division.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
His embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Adaptive change follow the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by claim
It points out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the attached claims.
Claims (10)
1. a kind of date storage method based on distributed data base, which is characterized in that the distributed data base includes agency
Layer, the first data Layer and the second data Layer；The method includes：
The Agent layer receives the target data of request write-in, and the target data includes at least first property value and the second attribute
Value；
The target data is distributed the first object number into the first data Layer according to the first property value by the Agent layer
The target data is distributed to the second target data section into the second data Layer according to node, and according to second attribute value
Point；
The first object back end and the second target data node store the assigned target data respectively.
2. the method according to claim 1, wherein the Agent layer is according to the first property value by the mesh
Mark data distribute the first object back end into the first data Layer, and according to second attribute value by the number of targets
Include according to second target data node of the distribution into the second data Layer：
The Agent layer carries out the first subseries to the target data according to the first property value, and according to described second
Attribute value carries out the second subseries to the target data；
First data Layer is according to first subseries as a result, specifying the first object back end；
Second data Layer is according to second subseries as a result, specifying the second target data node；
The Agent layer distributes the target data to the first object back end and the second target data node.
3. according to the method described in claim 2, it is characterized in that, first subseries and second subseries include breathing out
Uncommon forwarding.
4. the method according to claim 1, wherein the Agent layer includes at least home agent node and one
A standby agent node；
Wherein, the home agent node is for periodically sending state representation information to the standby agent node；The standby generation
It manages node to be used for when judging the home agent node exception according to the state representation information, replaces the home agent node.
5. the method according to claim 1, wherein first data Layer and the second data Layer respectively include it is more
A back end, wherein each back end includes at least a primary data node and one from back end；
The primary data node is for periodically sending state representation information from back end to described；
It is described to be used for from back end when judging the primary data node exception according to the state representation information, described in substitution
Primary data node.
6. a kind of data distributing method, the Agent layer applied to distributed data base, which is characterized in that including：
The target data of request write-in is received, the target data includes at least first property value and the second attribute value；
According to the first property value, the target data is distributed to the number into the first data Layer of the distributed data base
It is stored according to node；
According to second attribute value, the target data is distributed to the number into the second data Layer of the distributed data base
It is stored according to node.
7. a kind of data-storage system based on distributed data base, which is characterized in that including：
Agent layer divide for the first time for receiving the target data of write-in, and according to the first property value of the target data
Match, and the second sub-distribution is carried out according to the second attribute value of the target data；
First data Layer, for according to first sub-distribution as a result, specified first object back end stores the target
Data；
Second data Layer, for according to second sub-distribution as a result, specified second target data node stores the target
Data.
8. a kind of data distribution device, the Agent layer applied to distributed data base, which is characterized in that including：
Target data receiving module, for receiving the target data of request write-in, the target data includes at least the first attribute
Value and the second attribute value；
First distribution module, for according to the first property value, the target data to be distributed to the distributed data base
The first data Layer in back end stored；
Second distribution module, for according to second attribute value, the target data to be distributed to the distributed data base
The second data Layer in back end stored.
9. a kind of electronic equipment, which is characterized in that including：
Processor；And
Memory, for storing the executable instruction of the processor；
Wherein, the processor is configured to require 1-6 described in any item via executing the executable instruction and carry out perform claim
Method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
Method described in any one of claims 1-6 is realized when being executed by processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810570806.2A CN108829798B (en) | 2018-06-05 | 2018-06-05 | Data storage method and system based on distributed database |
| PCT/CN2018/101024 WO2019232933A1 (en) | 2018-06-05 | 2018-08-17 | Data storage method and system employing distributed database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810570806.2A CN108829798B (en) | 2018-06-05 | 2018-06-05 | Data storage method and system based on distributed database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108829798A true CN108829798A (en) | 2018-11-16 |
| CN108829798B CN108829798B (en) | 2024-02-02 |
Family
ID=64144075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810570806.2A Active CN108829798B (en) | 2018-06-05 | 2018-06-05 | Data storage method and system based on distributed database |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108829798B (en) |
| WO (1) | WO2019232933A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021031527A1 (en) * | 2019-08-20 | 2021-02-25 | 蚂蚁金服（杭州）网络技术有限公司 | Distributed database table join method and device, system, server, and medium |
| CN115662181A (en) * | 2022-12-12 | 2023-01-31 | 深圳市中智车联科技有限责任公司 | Multi-beacon control method, device and system for parking lot |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514229A (en) * | 2012-06-29 | 2014-01-15 | 国际商业机器公司 | Method and device used for processing database data in distributed database system |
| US20150220463A1 (en) * | 2014-01-31 | 2015-08-06 | International Business Machines Corporation | Prioritizing storage operation requests utilizing data attributes |
| CN105335448A (en) * | 2014-08-15 | 2016-02-17 | 中国银联股份有限公司 | Distributed environment-based data storage and processing system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100461689C (en) * | 2005-01-19 | 2009-02-11 | 华为技术有限公司 | Method of protection of agent nodes in multi-layered automatic switching optical network |
| CN104765749B (en) * | 2014-01-07 | 2019-04-16 | 阿里巴巴集团控股有限公司 | A kind of date storage method and device |
| CN106557498A (en) * | 2015-09-25 | 2017-04-05 | 北京国双科技有限公司 | Date storage method and device and data query method and apparatus |
| CN106873911A (en) * | 2017-02-10 | 2017-06-20 | 济南浪潮高新科技投资发展有限公司 | The implementation method of Distributed Storage is carried out using Classfication of containers pattern |
-
2018
- 2018-06-05 CN CN201810570806.2A patent/CN108829798B/en active Active
- 2018-08-17 WO PCT/CN2018/101024 patent/WO2019232933A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514229A (en) * | 2012-06-29 | 2014-01-15 | 国际商业机器公司 | Method and device used for processing database data in distributed database system |
| US20150220463A1 (en) * | 2014-01-31 | 2015-08-06 | International Business Machines Corporation | Prioritizing storage operation requests utilizing data attributes |
| CN105335448A (en) * | 2014-08-15 | 2016-02-17 | 中国银联股份有限公司 | Distributed environment-based data storage and processing system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021031527A1 (en) * | 2019-08-20 | 2021-02-25 | 蚂蚁金服（杭州）网络技术有限公司 | Distributed database table join method and device, system, server, and medium |
| CN115662181A (en) * | 2022-12-12 | 2023-01-31 | 深圳市中智车联科技有限责任公司 | Multi-beacon control method, device and system for parking lot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108829798B (en) | 2024-02-02 |
| WO2019232933A1 (en) | 2019-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102947796B (en) | For the method and apparatus of mobile virtual resource in thimble border in the data | |
| JP5341184B2 (en) | Storage system and storage system operation method | |
| JP5052376B2 (en) | Storage system and logical unit takeover method in storage system | |
| US7299322B2 (en) | Computer system, storage device and method providing intra-storage volume identifiers corresponding to intra-host volume identifiers through volume switching commands | |
| CN109062512A (en) | A kind of distributed storage cluster, data read-write method, system and relevant apparatus | |
| CN108351860A (en) | The distributed storage devices based on RDMA of low latency | |
| CN109040212A (en) | Equipment access server cluster method, system, equipment and storage medium | |
| CN107515795A (en) | Multi-task parallel data processing method, device, medium and equipment based on queue | |
| JP2004302713A (en) | Storage system and its control method | |
| US20060156055A1 (en) | Storage network that includes an arbiter for managing access to storage resources | |
| JP2021002172A (en) | Daisy chain connection system and system control method | |
| CN103329105B (en) | Application recovery in file system | |
| JP2007094505A (en) | Data processing system, data management method, and storage system | |
| CN104838374A (en) | Decentralizing a HADOOP cluster | |
| JP5738471B2 (en) | Storage apparatus and memory control method thereof | |
| CN103345502A (en) | Transaction processing method and system of distributed type database | |
| CN108829798A (en) | Date storage method and system based on distributed data base | |
| CN108845877A (en) | The methods, devices and systems of managing internal memory | |
| CN109800234A (en) | Service platform database system, upgrade method, equipment and storage medium | |
| CN109857723A (en) | Dynamic date migration method and relevant device based on extendible capacity data-base cluster | |
| JP2020154587A (en) | Computer system and data management method | |
| CN115878046B (en) | Data processing method, system, device, storage medium and electronic equipment | |
| CN107220375A (en) | Data read-write method and server | |
| CN100365600C (en) | Addressing logical subsystems in a data storage system | |
| RU2721235C2 (en) | Method and system for routing and execution of transactions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |