CN105049475A - High-efficiency data storage and optimization method and system for large-scale community - Google Patents
High-efficiency data storage and optimization method and system for large-scale community Download PDFInfo
- Publication number
- CN105049475A CN105049475A CN201510273778.4A CN201510273778A CN105049475A CN 105049475 A CN105049475 A CN 105049475A CN 201510273778 A CN201510273778 A CN 201510273778A CN 105049475 A CN105049475 A CN 105049475A
- Authority
- CN
- China
- Prior art keywords
- data
- server
- central server
- algorithm
- memory space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000005457 optimization Methods 0.000 title claims abstract description 30
- 238000013500 data storage Methods 0.000 title abstract description 6
- 238000004891 communication Methods 0.000 claims description 71
- 238000013138 pruning Methods 0.000 claims description 8
- 230000008520 organization Effects 0.000 claims description 4
- 239000002699 waste material Substances 0.000 abstract 1
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a high-efficiency data storage and optimization method for a large-scale community, and the method is used in a storage structure of a multistage server. The method comprises the steps: reporting a current storage capacity and use frequency of local type data through a multi-type project server; obtaining the number of currently-needed central servers and the combination information of a plurality of projection servers after determining an algorithm to be a current algorithm, and storing the local storage data according to the number of currently-needed central servers and the combination information of the plurality of projection servers. Therefore, the invention solves problems of energy waste and unbalanced system distribution in a process of big data storage of the large-scale community. Accordingly, the method and system improve the utilization rate of the central servers through the combination and optimization of storage capacities of project servers, effectively reduce the number of central servers, and improves the safety and stability.
Description
Technical field
The present invention relates to the technical field of wireless telecommunications, the data efficient storage optimization method of especially extensive community and system.
Background technology
Along with the progressively development of Intelligent Community, its Du's district service data amount significantly promotes, and for realizing fail safe and the reliability of community data, usually needs to carry out comprehensively community data, reliable multi-level backup structure backs up.As adopted: community-level server, project level server and central server backup architecture, but in the use procedure of data backup, first, each community server join this and actual storage amount and inconsistent.Next, in storing process, easily occur that larger central server leaves unused, and on the one hand, easily causes the decline of central server utilization rate, maintenance cost rising.On the other hand, cause security of system and storage efficiency to reduce, the flexibility of system and dilatancy are declined, thus the normal operation of influential system.
Summary of the invention
For above-mentioned defect of the prior art, the invention solves in the large data storage procedure of extensive community, the wasting of resources is many, the uneven problem of system disposition.
In order to achieve the above object, the invention provides following technical scheme:
The data efficient storage optimization method of the extensive community in the present invention, the method is implemented in the storage organization of multistage server, comprises,
Step S101: polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data;
Step S102: according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm;
Step S103: the pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
Step S104: according to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
In a preferred embodiment, also comprise before described step S101:
Step S100, described polymorphic type project level server receives from multiple community server and stores data.
In a preferred embodiment, described step S101 comprises,
The type of described local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
In a preferred embodiment, the current memory space of the described pre-storage according to separate unit central server, described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the step obtaining the first total communication cost and the second total communication cost respectively comprises:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
In a preferred embodiment, the pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, the step obtaining the combined information of multiple project level server comprises:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
Meanwhile, the invention provides the data store optimization system of extensive community, wherein, comprise, polymorphic type project level server, central server and storage optimization processor;
Described polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data;
Described polymorphic type project level server is according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm;
Pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
According to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
In a preferred embodiment, described polymorphic type project level server is also configured to receive from multiple Du districts server store data.
In a preferred embodiment, the type of described local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
In a preferred embodiment, described polymorphic type project level server is also configured to:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
In a preferred embodiment, polymorphic type project level server is also configured to:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
Beneficial effect of the present invention is: the data efficient storage optimization method of extensive community provided by the present invention and system, by combination and the optimization of the memory space to project level server, improve the utilance of central server, effectively reduce the usage quantity of central server, this improves fail safe and the stability of system.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is in one embodiment of the present invention, the system composition frame diagram of the data efficient storage optimization method of extensive community;
Fig. 2 is in one embodiment of the present invention, the schematic flow sheet of the data efficient storage optimization method of extensive community;
Fig. 3 is in another embodiment of the present invention, the schematic flow sheet of the data efficient storage optimization method of extensive community.
Embodiment
Below in conjunction with accompanying drawing of the present invention, be clearly and completely described technical scheme of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
As shown in Figure 1, 2, in one embodiment of the present invention, the data efficient storage optimization method of the extensive community in the present invention, is implemented in the method and is implemented in the storage organization of multistage server, this storage organization is illustrated in figure 1 A, B, C three tier server, wherein:
(1) A level server: central server.
Back up all data of B level server, each B level server can find unique backup place on A level server; In addition, the storage data of A level server, will be used for large data analysis, carry out commercial operation.
(2) B level server: project level server.
For storing the data of each smart city, intelligence community.
Each B level server only stores a certain data of some intelligence community.
The storage data class of B level server comprises:
1) property data; 2) finance data; 3) medical data; 4) home control data; 5) agreement and instruction
(3) C level server: community-level server.
The data class of B level server stores is identical with the data class of C level server stores.
B level server and C level server sync back up, and have one-to-one relationship.
In one embodiment of the invention, the data efficient storage optimization method of extensive community, comprising:
Step S101: report polymorphic type project level server info.
In this step, polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data.
Step S102: determine current algorithm.
In this step, according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm.
Step S103: the storage information obtaining central server.
Pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
Step S104: carry out data storage.
According to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
The type of above-mentioned local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
As shown in Figure 3, one of the present invention preferred embodiment in, also comprise before described step S101: step S100, described polymorphic type project level server receives from multiple community server and stores data.
One of the present invention preferred embodiment in, the current memory space of the above-mentioned pre-storage according to separate unit central server, described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the step obtaining the first total communication cost and the second total communication cost respectively comprises:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
One of the present invention preferred embodiment in, the above-mentioned pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, the step obtaining the combined information of multiple project level server comprises:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
Above-mentioned algorithm specifically realizes by following computational process:
First, the Mathematical Modeling of the server stores that Service Operation is optimized is set up
Suppose that the memory capacity of N platform B level server is respectively: w1, w2, w3 ..., wN;
The COS of N platform B level server is respectively: s1, s2, s3 ..., sN (value of COS is 1,2,3,4,5, wherein: 1 represents that property data, 2 represent that finance data, 3 represents that medical data, 4 represents home control data, 5 presentation protocol and instructions);
The frequency of operation of each COS is P1, P2, P3, P4, P5;
The memory capacity of each A level server is W;
The communication connection cost connecting any two A level servers is C;
B level server B i (1≤i≤n) will be assigned to BX
i(1≤BX
i≤ m) on K platform A level server;
Make function
Require: provide a server stores scheme, guarantee the communication cost of the Various types of data service access of A level server and minimum.
The solution vector making n platform B level server is BX={BX
1, BX
2..., BX
n, (1≤BX
i≤ m), during initialization, by BX
i(1≤i≤n) is all set to 0.
Make function F
i(1≤i≤5) represent, store the A level number of servers of i-th (1≤i≤5) class data.
Then can provide following Mathematical Modeling,
In conjunction with above Mathematical Modeling, following algorithm (as algorithm one and algorithm two) can be provided:
Algorithm one: under given number of servers condition, the optimized algorithm of the server stores that Service Operation is optimized
// note: this algorithm adopts tree to carry out searching for and beta pruning, and this tree is virtual presence in algorithm running, does not need clear and definite the building of a tree.
Input: the memory capacity array of N platform B level server
the COS of N platform B level server is respectively: S={s
1, s
2..., s
n, s
i∈ { 1,2,3,4,5}; The memory capacity W of each A level server, given number of servers m; The frequency of operation P={p of each COS
1, p
2, p
3, p
4, p
5}
Export: the A level server location that total communication cost FinalCommuCost, N platform B level server stores: BX={BX
1, BX
2..., BX
n, (1≤BX
i≤ m).
Step 1:(initialization)
FinalCommuCost＝+∞；
The number of plies j=1 of initialization tree extreme saturation;
Initialization solution vector: FORi=1TONDO
BX
i＝0；
ENDFOR
Step 2:(has searched for)
IFj＝＝0THEN
Go to step 6;
ENDIF
The end condition of step 3:(tree deep search)
IFj＝＝nTHEN
Judge the n-th component BX separating BX
nwhether meet constraint (1), (2), (3), (4);
IFBX
nsatisfy condition, THEN
Call function calculates present communications and connects total cost CommuCost (BX);
IFCommuCost(BX)<FinalCommuCostTHEN
FinalCommuCost＝CommuCost(BX)；
Preserve and separate BX={BX
1, BX
2..., BX
n;
ENDIF
ENDIF
j＝j-1；BX
j＝BX
j+1；
Go to step 2;
ENDIF
The end condition of the downward one deck search of step 4:(tree)
IFj<nTHEN
Judge the n-th component BX separating BX
nwhether meet constraint (1), (2), (3), (4);
IFBX
nsatisfy condition, then j=j+1; Go to step 3;
ENDIF
ENDIF
The beta pruning of step 5:(tree, and to last layer backtracking)
IFj<nTHEN
Judge the n-th component BX separating BX
nwhether meet constraint (1), (2), (3), (4);
IFBX
ndo not satisfy condition, then
IFBX
j<mTHENBX
j=BX
j+ 1; (continuing search when other branch of layer)
ELSE
J=j-1, BX
j=BX
j+ 1; (tracing back to the next branch of last layer)
ENDIF
Go to step 3;
ENDIF
ENDIF
The output that step 6:(separates)
Total communication cost FinalCommuCost, exports BX={BX successively
1, BX
2..., BX
n.
IntFunctionFi(inti)
The A level number of servers of // calculating storage i-th (1≤i≤5) class data.
Definition set: SetFi=φ;
FORj＝1TONDO
IFs
jif==iTHEN//B level server B X
jthe service data type of storage be i
SetFi＝SetFi∪{BX
j}；
ENDIF
ENDFOR
Element number Sum in statistics S set etFi;
ReturnSum；
ENDFunction
IntFunctionCommuCost(BX)
// add up the total cost of communication connection of the COS of the A level server of current solution
IntCommuTotal＝0；
FORi＝1TO5DO
CommuTotal＝CommuTotal+(F
i(i)-1)×p
i；
ENDFOR
ReturnCommuTotal；
ENDFunction
Algorithm two: the optimized algorithm of the server stores that Service Operation is optimized
Input: the memory capacity array of N platform B level server
the COS of N platform B level server is respectively: S={s
1, s
2..., s
n, s
1∈ { 1,2,3,4,5}; The memory capacity W of each A level server, the frequency of operation P={p of each COS
1, p
2, p
3, p
4, p
5}
Export: A level number of servers, the A level server location that total communication cost FinalCommuCost, N platform B level server stores: BX={BX
1, BX
2..., BX
n, (1≤BX
i≤ m).
Step 1: tentatively determine the A level number of servers MIN meeting B level server stores capacity;
Calculate all data storage capacity values of B level server:
IFTotalMODW＝＝0THENMIN＝Total/W；
ELSEMIN＝Total/W+1；
ENDIF
Step 2: search for minimum A level number of servers m successively;
P=MIN; (to temporary variable P initialize)
WHILE(P≥MIN)DO
Make m=P; Call algorithm one;
IFBX={0,0 ..., 0}THEN (algorithm one does not search suitable solution)
P＝P+1；
ELSE
Record BX value successively; Go to step 3;
ENDIF
ENDWHILE
Step 3: the quantity m of export server; Export total communication cost FinalCommuCost successively, export BX={BX successively
1, BX
2..., BX
n.
Embodiment:
Provide the memory capacity table (as shown in table 1) of one 13 B level servers.
Table 2: the memory capacity table of one 13 B level servers
| Numbering | B level server name | Memory capacity | Store content |
| 1 | B1 | 10T | Property data |
| 2 | B2 | 10T | Property data |
| 3 | B3 | 5T | Finance data |
| 4 | B4 | 10T | Property data |
| 5 | B5 | 9T | Finance data |
| 6 | B6 | 4T | Medical data |
| 7 | B7 | 3T | Protocol integrated test system and instruction |
| 8 | B8 | 9T | Home control data |
| 9 | B9 | 8T | Home control data |
| 10 | B10 | 8T | Home control data |
| 11 | B11 | 8T | Finance data |
| 12 | B12 | 4T | Medical data |
| 13 | B13 | 9T | Property data |
The access frequency of all kinds of service of table 3.
| COS | Property data | Finance data | Medical data | Home control data | Agreement and instruction |
| Access frequency | 3000 | 10000 | 3000 | 20000 | 3000 |
Suppose that the communication connection cost between different A level server is C;
The total storage capacity supposing every platform A level server is 25T;
If employing non-optimal way, by B level sequence server stored in A level server, then, the storage mode of B level server is:
A1 server is stored in B1 (10T)+B2 (10T)+B3 (5T);
A2 server is stored in B4 (10T)+B5 (9T)+B6 (4T);
A3 server is stored in B7 (3T)+B8 (9T)+B9 (8T);
A4 server is stored in B10 (8T)+B11 (8T)+B12 (4T);
A5 server is stored in B13 (9T).
Needs 5 A level servers altogether.
1) the A level server that property data relate to is: A1, A2, A5, and property data, services cost is:
3000×(3-1)×C＝6000C；
2) the A level server that finance data relates to is: A1, A2, A4, and finance data service cost is:
10000×(3-1)×C＝20000C
3) the A level server that medical data relates to is: A2, A4, and medical data service cost is:
3000×(2-1)×C＝3000C
4) the A level server that home control data relate to is: A3, A4, and home control data, services cost is:
20000×(2-1)×C＝20000C
5) the A level server that agreement and instruction data relate to is: A3, and agreement and instruction data, services cost is:
3000×(1-1)×C＝0
Therefore, total communication cost is: 6000C+20000C+3000C+20000C=49000C
And adopt algorithm two, then only need 4 A level servers, the storage mode adopted is:
A1 server is stored in B1 (10T)+B2 (10T)+B3 (5T);
A2 server is stored in B4 (10T)+B7 (3T)+B13 (9T);
A3 server is stored in B5 (9T)+B6 (4T)+B11 (8T)+B12 (4T);
A4 server is stored in B8 (9T)+B9 (8T)+B10 (9T);
1) the A level server that property data relate to is: A1, A2, and property data, services cost is:
3000×(2-1)×C＝4000C；
2) the A level server that finance data relates to is: A1, A3, and finance data service cost is:
10000×(2-1)×C＝10000C
3) the A level server that medical data relates to is: A3, and medical data service cost is:
3000×(1-1)×C＝0
4) the A level server that home control data relate to is: A4, and home control data, services cost is:
20000×(1-1)×C＝0
5) the A level server that agreement and instruction data relate to is: A2, and agreement and instruction data, services cost is:
3000×(1-1)×C＝0
Therefore, total communication cost is: 4000C+10000C=14000C
Therefore, adopt algorithm two, effectively can save the memory space of server; The integrated access of A level server, disposal ability can also be improved.
Meanwhile, present invention also offers the data store optimization system of extensive community, comprise, polymorphic type project level server, central server and storage optimization processor;
Meanwhile, the invention provides the data store optimization system of extensive community, wherein, comprise, polymorphic type project level server, central server and storage optimization processor;
Described polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data;
Described polymorphic type project level server is according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm;
Pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
According to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
In a preferred embodiment, described polymorphic type project level server is also configured to receive from multiple Du districts server store data.
In a preferred embodiment, the type of described local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
In a preferred embodiment, described polymorphic type project level server is also configured to:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
In a preferred embodiment, polymorphic type project level server is also configured to:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, is anyly familiar with those skilled in the art in the technical scope that the present invention discloses; change can be expected easily or replace, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should described be as the criterion with the protection range of claim.
Claims (10)
1. the data efficient storage optimization method of extensive community, it is characterized in that, the method is implemented in the storage organization of multistage server, comprises,
Step S101: polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data;
Step S102: according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm;
Step S103: the pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
Step S104: according to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
2. the data efficient storage optimization method of extensive community according to claim 1, is characterized in that, also comprises before described step S101:
Step S100, described polymorphic type project level server receives from multiple community server and stores data.
3. the data efficient storage optimization method of extensive community according to claim 2, is characterized in that, described step S101 comprises,
The type of described local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
4. the data efficient storage optimization method of extensive community according to claim 3, it is characterized in that, the current memory space of the described pre-storage according to separate unit central server, described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the step obtaining the first total communication cost and the second total communication cost respectively comprises:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
5. the data efficient storage optimization method of the extensive community according to claim 3 or 4, it is characterized in that, pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, the step obtaining the combined information of multiple project level server comprises:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
6. the data store optimization system of extensive community, is characterized in that, comprise, polymorphic type project level server, central server and storage optimization processor;
Described polymorphic type project level server reports current memory space and the frequency of utilization of local categorical data;
Described polymorphic type project level server is according to the pre-storage of separate unit central server, the current memory space of described local categorical data and described frequency of utilization, by algorithm one and algorithm two, the communication cost that acquisition first is total respectively and second total communication cost, the algorithm corresponding to the little value of this communication cost is current algorithm;
Pre-storage according to described current algorithm and separate unit central server combines multiple described current memory space, obtains current required central server quantity; According to the corresponding described multiple project level server info of the combined information of described current memory space, obtain the combined information of multiple project level server;
According to the combined information of current required central server quantity and polymorphic type project level server, data are stored to described this locality and stores.
7. the data efficient storage optimization system of extensive community according to claim 6, is characterized in that, described polymorphic type project level server is also configured to receive from multiple community server store data.
8. the data efficient storage optimization method of extensive community according to claim 7, is characterized in that, the type of described local categorical data comprises: property data, finance data, medical data, home control data and agreement and instruction data.
9. the data efficient storage optimization method of extensive community according to claim 5, is characterized in that, described polymorphic type project level server is also configured to:
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm one, obtain the communication cost of diversiform data respectively, obtain first total communication cost according to described diversiform data communication cost;
According to the pre-storage of separate unit central server, the current memory space of multiple described local categorical data and described frequency of utilization, by algorithm two, obtain the communication cost of diversiform data respectively, obtain second total communication cost according to described diversiform data communication cost.
10. the data efficient storage optimization method of extensive community according to claim 8 or claim 9, it is characterized in that, polymorphic type project level server is also configured to:
Polymorphic type project level server described in poll, combines the current memory space of any two or polymorphic type project level server, and after obtaining combination, capability value is less than and closest to multiple item servers combinations of separate unit central server; Quantity according to described multiple item server combination determines current required central server quantity; Or;
Tree data structure is set up according to default Current central number of servers, the current memory space of polymorphic type project level server and the pre-storage of platform central server, search for and beta pruning this structure, after obtaining combination, capability value is less than and closest to the multiple item servers combination of separate unit central server; The second current required central server quantity is determined according to the quantity of described multiple item server combination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510273778.4A CN105049475B (en) | 2015-05-26 | 2015-05-26 | The data efficient storage optimization method and system of extensive community |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510273778.4A CN105049475B (en) | 2015-05-26 | 2015-05-26 | The data efficient storage optimization method and system of extensive community |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105049475A true CN105049475A (en) | 2015-11-11 |
| CN105049475B CN105049475B (en) | 2018-10-30 |
Family
ID=54455678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510273778.4A Expired - Fee Related CN105049475B (en) | 2015-05-26 | 2015-05-26 | The data efficient storage optimization method and system of extensive community |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105049475B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105045527A (en) * | 2015-06-16 | 2015-11-11 | 重庆本原致创科技发展有限公司 | Optimization method and system for data storage of large-scale community |
| CN105260127A (en) * | 2015-06-16 | 2016-01-20 | 重庆本原致创科技发展有限公司 | Efficient storage optimization method and system of data of large-scale community |
| CN110019400A (en) * | 2017-12-25 | 2019-07-16 | 深圳云天励飞技术有限公司 | Date storage method, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103037004A (en) * | 2012-12-21 | 2013-04-10 | 曙光信息产业(北京)有限公司 | Implement method and device of cloud storage system operation |
| CN103744617A (en) * | 2013-12-20 | 2014-04-23 | 北京奇虎科技有限公司 | Merging and compressing method and device for data files in key-value storage system |
| CN103763363A (en) * | 2014-01-14 | 2014-04-30 | 南京邮电大学 | Cloud storage system and file copy deployment method based on cloud storage system |
| US20140215073A1 (en) * | 2013-01-28 | 2014-07-31 | International Business Machines Corporation | Computing optimized virtual machine allocations using equivalence combinations |
| CN104536803A (en) * | 2014-12-23 | 2015-04-22 | 西安电子科技大学 | Virtual machine scheduling method based on combination optimization |
-
2015
- 2015-05-26 CN CN201510273778.4A patent/CN105049475B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103037004A (en) * | 2012-12-21 | 2013-04-10 | 曙光信息产业(北京)有限公司 | Implement method and device of cloud storage system operation |
| US20140215073A1 (en) * | 2013-01-28 | 2014-07-31 | International Business Machines Corporation | Computing optimized virtual machine allocations using equivalence combinations |
| CN103744617A (en) * | 2013-12-20 | 2014-04-23 | 北京奇虎科技有限公司 | Merging and compressing method and device for data files in key-value storage system |
| CN103763363A (en) * | 2014-01-14 | 2014-04-30 | 南京邮电大学 | Cloud storage system and file copy deployment method based on cloud storage system |
| CN104536803A (en) * | 2014-12-23 | 2015-04-22 | 西安电子科技大学 | Virtual machine scheduling method based on combination optimization |
Non-Patent Citations (2)
| Title |
|---|
| CRAIG ARMSTRONG.ETC: "Distributed Storage with Communication Costs", 《FORTY-NINTH ANNUAL ALLERTON CONFERENCE ALLERTON HOUSE, UIUC, ILLINOIS, USA》 * |
| 范宗勤: "云环境下的资源调度算法研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105045527A (en) * | 2015-06-16 | 2015-11-11 | 重庆本原致创科技发展有限公司 | Optimization method and system for data storage of large-scale community |
| CN105260127A (en) * | 2015-06-16 | 2016-01-20 | 重庆本原致创科技发展有限公司 | Efficient storage optimization method and system of data of large-scale community |
| CN110019400A (en) * | 2017-12-25 | 2019-07-16 | 深圳云天励飞技术有限公司 | Date storage method, electronic equipment and storage medium |
| CN110019400B (en) * | 2017-12-25 | 2021-01-12 | 深圳云天励飞技术有限公司 | Data storage method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105049475B (en) | 2018-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110047014B (en) | User electric quantity data restoration method based on load curve and historical electric quantity | |
| CN105468435A (en) | NFV dynamic resource distribution method | |
| CN107026877A (en) | The method and apparatus that resource is managed in cloud platform | |
| CN103841647A (en) | Multi-granularity telescopic packaging device and method based on base station resource pool | |
| CN111125887A (en) | Resource optimization configuration model establishment method for emergency recovery of power distribution network | |
| CN104202176A (en) | Computer automatic constructing method of optical network topology | |
| CN102394903B (en) | A kind of active reconstruction calculating system constructing system | |
| CN102904824A (en) | Service providing entity selection method and device | |
| CN105049475A (en) | High-efficiency data storage and optimization method and system for large-scale community | |
| CN113193996B (en) | Power optical transmission network optimization method, device, equipment and storage medium | |
| CN108322318B (en) | Alarm analysis method and equipment | |
| CN109195178A (en) | signal quality monitoring method and terminal device | |
| CN107918561A (en) | A kind of method for allocating tasks in industrial allocation formula data collecting system | |
| CN114745791A (en) | Power business slice arranging and resource configuring method and device | |
| CN109743202A (en) | Management method, device, equipment and the readable storage medium storing program for executing of data | |
| CN109586281B (en) | Power distribution network reliability assessment method, device and medium based on node optimization number | |
| CN104869011A (en) | Circuit dispatching method and circuit dispatching device | |
| CN107172193A (en) | A kind of load-balancing method and its device based on cluster | |
| CN106855991A (en) | The Accident Handling Method and system of a kind of main distribution one | |
| CN104363129B (en) | A kind of network event association analysis and dynamic early-warning method | |
| US9565101B2 (en) | Risk mitigation in data center networks | |
| CN105512895A (en) | Power transaction management method based on multi-factor combination modeling and power transaction management system thereof | |
| CN105373451A (en) | Virtual machine placement method and apparatus | |
| CN104994136A (en) | Data storage optimization method and system of large-scale community | |
| CN105260127A (en) | Efficient storage optimization method and system of data of large-scale community |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 401120 Chongqing city Yubei District Longxi Street Road No. 6 are the first new irrigation station 9 building 12-15 Applicant after: CHONGQING FANGHUI TECHNOLOGY Co.,Ltd. Address before: 401120, Yubei District, Chongqing red street street 99, Huang Guandong and garden C1 building 13-4 Applicant before: CHONGQING FANGHUI TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Guo Yajun Inventor before: Shu Haidong |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181030 |