CN104967660A - Network performance improving method oriented to multi-cloud architecture - Google Patents
Network performance improving method oriented to multi-cloud architecture Download PDFInfo
- Publication number
- CN104967660A CN104967660A CN201510239628.1A CN201510239628A CN104967660A CN 104967660 A CN104967660 A CN 104967660A CN 201510239628 A CN201510239628 A CN 201510239628A CN 104967660 A CN104967660 A CN 104967660A
- Authority
- CN
- China
- Prior art keywords
- file
- cloud storage
- storage server
- client
- section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/101—Server selection for load balancing based on network conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a network performance improving method oriented to multi-cloud architecture and belongs to the field of cloud computing. The method solves a problem of a conventional multi-cloud architecture system does not pay attention to performance difference of different cloud storage servers in order to improve network performance. The network performance improving method uses the multi-cloud architecture. A user file is stored redundantly in multiple cloud storage servers after being encoded by erasure codes. Each cloud storage server may just obtain a part of the data of the user file. The cloud storage server with good performance can be used as much as possible according to an actual network environment when file transmission is performed and the file transmission is prevented from being encumbered by the cloud storage server with poor performance. Thus, the network performance of a system is better than the conventional multi-cloud architecture system and any cloud storage server. While guaranteeing the high reliability, high security, and low operator locking risk of the multi-cloud architecture system, the method may dynamically adapt to the network performance of different cloud storage servers and greatly improves the network performance of the system.
Description
Technical field
The invention belongs to field of cloud calculation, be specifically related to a kind of network performance method for improving towards cloudy framework.
Background technology
Along with the develop rapidly of Internet technology, the network-termination device that user has also constantly increases, net dish as a kind of for user provide multiple device file to store and synchronous cloud storage products fashionable gradually.By installing client on a user device, the file in user's local synchronization file can automatically be stored into cloud storage server, and is synchronized on other equipment that he has.This emerging cloud storage products of net dish has just attracted a large amount of users, Typical Representative Dropbox wherein at once once release, has had more than 300,000,000 users in May, 2014.And the IT giant as Google and Microsoft, also add one after another and compete and be proposed similar cloud storage products.
Different from this enterprise-level cloud storage products with clear and definite service level agreement of Amazon S3, net dish provides free or only collects the expense of seldom amount often, but also lacks enough quality of service guarantees simultaneously.Therefore user often runs into the problems such as the slow and file of such as service disruption, file transfer speed is stolen in the process using net dish.According to famous scientific and technological blog the next web, the OneDrive net dish of Microsoft once experienced by August, 2013 service disruption reaching three days.In addition, also have research to point out, the synchronizing speed of Dropbox just constantly declines along with the growth of its customer volume.
The proposition of cloudy associating concept compensate for the deficiency existing for single cloud storage products.By being stored on multiple cloud storage server by data redundancy, the system of cloudy framework can reach higher reliability and fail safe, and can effectively reduce carrier-lock risk.Existing cloudy architecture system is To enterprises level cloud storage products often, and they do not consider the performance difference between different cloud storage server, but directly redundant data is evenly distributed on multiple cloud storage server.This simple data transmission method cannot obtain good network performance, even cannot surmount the cloud storage server that wherein performance is best.Except data store, the synchronizing function that net dish also needs intended for end consumers to provide extra, the network condition faced by it is also more complicated and changeable compared to enterprise-level cloud storage products.Therefore, traditional cloudy architecture system is difficult to meet the demand adapting to emerging net dish product.
For ease of understanding the present invention, below relational language is made an explanation:
In the present invention, client integrates N number of cloud storage server, N > 1,
File data: in synchronizing process, file in local synchronization file is cut into one or more file section, each file section is generated the coded data block of (k-1) N number of redundancy by correcting and eleting codes coding by k original data block, 1 < k≤20, these coded data blocks are referred to as file data, it is the base unit that file in local synchronization file carries out files passe and download between client and multiple cloud storage server, utilizes wherein any k coded data block can recover original document section;
In reliability, client is passed through used correcting and eleting codes and is ensured to be no less than K when having in N number of cloud storage server
rindividually can normally to run, 0 < K
r≤ N, then the file of user can normally carry out synchronously, that is, tolerating maximum N-K
rindividual cloud storage server breaks down; Each cloud storage server minimum needs stores
individual data block to meet reliability requirement,
it is right to represent
result round up;
In fail safe, client ensures that any single cloud storage server victim is broken through, and assailant cannot recover original document; Therefore, the most multipotency of each cloud storage server stores k-1 data block to meet security requirement;
Metadata: metadata record the bibliographic structure of local synchronization file and the metamessage of All Files, the metamessage of each file comprises the file data metamessage of file path, increase or modification time, file size and correspondence, and file data metamessage comprises the local path of all coded data blocks of this file and stores the title of cloud storage server of each coded data block;
The metadata that client stores, is called local metadata; The metadata stored in cloud storage server is called high in the clouds metadata, and same metadata will preserve backup in all cloud storage servers.
Summary of the invention
The invention provides a kind of network performance method for improving towards cloudy framework, solve the problem that traditional cloudy architecture system does not consider performance difference between different cloud storage server, to promote network performance.
A kind of network performance method for improving towards cloudy framework provided by the present invention, comprises the following steps:
(1) client collects the local change record e of local synchronization file, comprises the increase of file, amendment and deletion record; Increase, amendment and deletion record include file path and time of origin respectively;
(2) local change record e is applied to the local metadata V of local cache
o, thus obtain local new metadata V
l;
(3) from multiple cloud storage server, high in the clouds metadata V is downloaded
c, and by high in the clouds metadata V
cincrease or modification time, judge high in the clouds metadata V
cwhether once upgraded by other client, and be, and carried out step (4), otherwise go to step (10);
(4) by local metadata V
owith high in the clouds metadata V
ccompare the difference finding out the two, obtain from V
obe altered to V
crequired high in the clouds change record e ';
(5) high in the clouds change record e ' is applied to local new metadata V
l, thus upgrade local new metadata V
l;
(6) client judges that whether there is file in the change record e ' of high in the clouds increases or amendment record, is carry out step (7), otherwise go to step (8);
(7) client increases or amendment record according to the file in the change record e ' of high in the clouds, the coded data block of its respective file is downloaded to this locality from multiple cloud storage server, and be reduced into original document after being decoded by correcting and eleting codes, carry out step (8);
(8) client judges whether there is file erase record in the change record e ' of high in the clouds, is, carries out step (9), otherwise goes to step (10);
(9) client is according to the file erase record in the change record e ' of high in the clouds, is deleted by the file of its correspondence from local synchronization file, carries out step (10);
(10) client judges that whether there is file in local change record e increases or amendment record, is carry out step (11), otherwise go to step (12);
(11) client increases or amendment record according to the file in local change record e, its respective file is carried out cutting and correcting and eleting codes coding, generates the coded data block of redundancy, then upload to multiple cloud storage server by them from this locality; Carry out step (12);
(12) client judges whether there is file erase record in local change record e, is, carries out step (13), otherwise goes to step (14);
(13) client is according to the file erase record in local change record e, proposes file deletion requests to multiple cloud storage server; By multiple cloud storage server respectively by the coded data block delete of file erase record respective file;
(14) client by uploading lock file in multiple cloud storage server, obtains high in the clouds metadata V
crenewal authority;
(15) client is by local new metadata V
lupload in all cloud storage servers, by high in the clouds metadata V
ccontent update be V
l;
(16) client proposes file deletion requests to multiple cloud storage server, and by multiple cloud storage server respectively by the lock file erase of this client upload, thus release is to high in the clouds metadata V
crenewal authority, it is synchronous to complete this locality-high in the clouds.
Described step (7) comprises following sub-step:
(7.1) increase or amendment record according to the file in the change record e ' of high in the clouds, the All Files section of its respective file is sequentially formed download queue;
(7.2) the average speed of download of all cloud storage servers according to its all download interface channel is sorted from high to low;
(7.3) described download queue is from the beginning sequentially traveled through, for each file section, successively the available downloads interface channel of each cloud storage server after sequence is distributed to this file section and carried out the download of coded data block, available downloads interface channel quantity is no more than the coded data number of blocks of the file section that this cloud storage server stores, until this file section is assigned with k download interface channel altogether, 1 < k≤20; Often download a coded data block, just discharge the download interface channel of the cloud storage server of its correspondence, when a file section completes the download of k coded data block, namely from download queue, delete this file section;
(7.4) by correcting and eleting codes, the coded data block of described file section is decoded, recover original document section;
(7.5) iteron step (7.2) ~ (7.4) are until it is empty for downloading queue, and the All Files section of a file all reverts to original document section, and they are spliced to form original document again.
Transmission speed constantly according to the data block downloaded in transmitting procedure assesses the download performance of each cloud storage server, thus the good cloud storage server of download performance downloading data block as much as possible can be utilized, highly shortened the file download time, improve network performance.
Described step (11) comprises following sub-step:
(11.1) increase or amendment record according to the file in local change record e, its respective file is cut into one or more file section according to the size of 4MB, and All Files section is sequentially formed uploads queue;
(11.2) for each file section, correcting and eleting codes is used to encode to file, each file section is generated the coded data block of (k-1) N number of redundancy by correcting and eleting codes coding by k original data block, 1 < k≤20, N > 1;
(11.3) coded data block of each file section is submitted to and uploads in the corresponding document section of queue;
(11.4) from the beginning sequentially traversal uploads queue, for each file section, respectively from each cloud storage server distribute one and availablely upload interface channel to this file section and carry out uploading of coded data block, the coded data block making file section corresponding can be uploaded to equably in each cloud storage server and be stored; Often upload a coded data block, what just discharge the cloud storage server of its correspondence uploads interface channel;
(11.5) from the beginning sequentially traversal uploads queue, judge whether to there is a certain file section, the coded data block that its cloud storage server uploading interface channel to release is uploaded is individual more than k-1, be the interface channel of release is distributed to this file section and carries out uploading of coded data block, carry out sub-step (11.6); Otherwise directly carry out sub-step (11.6);
(11.6) judge whether a file section has had k to be uploaded in multiple cloud storage server according to block, is, this file section is scheduled for uploads queue tail, carries out sub-step (11.7); Otherwise rotor step (11.4);
(11.7) judge whether a file section stores in each cloud storage server to be no less than
individual data block, 0 < K
r≤ N, is, deletes this file section from uploading queue, rotor step (11.8); Otherwise rotor step (11.4);
it is right to represent
result round up;
(11.8) iteron step (11.4) ~ (11.7) are until upload queue for empty.
The cloud storage server uploading data block as much as possible uploading better performances can be discharged in transmitting procedure; Under the prerequisite ensureing reliability and fail safe, making file section complete rapidly k uploading according to block, so that other client is downloaded early, avoiding the cloud storage server by uploading poor performance to tie down.
Described step (14) comprises following sub-step:
(14.1) generate in client the lock file lock_d that is sky with the content of client name d name, and lock file lock_d is uploaded to each cloud storage server;
(14.2) to each cloud storage server, whether inquiry only exists the lock file lock_d that client d uploads, and is think that client d obtains the lock of this cloud storage server, carries out sub-step (14.3); Otherwise think that client d does not obtain the lock of this cloud storage server, carry out sub-step (14.3);
(14.3) judging whether client d obtains the lock exceeding cloud storage server more than half, is that then client d obtains high in the clouds metadata V
crenewal authority; Otherwise carry out sub-step (14.4);
(14.4) client d proposes file deletion requests to all cloud storage servers, is deleted by the lock file lock_d of this client upload respectively by each cloud storage server, goes to step (14.1) after waiting for one random period.
This lock mechanism ensure that at one time, only has at most a client to have high in the clouds metadata V
crenewal authority, ensure that high in the clouds metadata V
cconsistency.
Compared with prior art, the present invention has following beneficial effect:
1, owing to adopting cloudy framework, be stored in redundantly in multiple cloud storage server after being encoded by correcting and eleting codes by user file, system has very high reliability, even if part cloud storage server lost efficacy also normally can access user file; Simultaneously because each cloud storage server provides commercial city can only obtain the partial data of file, thus drastically increase the fail safe of file, and reduce the risk of carrier-lock.
2, can according to the network environment of the reality cloud storage server that utility is good as much as possible when carrying out file transfer, avoid being tied down by the cloud storage server of poor performance simultaneously, make the network performance of system than traditional cloudy framework and arbitrary cloud storage server all good.
The present invention is while guaranteeing the high reliability of cloudy architecture system, high security and low carrier-lock risk, and dynamic ground adapts to the network performance of different cloud storage server isomery, the greatly network performance of elevator system.
Accompanying drawing explanation
Fig. 1 is the FB(flow block) of the embodiment of the present invention;
Fig. 2 is the FB(flow block) of step 7 in embodiment;
Fig. 3 is the FB(flow block) of step 11 in embodiment;
Fig. 4 is the FB(flow block) of step 14 in embodiment.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in more detail.
In embodiments of the invention, client integrates 5 cloud storage servers,
As shown in Figure 1, embodiments of the invention, comprise the following steps:
(1) client collects the local change record e of local synchronization file, comprises the increase of file, amendment and deletion record; Increase, amendment and deletion record include file path and time of origin respectively;
(2) local change record e is applied to the local metadata V of local cache
o, thus obtain local new metadata V
l;
(3) from multiple cloud storage server, high in the clouds metadata V is downloaded
c, and by high in the clouds metadata V
cincrease or modification time, judge high in the clouds metadata V
cwhether once upgraded by other client, and be, and carried out step (4), otherwise go to step (10);
(4) by local metadata V
owith high in the clouds metadata V
ccompare the difference finding out the two, obtain from V
obe altered to V
crequired high in the clouds change record e ';
(5) high in the clouds change record e ' is applied to local new metadata V
l, thus upgrade local new metadata V
l;
(6) client judges that whether there is file in the change record e ' of high in the clouds increases or amendment record, is carry out step (7), otherwise go to step (8);
(7) client increases or amendment record according to the file in the change record e ' of high in the clouds, the coded data block of its respective file is downloaded to this locality from multiple cloud storage server, and be reduced into original document after being decoded by Reed Solomon code, carry out step (8);
(8) client judges whether there is file erase record in the change record e ' of high in the clouds, is, carries out step (9), otherwise goes to step (10);
(9) client is according to the file erase record in the change record e ' of high in the clouds, is deleted by the file of its correspondence from local synchronization file, carries out step (10);
(10) client judges that whether there is file in local change record e increases or amendment record, is carry out step (11), otherwise go to step (12);
(11) client increases or amendment record according to the file in local change record e, and its respective file is carried out cutting and Reed-Solomon codes, generates the coded data block of redundancy, then uploads to multiple cloud storage server by them from this locality; Carry out step (12);
(12) client judges whether there is file erase record in local change record e, is, carries out step (13), otherwise goes to step (14);
(13) client is according to the file erase record in local change record e, proposes file deletion requests to multiple cloud storage server; By multiple cloud storage server respectively by the coded data block delete of file erase record respective file;
(14) client by uploading lock file in multiple cloud storage server, obtains high in the clouds metadata V
crenewal authority;
(15) client is by local new metadata V
lupload in all cloud storage servers, by high in the clouds metadata V
ccontent update be V
l;
(16) client proposes file deletion requests to multiple cloud storage server, and by multiple cloud storage server respectively by the lock file erase of this client upload, thus release is to high in the clouds metadata V
crenewal authority, it is synchronous to complete this locality-high in the clouds.
As shown in Figure 2, described step (7) comprises following sub-step:
(7.1) increase or amendment record according to the file in the change record e ' of high in the clouds, the All Files section of its respective file is sequentially formed download queue;
(7.2) the average speed of download of all cloud storage servers according to its all download interface channel is sorted from high to low;
(7.3) described download queue is from the beginning sequentially traveled through, for each file section, successively the available downloads interface channel of each cloud storage server after sequence is distributed to this file section and carried out the download of coded data block, available downloads interface channel quantity is no more than the coded data number of blocks of the file section that this cloud storage server stores, until this file section is assigned with 3 download interface channels altogether; Often download a coded data block, just discharge the download interface channel of the cloud storage server of its correspondence, when a file section completes the download of 3 coded data blocks, namely from download queue, delete this file section;
(7.4) by Reed Solomon code, the coded data block of described file section is decoded, recover original document section;
(7.5) iteron step (7.2) ~ (7.4) are until it is empty for downloading queue, and the All Files section of a file all reverts to original document section, and they are spliced to form original document again.
As shown in Figure 3, described step (11) comprises following sub-step:
(11.1) increase or amendment record according to the file in local change record e, its respective file is cut into one or more file section according to the size of 4MB, and All Files section is sequentially formed uploads queue;
(11.2) for each file section, use Reed Solomon code to encode to file, each file section is generated the coded data block of (3-1) × 5 redundancy by Reed-Solomon codes by 3 original data blocks;
11.3) coded data block of each file section is submitted to and uploads in the corresponding document section of queue;
(11.4) from the beginning sequentially traversal uploads queue, for each file section, respectively from each cloud storage server distribute one and availablely upload interface channel to this file section and carry out uploading of coded data block, the coded data block making file section corresponding can be uploaded to equably in each cloud storage server and be stored; Often upload a coded data block, what just discharge the cloud storage server of its correspondence uploads interface channel;
(11.5) from the beginning sequentially traversal uploads queue, judge whether to there is a certain file section, the coded data block that its cloud storage server uploading interface channel to release is uploaded is individual more than 3-1, be the interface channel of release is distributed to this file section and carries out uploading of coded data block, carry out sub-step (11.6); Otherwise directly carry out sub-step (11.6);
(11.6) judge whether a file section has had 3 data blocks to be uploaded in multiple cloud storage server, is, this file section is scheduled for uploads queue tail, carries out sub-step (11.7); Otherwise rotor step (11.4);
(11.7) judge whether a file section stores in each cloud storage server to be no less than
individual data block, k=3, K
r=3, be delete this file section from uploading queue, rotor step (11.8); Otherwise rotor step (11.4);
(11.8) iteron step (11.4) ~ (11.7) are until upload queue for empty.
As shown in Figure 4, described step (14) comprises following sub-step:
(14.1) generate in client the lock file lock_d that is sky with the content of client name d name, and lock file lock_d is uploaded to each cloud storage server;
(14.2) to each cloud storage server, whether inquiry only exists the lock file lock_d that client d uploads, and is think that client d obtains the lock of this cloud storage server, carries out sub-step (14.3); Otherwise think that client d does not obtain the lock of this cloud storage server, carry out sub-step (14.3);
(14.3) judging whether client d obtains the lock exceeding cloud storage server more than half, is that then client d obtains high in the clouds metadata V
crenewal authority; Otherwise carry out sub-step (14.4);
(14.4) client d proposes file deletion requests to all cloud storage servers, is deleted by the lock file lock_d of this client upload respectively by each cloud storage server, goes to step (14.1) after waiting for one random period.
Claims (4)
1., towards a network performance method for improving for cloudy framework, it is characterized in that, comprise the following steps:
(1) client collects the local change record e of local synchronization file, comprises the increase of file, amendment and deletion record; Increase, amendment and deletion record include file path and time of origin respectively;
(2) local change record e is applied to the local metadata V of local cache
o, thus obtain local new metadata V
l;
(3) from multiple cloud storage server, high in the clouds metadata V is downloaded
c, and by high in the clouds metadata V
cincrease or modification time, judge high in the clouds metadata V
cwhether once upgraded by other client, and be, and carried out step (4), otherwise go to step (10);
(4) by local metadata V
owith high in the clouds metadata V
ccompare the difference finding out the two, obtain from V
obe altered to V
crequired high in the clouds change record e ';
(5) high in the clouds change record e ' is applied to local new metadata V
l, thus upgrade local new metadata V
l;
(6) client judges that whether there is file in the change record e ' of high in the clouds increases or amendment record, is carry out step (7), otherwise go to step (8);
(7) client increases or amendment record according to the file in the change record e ' of high in the clouds, the coded data block of its respective file is downloaded to this locality from multiple cloud storage server, and be reduced into original document after being decoded by correcting and eleting codes, carry out step (8);
(8) client judges whether there is file erase record in the change record e ' of high in the clouds, is, carries out step (9), otherwise goes to step (10);
(9) client is according to the file erase record in the change record e ' of high in the clouds, is deleted by the file of its correspondence from local synchronization file, carries out step (10);
(10) client judges that whether there is file in local change record e increases or amendment record, is carry out step (11), otherwise go to step (12);
(11) client increases or amendment record according to the file in local change record e, its respective file is carried out cutting and correcting and eleting codes coding, generates the coded data block of redundancy, then upload to multiple cloud storage server by them from this locality; Carry out step (12);
(12) client judges whether there is file erase record in local change record e, is, carries out step (13), otherwise goes to step (14);
(13) client is according to the file erase record in local change record e, proposes file deletion requests to multiple cloud storage server; By multiple cloud storage server respectively by the coded data block delete of file erase record respective file;
(14) client by uploading lock file in multiple cloud storage server, obtains high in the clouds metadata V
crenewal authority;
(15) client is by local new metadata V
lupload in all cloud storage servers, by high in the clouds metadata V
ccontent update be V
l;
(16) client proposes file deletion requests to multiple cloud storage server, and by multiple cloud storage server respectively by the lock file erase of this client upload, thus release is to high in the clouds metadata V
crenewal authority, it is synchronous to complete this locality-high in the clouds.
2. network performance method for improving as claimed in claim 1, it is characterized in that, described step (7) comprises following sub-step:
(7.1) increase or amendment record according to the file in the change record e ' of high in the clouds, the All Files section of its respective file is sequentially formed download queue;
(7.2) the average speed of download of all cloud storage servers according to its all download interface channel is sorted from high to low;
(7.3) described download queue is from the beginning sequentially traveled through, for each file section, successively the available downloads interface channel of each cloud storage server after sequence is distributed to this file section and carried out the download of coded data block, available downloads interface channel quantity is no more than the coded data number of blocks of the file section that this cloud storage server stores, until this file section is assigned with k download interface channel altogether, 1 < k≤20; Often download a coded data block, just discharge the download interface channel of the cloud storage server of its correspondence, when a file section completes the download of k coded data block, namely from download queue, delete this file section;
(7.4) by correcting and eleting codes, the coded data block of described file section is decoded, recover original document section;
(7.5) iteron step (7.2) ~ (7.4) are until it is empty for downloading queue, and the All Files section of a file all reverts to original document section, and they are spliced to form original document again.
3. network performance method for improving as claimed in claim 1, it is characterized in that, described step (11) comprises following sub-step:
(11.1) increase or amendment record according to the file in local change record e, its respective file is cut into one or more file section according to the size of 4MB, and All Files section is sequentially formed uploads queue;
(11.2) for each file section, correcting and eleting codes is used to encode to file, each file section is generated the coded data block of (k-1) N number of redundancy by correcting and eleting codes coding by k original data block, 1 < k≤20, N > 1;
(11.3) coded data block of each file section is submitted to and uploads in the corresponding document section of queue;
(11.4) from the beginning sequentially traversal uploads queue, for each file section, respectively from each cloud storage server distribute one and availablely upload interface channel to this file section and carry out uploading of coded data block, the coded data block making file section corresponding can be uploaded to equably in each cloud storage server and be stored; Often upload a coded data block, what just discharge the cloud storage server of its correspondence uploads interface channel;
(11.5) from the beginning sequentially traversal uploads queue, judge whether to there is a certain file section, the coded data block that its cloud storage server uploading interface channel to release is uploaded is individual more than k-1, be the interface channel of release is distributed to this file section and carries out uploading of coded data block, carry out sub-step (11.6); Otherwise directly carry out sub-step (11.6);
(11.6) judge whether a file section has had k to be uploaded in multiple cloud storage server according to block, is, this file section is scheduled for uploads queue tail, carries out sub-step (11.7); Otherwise rotor step (11.4);
(11.7) judge whether a file section stores in each cloud storage server to be no less than
individual data block, 0 < K
r≤ N, is, deletes this file section from uploading queue, rotor step (11.8); Otherwise rotor step (11.4);
it is right to represent
result round up;
(11.8) iteron step (11.4) ~ (11.7) are until upload queue for empty.
4. network performance method for improving as claimed in claim 1, it is characterized in that, described step (14) comprises following sub-step:
(14.1) generate in client the lock file lock_d that is sky with the content of client name d name, and lock file lock_d is uploaded in each cloud storage server;
(14.2) to each cloud storage server, whether inquiry only exists the lock file lock_d that client d uploads, and is think that client d obtains the lock of this cloud storage server, carries out sub-step (14.3); Otherwise think that client d does not obtain the lock of this cloud storage server, carry out sub-step (14.3);
(14.3) judging whether client d obtains the lock exceeding cloud storage server more than half, is that then client d obtains high in the clouds metadata V
crenewal authority; Otherwise carry out sub-step (14.4);
(14.4) client d proposes file deletion requests to all cloud storage servers, is deleted by the lock file lock_d of this client upload respectively by each cloud storage server, goes to step (14.1) after waiting for one random period.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510239628.1A CN104967660B (en) | 2015-05-11 | 2015-05-11 | A kind of network performance method for improving towards more cloud frameworks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510239628.1A CN104967660B (en) | 2015-05-11 | 2015-05-11 | A kind of network performance method for improving towards more cloud frameworks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104967660A true CN104967660A (en) | 2015-10-07 |
| CN104967660B CN104967660B (en) | 2018-01-05 |
Family
ID=54221601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510239628.1A Active CN104967660B (en) | 2015-05-11 | 2015-05-11 | A kind of network performance method for improving towards more cloud frameworks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104967660B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354830A (en) * | 2016-08-31 | 2017-01-25 | 天津南大通用数据技术股份有限公司 | Device and method for data synchronization of database cluster nodes |
| CN106657266A (en) * | 2016-11-10 | 2017-05-10 | 南京云创大数据科技股份有限公司 | Architecture and method of secure and reliable public cloud storage system based on public clouds |
| CN107124304A (en) * | 2017-04-19 | 2017-09-01 | 千寻位置网络有限公司 | Resources configuration management method and system, cloud terminal based on cloud resource |
| CN110032544A (en) * | 2018-05-29 | 2019-07-19 | 普安科技股份有限公司 | The right management method and its cloud data-storage system of cloud service |
| CN110851080A (en) * | 2019-11-04 | 2020-02-28 | 紫光云技术有限公司 | Distributed storage management system of multi-cloud-disk platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050097144A1 (en) * | 2003-11-04 | 2005-05-05 | Taiwan Semiconductor Manufacturing Co. | Performance tuning at CM loader program while replicating EQP list for IBM SiView |
| CN103716370A (en) * | 2013-10-13 | 2014-04-09 | 北界创想（北京）软件有限公司 | Method and device for data synchronization between multiple clients |
| CN103916479A (en) * | 2014-04-15 | 2014-07-09 | 大连理工大学 | Cloud synchronous local area network accelerating system based on working group document |
| CN103942208A (en) * | 2013-01-18 | 2014-07-23 | 联想(北京)有限公司 | File processing method and device |
| CN104156377A (en) * | 2013-05-13 | 2014-11-19 | 孙良茵 | Virtual file system achieving reliability through multi-network-disk redundant storage |
-
2015
- 2015-05-11 CN CN201510239628.1A patent/CN104967660B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050097144A1 (en) * | 2003-11-04 | 2005-05-05 | Taiwan Semiconductor Manufacturing Co. | Performance tuning at CM loader program while replicating EQP list for IBM SiView |
| CN103942208A (en) * | 2013-01-18 | 2014-07-23 | 联想(北京)有限公司 | File processing method and device |
| CN104156377A (en) * | 2013-05-13 | 2014-11-19 | 孙良茵 | Virtual file system achieving reliability through multi-network-disk redundant storage |
| CN103716370A (en) * | 2013-10-13 | 2014-04-09 | 北界创想（北京）软件有限公司 | Method and device for data synchronization between multiple clients |
| CN103916479A (en) * | 2014-04-15 | 2014-07-09 | 大连理工大学 | Cloud synchronous local area network accelerating system based on working group document |
Non-Patent Citations (2)
| Title |
|---|
| SHVACHKO K,KUANG H,RADIA S,ET AL.: "the hadoop distributed file system", 《2010 IEEE 26TH SYMPOSIUM ON MASS STORAGE SYSTEMS AND TECHNOLOGIES》 * |
| 王意洁，孙伟东，周松等: "云计算环境下的分布式存储关键技术", 《软件学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354830A (en) * | 2016-08-31 | 2017-01-25 | 天津南大通用数据技术股份有限公司 | Device and method for data synchronization of database cluster nodes |
| CN106657266A (en) * | 2016-11-10 | 2017-05-10 | 南京云创大数据科技股份有限公司 | Architecture and method of secure and reliable public cloud storage system based on public clouds |
| CN107124304A (en) * | 2017-04-19 | 2017-09-01 | 千寻位置网络有限公司 | Resources configuration management method and system, cloud terminal based on cloud resource |
| CN110032544A (en) * | 2018-05-29 | 2019-07-19 | 普安科技股份有限公司 | The right management method and its cloud data-storage system of cloud service |
| CN110032544B (en) * | 2018-05-29 | 2023-09-15 | 普安科技股份有限公司 | Authority management method of cloud service and cloud data storage system thereof |
| CN110851080A (en) * | 2019-11-04 | 2020-02-28 | 紫光云技术有限公司 | Distributed storage management system of multi-cloud-disk platform |
| CN110851080B (en) * | 2019-11-04 | 2023-07-04 | 紫光云技术有限公司 | Distributed storage management system of multi-cloud-disk platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104967660B (en) | 2018-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8788831B2 (en) | More elegant exastore apparatus and method of operation | |
| CN104967660A (en) | Network performance improving method oriented to multi-cloud architecture | |
| US9984093B2 (en) | Technique selection in a deduplication aware client environment | |
| EP3304310B1 (en) | System and method for automatic cloud-based full-data backup and restore on mobile devices | |
| US20120089579A1 (en) | Compression pipeline for storing data in a storage cloud | |
| US9990352B2 (en) | Chunk compression in a deduplication aware client environment | |
| US20120089775A1 (en) | Method and apparatus for selecting references to use in data compression | |
| EP2735972A1 (en) | Data transmission and reception system | |
| CN107154945A (en) | A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes | |
| CN110636141B (en) | Multi-cloud storage system based on cloud and mist cooperation and management method thereof | |
| CN104573064B (en) | A kind of data processing method under big data environment | |
| CN102281314A (en) | Realization method and apparatus for high-efficient and safe data cloud storage system | |
| US10459886B2 (en) | Client-side deduplication with local chunk caching | |
| CN102024005B (en) | Issuance system and method of information item set directories | |
| US20160044100A1 (en) | Accelerating transfer protocols | |
| US10339124B2 (en) | Data fingerprint strengthening | |
| US20120296871A1 (en) | File managing apparatus for processing an online storage service | |
| WO2016130091A1 (en) | Methods of encoding and storing multiple versions of data, method of decoding encoded multiple versions of data and distributed storage system | |
| US10346256B1 (en) | Client side cache for deduplication backup systems | |
| CN109451004A (en) | Cloud storage system and method based on cloud bridge | |
| CN110419029B (en) | Method for partially updating data content in distributed storage network | |
| Sipos et al. | Adaptive network coded clouds: High speed downloads and cost-effective version control | |
| CN104902000A (en) | Method for using p2p technology to fast transmit virtual machine template | |
| CN104270457A (en) | Family cloud center file sharing system based on mobile cross-platform architecture | |
| US20200097445A1 (en) | Deduplicating distributed erasure coded objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |